|
SAUL ROCKMAN regularly transforms data and information into knowledge and wisdom - with style, clarity, and humor. A long-time member of the educational computing community, he is founder and president of ROCKMAN ET AL, which provides educational technology research and evaluation along with consulting on policy development to educational organizations and agencies (www.rockman.com). Having closely studied classrooms, programs, and systems, Saul has particularly practical insights into how to observe, contemplate, and assess what does - or doesn't - happen after the equipment is delivered. He has a lot to offer those of us who need to plan and document our own efforts in these regards. Here are some of Saul's suggestions. Whatever happens: • MAKE SURE THAT... if you are evaluating the impact of technology in schools, that you actually have a meaningful project. Using Math Blaster a half hour every other day for two weeks is not likely to change standardized test scores dramatically. If you want to attribute change to the influence of technology, be certain that students have enough access to it and that the access is used for important things - before you apply measures to assess impact. "No difference" is the most likely outcome. But take some comfort in knowing that the scores do not go down! • LOOK CAREFULLY FOR... the aspects of classroom activities where technology can have an impact. Be specific, not global. Look at innovative ways to capture quantitative outcomes: people are often happier with a number than with meaningful, but complex information. If you are using technology-based student presentations, find a way of having the students use a rubric to assess each other's presentations and use that data to portray the technology program. Assess how well the information is being transmitted, as well as how fancy the presentations are. Assessing writing is an easy way to show student improvement, but you have to teach writing as well as provide computer access time. • MAKE SURE YOU DOCUMENT... what you do in your project. Administrators, policymakers, funders, all want to know what happened, and some of "what happened" is doing what you said you were going to do. While you want to describe student impact and activities, you can also report some of the more mundane aspects of your computer program, such as, "well you gave us money to purchase computers and we developed the specs, put it out for bid, acquired these lovely objects, installed them, and they are now being used." Put some numbers to it, and you have the start of a report. Add some training numbers, hours of use, curriculum content, and your report is about half finished. • REMEMBER... it is not likely that your ambitious goals will be measureable after the first year of a major technology intervention. Give yourself, and the project, some time to have an impact. We often set lofty goals - and forget or ignore the realities of the classrooms we are given. And it never happens as quickly as we promise. Give significant change two or three years to happen. • REMIND YOURSELF... that children are in school for about 20% of their waking hours each year. The other 80% of the time they are doing something else - and it probably isn't homework. You and your computer project can't be held accountable for all of the things young people learn or do. But you can help start an after-school program or encourage better use of home computers to influence some of that 80%. • ALWAYS ... ask what teachers see as the outcomes of their use of technology. Their eyes are the best instruments we have -- and we all need some instruments as good as their eyes. Look carefully at things that teachers tell you are different from the previous year's work. • YOU CAN'T HAVE ENOUGH... numbers to report to a funding source or an administrator, so always count things. There are easier and harder things to count, so always go for the easiest. Numbers of hours the lab is open and used, student contact hours, hours of staff development offered and taken, numbers of participants at workshops, numbers of Internet connections added, etc. • DON'T BE SURPRISED WHEN... people ask for data that don't make sense. "Great you have really expanded the technology program this fall," they'll say. "As a consequence, should we expect that this fall's SAT scores will be going up, too?" Or, after you have installed a new writing program, are you being judged by all the measures on the standardized tests or just the writing sections? Let them know that the ultimate goal is always to enhance student achievement, but the project has defined what it will accomplish and really can't tackle all the problems of schooling. 415/543-4144 www.rockman.com FAX: 543-4145 |