|
|
|
|
|
|
Communicating our Successes: Issues and Tactics |
|
|
|
by Saul Rockman
|
|
|
|
A paper written for the Council of Chief State School Officers State Educational Technology Leadership Conference November12-13, 1998
|
|
|
|
|
|
|
|
"When governments put large amounts of money into programs, they want results that show improvement. It's a quid pro quo: significant dollars, significant differences."
By articulating the role of technology in the overall education program, we will help decision makers understand the potentials and limitations of our technology programs and ideas for how to develop and substantiate its impacts. |
|
State allocations for technology are reaching new highs at the same time that the US Department of Education technology budgets and the Universal Service Fund are putting well over two billion dollars into state technology and telecommunications programs. It must be our lucky day. However, the confluence of two policy themes have stirred up the dust and resulted in substantial problems for state and district technology programs. These two themes - accelerating legislative pressures for educational accountability and corresponding efforts to establish standards for curriculum and student performance - are visibly shaping our discussions and our activities. So why, with all this money for technology, are states, districts, and federal technology initiatives having problems? Well, it's the money! All of these initiatives come with a mandate, implied or explicit, to demonstrate that expenditures are influencing student achievement. States and district technology programs have reached the point where, by successfully attracting state and federal resources, they must now begin to demonstrate some corresponding outcomes for students. When governments put large amounts of money into programs, they want results that show improvement. It's a quid pro quo: significant dollars, significant differences. What we're up against: It's no easy job getting information that will satisfy the desires of legislators and policy makers. They seem to want simple answers to complex questions - and they want the answers quickly. In the first place, we really haven't been collecting too much information, and what we have collected hasn't been all that sound or useful. Second, technology programs vary so greatly in their goals and their implementation - from class to class, school to school, district to district, state to state - that generalizing about the impact of technology is like figuring out what you have when you place apples and oranges in the same basket with cabbages and cauliflower. Third, there isn't really all that much technology in schools and students don't often use it for any substantial amounts of time or for important purposes. And fourth, most of the time schools don't use the technology for bettering student content knowledge directly and concomitantly improving test scores. State data are rarely reflective of the range of school technology programs directed toward instruction. States usually focus on the easily counted and not unreasonably stay away from more challenging accountability issues. While counting computers and Internet connections provide indirect indicators of access (that is, potential access), data about web site selection and software use don't furnish any real information about improved skills, knowledge, and abilities. Program and project data are not usually collected in systematic ways and only a few of the federal programs are collecting any kind of consistent information about impact - and that is a recent phenomenon. Given the range of technologies and the purposes to which they are put in schooling, it is profoundly difficult to define what we mean by a technology program and even more difficult to measure its impact. Do we include anything in a classroom that we plug into a wall? Do we limit ourselves to computers and television and the Internet? And if only computers, is it sophisticated multimedia production or academic skills development and remediation or mastering technology skills that we should be measuring? Or should the definition include distance learning programs for rural schools and college courses for advanced secondary school students? Well, it should include all of that and more. The lack of a widely acceptable definition of technology exacerbates the difficulties of capturing and assessing related outcomes that influence school success, whatever that means. When we look at education's technology investment, it appears to be quite large - especially considering the opportunity costs. But relatively speaking, it is a modest allocation of scarce capital resources when compared to business and industry, where leveraging resources can lead to powerful returns. Just what is the level of technology use, an intervention that promises to dramatically improve student achievement? Well, eighty percent of the year, children are not in school; sitting in school accounts for about twenty percent of a child's waking time over a year. And computer use accounts for about five percent of school hours. So, what level of influence should that modest experience have on test scores? Moreover, when we look at ways in which technology is used, very little of its implementation is directly associated with improving the kind of student knowledge that is covered on standardized tests. Teachers often see technology as something that will extend what they do in class, enhance content coverage, provide opportunities for the introduction and practice of higher-order thinking skills, or develop students' ability to use technology&emdash;for its own sake. Except for remediation and modest subject-related units, we are seeing that the most predominant uses of technology are for writing and researching, and in most assessments we give greater credence to writing mechanics and the form of the bibliography than to the ability to communicate an idea or an argument and substantiate it. Given these issues that form a tremendous barrier to providing the simple answers that lawmakers and policy makers want to see, what do we have to offer and how can we offer it. To illustrate the problem: The conflicting headlines of a few weeks ago make it quite evident how hard it is to get our real story into the minds of policy makers. a) Computers Help Math Learning, Study Finds (New York Times) b) Study Links Lower Grades to Computer Use (Washington Post) c) Neither of the above d) Both of the above When these two contrasting stories appeared (pick your answer), the resulting confusion among educators and the defensive posturing from educational technologists served to mask the difficulty of portraying the outcomes and impacts of using technology in schools. We - and the policy makers and parents - are still looking for a simple answer to complex questions. It is easy to take a single study and portray the global outcomes and issues we hope to find and then generalize them to all of technology and schools. Few people seem willing to look deeply into this correlational study, most taking the headlines at face value. But was the NAEP the right outcome measure? Does it measure the characteristics of achievement that we look for when technology is used to support and improve learning? We are told that eighth grade students scored better on problem solving when they were doing simulations on the computer. Do we know anything about the kinds of classrooms that participated at either the 4th or 8th grades? Depending whether the eighth grade students were in algebra, pre-algebra, regular or remedial classes, they would be treated differently by their teachers, given substantively different curricula, and likely be given different tasks on computers. Perhaps student characteristics rather than computers had more to do with achievement. The fourth graders were found to do worse when they were using computer-based drill-and-practice programs. Now, I can't think of a state that uses the NAEP to assess student progress in the fourth grade. It's not a test that focuses on basic skills such as computation and math facts. But if my students were taking the Iowa, the Stanford, or similar state- or nationally-normed assessments - and my school and my career were on the line - I certainly would be using drill-and-practice programs to improve computational skills, rather than using problem solving exercises that would not be reflected on the tests. There are no simple answers. The ETS study did not help answer the question of whether technology improves (or impedes) student achievement, although headline readers now have a basis on which to judge school computer use. It may have shed some light on the importance of professional development; it certainly helped reiterate the concerns for student and teacher equity - especially for poor rural and urban schools; and it encouraged the application of more research dollars to the problem, an outcome dear to the hearts of researchers. Ideas and tactics for success: So what should we do? How do we engage - in a positive fashion - the questions raised about the impact of technology in schools? How do we justify the increasing budgets for acquiring and upgrading technology and telecommunications services? There are a number of steps state and district leaders might consider as they prepare to explain their past actions, their plans, and their expectations for improved student achievement.
By articulating the role of technology in the overall education program, we will help decision makers understand the potentials and limitations of our technology programs and ideas for how to develop and substantiate its impacts. States and districts can bring in decision makers and obtain their support early in the process; when all stake-holders have a say in planning, identifying outcomes, and selecting measures, they will support the decisions. A clear assessment strategy - one that goes beyond standardized tests to measure what we actually do with technology - can enable school leaders, policy makers, and the community to understand the impact of technology on teaching and learning.
If our legislators continue to ask simple questions about complex activities, not only are they constraining our abilities to try new strategies, but they are missing some of the more powerful impacts of our technology programs. We need to move them from standardized test scores, that do not reflect what we use technology to do, to more sophisticated measures of outcomes that better tap into our applications of technology. Consequently, we may see improved writing, better organization of ideas, more strategic thinking, enhanced communications and presentation skills, and the emergence of collaborative work products, as different indicators of technology's impact. One approach for institutional reporting that can inform decision makers is to break out the school reform and curriculum reform programs that leverage technology and look for impacts within each. Some strong programs are likely to emerge that can represent and portray important outcomes that are not simple test scores. We can also look to changes in teaching that are the consequence of technology access and that are necessary precursors to changes in learning and achievement. Such indirect indicators can have great explanatory powers about the power of technology to change classroom environments.
In the long run, the only way technology will be successfully used in elementary and secondary schools is through the efforts of teacher preparation institutions. They should be made aware of their economic stake in doing this well, since the state education departments and the school districts can set the certification and hiring criteria in ways that limit the ability of state university graduates to get jobs in the state. Pre-service education can certainly be tied more closely to K-12 classrooms, especially for preparation to use technology in support of curriculum and instruction. We should assume computer competencies for college students - just as other academic departments do - and focus our educational efforts on curriculum integration, not technology skill development.
Consider a Rotunda Day, where parents and their students come to the state capitol to display their technology products and talk with legislators or even help state assembly members set up their new computers. Use the opportunity to discuss what the program has accomplished - and help the policy makers see what you see in the schools. Think about a technology fair as a way to honor the assembly speaker's or superintendent's or governor's birthday. Connect students to legislators or key government executives for curriculum tasks in civics and history. These are all ways of linking legislators to the most visible impacts of technology - the abilities developed by our students. Parents can help tell your story if you help them learn about their children's accomplishments. They become the story when they learn technology skills as part of your schools' technology programs. State policy and resources can help school computer labs become available for adult learning in non-school hour - afternoons, evening, and weekends. Schools can also connect more to the community through partnerships with public libraries, community groups, and afterschool programs. Increasingly software licenses will cover these out of school programs if they are sponsored by and connect to school. Splitting the cost of the public library's computers to provide Internet training to parents and other adults, can help garner support for school technology programs and bond issues. The more public school technology efforts become, the more likely they will be supported by voters.
As State Departments of Education are required to undertake or commission more substantive studies of technology and its impact on teaching and learning, their staff need to be much more thoughtful of the data collection strategy, its burden on the schools, and the answers that are likely to emerge. State-level data collection staffers are often more optimistic about response rates, assume common understanding of terms, and ignore the need for incentives for teachers and administrators to respond to their instruments. Informed and politically-sensitive advisors should define the limitations at the beginning of the study, not come in at the end and ask for substantial revisions. It is easy to do a study that should never have been done. And in conclusion: The need to respond meaningfully to the demands of state legislators, policy leaders, parents and the public is clear. But giving answers to questions that should not have been asked is doing neither the technology programs nor the stakeholders any favors. We should be educating these leaders to ask better questions, not only ones for which we have answers, but questions that lead to more powerful impacts for technology within the standards movement because they push us to do more important things with technology. We need to be standing up for what we know we are accomplishing with technology, even if it means working harder to gain the understanding of our partners. Once they accept the valuable role that technology plays in school reform, once they see the changes in the way teachers teach and students learn, once they see students' increased motivation to learn and to succeed in school, they will be better able to judge whether we are doing a credible job. We need to demonstrate the improved writing skills, the simulations of complex scientific processes that permit more students to grasp complex ideas, the interactive tools that allow students to learn by manipulating referents of mathematics concepts. We often strive to improve student communication skills though our technology programs. We ask students to write more, to use the Internet for research on the topics, to be more thoughtful when they send email, to analyze their audience before they begin their preparation, to develop effective presentations that use technology. We need to do more of the same. Improved communications - as well as powerful data - will further our case.
|
|
|
|
As Published In: Council of Chief State School Officers. Investing, Assessing and Communicating Results of Learning Technologies. Washington, DC 1999. |
|
|
|