US Department of Education

USEDRockman et al frequently serves as external evaluator for multi-year projects funded by the U.S. Department of Education which are designed to drive whole-school reform and measurable gains in student achievement. The substantial investment in these projects means they require rigorous evaluations to assess evidence of impact, working with complex data sets, and analyses that examine the critical links between implementation and impact. Evidence to document and inform implementation throughout the life of the grant are also often components of our evaluations.

Rockman has a long history of studying how technology can inspire teachers, engage students, and enhance learning. From the introduction of technology into schools, to the advent of portable devices and one-to-one computing, Rockman has studied what characterizes and sustains effective use of technology in K12 schools.

Whether gauging the impact of broad-based reform initiatives or examining the use of innovative technology tools, we have looked at the context of implementation:

  • the school leadership needed to spearhead changes;
  • the training and conditions that give teachers the skills, comfort, support, and collaboration opportunities needed to develop lessons and integrate technology into practice; and
  • the value of engaging the larger school community in the efforts.

Selected examples of our federally-funded, external evaluations of K12 school initiatives are provided below. We welcome the opportunity to discuss your project with you and to provide you with more information about our tailored services. Please Contact Us to schedule a free 30-minute Q&A session with a consultant.

Selected Projects Include

  • PBS Kids Transmedia Games Usability Testing

    Rockman et al has run a series of usability tests for media products developed by PBS Kids and its partners as part of the Ready To Learn initiative, including several rounds of testing in Boston, Washington DC, and Bloomington IN.

  • East Bay Center for the Performing Arts Learning Without Borders

    The East Bay Center for the Performing Arts (EBCPA) Learning without Borders (LWOB) professional development program served elementary school teachers in the West Contra Costa Unified School District. Through professional development workshops, coaching, and in-class visits from artists, the program aimed to increase the capacity and confidence of the teachers to integrate arts with other core subject areas. REA collected survey, focus group, and pre- and post-test data from the participating teachers, and this data led to important insights about the teachers’ expectations for the program, feedback about the program, perceptions of the program’s impacts, and recommendations for the program moving forward.

  • Leading with Learning

    Rockman et al is the external evaluator for the U.S. Department of Education Investing in Innovation Fund (i3) Development award to WestEd for the Leading with Learning (LWL) project, (formerly known as the English Learner Professional Learning (ELPL) Project). The three-year evaluation includes both a quasi-experimental impact study with matched comparison schools, and an implementation study in six California elementary schools in two districts. For the impact study, REA is using hierarchical linear modeling to compare student and teacher outcomes. Expected outcomes for students—as measured by state tests—are improved reading comprehension, speaking and listening competencies, oral and written production of English, and content understanding. Participating teachers are expected to develop greater knowledge and skills in new practices for accelerating EL students’ understanding and use of academic language. To study implementation, REA will use surveys, interviews, focus groups, observations, and logs. The LWL model aims to develop teachers’ and instructional coaches’ knowledge and skills for implementing new instructional approaches that are sensitive to the language demands necessary for ELs to excel in academically and linguistically challenging content.

  • New Visions Accessing Algebra through Inquiry (a2i)

    The Accessing Algebra Through Inquiry (a2i) project is an i3 validation grant conducted by New Visions for Public Schools, in partnership with the Silicon Valley Mathematics Initiative and the New York City Department of Education. The project provides formative assessment lessons, ongoing coaching and training, and opportunities to examine student work in school-based inquiry team meetings. As the external evaluator, REA will conduct both implementation and impact studies to assess how students in a2i project classrooms perform on annual standardized achievement tests compared to matched groups of students whose teachers are not participating.

    Report: a2i narrative
  • New Visions Urban Teacher Residency Program

    The New Visions for Public Schools-Hunter College Urban Teacher Residency (UTR) project is a partnership between New Visions, the Hunter College Schools of Education and Arts and Sciences, and the New York City Department of Education. The 5-year project is designed to increase the number of well-prepared teachers entering the city’s classrooms, provide ongoing support during the critical early years of teaching, and ultimately raise teacher retention rates and student achievement. To assess the impact of the UTR model, REA conducted surveys and focus groups with each cohort of UTR residents at key points during their 14-month program, and during their first years as NYC public school teachers. REA also gathered data from Hunter faculty and host school mentors and principals focusing on residents’ preparation, readiness, confidence, and integration into school culture. Finally, REA reviewed achievement and other institutional data to assess the program’s impact on student performance, including state licensing and certification records, graduation rates, and NYC teacher retention rates.

  • West Virginia Department of Education: ED PACE

    The Educational Development for Planning and Conducting Evaluations (ED PACE) was a three-year research project funded by the U.S. Department of Education to develop, implement, and refine a set of research tools and methods that would enable project personnel at the state and local level to compare student outcomes and achievement data using a quasi-experimental design. The focus of this study was the West Virginia Virtual School Spanish 1A and 1B courses at the middle school level. These courses provide rural students an opportunity to study a foreign language at sites that would not otherwise be able to offer foreign language. For this project, it was necessary to develop and validate a Spanish Proficiency Assessment to use as the outcome measure to compare the Virtual Spanish program with face-to-face Spanish classes.

  • Project Jericho

    In collaboration with International Society for Technology in Education (ISTE), REA evaluated a pilot effort of the South Carolina State Department of Education to test the feasibility of virtual schooling for the state, examining its online learning course management system (Blackboard), registration system, quality assurance measures, and the need for and response to the program around the state. REA conducted site visits in pilot sites and interviews with virtual school coordinators, and consulted on surveys of students, teachers, and counselors involved in the pilot.

  • Tech Know Build

    The TechKnowBuild project was a partnership between the Crawfordsville Community Schools and Indianapolis Public Schools that provided “one to one” laptop computers and wireless Internet access to 2,500 middle school students and approximately 100 teachers, combining ubiquitous computing and problem-based learning. As the external evaluator, REA assessed impacts on teachers and students through suveys and secondary data analyses. During the final year, student achievement was measured by standardized test scores and writing prompts. Almost all teachers incorporated internet research and exploration, one of the most popular and regular use of laptops, and generally found problem-based learning to be an effective way to meet their academic content standards. REA found that problem-based learning activities increased students’ engagement in school, and gave students a sense of ownership for the topics they researched and a deeper awareness of community issues and their roles as citizens.

  • Star Schools

    Rockman et al conducted a three-year study of the impact of the online, Supplementary Education Services (SES) provided by Educate Online, Inc. (EO), funded by the Star Schools Program of the U.S. Department of Education. The evaluation activities under the grant measured the effectiveness of Catapult Online, a real-time, one-on-one tutoring service designed to address the needs of underserved rural, middle and high school students. The formative and summative evaluations examined online student assessment instruments, content, delivery mechanism, and the enhanced parental communication via mobile technology. REA also conducted a five-year evaluation of critical issues associated with the delivery of SES via online technologies in rural and urban areas. Among these issues were strategies for motivating and engaging middle and high schools students in remedial and supplemental programs, serving the needs of students with limited English proficiency, the impact of evolving technology platforms on the delivery of instructional programs, and emerging online assessments.