US Department of Education

USEDRockman et al frequently serves as external evaluator for multi-year projects funded by the U.S. Department of Education which are designed to drive whole-school reform and measurable gains in student achievement. The substantial investment in these projects means they require rigorous evaluations to assess evidence of impact, working with complex data sets, and analyses that examine the critical links between implementation and impact. Evidence to document and inform implementation throughout the life of the grant are also often components of our evaluations.

Rockman has a long history of studying how technology can inspire teachers, engage students, and enhance learning. From the introduction of technology into schools, to the advent of portable devices and one-to-one computing, Rockman has studied what characterizes and sustains effective use of technology in K12 schools.

Whether gauging the impact of broad-based reform initiatives or examining the use of innovative technology tools, we have looked at the context of implementation:

  • the school leadership needed to spearhead changes;
  • the training and conditions that give teachers the skills, comfort, support, and collaboration opportunities needed to develop lessons and integrate technology into practice; and
  • the value of engaging the larger school community in the efforts.

Selected examples of our federally-funded, external evaluations of K12 school initiatives are provided below. We welcome the opportunity to discuss your project with you and to provide you with more information about our tailored services. Please Contact Us to schedule a free 30-minute Q&A session with a consultant.

Selected Projects Include

  • Photo courtesy of Lucélia Ribeiro via CC licensing.

    PBS Ready to Learn

    For nearly two decades, REA has conducted formative evaluations on behalf of PBS Kids and their partners as part of the US Department of Education “Ready to Learn” grant program. Through the years, our evaluation efforts have included youth and family playtesting for digital games at various stages during the design process, formative testing of stories for broadcast programming, and extended play studies that examine the impacts of longer-term product-usage, including gameplay and video programming.

  • Photo courtesy of Dean Hochman via CC licensing.

    East Bay Center for the Performing Arts Learning Without Borders

    The East Bay Center for the Performing Arts (EBCPA) Learning without Borders (LWOB) professional development program served elementary school teachers in the West Contra Costa Unified School District. Through professional development workshops, coaching, and in-class visits from artists, the program aimed to increase the capacity and confidence of the teachers to integrate arts with other core subject areas. REA collected survey, focus group, and pre- and post-test data from the participating teachers, and this data led to important insights about the teachers’ expectations for the program, feedback about the program, perceptions of the program’s impacts, and recommendations for the program moving forward.

  • Photo courtesy of Enokson via CC licensing.

    Leading with Learning

    To improve teaching for English learners (ELs), WestEd partnered with two, large school districts in California on a multiyear project called Leading with Learning: Systematically Transforming Teaching for English Learners. The Leading with Learning (LwL) project was awarded a three-year development grant from the U.S. Department of Education’s Investing in Innovation (i3) fund. Rockman et al conducted the independent evaluation of LwL to study its implementation and to test its effectiveness on elementary student outcomes using a quasi-experimental design. The LwL model involved an intensive, two-year program of professional learning for K–6 teachers, instructional coaches, principals, district co-facilitators, and district leaders. Teacher practice changes were aimed at improving students’ understanding of complex texts, engagement with complex tasks, and understanding of academic English; which was expected to lead to improved academic achievement. Results showed that while the state assessment outcomes did not provide evidence of effectiveness, other data sources revealed important improvements in outcomes for students, teachers, and coaches.

    Report: Leading with Learning Final Report

  • Appalachian Support for Specialized Education Training (ASSET)

    The Appalachian Support for Specialized Education Training (ASSET) project was funded through the Department of Education’s Supporting Effective Educator Development (SEED) grant program and sought to develop and provide high-quality professional development micro-credentials for Appalachian teachers. The REA research team designed and implemented a randomized control trial with more than 600 participants. In addition to the RCT, we gathered formative evaluation data on each micro-credential from a teacher advisory board and early micro-credential completers and studied iterative versions of simulations designed to assess participants’ performance within one of the mathematics micro-credentials.

  • Photo courtesy of Tim Farrell, New Visions.

    New Visions Accessing Algebra through Inquiry (a2i)

    Accessing Algebra Through Inquiry (a2i) was a high school mathematics project conducted by New Visions for Public Schools (New Visions), in partnership with the Silicon Valley Mathematics Initiative (SVMI) and the New York City Department of Education. The project was supported with a 2013 i3 validation grant (#U411B110116) from the U.S. Department of Education. Rockman et al (REA) conducted the independent evaluation of a2i. The central focus of a2i was on the use of high-quality formative assessment processes to help teachers implement high standards and create effective math learning environments. The major categories of support for teachers included: (a) inquiry teams focused on data-driven instructional practices, (b) adaptable formative assessment lessons and strategies, and (c) instructional coaches. A quasi-experimental study involving 40 schools (across 2 cohorts) compared the effectiveness of a2i on students’ achievement in algebra and geometry compared to business-as-usual instruction in matched comparison schools. While the main findings did reveal statistically positive effects of a2i instruction on state standardized test performances, additional analyses by REA found evidence of improved achievement during the second year of the study after professional development was restructured and day-to-day pedagogical practices were introduced. REA evaluation of classroom changes revealed that students engaged in more group work, talked more about math, and solved problems collaboratively.

    Report: Accessing Algebra Through Inquiry Final Report
  • Photo courtesy of Tim Farrell, New Visions.

    New Visions Hunter College: Urban Teacher Residency Program

    A 10-year partnership between New Visions for Public Schools, Hunter College, and the New York City Department of Education, the Urban Teacher residency (UTR) program was funded through multiple U.S. Department of Education grants, along with the Mathematics and Science Teacher Education Residency (MASTER), funded by the National Science Foundation. The programs focused on the placement of new secondary teachers in high-needs classrooms in New York City where resident teachers were trained alongside a mentor teacher during a year-long residency. Rockman et al, served as the external evaluator to assess the impact and implementation of training on student performance, teacher practice and school culture.
    Report: 2018 Summative Report for New Visions for Public Schools – Hunter College Urban Teacher Residency Project

  • West Virginia Department of Education: ED PACE

    The Educational Development for Planning and Conducting Evaluations (ED PACE) was a three-year research project funded by the U.S. Department of Education to develop, implement, and refine a set of research tools and methods that would enable project personnel at the state and local level to compare student outcomes and achievement data using a quasi-experimental design. The focus of this study was the West Virginia Virtual School Spanish 1A and 1B courses at the middle school level. These courses provide rural students an opportunity to study a foreign language at sites that would not otherwise be able to offer foreign language. For this project, it was necessary to develop and validate a Spanish Proficiency Assessment to use as the outcome measure to compare the Virtual Spanish program with face-to-face Spanish classes.

    Report: ED-PACE Final Report
  • Photo: Wikimedia Commons

    Project Jericho

    In collaboration with International Society for Technology in Education (ISTE), REA evaluated a pilot effort of the South Carolina State Department of Education to test the feasibility of virtual schooling for the state, examining its online learning course management system (Blackboard), registration system, quality assurance measures, and the need for and response to the program around the state. REA conducted site visits in pilot sites and interviews with virtual school coordinators, and consulted on surveys of students, teachers, and counselors involved in the pilot.

  • Tech Know Build

    The TechKnowBuild project was a partnership between the Crawfordsville Community Schools and Indianapolis Public Schools that provided “one to one” laptop computers and wireless Internet access to 2,500 middle school students and approximately 100 teachers, combining ubiquitous computing and problem-based learning. As the external evaluator, REA assessed impacts on teachers and students through suveys and secondary data analyses. During the final year, student achievement was measured by standardized test scores and writing prompts. Almost all teachers incorporated internet research and exploration, one of the most popular and regular use of laptops, and generally found problem-based learning to be an effective way to meet their academic content standards. REA found that problem-based learning activities increased students’ engagement in school, and gave students a sense of ownership for the topics they researched and a deeper awareness of community issues and their roles as citizens.

    Report: One to One Laptop Programs

  • Star Schools

    Rockman et al conducted a three-year study of the impact of the online, Supplementary Education Services (SES) provided by Educate Online, Inc. (EO), funded by the Star Schools Program of the U.S. Department of Education. The evaluation activities under the grant measured the effectiveness of Catapult Online, a real-time, one-on-one tutoring service designed to address the needs of underserved rural, middle and high school students. The formative and summative evaluations examined online student assessment instruments, content, delivery mechanism, and the enhanced parental communication via mobile technology. REA also conducted a five-year evaluation of critical issues associated with the delivery of SES via online technologies in rural and urban areas. Among these issues were strategies for motivating and engaging middle and high schools students in remedial and supplemental programs, serving the needs of students with limited English proficiency, the impact of evolving technology platforms on the delivery of instructional programs, and emerging online assessments.

    Report: starschoolsreport