Assessment Development

Selected Projects Include

  • Appalachian Support for Specialized Education Training (ASSET)

    The Appalachian Support for Specialized Education Training (ASSET) project was funded through the Department of Education’s Supporting Effective Educator Development (SEED) grant program and sought to develop and provide high-quality professional development micro-credentials for Appalachian teachers. The REA research team designed and implemented a randomized control trial with more than 600 participants. In addition to the RCT, we gathered formative evaluation data on each micro-credential from a teacher advisory board and early micro-credential completers and studied iterative versions of simulations designed to assess participants’ performance within one of the mathematics micro-credentials.

  • PBS Kids iPod App Study

    Rockman et al was commissioned by PBS Kids to assess learning outcomes and usage patterns among 3-7 year-olds exposed to two iPod Apps. Ninety iPod Touch devices were given to children from two Title I Schools in Washington D.C. and Bloomington, Indiana. The children were asked to use two apps: Super Why and Martha Speaks: Dog Party for a two-week period during the spring of 2010. Parents were asked to submit daily observation forms and complete short pre- and post-study surveys. Children were given pre- and post-study tests to assess their reading skills and specific content areas covered in the apps. Findings from this study were featured as part of the report by the Joan Ganz Cooney Center at Sesame Workshop’s report entitled: “Learning: Is there an app for that?

  • Star Schools

    Rockman et al conducted a three-year study of the impact of the online, Supplementary Education Services (SES) provided by Educate Online, Inc. (EO), funded by the Star Schools Program of the U.S. Department of Education. The evaluation activities under the grant measured the effectiveness of Catapult Online, a real-time, one-on-one tutoring service designed to address the needs of underserved rural, middle and high school students. The formative and summative evaluations examined online student assessment instruments, content, delivery mechanism, and the enhanced parental communication via mobile technology. REA also conducted a five-year evaluation of critical issues associated with the delivery of SES via online technologies in rural and urban areas. Among these issues were strategies for motivating and engaging middle and high schools students in remedial and supplemental programs, serving the needs of students with limited English proficiency, the impact of evolving technology platforms on the delivery of instructional programs, and emerging online assessments.

  • Elmo Loves ABCs and Elmo Loves 123s

    Rockman et al worked with the Sesame Workshop to develop and administer a basic literacy and mathematics assessment for preschool-aged students in order to test outcomes of using either Elmo Loves ABC’s or Elmo Loves 123’s for a month-long period as part of the normal school-day experience. To facilitate this project, the Rockman team managed a set of 10 iPad devices (with unique copies of apps for each of 90 students in the study) and a team of 8 researchers.

  • Genetic Science Learning Center (GSLC) at University of Utah

    Since 2006, REA has worked with the University of Utah’s Genetic Science Learning Center (GSLC) on evaluation projects for the National Institutes of Health (NIH), conference presentations, and peer-reviewed publications. Our collaborations have included conducting randomized controlled trials (RCTs) to evaluate the impact of high school curriculum supplements on cell biology and epigenetics, and documenting the process of curriculum and assessment development for a National Science Foundation Discovery Research K-12 project on evolution and common ancestry. REA and the GSLC co-authored a paper on randomized controlled trials and a primer on instrument development. They have co-presented on evaluation and educational research topics, such as mixed-methods designs, program implementation and assessment development, at the National Institutes of Health’s Science Education annual meetings.

  • National Study of Bill Nye the Science Guy

    Bill Nye the Science Guy is a high energy, inquiry-based educational television program presenting science content directed at fourth grade students. REA’s study investigated learning outcomes among school-age viewers and assessed the impacts of various outreach efforts for youth and adult viewers. Our findings suggest that program viewers were able to provide more complete and more complex explanations of science concepts and were able to learn facts presented in the program to support explanations of science concepts. Findings also suggest that the program was successful in helping to reduce gaps in science knowledge, understanding and aptitude between boys and girls, and for students traditionally underrepresented in the sciences.

  • Virtual Clinical Trials at Rice University

    In collaboration with Rice University’s Center for Technology in Teaching and Learning (CTTL), with funding from the National Institutes of Health (NIH) Blueprint for Neuroscience Research, REA evaluated the impact of three Virtual Clinical Trial simulations on high schoolers’ understanding of research design and neuroscience, and attitudes toward clinical trials. The studies, which used a randomized-controlled trial design, also investigated some of the program-related factors (e.g., satisfaction with the simulation) that may have influenced the observed outcomes.

  • Common Sense Media

    Rockman et al conducted an evaluation to help Common Sense Media (CSM) review existing instruments and evaluation methodologies in the Parent Media Education program and to measure the outcomes of CSM’s educational programs in terms of parent and teacher satisfaction and implementation.

  • Testing outcomes of Cyberchase viewing among elementary school students

    REA conducted a quasi-experimental design study with students in four schools located in Indiana and Arizona. Nearly 300 children participated and findings revealed the impacts of Cyberchase programming on a variety of STEM learning and attitudinal outcomes as well as environmental science knowledge, attitudes and beliefs.

  • Game-Enhanced Interactive Physical Science (GIPS)

    Through funding from the NSF’s SBIR program, Filament Games developed a suite of online educational games and curricular activities for middle school classrooms that target core scientific concepts, such as engineering, plant genetics, and planetary characteristics. Rockman et al evaluated the “promise of efficacy” from each game via pre-post surveys, classroom observations, and embedded assessments, comparing content learning and engagement between students who played the game and engaged with the curricular activities, only played the game, or received traditional instruction. Differences in outcomes based on students’ interest in and aptitude for reading were also explored to ensure that the games were inclusive of a range of student abilities.