Surveys

Selected Projects Include

• 

  Photo courtesy of Maryland GovPics via CC licensing.

  Oakland Unified School District Salad Program

  REA conducted an evaluation of the impact of a salad bar implementation at 5 schools within the Oakland Unified School District (OUSD). Project evaluation included pre-implementation student surveys and interviews, lunchroom observations, and post-implementation interviews with key project stakeholders (students, teachers, and administrators). The evaluation focused on three key areas: 1) Implementation (training, staffing, promotion, availability, methods for sustainability, curricular connections), 2) Participation (methods of adoption/education, level of participation, parent/teacher/food service worker roles) and 3) Attitudes (changes in diet/nutrition-related knowledge, behavior, impact of cultural differences, food likes/dislikes, expectations, influences on food selection, etc.).
• CancerCare’s Ocular Cancer Patient Support Program

  CancerCare® is the leading national organization providing free, professional support services and information to help people manage the emotional, practical and financial challenges of cancer. REA conducted an evaluation of the program’s financial, medical, and psycho-social outcomes for patients in the Tri-state New York area with ocular cancer and other vision-affecting cancers. The patients studied included current program participants and those who participated in the program over the past 2-3 years.
• Red Hill Studios Wii game

  REA conducted a usability study of a Wii-based video game designed to improve gait and balance in patients with Parkinson’s Disease (PD). Red Hill Studios, developers of the game’s hardware and software elements, in collaboration with researchers at the University of California, San Francisco (UCSF), received funding from the National Institute of Health’s (NIH) Small Business Innovation and Research (SBIR) program to conduct a Phase 1 feasibility study.
• 

  Photo courtesy of Dean Hochman via CC licensing.

  East Bay Center for the Performing Arts Learning Without Borders

  The East Bay Center for the Performing Arts (EBCPA) Learning without Borders (LWOB) professional development program served elementary school teachers in the West Contra Costa Unified School District. Through professional development workshops, coaching, and in-class visits from artists, the program aimed to increase the capacity and confidence of the teachers to integrate arts with other core subject areas. REA collected survey, focus group, and pre- and post-test data from the participating teachers, and this data led to important insights about the teachers’ expectations for the program, feedback about the program, perceptions of the program’s impacts, and recommendations for the program moving forward.
• 

  Photo courtesy of fabola via CC licensing.

  National Endowment for Arts: Teacher Art Institute

  Rockman et al conducted an evaluation of the National Endowment for the Arts (NEA) Teacher Arts Institute, providing an evaluation symposium and coordinating with local evaluators for each of the Institutes. The evaluation included Institute observations, informal interviews, pre- and post-surveys, and participant reflections. Rockman also convened an expert panel to review curriculum units developed subsequent to the Institutes. In 2006, the Endowment awarded Year 2 extensions to 4 of the 12 original Teacher Institute grant recipients to further develop activities around “anchor” works of art that served as inspiration for instructional activities. Rockman conducted surveys, site visits, interviews, and classroom observations to chronicle the activities of the institutes and to assess teachers’ use of “anchor” artworks in their instruction.
• 

  Photo courtesy of Nic McPhee via CC licensing.

  Poetry Out Loud

  The Poetry Out Loud National Recitation Contest, sponsored by the Poetry Foundation and the National Endowment for the Arts, is designed to inspire a love of poetry in a new generation of teachers and students. With the help of teachers, web-based resources, and state arts agencies, students learn poems by heart and engage in a series of poetry-related activities that culminate in local, state, and national recitation contests. REA evaluated the Poetry Out Loud curriculum, the program’s implementation at the local, state, and national levels, and its impact on students and teachers. Findings indicated that the program changed the ways teachers engage students in poetry, and reached rural, urban, and suburban schools and a diverse group of students.
• 

  Photo courtesy of Wars via CC licensing.

  Water: Exploring Science in the Studio

  In collaboration with California College of the Art (CCA), Rockman et al conducted an evaluation of “Water: Exploring Science in the Studio,” including course development and delivery processes, and instructional and assessment elements. The broad objective of this project was to increase the ability of CCA students to interact with scientists within the context of a series of four design courses. REA developed and administrated online surveys to CCA art faculty and visiting science faculty who collaborated on the development and delivery of these courses.
• 

  Photo: Courtesy of Keith Ellison via CC licensing.

  National History Day

  REA conducted a year-long evaluation of the National History Day (NHD) program, which serves more than half a million students annually. A diverse group of 500 students in four states in the NHD and comparison-group reported that the NHD program helped them to acquire critical thinking and communication skills, a mature understanding of past and current events, and college-level research skills. NHD students also outperformed their peers on statewide standardized assessments.

  Report: Evaluation Report
• 

  Image courtesy of Center for the Future of Teaching & Learning at WestEd

  Teacher Practice Networks

  Rockman et al served as an evaluation partner for the 5-year Teacher Practice Networks (TPN) initiative that was led by the Center for the Future of Teaching and Learning at WestEd on behalf of the Bill & Melinda Gates Foundation. Over 35 educational organizations with large networks of K–12 teachers were awarded grants to help spread innovative, standards-aligned approaches to college readiness across their teacher networks. REA designed and conducted annual surveys across the networks’ TPN teachers and their teacher leaders to develop an understanding about implementation and outcomes of the teacher practice networks each year. Findings revealed that teacher leaders’ self-efficacy for instructional leadership and instruction increased over time and teachers reported valuable impacts on their standards-aligned teaching.
• 

  Photo courtesy of Tim Farrell, New Visions.

  New Visions Hunter College: Urban Teacher Residency Program

  A 10-year partnership between New Visions for Public Schools, Hunter College, and the New York City Department of Education, the Urban Teacher residency (UTR) program was funded through multiple U.S. Department of Education grants, along with the Mathematics and Science Teacher Education Residency (MASTER), funded by the National Science Foundation. The programs focused on the placement of new secondary teachers in high-needs classrooms in New York City where resident teachers were trained alongside a mentor teacher during a year-long residency. Rockman et al, served as the external evaluator to assess the impact and implementation of training on student performance, teacher practice and school culture.

  Report: 2018 Summative Report for New Visions for Public Schools – Hunter College Urban Teacher Residency Project
• Tech Know Build

  The TechKnowBuild project was a partnership between the Crawfordsville Community Schools and Indianapolis Public Schools that provided “one to one” laptop computers and wireless Internet access to 2,500 middle school students and approximately 100 teachers, combining ubiquitous computing and problem-based learning. As the external evaluator, REA assessed impacts on teachers and students through suveys and secondary data analyses. During the final year, student achievement was measured by standardized test scores and writing prompts. Almost all teachers incorporated internet research and exploration, one of the most popular and regular use of laptops, and generally found problem-based learning to be an effective way to meet their academic content standards. REA found that problem-based learning activities increased students’ engagement in school, and gave students a sense of ownership for the topics they researched and a deeper awareness of community issues and their roles as citizens.

  Report: One to One Laptop Programs
• 

  Photo courtesy of Tim Farrell, New Visions.

  New Visions Accessing Algebra through Inquiry (a2i)

  Accessing Algebra Through Inquiry (a2i) was a high school mathematics project conducted by New Visions for Public Schools (New Visions), in partnership with the Silicon Valley Mathematics Initiative (SVMI) and the New York City Department of Education. The project was supported with a 2013 i3 validation grant (#U411B110116) from the U.S. Department of Education. Rockman et al (REA) conducted the independent evaluation of a2i. The central focus of a2i was on the use of high-quality formative assessment processes to help teachers implement high standards and create effective math learning environments. The major categories of support for teachers included: (a) inquiry teams focused on data-driven instructional practices, (b) adaptable formative assessment lessons and strategies, and (c) instructional coaches. A quasi-experimental study involving 40 schools (across 2 cohorts) compared the effectiveness of a2i on students’ achievement in algebra and geometry compared to business-as-usual instruction in matched comparison schools. While the main findings did reveal statistically positive effects of a2i instruction on state standardized test performances, additional analyses by REA found evidence of improved achievement during the second year of the study after professional development was restructured and day-to-day pedagogical practices were introduced. REA evaluation of classroom changes revealed that students engaged in more group work, talked more about math, and solved problems collaboratively.

  Report: Accessing Algebra Through Inquiry Final Report
• 

  Photo courtesy of spacedust2019 via CC licensing.

  California Academy of Sciences

  Over a 4 year period, Rockman et al has conducted more than a dozen evaluation and research projects with the California Academy of Sciences in San Francisco using onsite and online surveys, visitor intercepts, interviews, focus groups and observations with members and visitors. Evaluation projects include formative and summative evaluation of exhibits, planetarium shows, museum websites, and K-12 school field trip programs, as well as summative evaluation of teacher professional development programs, curriculum kits and evening programs for adults. REA has also delivered research studies on capacity building, institutional partnerships, and organizational development.
• PBS Kids iPod App Study

  Rockman et al was commissioned by PBS Kids to assess learning outcomes and usage patterns among 3-7 year-olds exposed to two iPod Apps. Ninety iPod Touch devices were given to children from two Title I Schools in Washington D.C. and Bloomington, Indiana. The children were asked to use two apps: Super Why and Martha Speaks: Dog Party for a two-week period during the spring of 2010. Parents were asked to submit daily observation forms and complete short pre- and post-study surveys. Children were given pre- and post-study tests to assess their reading skills and specific content areas covered in the apps. Findings from this study were featured as part of the report by the Joan Ganz Cooney Center at Sesame Workshop’s report entitled: “Learning: Is there an app for that?”

  Report: Findings From Ready To Learn 2005-2010
• 

  Photo courtesy of CERDEC Math and Science Summer Camp via CC licensing.

  Odd Squad Summer Camp

  Rockman et al worked with PBS KIDS to evaluate a pilot of the Odd Squad Camp program, including observations at two summer camps, youth and parent surveys, and interviews with participants, program facilitators, and participants’ parents.
• 

  Dinosaur Train Evaluation

  Rockman et al conducted a series of evaluative studies for Dinosaur Train, an animated series geared towards a pre-school aged audience, produced by the Jim Henson Company for PBS Kids. The evaluation included an online survey, focus group sessions with parents and children, data collection from families at the Pittsburgh Children’s Museum, and in-home observations and interviews with families children viewing Dinosaur Train episodes in Bloomington, IN.
• 

  Photo courtesy of AfroTech.

  Black Girls Code

  Black Girls Code (BGC) is an out-of-school program designed to provide girls of color opportunities to learn in-demand skills in technology and computer programming during a time when they are considering career paths. The goal of this program is twofold: To introduce programming and technology to a new generation of coders and to sustain this interest well into high school and college. Since 2012, REA has examined the impact of BGC’s Culturally-Relevant STEM programming that embraces the intersectionality of its students (culture, race/ethnicity/gender), sense of belonging for girls of color, short and long-term program impact, and career inspirations and pathways. We have utilized case study design, focus groups, interviews, and surveys to measure BGC’s impact.

  Report: Black Girls Code Final Report
• 

  Photo: Wikimedia Commons

  Project Jericho

  In collaboration with International Society for Technology in Education (ISTE), REA evaluated a pilot effort of the South Carolina State Department of Education to test the feasibility of virtual schooling for the state, examining its online learning course management system (Blackboard), registration system, quality assurance measures, and the need for and response to the program around the state. REA conducted site visits in pilot sites and interviews with virtual school coordinators, and consulted on surveys of students, teachers, and counselors involved in the pilot.
• 

  Photo courtesy of mysapl via CC licensing.

  The Big Read

  The Big Read is an initiative of the National Endowment for the Arts, in partnership with the Institute of Museum and Library Services and Arts Midwest designed to bring communities together to celebrate reading. REA conducted a 19-month study of The Big Read to learn more about how communities held a Big Read—what books they chose, what partners they enlisted, what promotional and programming strategies they devised, what NEA resources they used, and what audiences they reached. The study, based on data from over 300 communities, also explored the project’s success in its overarching goal of restoring reading to the center of American culture. REA used a mixed-methods design that included feedback forms from Big Read events, surveys of participants and grantees, reviews of grantee reports required by the project, and tabular data on attendance, circulation, and partnerships from Arts Midwest. We also conducted a series of 36 case studies.

  Report: The-Big-Read-Final-Report
• 

  Photo: Dr. Peter Chang.

  Jackson State University: Mississippi Academy for Science Teaching (MAST)

  Rockman et al has served as the external evaluator for four professional development programs provided by Jackson State University for science teachers in elementary, middle and high school. REA has employed a mixed methods approach, including pre- and post surveys, qualitative interviews, and observations to measure the quality of implementation and impacts on teachers and students. Our most recent evaluation, a long-term impact study of MAST alums between 2004-2017, showed that MAST can achieve meaningful and sustained impacts on teaching and learning.
• 

  Photo courtesy of Juan Cristóbal Cobo via CC licensing.

  PBS Kids: Raising Readers

  REA conducted a study of PBS Kids Island with twenty-one English-speaking children ages 3-8 to learn about site usage patterns over an extended period of time, whether children found the content to be age-appropriate, fun, and accessible, and the overall appeal of the site to children. REA conducted usability testing for the PBS Kids Raising Readers website, including versions in Spanish and English. Working with PBS contacts, Hispanic community organizations, and public libraries in three US sites, we conducted more than 69 user testing sessions with parents, teachers and caregivers of children ages 2-8, focusing on children from low-income families and including bilingual teachers and caregivers of Spanish-speaking children.
• 

  Photo: Jennifer Borland

  Museum of Science and Industry “Our Place in Space”

  Rockman et al is the external evaluator for “Our Place in Space,” a professional development program for formal and informal science educators designed and implemented by the Museum of Science and Industry in Chicago, with funding from NASA. Through observations, surveys, and interviews, REA is studying how the program helps prepare educators to cover Next Generation Science Standards and implement a series of high-quality STEM instructional activities. REA’s evaluation seeks to demonstrate the value of the program as well as suggest potential areas for improvements in subsequent programming.
• Microsoft Anytime Anywhere Learning

  Rockman et al conducted a three-year study of the introduction of “one to one” laptop computing in U.S. public and independent schools (1996-1999), reporting on types of implementation, classroom and home laptop use, and changes in pedagogy and teacher and student attitudes. The study was funded by Microsoft and Toshiba.

  Report: Anytime Anywhere Learning Report
• 

  Photo courtesy of Mark Richards via CC licensing.

  Computer History Museum “Get Invested”

  The Computer History Museum (CHM), located in Mountain View, CA, showcases the history and future of computing and technology. REA conducted evaluation projects for a number of CHM education programs and exhibits. These include an evaluation of CHM’s Get Invested program that provided high school students in the US and Mexico opportunities to develop apps to address critical social challenges; a front-end evaluation with museum visitors for CHM’s Make Software exhibit, and case study documentation of the planning and design of CHM’s multi-modal Learning Lab.
• 

  Photo courtesy of US ED via CC licensing.

  ReadWorks

  ReadWorks worked to support reading comprehension instruction by providing research-based, grade-level aligned instructional materials and tools to over 1 million K–12 teachers and 13 million students. REA partnered with ReadWorks to design and conduct a series of multi-year evaluation studies investigating both the implementation and impact of ReadWorks’ curriculum and teacher training materials. The evaluation research included an investigation of web analytics data, a mixed-methods retrospective study focused on teacher usage of the website and curriculum, a multi-year study of ReadWorks-supported teacher practice networks, and a prospective efficacy study of their Article-A-Day curriculum in high-needs, urban classrooms. Teachers reported valuable gains from using ReadWorks to develop students’ content knowledge, vocabulary, reading comprehension skills, and reading motivation.
• 

  Photo by National Cancer Institute on Unsplash

  Common Sense Media Educator Census

  Together with Common Sense Media and Veraquest Inc., REA developed and conducted a survey of U.S. teachers’ technology use in schools. The Educator Census Survey provided an update on the use and perceived impacts of various classroom technologies, understandings about digital citizenship curricula use, an overview of technology-related policies in schools, and insights into issues of access to educational technology within and outside classrooms. Among the most notable survey findings was that there was a misalignment between the digital tools that teachers used most frequently and the digital tools teachers perceived to be most effective for students.

  Report: Common Sense Media Educator Report
• 

  Photograph courtesy of Jared Kelly via creative commons licensing.

  RadioLab

  Radiolab’s “What Curiosity Sounds Like: Discovering, Challenging, and Sharing Scientific Ideas” project, funded by the NSF Advancing Informal Science Learning (AISL) program, explored new ways to actively engage both lay and professional audiences and foster meaningful communication between scientists and the general public. As the external evaluator for the project, REA examined participants’ experiences and explored project outcomes through a series of surveys and interviews.

  Report: Radiolab. What Curiosity Sounds Like
• 

  Photo courtesy of KQED QUEST via CC licensing.

  KQED

  REA has a long history of partnering with KQED, the PBS affiliate in Northern California, on diverse research and evaluation projects involving teacher professional development, organizational development, environmental education, public science communication, and science journalism. Specific projects include evaluation of KQED Teach (an online professional development tool for teachers), a two-year organizational development study, and two NSF-funded projects applying social science research methods to investigate best practices for engaging millennial audiences in science content, and improving science media production and journalism practices.

  Report: QUEST Summative Evaluation Report
• 

  Photo courtesy of Ryan Somma via CC licensing.

  Astronomical Society of the Pacific Galileo Educator Network (GEN)

  With funding from NASA, the Astronomical Society of the Pacific (ASP) developed the Galileo Educator Network (GEN), which prepared teacher educators nationwide to design and deliver professional development workshops on astronomy content and science practices aligned with Next Generation Science Standards (NGSS). REA collected surveys and observations to describe the ways in which GEN engaged K-12 teachers and promoted the effective use of NASA-developed resources. REA presented its evaluation findings at two ASP annual meetings.
• 

  Photo courtesy of Enokson via CC licensing.

  Leading with Learning

  To improve teaching for English learners (ELs), WestEd partnered with two, large school districts in California on a multiyear project called Leading with Learning: Systematically Transforming Teaching for English Learners. The Leading with Learning (LwL) project was awarded a three-year development grant from the U.S. Department of Education’s Investing in Innovation (i3) fund. Rockman et al conducted the independent evaluation of LwL to study its implementation and to test its effectiveness on elementary student outcomes using a quasi-experimental design. The LwL model involved an intensive, two-year program of professional learning for K–6 teachers, instructional coaches, principals, district co-facilitators, and district leaders. Teacher practice changes were aimed at improving students’ understanding of complex texts, engagement with complex tasks, and understanding of academic English; which was expected to lead to improved academic achievement. Results showed that while the state assessment outcomes did not provide evidence of effectiveness, other data sources revealed important improvements in outcomes for students, teachers, and coaches.

  Report: Leading with Learning Final Report
• 

  Photo courtesy of NOAA Photo Library via CC licensing.

  Exploratorium Carbon Networks

  REA is the external evaluator for ‘Carbon Networks,’ a federally-funded project which is a collaboration between the Exploratorium, the Pacific Science Center, and the Waikiki Aquarium. This project aims to assist museum educators in building capacity for developing educational programs and activities for public audiences and formal educators on the topics of climate change and ocean acidification. REA is conducting formative evaluations of educator workshops and summative evaluation of program impacts at each of the participating institutions, using online surveys, interviews and observations.
• 

  Photo courtesy of Hudson Alpha.

  It’s Complex! Engaging Student Discussions around Complex Genetics and Individualized Medicine

  Rockman et al is studying the impact of Touching Triton, an online space-themed interactive created by the HudsonAlpha Institute for Biotechnology. The game is designed to increase high school students’ knowledge about and interest in genetic and environmental factors that might influence the likelihood of developing a complex disease. To inform the design, REA conducted student focus groups, teacher interviews, and classroom observations. REA has also provided consulting on quasi-experimental studies to assess summative impacts. Touching Triton was selected as a finalist in the 2015 Serious Games Showcase and Challenge in the Business Category.
• 

  Photo courtesy of MusikAnimal via CC licensing.

  NYSCI Design Lab

  REA provided formative and summative evaluation for The New York Hall of Science’s (NYSCI) Design Lab, a multi-faceted project which consists of a unique, museum-based innovation laboratory, a series of permanent onsite exhibits and teacher professional development program that supports design-based approaches to science, technology, engineering, and math (STEM) teaching and learning. Over a three year period, REA staff conducted formative evaluation of onsite Design Lab exhibits, summative evaluation of Teacher Summer Institutes. and documented the impact of ongoing professional development through observations of participating teachers and their students when back in the classroom. REA also completed a retrospective study to assess long term impact with teacher graduates from each of program’s four cohorts. Evaluation methods included online surveys, interviews and observations.
• DragonflyTV’s Informal Science Education Project

  Rockman et al developed an impact study for the educational television program: DragonflyTV, a science program for and about 9-12 year old children produced by Twin Cities Public Television. A controlled classroom viewing protocol was established for schools in Chicago and San Francisco. Students were surveyed following each of three episodes selected for the study. Pre and post- interviews with students introduced a hypothetical scenario in which they were asked to design an experimental study. An additional in-class viewing study was also implemented with educators in a variety of locations across the United States.
• 

  Photo: Science Museum of Minnesota

  Science Museum of Minnesota “Journey to Space”

  REA conducted a summative evaluation for the Science Museum of Minnesota‘s ‘Journey to Space’ exhibit, examining the extent to which the exhibit gave visitors a memorable experience about what it is like to be in space, and increased their awareness and understanding of the systems engineering required to support living and working in space. Evaluation methods included visitor intercepts, onsite and online surveys, think alouds, and pre- and post-group activities. The exhibition will tour twelve major science museums across North America and reach upwards of three and a quarter million families, adults, teachers, and students over six years.
• UCMP’s Understanding Evolution Website

  REA conducted a multi-component evaluation of University of California Museum of Paleontology’s (UCMP) Understanding Evolution website, which aims to teach the science and history of evolutionary biology. The website includes evaluated lessons appropriate for different grade spans and strategies to avoid or overcome roadblocks related to teaching evolution. REA has evaluated and informed the development of multiple aspects of the website, ranging from its appeal and utility, to its ability to help users understand new concepts and acquire strategies for, and confidence in teaching evolution. REA also conducted evaluations of the interactive student module “The Arthropod Story,” the comic strip “Survival of the Sneakiest,” and the “Tree Room” tool.
• 

  Photo: BAVC News

  Bay Area Video Coalition Bridges to STEM Careers

  REA conducted a formative and summative evaluation of Bay Area Video Coalition’s Bridges to STEM Careers, a three-year Advanced Technological Education (ATE) Program Improvement Project designed to enhance the relevance of technician education in Computer Science and Multimedia. The project targeted two-year college students, their families and their college and workplace mentors. The ultimate goal of the Bridges project was to keep traditionally underrepresented and low-income students in the STEM pipeline by demystifying the process of preparing for a STEM career through internships and informational videos from past internship participants.
• Euteleost Tree of Life

  The Euteleost Tree of Life (EToL) is an interactive Web module created through a collaborative National Science Foundation research project between eight institutions, including Kansas University and the University of California Museum of Paleontology. The module is designed to for support users’ exploration of the evolutionary history of fish. Rockman et al conducted an independent summative evaluation of the EToL Web module that was designed to gather users’ feedback about the website, and assess impacts on users’ knowledge and attitudes about fish and fish evolution. REA recruited two groups of high school students to perform a series of tasks on the website and then provide feedback during a focus group discussion. Findings indicated that students were able to expand upon previous knowledge about evolution and natural selection by learning about how fish evolved and by gaining familiarity with traits common to specific fish lineages.
• 

  Photo courtesy of NWABR via CC licensing.

  Coastal Marine Biolabs

  Coastal Marine Biolabs’ NeuroLab 2.0 project has developed a multidisciplinary, storyline-based high school curriculum unit about a complex movement disorder, congenital mirror movement disorder (CMM). Over the ten lesson sequence, students generate questions, analyze and interpret data, and construct and revise explanatory models. The project seeks to expand access to investigative research experiences for students by training teachers on the new curriculum. REA is conducting a small-scale classroom study to evaluate the efficacy of the NeuroLab 2.0 unit on students’ science knowledge, beliefs, and attitudes. Evaluators are also studying teachers’ implementation of the materials.
• 

  Photo courtesy of joaquin uy via CC licensing.

  YR Media

  Since 2006, REA has served as the independent evaluator on YR Media’s education initiatives to engage under-represented youth in informal STEM learning through interest-driven inquiry that combines journalism, design, data, and code. Building on YR Media and MIT App Inventor’s longstanding collaboration to democratize youth media production, the Innovative Approaches to Informal Education in Artificial Intelligence project, funded by NSF-AISL, aims to improve public understanding of artificial intelligence (AI) among young creators and their audiences through youth-led journalism about AI, digital media powered by AI, and open educational resources about how AI works and its role in society. REA conducted the project’s independent evaluation, which included project team interviews, participant observations, focus groups with youth producers, online youth surveys, and audience testing to capture outcomes and lessons learned about effectively engaging youth in learning about AI.

  Report: Youth Radio Innovation Lab Final Evaluation Report
• 

  Photo: Hack the Hood

  Hack the Hood

  REA worked with Hack the Hood to develop a self-evaluation framework for collecting, analyzing and reporting qualitative and quantitative data to inform stakeholders and program development at scale.
• 

  Photo courtesy of crowdandcloud.org

  The Crowd & the Cloud

  The Crowd & The Cloud, a three-year project developed by Passport To Knowledge and funded by the NSF, used multimedia to engage different audiences around citizen science and crowdsourcing. The project team created four episodes of a broadcast television series, which appeared on the WORLD channel and PBS stations, an interactive website, and a robust social media presence in an attempt to reach three target audiences: the general public, scientists, and citizen scientists. REA conducted an external evaluation of The Crowd & The Cloud’s broadcast series via online surveys and in-person focus groups, and The Crowd & The Cloud’s website and social media via online analytics, surveys, and focus groups.
• 

  Photo by Roméo A. on Unsplash

  Connor Prairie: 1863 Civil War Journey

  Funded by The National Endowment for the Humanities, the 1863 Civil War Journey: Raid on Indiana, was an interactive exhibit area at Connor Prairie Historical Park that incorporated live actors, projected screen technology, and touchscreen activities to engage visitors in stories from multiple historical perspectives focused on a raid in Indiana during the Civil War. REA collected exhibit exit surveys and interviews, and pre- and follow-up surveys from visitors to determine changes in their knowledge of and interest in Civil War topics and the history of Indiana, as well as their opinions about the narrative content and use of technology within the exhibit.
• 

  Photo courtesy of Daniel Torres Bonatto via CC licensing.

  American Museum of Natural History Space Shows

  Rockman et al has conducted evaluative studies on four dome-style planetarium shows created by the American Museum for Natural History. Rockman’s evaluations of “The Search for Life,” “Cosmic Collisions,” “Journey to the Stars” and “Dark Universe” have examined audience responses to the shows and a range of outcomes including attitudes toward science, interest in science, and content knowledge. We’ve conducted research at several planetaria around the United States to better understand the impact of presentation context, and with a variety of audiences in order to understand the impacts that programming has on audience members of all ages and prior knowledge levels.
• 

  Photo courtesy of Chris Betcher via CC licensing.

  Convergence Academies Project

  The Center for Community Arts Partnerships at Columbia College Chicago, in partnership with Chicago Public Schools, received a development grant from the Investing in Innovation Fund (i3) of the U.S. Department of Education for their Convergence Academies project (http://convergenceacademies.org/ourorigins.html). To support 21st Century learning, CCAP developed a whole school reform model, inspired by principles and values of connected learning, that involved inquiry-based instruction and integration of digital media arts and technology throughout the school. The model was implemented in Chicago in both a high school and a feeder elementary school (K-8). As the external evaluator for the project, Rockman et al conducted studies of both implementation and impact, including an interrupted time-series design to investigate gains on state tests in math and reading.

  Report: Evaluation of the Convergence Academies Initiative Final Report
• 

  Kaoru Watanabe © Akiko Nishimura

  American Museum of Natural History “Traveling the Silk Road”

  Rockman et al faciliated a collaborative evaluation with staff at the American Museum of Natural History for the exhibition: “Traveling the Silk Road: Ancient Pathway to the Modern World.” This evaluation explored visitors’ interest in and understanding of the exhibition and related programming and sought to integrate museum staff and volunteers throughout the process of instrument development, data collection, analysis, and reporting. The collaborative, cross-departmental evaluation effort utimately yielded greater understanding and buy-in for evaluation findings and identified meaningful outcomes, such as connections between special programming and subsequent visitor engagement and learning.

  Report: American Museum of Natural History Silkroad Exhibition
• 

  Photo courtesy of ENERGY.GOV via CC licensing.

  Systems Approach to Green Energy (SAGE) Program

  With a grant award from the NSF IGERT program, the University of California, Berkeley’s (UCB) Systems Approach to Green Energy (SAGE) program aims to provide interdisciplinary education and training for UCB graduate students, in order to develop a workforce capable of achieving greener energy technologies. As the external evaluator of the program, Rockman et al is conducting a five-year, mixed-methods study to track progress towards and attainment of SAGE program goals. The study involves multiple questionnaires and focus groups with students and faculty, and focuses on three areas: (1) program infrastructure, (2) faculty collaboration and productivity, and (3) student expectations and experiences.
• 

  Photo by Kenny Eliason on Unsplash

  Mount Vernon: George Washington Teacher Institute

  Mount Vernon’s George Washington Teacher Institute hosts a residency program that has been providing professional development to teachers since 1999. REA collected feedback from past participants to help Mount Vernon better understand how the Institute has impacted educators over the years and the extent to which these have endured through time. Feedback was collected via surveys and interviews with educators, many of whom reported that the Institute had a profound and lasting influence on their teaching, their outlook on Washington, and the way they think about history as a whole. Major themes from the study include the power of place in teaching history, the value of teaching with diverse primary sources, and the benefits of providing professional development that treats teachers as true professionals.
• 

  Photo by J. Amill Santiago on Unsplash

  Lineage

  The Lineage project was a collaboration between Twin Cities Public Television and the Smithsonian’s National Museum of Natural History. The project included creation of a feature-length film called “When Whales Walked,” hands-on activities, and a Virtual Reality game—all of which were incorporated as part of a series of fossil festivals at museums and other sites around the US. REA conducted evaluations of fossil festival experiences and attendee outcomes and tested the impact of film viewership on participants’ VR and hands-on experiences.
• 

  Photo courtesy of PBS KIDS.

  Testing outcomes of Cyberchase viewing among elementary school students

  REA conducted a quasi-experimental design study with students in four schools located in Indiana and Arizona. Nearly 300 children participated and findings revealed the impacts of Cyberchase programming on a variety of STEM learning and attitudinal outcomes as well as environmental science knowledge, attitudes and beliefs.

  Report: Rockman_2001_SummEval_CybPhase1PilotStudy-Vol1
• Game-Enhanced Interactive Physical Science (GIPS)

  Through funding from the NSF’s SBIR program, Filament Games developed a suite of online educational games and curricular activities for middle school classrooms that target core scientific concepts, such as engineering, plant genetics, and planetary characteristics. Rockman et al evaluated the “promise of efficacy” from each game via pre-post surveys, classroom observations, and embedded assessments, comparing content learning and engagement between students who played the game and engaged with the curricular activities, only played the game, or received traditional instruction. Differences in outcomes based on students’ interest in and aptitude for reading were also explored to ensure that the games were inclusive of a range of student abilities.
• 

  Photo courtesy of NASA's Earth Observatory via CC licensing.

  American Museum of Natural History “Our Earth’s Future”

  Rockman et al collaborated with the American Museum of Natural History to evaluate onsite and online course offerings as part of the “Our Earth’s Future” series of educational programming about climate change. Evaluation efforts included observations, surveys, and focus groups with face-to-face as well as online course participants, and in addition to examining the unique outcomes for each course offering, compared findings across programs of different lenghts and formats.
• 

  Photo courtesy of marco antonio torres via CC licensing.

  A. James Clark Excellence in History Teaching Program

  Rockman et al conducted a study for The Smithsonian National Museum of American History (NMAH) Excellence in History Teaching Program, a two-part program including Teach It Forward, (held at NMAH for teachers in the D.C. Maryland, and Virginia area), and Let’s Do History, (for teachers around the country). The two programs provided professional development for over 3,000 teachers, introducing object- and primary source-based instructional strategies, the Smithsonian’s rich array of resources, and ways teachers could use these to kindle an interest in history among K-12 students. REA explored teachers’ usage of project resources, classroom implementation of new strategies for teaching history, district-level curriculum and instruction changes, and changes in students’ interest in history and historical thinking, using surveys, interviews, case studies, and online activities.
• 

  Photo courtesy of Ilmicrofono Oggiono via CC Licensing.

  Clark Let’s Do History Program and The Smithsonian Institution 3D Resource Pilot

  In this two-part study, Rockman et al (REA) evaluated outcomes of The Smithsonian Institution’s ongoing Let’s Do History teacher training program and an educational iBook which incorporates 3D technology in an interdisciplinary exploration of Abraham Lincoln and his era in history. REA gathered feedback on the Let’s Do History program from teachers who attended the training and students experiencing their teaching techniques in the classroom. REA’s evaluation provided information on how The Smithsonian Institution can combine 3D technology with their considerable artifact collection to bring history to life.