NSF Evaluations

NSFRockman et al has served as the external evaluator for dozens of National Science Foundation (NSF) grants, including:

  • Advanced Technological Education (ATE)
  • Advancing Informal STEM Learning (AISL)
  • Discovery Research K-12 (DR-K12)
  • Early Concept Grants for Exploratory Research (EAGER)
  • Historically Black Colleges and Universities Undergraduate Program (HBCUUP)
  • Informal Science Education Program (ISE)
  • Information Technology Experiences for Students and Teachers (ITEST)
  • Math and Science Partnership (MSP)
  • Small Business Innovation Research (SBIR)
  • Transforming Undergraduate Education in Science (TUES)
  • Improving Undergraduate Education in Science (IUES)

Our team offers experience and expertise in working closely with clients to design and conduct science learning evaluations that meet rigorous standards for NSF external evaluations. Our evaluations have examined science learning across grade levels, disciplines, and learning environments, ranging from museums to k-12 and post-secondary schools, to summer camps and after-school programs, to digital media, and radio and television series. Our team is experienced in facilitating partnerships between scientists and educators, and in providing technical assistance to scientists who are interested in learning more about evaluation. Several of our NSF projects have resulted in publications and conference presentations. We invite you to browse selected examples of our rigorous, independent evaluations for NSF grant programs, below.

We welcome the opportunity to discuss your project with you and to provide you with more information about our tailored services. Please Contact Us to schedule a free 30-minute Q&A session with a consultant.

Selected Projects Include

  • Photograph courtesy of Jared Kelly via creative commons licensing.

    RadioLab

    Radiolab’s “What Curiosity Sounds Like: Discovering, Challenging, and Sharing Scientific Ideas” project, funded by the NSF Advancing Informal Science Learning (AISL) program, explored new ways to actively engage both lay and professional audiences and foster meaningful communication between scientists and the general public. As the external evaluator for the project, REA examined participants’ experiences and explored project outcomes through a series of surveys and interviews.

    Report: Radiolab. What Curiosity Sounds Like
  • Photo courtesy of KQED QUEST via CC licensing.

    KQED

    REA has a long history of partnering with KQED, the PBS affiliate in Northern California, on diverse research and evaluation projects involving teacher professional development, organizational development, environmental education, public science communication, and science journalism. Specific projects include evaluation of KQED Teach (an online professional development tool for teachers), a two-year organizational development study, and two NSF-funded projects applying social science research methods to investigate best practices for engaging millennial audiences in science content, and improving science media production and journalism practices.

    Report: QUEST Summative Evaluation Report
  • Photo courtesy of Julia Li.

    Peg + Cat: Adventures in Learning

    Peg + Cat is a math-based PBS television program for preschoolers, supplemented by online games, digital storybooks, and apps. We conducted formative and summative evaluation of Peg + Cat interactive media and themed science camp activities, developed by Fred Rogers Productions (formerly The Fred Rogers Company) and Carnegie Science Center as part of an NSF-funded AISL project. We examined pre-schoolers’, families’, and educators’ experiences using project resources, as well as pre-schoolers’ social-emotional and learning outcomes via observations, focus groups, interviews, and a quasi-experimental design that incorporated pre-post surveys and content assessments.

  • Photo: BAVC News

    Bay Area Video Coalition Bridges to STEM Careers

    REA conducted a formative and summative evaluation of Bay Area Video Coalition’s Bridges to STEM Careers, a three-year Advanced Technological Education (ATE) Program Improvement Project designed to enhance the relevance of technician education in Computer Science and Multimedia. The project targeted two-year college students, their families and their college and workplace mentors. The ultimate goal of the Bridges project was to keep traditionally underrepresented and low-income students in the STEM pipeline by demystifying the process of preparing for a STEM career through internships and informational videos from past internship participants.

  • Photo courtesy of crowdandcloud.org

    The Crowd & the Cloud

    The Crowd & The Cloud, a three-year project developed by Passport To Knowledge and funded by the NSF, used multimedia to engage different audiences around citizen science and crowdsourcing. The project team created four episodes of a broadcast television series, which appeared on the WORLD channel and PBS stations, an interactive website, and a robust social media presence in an attempt to reach three target audiences: the general public, scientists, and citizen scientists. REA conducted an external evaluation of The Crowd & The Cloud’s broadcast series via online surveys and in-person focus groups, and The Crowd & The Cloud’s website and social media via online analytics, surveys, and focus groups.

  • Photo courtesy of ENERGY.GOV via CC licensing.

    Systems Approach to Green Energy (SAGE) Program

    With a grant award from the NSF IGERT program, the University of California, Berkeley’s (UCB) Systems Approach to Green Energy (SAGE) program aims to provide interdisciplinary education and training for UCB graduate students, in order to develop a workforce capable of achieving greener energy technologies. As the external evaluator of the program, Rockman et al is conducting a five-year, mixed-methods study to track progress towards and attainment of SAGE program goals. The study involves multiple questionnaires and focus groups with students and faculty, and focuses on three areas: (1) program infrastructure, (2) faculty collaboration and productivity, and (3) student expectations and experiences.

  • University of California, Berkeley Museum of Paleontology (UCMP)

    REA conducted focus groups and usability studies to capture the potential instructional use of the University of California Museum of Paleontology‘s (UCMP) Understanding Evolution (UE) website. REA examined user profiles and usage patterns, reporting on the value that the web-based resources provided to users for teaching science concepts. REA also conducted market research for areas of further website development.

  • Photo courtesy of joaquin uy via CC licensing.

    YR Media

    Since 2006, REA has served as the independent evaluator on YR Media’s education initiatives to engage under-represented youth in informal STEM learning through interest-driven inquiry that combines journalism, design, data, and code. Building on YR Media and MIT App Inventor’s longstanding collaboration to democratize youth media production, the Innovative Approaches to Informal Education in Artificial Intelligence project, funded by NSF-AISL, aims to improve public understanding of artificial intelligence (AI) among young creators and their audiences through youth-led journalism about AI, digital media powered by AI, and open educational resources about how AI works and its role in society. REA conducted the project’s independent evaluation, which included project team interviews, participant observations, focus groups with youth producers, online youth surveys, and audience testing to capture outcomes and lessons learned about effectively engaging youth in learning about AI.

    Report: Youth Radio Innovation Lab Final Evaluation Report