NSF Evaluations

NSFRockman et al has served as the external evaluator for dozens of National Science Foundation (NSF) grants, including:

  • Advanced Technological Education (ATE)
  • Advancing Informal STEM Learning (AISL)
  • Discovery Research K-12 (DR-K12)
  • Early Concept Grants for Exploratory Research (EAGER)
  • Historically Black Colleges and Universities Undergraduate Program (HBCUUP)
  • Informal Science Education Program (ISE)
  • Information Technology Experiences for Students and Teachers (ITEST)
  • Math and Science Partnership (MSP)
  • Small Business Innovation Research (SBIR)
  • Transforming Undergraduate Education in Science (TUES)
  • Improving Undergraduate Education in Science (IUES)

Our team offers experience and expertise in working closely with clients to design and conduct science learning evaluations that meet rigorous standards for NSF external evaluations. Our evaluations have examined science learning across grade levels, disciplines, and learning environments, ranging from museums to k-12 and post-secondary schools, to summer camps and after-school programs, to digital media, and radio and television series. Our team is experienced in facilitating partnerships between scientists and educators, and in providing technical assitance to scientists who are interested in learning more about evaluation. Several of our NSF projects have resulted in publications and conference presentations. We invite you to browse selected examples of our rigorous, independent evaluations for NSF grant programs, below.

We welcome the opportunity to discuss your project with you and to provide you with more information about our tailored services. Please Contact Us to schedule a free 30-minute Q&A session with a consultant.

Selected Projects Include

  • Radiolab

    Rockman et al conducted a comprehensive evaluation of Radiolab initiatives funded as part of its “What does curiosity sound like?” project. The project included evaluation of live events, online content, Radiolab’s social media and mobile phone app, and citizen science activities. The evaluation also sought to differentiate outcomes on STEM professionals and lay-audiences, and the extent to which Radiolab programming can facilitate better communication between these two audiences.

  • KQED QUEST

    QUEST, initiated by KQED in San Francisco, is a unique cross-platform science journalism collaborative designed to tap into the region’s interest in science, environment, and nature, while providing entertaining and informative content about current local issues in science-environment-nature, and engaging users in conversations about actions for these issues. Over a six-year period, REA evaluated pilot materials, including radio, TV and web content, and educational outreach workshops, and documented internal operations and partnerships with external organizations. The project served to identify the internal collaboration processes and resources needed for full production of a weekly half hour magazine show on KQED-TV, a weekly insert in Morning Edition, and Web site and education materials for use by general audiences. The formative and summative evaluation of QUEST’s six-station collaborative over a three-year period included focus groups and interviews with diverse stakeholders, and revealed that QUEST successfully served a number of interests across diverse groups, addressing environmental issues with high quality content and current science information.

  • Peg + Cat: Adventures in Learning

    As external evaluators for an NSF-funded AISL grant awarded to The Fred Rogers Company, Rockman et al tested early versions of Peg + Cat interactive media and science camp activities with pre-schoolers, their families, and their teachers. REA also evaluated youth experiences and outcomes in an affiliated summer camp, including observations, focus groups, and interviews with participating educators and children. Peg + Cat is a PBS math-based television program for preschoolers, which is accompanied by interactive media including digital storybooks, online games and apps.

  • Bay Area Video Coalition Bridges to STEM Careers

    REA conducted a formative and summative evaluation of Bay Area Video Coalition’s Bridges to STEM Careers, a three-year Advanced Technological Education (ATE) Program Improvement Project designed to enhance the relevance of technician education in Computer Science and Multimedia. The project targeted two-year college students, their families and their college and workplace mentors. The ultimate goal of the Bridges project was to keep traditionally underrepresented and low-income students in the STEM pipeline by demystifying the process of preparing for a STEM career through internships and informational videos from past internship participants.

  • The Crowd & the Cloud

    The Crowd & the Cloud is designed to increase the visibility and credibility of, and participation in projects using citizen science as well as Big Data to address key global and local challenges, including: public health, air and water quality, climate change, and biodiversity. The series will consist of four nationally broadcast documentary programs, a 2nd screen app, website, and social media materials. Rockman et al will conduct audience research and usability testing with the general public, citizen science participants, and scientists to inform development. REA will also conduct the summative evaluation to assess the impact of the project’s media, which includes pre-post surveys, a quasi-experimental study of the traditional broadcast versus the app-enhanced version, phone interviews, and a series of focus groups. To evaluate engagement with online materials, REA will examine the program’s social media and website analytics and samples of website discourse. Read more about the series in this article from the Center for the Advancement of Informal Science Education.

  • Systems Approach to Green Energy (SAGE) Program

    With a grant award from the NSF IGERT program, the University of California, Berkeley’s (UCB) Systems Approach to Green Energy (SAGE) program aims to provide interdisciplinary education and training for UCB graduate students, in order to develop a workforce capable of achieving greener energy technologies. As the external evaluator of the program, Rockman et al is conducting a five-year, mixed-methods study to track progress towards and attainment of SAGE program goals. The study involves multiple questionnaires and focus groups with students and faculty, and focuses on three areas: (1) program infrastructure, (2) faculty collaboration and productivity, and (3) student expectations and experiences.

  • University of California, Berkeley Museum of Paleontology (UCMP)

    REA conducted focus groups and usability studies to capture the potential instructional use of the University of California Museum of Paleontology’s (UCMP) Understanding Evolution (UE) website. REA examined user profiles and usage patterns, reporting on the value that the web-based resources provided to users for teaching science concepts. REA also conducted market research for areas of further website development.

    Services:
  • Youth Radio

    REA has evaluated four of Youth Radio’s NSF-funded initiatives to provide training in science journalism and technology development to underserved young people, and to create digital media toolkits for educators. REA has triangulated data from various sources, including surveys, interviews, focus groups and observations, to document the evolution of Youth Radio’s STEM programming, and its effects on the skills, knowledge, and attitudes young adults need to produce and communicate STEM ideas.