Rockman et al (REA) recently served as the external evaluator on the NSF-sponsored project “A Massive Open Online Platform for Language Learning Content” (No. 1632488), aimed at studying the impact of Speak Agent, an academic language acquisition tool, on science language development for elementary students with variable English language proficiency. This online tool delivers digital lessons that align to curricular content at each grade level for a particular school district. Lessons in Speak Agent each consist of a variety of multimodal activities that develop academic language components such as vocabulary and syntax through reading, writing, listening, and speaking.
The project implemented a quasi-experimental design to study the differences in academic language development, as measured by changes in science vocabulary comprehension between students with and students without access to the Speak Agent tool. REA worked with 2nd and 3rd grade science classes from an independent public school district in central Texas. The majority of students were identified by the district as English as a Second Language (ESL) learners. REA developed science vocabulary assessments for each grade, based on the school district’s science curriculum learning objectives.
For the first academic semester, half of the classes from each grade started the school year with access to the Speak Agent tool while the remaining classes continued with their regularly scheduled curriculum. In the second half of the year, all classes were given access to the tool, but half of the teachers had opportunities to customize the activities to fit the needs of their students. All students completed the pre and post vocabulary assessments for each term. This design allowed REA to compare differences in vocabulary comprehension between those who did or did not have access to Speak Agent, as well as between classes where teachers could or could not tailor the activities.
Results from the first term illustrated statistically significant differences in science vocabulary growth between classes who did and did not use the Speak Agent tool. Second grade students who had access to Speak Agent exhibited more than three times the growth in science content knowledge than students who did not use the tool.
In the second semester, there were no statistically significant differences between classes whose teachers did or did not customize the tool. This is due to the fact that there was little to no customization implemented by the teachers, resulting in no procedural differences between conditions.
The results from the study suggest that use of the Speak Agent tool facilitated science language development for ESL students at a faster pace than teacher instruction alone. However, there were implementation issues that arose during the study that could have hindered the full impact of the study design. Mainly, teachers were asked to give students at least one hour of Speak Agent usage time per week, but due to restrictions in school schedules and technology availability, a majority of students had limited experience (average of 2-3 hours of total usage per semester) with the tool. Future research should focus on ensuring fidelity of the Speak Agent implementation in order to observe the full impact of the tool on language acquisition.
Please read the full summative report evaluation for Speak Agent here.