Grantee: WestEd. The study report is linked here.
Description of the Intervention: ASSISTments is an online study tool that assists seventh grade students with math homework and produces real-time reports for teachers on student performance. As students work through homework problems and enter their answers into ASSISTments, the system provides immediate feedback on the correctness of answers and offers additional assistance in the form of hints or scaffolds. ASSISTments provides teachers with real-time, easily accessible reports that summarize student work for a particular assignment, which teachers can use to target their homework review in class and tailor instruction to their students’ needs. A prior well-conducted RCT in Maine found that ASSISTments produced a statistically significant positive impact on state test scores in math at the end of seventh grade (effect size 0.18). The program’s cost is less than $100 per student, based on a cost study conducted alongside the RCT.
Study Design: The researchers randomly assigned 63 public schools in North Carolina to a treatment group that implemented ASSISTments in seventh grade math classrooms versus a control group that did not. The student sample comprised all 9,073 seventh graders in these schools in the 2019 – 2020 school year whose sixth grade (baseline) test scores were available to researchers. Sample members were 17% Black, 19% Hispanic, and 56% White, and 53% were economically disadvantaged. The study’s pre-registered primary outcome was math achievement on the North Carolina state test (EOG-MA08) at the end of eighth grade – one year after program completion. f1
Impact on the Primary Outcome: At the end of eighth grade (one year post-program), the study found a statistically significant positive impact on student math achievement on the state test. The impact (0.10 standard deviations, p=0.01) represents approximately a 31% improvement over the annual gain in math otherwise expected for eighth graders. f2 The presence of this impact one year after program completion is evidence of a sustained learning gain. The study also examined, as a primary research question, whether the program’s impact for lower-performing students (those with baseline math scores below the median) differed from its impact for higher-performing students. It found suggestive evidence (near statistically significant, p<0.10) of a larger impact for higher-performing students.
Study Quality. Based on a careful review, we believe the study was well-conducted and produced valid findings. f3
