Grantee: MDRC. The study report is linked here.
Description of the Intervention: This study is a replication randomized controlled trial (RCT) of ASAP, as implemented by three Ohio community colleges. Originally developed by the City University of New York (CUNY), ASAP provides academic, personal, and financial supports to low-income students to help them earn an associate’s degree within three years. Core program components include (i) required full-time enrollment; (ii) a range of resources that ASAP students are required to use including a dedicated advisor who helps students with academic, social, and interpersonal issues, a career counselor, and (if needed) tutoring services; (iii) special class scheduling options to ensure that ASAP students secure the classes they need and that they take remedial classes (if needed) early in college; and (iv) financial supports such as tuition waivers, free textbooks, and a monthly financial incentive to assist with transportation or groceries. The program’s cost in Ohio is approximately $8,030 total per student over three years.1 ASAP was found in an earlier well-conducted RCT at CUNY to substantially increase the rate of degree completion.
Study Design: The researchers randomly assigned 1,501 low-income students across three Ohio community colleges 2 to either a treatment group that was eligible to receive the program, or to a control group that was not (but had access their college’s usual services). Sample members averaged 23 years of age, and were 64% female, 35% Black, and 46% white. 34% were the first member of their family to attend college. The study’s pre-registered primary outcomes for the six-year follow-up were (i) degree completion, measured with National Student Clearinghouse data; and (ii) workforce earnings, measured with Ohio Unemployment Insurance wage records. The study is ongoing, and will measure program impacts through 10 years after random assignment.
Impact on the Primary Outcome:
At follow-up six years after random assignment, the study found that the Ohio programs increased the rate of degree completion (the first primary outcome) by 15 percentage points. Specifically, 44% of the treatment group completed a college degree versus 29% of the control group – a difference that was statistically significant (p<0.01). In exploratory analyses examining degree type, the study found statistically significant impacts on both two-year associate’s degree completion (42% in the treatment group versus 26% in the control group, p<0.01) and four-year bachelor’s degree completion (14% versus 9%, p<0.01).
On the other primary outcome – earnings – the study found that ASAP Ohio produced a statistically significant 11% ($1,948) increase in average annual earnings in the sixth year after random assignment ($19,573 in the treatment group versus $17,626 in the control group, p<0.053).4
Study Quality. Based on a careful review, we believe the study was well-conducted and produced valid findings. 5
-
1
2018 dollars. This total cost of $8,030 includes the direct cost of program services in Ohio ($5,521) plus the higher cost associated with treatment group students taking more classes than control group students ($2,510). For reference, the current cost of ASAP at CUNY is $10,200 total per student, which is lower than its initial, start-up cost ($16,000) during the CUNY ASAP RCT.
↩︎ -
2
Participating community colleges were Lorain Community College, Cuyahoga Community College, and Cincinnati State Technical and Community College.
↩︎ -
3
The p-value was 0.047, which meets the conventional 0.05 benchmark for statistical significance. After being adjusted for multiple comparisons (across the two primary outcomes) using the Westfall-Young method, the p-value was 0.059 – just shy of the benchmark.
↩︎ -
4
Because the study measured earnings using Ohio Unemployment Insurance wage records, it captured only wages earned in Ohio and therefore may miss earnings of sample members who moved to other states. The researchers hope to address this limitation in future study reports by obtaining Unemployment Insurance wage records from neighboring states.
↩︎ -
5
For example, the study had successful random assignment (as evidenced by highly similar treatment and control groups), negligible sample attrition, and valid analyses that were appropriately pre-specified.
↩︎
