Skip to content
Q&A

Designing An Intuitive Measure of Value’ for Higher Education Programs

A conversation with Professor Stephanie Cellini on how the federal government can help students steer clear of low-quality schools and get a better education for their investment.

New graduates line up before the start of the Bergen Community College commencement at MetLife Stadium in East Rutherford, N.J. (Associated Press/Seth Wenig)

In January, the U.S. Department of Education confirmed that it will create a list of low-value programs, publicly naming and shaming institutions that offer students junk degrees and saddle them with unpayable debt. The goal is to help students steer clear of programs with poor outcomes. The question now: How should the Department fairly identify these schools?

January’s announcement also came with a formal request for information, or RFI: The Department is accepting feedback from experts, advocates, and policymakers on which metrics it should use to make its list. 

The RFI is an opportunity for the Education Department to hear from the higher ed policy community and other stakeholders about what a low-value program looks like, and about how to get that information in front of students in useful ways,” said Clare McCann, a higher education fellow at Arnold Ventures. 

Comments are due to the Department by February 102023.

One expert who will be submitting feedback is Stephanie Cellini, a researcher and professor in the Trachtenberg School of Public Policy and Public Administration at George Washington University, a recent senior consultant to the White House Office of Information and Regulatory Affairs and an all-around guru in the higher education sector. We spoke to Cellini, whose research includes close analysis of for-profit institutions, about how the Education Department should identify low-value programs and make that information available to students and families.


This interview has been edited for length and clarity.


Arnold Ventures

How did you get into this area of research? 

Headshot of Stephanie Cellini
Stephanie Cellini

Before graduate school, I worked on higher education and job training issues at the Urban Institute, and I knew I wanted to continue to study labor economics and education policy. So in graduate school, like many Ph.D. students, I was thinking about dissertation topics and research areas that people hadn’t looked at before. 

I was watching cable TV one night and saw ads for very quick training programs at what seemed to be a newer type of school. I thought, What are these programs, and has anyone studied them?” That led me down a path to study for-profit programs and institutions. My interest was in seeing what the labor market outcomes were for students in these programs, and there was very little research on them at the time.

Arnold Ventures

The Education Department is asking experts like you for feedback on how to create a watch list of low-performing higher education programs. What’s the big-picture potential for reform here?

Headshot of Stephanie Cellini
Stephanie Cellini

First and foremost, the Education Department needs to ensure that programs and institutions are held accountable for student outcomes. The watch list seems like it could be a move in that direction, but it is not quite there. Right now, one of the only accountability metrics is the cohort default rate on student loans, and it’s a somewhat weak instrument. Soon, changes to the administration’s income-driven repayment program will likely put fewer students into default, and that will make it harder to hold institutions accountable based on a metric of loan default. A watch list is a way to take one step toward greater institutional accountability, but to be effective there need to be consequences for poor-performing programs.

Arnold Ventures

What problem is the department trying to remedy, and why is it so important for students?

Headshot of Stephanie Cellini
Stephanie Cellini

There are programs out there that are extremely expensive and don’t provide the return on investment that students expect. Some programs have very low earnings after graduation, or students can’t find a job in the field that they trained for. They’re going into debt for these very expensive programs and are struggling to repay that debt. That can lead to all kinds of financial struggles, including delinquency and default, which can hurt their credit score, stop them from getting other kinds of credit in the future, and make it difficult to attend another institution. This is especially a problem for students who don’t complete a degree, and it’s also particularly a problem for students in for-profit institutions. 

Students can’t always tell which programs are high-quality and low-quality — that’s a really difficult thing for anyone to observe. But the federal government has data on these programs, and it can use that data to look at the value that programs offer and protect students from high-cost, low-value programs.

Arnold Ventures

In order to create a list using this data, what metric do you recommend?

Headshot of Stephanie Cellini
Stephanie Cellini

To start, putting in place strong gainful employment rules should be a priority of the Education Department. Those rules would require that programs in for-profit institutions and non-degree programs in other sectors lead to employment with earnings high enough to allow students to repay their loans. In negotiated rulemaking, the Department proposed using debt-to-earnings rates and an earnings threshold to hold programs accountable under gainful employment. 

If programs fail to meet benchmark debt-to-earnings ratios or earnings thresholds, they would lose access to federal student aid. Having a consequence is important for accountability.

The Education Department’s watch list should be consistent with gainful employment rules and use similar metrics. A metric that I have focused on in my own research is what I call a high school earnings benchmark. It compares the earnings of students two to three years after they graduate from a higher education program to the earnings of students who have completed high school but not attended any college. It gives what we call a counterfactual, representing the earnings that students might expect if they don’t go to college at all.

If the Education Department uses that benchmark, it shows that it is expecting programs to increase student earnings above what the average high school graduate would earn with no college. This seems like a reasonable benchmark if the program is seeking access to funds – like Pell Grants and student loans – that are intended to support college-going.

Arnold Ventures

How would that metric work to name and shame low-performing schools?

Headshot of Stephanie Cellini
Stephanie Cellini

It is not entirely clear from the RFI how the watch list would work, but it seems that the Administration is proposing to publish an annual list of programs with low financial value. The goal is to signal to students that they should be careful and either consider a different program or learn more about the program before they enroll. 

Of course, this would only work if students see the list, and we know from previous research that information alone is not enough to change student behavior – additional accountability is needed. Still, it would be helpful if data on graduates’ earnings relative to high school graduates were widely available to students and connected to the College Scorecard, where students can easily compare student outcomes for programs they are interested in, whether or not they appear on a watch list.

Arnold Ventures

Others may advocate for different ways to measure school performance and value. Why do you like this one above others?

Headshot of Stephanie Cellini
Stephanie Cellini

It’s easy for students to understand. If you show students that at certain schools, the average graduate does not earn more than the average high school graduate with no college, that’s an intuitive measure of value. I’ve also done work showing that whether programs pass or fail a high school earnings threshold is not driven by student demographics, so it may be more equitable than other metrics. And finally, it is theoretically justified. Earnings metrics are central to estimating the returns to college in labor economics going back to the work of Gary Becker and Jacob Mincer. Under human capital theory, when people decide whether or not to go to college, they think about the benefits they accrue over their lifetime versus the cost of their education. 

While there are many other benefits to education, higher earnings is the largest. They are also critically important if we are concerned about students’ ability to repay debt.

Arnold Ventures

When you’re not thinking about these challenges of higher education policy, what do you do to unwind?

Headshot of Stephanie Cellini
Stephanie Cellini

I like to go running. I also like to travel and ski and hang out with my kids. I took my kids skiing yesterday to Whitetail in Pennsylvania. We almost got stuck in the snow! It was a harrowing drive, but we made it back safely. Last summer, I traveled to South Africa and Zambia with my family and really enjoyed that. We loved Cape Town. It’s just such a stunningly beautiful city with so much interesting history.

Arnold Ventures

And are there any good reports or books you would recommend to readers? 

Headshot of Stephanie Cellini
Stephanie Cellini

I would point people to my research project, the Postsecondary Equity and Economics Research Project. There are a number of articles by academics who are doing policy-relevant work, including Judith Scott-Clayton and Doug Webber. I have a paper with Kathryn Blanchard about the high-school earnings threshold also. 

I am about halfway through reading Nelson Mandela’s autobiography, Long Walk to Freedom. When we were in Cape Town, we learned a lot about Nelson Mandela’s fight against apartheid. We went to Robben Island, where he was held in prison for many years, and heard from a former political prisoner who took us on a tour. That was really moving and important — and a tremendous inspiration when I think about equity in my work.

Grants

Arnold Ventures funds projects to understand problems and identify policy solutions.