If educational choice programs are to be judged by testing gains—which they often are—it is vital to establish a link between a program’s effect on short-term achievement with its effect on long-term attainment.
This has been at the center of a recent debate, spurred by publication of a provocatively-titled meta-analysis, “Do Test Scores Even Matter? Lessons from Long-Run Outcomes in School Choice Research.” This paper presents a weak relationship between a choice program’s effect on achievement and attainment. American Enterprise Institute (AEI) authors Colin Hitt, Michael McShane and Patrick Wolf conclude that policymakers should thus be cautious about evaluating choice programs solely on test scores.
In response, Fordham Institute’s Michael Petrilli penned a five-part series questioning the authors’ methodology and arguing that, in fact, test scores correlate neatly with later-life outcomes—assuming you only analyze “bona-fide” choice programs (which excludes early college high schools, selective enrollment high school and career and technical programs).
Wolf responded, here, to Petrilli.
What should we make of this debate? At ExcelinEd, we believe there is value to traditional, test-based accountability as a short-term indicator of a child’s progress. It’s not reasonable to wait for a cohort of students to graduate college before measuring the academic success of a choice program. Long-term indicators are, indeed, long-term—and test-based accountability provides quicker feedback to parents, schools and policymakers.
Having said that, there are many indicators beyond test scores that should also be considered when analyzing the benefits of education choice. These include college enrollment, college attainment, career earnings, civic engagement and parental satisfaction, among others. When making the case for new or expanded programs, policymakers and advocates can rightly point to progress on these indicators, too. Especially if the AEI authors are correct about the tenuous relationship between achievement and attainment.
Although the application of test-based accountability to education choice will be an ongoing conversation, one thing most can agree on is the need for more research on the long-run effects of choice. Accordingly, it was fortuitous to attend last week’s conference at Harvard Kennedy School’s Program on Education Policy and Governance “Learning from the Long-term Effects of School Choice in America.”
In addition to the AEI analysis, research was presented and discussed on the following topics:
- Positive effects of Florida’s tax-credit scholarship program on college enrollment.
- Positive effects of a New York City Democracy Prep Public Charter School on civic engagement.
- Positive effects of religious schools on giving and volunteering in adulthood.
- Neutral effects of public school choice on high school and post-secondary outcomes in San Diego Unified School District.
Perhaps the disagreement between the AEI authors and Petrilli can be a catalyst for more research on how educational choice programs affect both achievement and attainment, in both the short- and long-term. This would be a welcome outcome from an important debate.
About the author
James Paul is an Associate Policy Director who focuses on expanding opportunity through private education choice. In this role, James provides analysis and support to state partners regarding the design and implementation of tax-credit scholarship and education savings account programs. Prior to joining ExcelinEd, James was a Policy Analyst at the Commonwealth Foundation in Harrisburg, Pennsylvania, where he worked on a variety of state education issues. James graduated from Syracuse University and resides in the Washington, D.C. area.