Why It's So Hard To Know Whether School Choice Is Working

Education Secretary Betsy DeVos has been a passionate proponent of expanding school choice, including private school vouchers and charter schools, and she has the clear backing of President Trump. But does the research justify her enthusiasm?

Experts say one single, overarching issue bedevils their efforts to study the impact of school choice programs. That is: It's hard to disentangle the performance of a school from the selection of its students.

Students are never randomly assigned to a school. A school's population is always affected by local demographics. With schools of choice, by definition, parents and students are making a decision to attend that school, so their enrollment is even less random.

Even when researchers carefully match students at different schools based on demographics, it's possible that families that are more organized and more invested in education are also more likely to seek out charters and voucher programs. Or, selection bias can also work the other way: The students who struggle in traditional public schools may be more likely to seek alternatives.

Further complicating matters is this: By law, most charter schools must have open enrollment, using a lottery if they have more applicants than spots. However, charters, and private schools, have sometimes been accused of using strict discipline rules or other measures to filter out underperforming or otherwise undesirable students.

The admissions policies of voucher-accepting private schools can also vary widely, depending on school policy and state law. Some must have open enrollment. Others retain the right to select students based on religion, academic achievement, artistic talent, conduct, or other factors.

And, no matter where you look, both private schools and charters tend to enroll fewer students with disabilities compared with public schools, a practice known as "creaming."

So, with the huge caveat that it's difficult to do an apples-to-apples comparison, where do researchers agree on the impact of choice?

Choice programs seem to push nearby public schools to improve.

"The results are consistent, and I don't think there's any debate," says Douglas Harris of the Education Research Alliance at Tulane University. "Charters, vouchers and tax credits create competition and positive spillovers." When school choices expand, public schools stand to lose students, and thus money, and they seem to respond by stepping up their game.

Martin Carnoy, a professor at Stanford University who has been studying the issue for decades, terms this response to competition "an accountability effect." In other words, "When you announce that there's a new sheriff in town, all of a sudden everybody perks up."

Carnoy's studies of cities including Milwaukee argue that this effect fades over time.

There's far less agreement among researchers on the effects of choice for the students who choose to leave their traditional public school.

First, charter schools. Charter students in city centers tend to do better than their public-school peers, according to the most recent research by the Center for Research on Education Outcomes at Stanford. But charter students elsewhere tend to do the same or worse.

And the learning progress of nearly 200,000 students enrolled in 100-percent online "virtual charter schools" is generally so bad that representatives of the charter sector recently called for states to close many of them.

Second, private schools. Whether students are using vouchers or tax-credit scholarships, their academic outcomes are a similarly mixed bag.

"Most studies find modestly positive or neutral impact on student scores, and that's generally limited to African-American students in large urban centers," says Micah Ann Wixom, a policy analyst with the Education Commission of the States.

Martin Carnoy wrote for the left-leaning Economic Policy institute in February: "Extensive research on educational vouchers in the United States over the past 25 years shows that gains in student achievement are at best small."

"Do voucher students perform better than they would have in their neighborhood school?" asks Josh Cunningham, a senior education policy specialist with the National Conference of State Legislatures. "At the most I can say, 'Maybe, a little bit.' But there's not a lot of evidence that they'll have any substantial academic gains."

Or, as the Center on Education Policy underscored in its 2011 survey of voucher research: "Achievement gains for voucher students are similar to those of their public school peers."

But more recent research has found that private schools, like charter schools, can actually hurt the academic progress of the students who choose them. Studies of programs in Louisiana, Ohio and Washington, D.C., found students who left for private schools did worse, not better, compared with their peers.

Scholarship tax-credit programs, like voucher programs, send students to private schools. Their impact is hard to determine, says Harris of the Education Research Alliance.

"The best-case scenario seems to be a slight positive effect, but most people would interpret it as no effect," he says.

Finally, when it comes to voucher and tax-credit programs, many states don't even require private schools to administer the state's test. In Florida, it can be any nationally normed test. So these programs are a bit of a black box. Going forward, if private school options expand, we may know less and less about the schools that more and more students are attending.

Oh, and there's another challenge: Most of these analyses rely on test scores alone. Test scores provide an imperfect and incomplete picture of student performance, as the most recent reauthorization of the federal education law acknowledges.

Copyright 2017 NPR. To see more, visit http://www.npr.org/.