Reformer ToolboxLogin

CancelLost your password?

I’ll Have a Dose of Confirmation Bias, Heavy on the Bias

• Dr. Matthew Ladner

So how do private school students do in Science compared to public school students? I wasn’t sure, so I went to the NAEP data explorer to find out.

Private school students outscore public school students, but private school students tend to be more affluent than public school students, and there can be differences in special need and language profiles. Fortunately the NAEP data explorer allows you to take such factors into account. To maximize the comparison, we will only look at the NAEP Science scores of children eligible for a Free or Reduced-Priced Lunch under federal guidelines and who have neither a special education nor an English Language Learner designation. This is about as close to apples-to-apples comparison you can hope for in NAEP data.

So NAEP changed the framework of their Science exam in 2009, making the 2009 and later exams incomparable to those given before 2009. The comparison of general education poor children between public and private schools is sporadically available in both NAEP Science frameworks. You can’t compare old NAEP Science to new NAEP Science, but you can compare public and private school scores within each year. So let’s start with 4th grade:

NAEP Science 4

Private school generic poor children outscored their peers in the public schools two out of three tries. Let’s look at 8th grade scores:

NAEP Science 8

Private school generic poor children outscored their public school peers three out of four times in 8th grade. Let’s have a look at 12th grade scores:

NAEP Science 12

So for those of you scoring at home, in eight possible comparisons, private school general education poor children outscored six times. It was close (within the margin of sampling error) a few times, but every time the result was lopsided it was lopsided in favor of the private school children. Quite frankly science scores should be higher in both public and private schools for low-income kids, but the available evidence does show an overall private school advantage. Unless you happen to be Stephanie Simon working through a sizable case of confirmation bias, in which case this is what you saw:

Taxpayers in 14 states will bankroll nearly $1 billion this year in tuition for private schools, including hundreds of religious schools that teach Earth is less than 10,000 years old, Adam and Eve strolled the garden with dinosaurs, and much of modern biology, geology and cosmology is a web of lies.

Gosh, a billion dollars—that sounds scary! At least until you think of it as less than 80 percent of the Dallas Independent School District’s budget. Still, this is an outrage! We should put a stop to it immediately!

Except…how is it that these kids at hillbilly flat-earther private schools keep managing to score about the same or more often better than their public school peers on the NAEP Science exams? Does the NAEP Science framework ask a battery of questions on the Book of Genesis? Does learning how to play Dueling Banjos wire the mind for multiple choice science exams?

Um, no. Not so much. Private schools just do a better job teaching science overall. Ms. Simon has written a hyperbolic story about a crisis that does not exist. The available evidence suggests that if we eliminated all funding for choice programs that it would result in a net decrease in knowledge of science.

If Ms. Simon wants to pull the funding for private schools based on science achievement, the river needs to flow both ways and we will have to pull the funding for an even larger number of public schools on the same basis. In the meantime, if Ms. Simon doesn’t like private schools, she always has the option of not enrolling her children in one. As an added bonus, her kids can learn science on Khan Academy if she happens to choose one of the many that do a poor job of teaching science.


[Editor’s note: Michael McShane at the American Enterprise Institute has also published a response to Ms. Simon’s article in the National Review Online.]

About the author

Dr. Matthew Ladner @MatthewLadner

Dr. Matthew Ladner is the Senior Advisor of Policy and Research for the Foundation for Excellence in Education. He previously served as Vice President of Research and Goldwater Institute. Prior to joining Goldwater, Dr. Ladner was director of state projects at the Alliance for School Choice. Dr. Ladner has written numerous studies on school choice, charter schools and special education reform. Most recently, Dr. Ladner authored the groundbreaking, original research Turn and Face the Strain: Age Demographic Change and the Near Future of American Education, outlining the future funding crisis facing America’s K-12 public education funding. He also coauthors the American Legislative Exchange Council's annual Report Card on American Education: Ranking State K-12 Performance, Progress and Reform. Dr. Ladner has testified before Congress, the United States Commission of Civil Rights and numerous state legislative committees. He is a graduate of the University of Texas at Austin and received both a Masters and a Ph.D. in political science from the University of Houston. Dr. Ladner is a Senior Fellow with the Foundation for Educational Choice. He lives in Phoenix, Arizona.