(Mis)Understanding the NAEP Results
Stories from the NY Times, Mother Jones, and the Washington Postbemoaned the flat National Assessment of Education Progress (NAEP) reading scores released Wednesday. Jay Matthews called it the epitaphof the No Child Left Behind era. The results aren’t quite so simple.
See, NAEP is different than most standardized tests. It takes a sample of the current population in every state, so this year’s population of kids is compared to the last time the test was administered. There’s an automatic correction for changing demographics, so as America has gotten less white, so has NAEP. In statistical terms this creates something called Simpson’s Paradox, which makes trend lines seem worse than they really are because of a hidden variable, in this case, race.
To show how this impacts NAEP scores, here are the results of the long-term trend NAEP results for fourth-grade reading from 1975 to 2008 (I’m using the long-term