Hocus Pocus and the History of High Stakes Tests
By John Thompson
Did you hear the one about “Voo Doo Economics?” President Ronald Reagan said that his “Supply Side Economics” would cut taxes, increase spending, and reduce the deficit!?!?
If a 22nd century historian were to uncover Reagan’s claim, and yet discover that all of the physical and digital records of the 1980s and several subsequent decades had been lost, it might be hard to prove that Reaganism didn’t raise the deficit.
Perhaps the same agnosticism could apply to claims that No Child Left Behind boosted student performance as measured by the reliable NAEP tests – except for one reason. NAEP records are readily obtainable by a quick Google search.
NAEP data may not prove what I believe to be the best summary of the evidence – that NCLB and subsequent NCLB-type testing caused more harm than good for students. But, NAEP metrics do prove the intellectual dishonesty of the true believers who claim that high stakes testing has improved so-called “student performance.”
In fact, NAEP scores were increasing before NCLB and their growth slowed after NCLB testing took effect. The American Institutes of Research’s Mark Schneider, known by the conservative Fordham Institute as the “Statstud,” is just one scholar who documented this pattern, concluding “pre-NCLB gains were greater than the post-NCLB gains.”
Curiously, Schneider was also one of the true believers who first pushed the silly claim that NCLB deserves credit for test score gains that occurred before the law was enacted. Illogically, reformers claim test score increases from 1999 to the winter of 2002 were the result of a law that was enacted in the winter of 2002. The actual passage of NCLB high stakes testing was the tail of a “meteor” that was dubbed “consequential accountability.” And that brings us to the latest convoluted spin trying to deny that test-driven reform has failed. The most recent example is Tom Loveless’s “Measuring Effects of the Common Core.”
At least Loveless’s approach to the pre-NCLB effects of NCLB is much more modest. He claims that it is “unlikely” that accountability efforts and increased reform-related spending did not “influence” pre-NCLB NAEP scores. Even so, Loveless offers no credible reason to believe that increases in 1999 test results should be attributed to stakes attached to tests that were imposed three years later.
So, what evidence does Loveless offer for his conclusion that NCLB might deserve credit for the test scores that preceded it? He cites Derek Neal and Diane Whitmore Schanzenbach who reported that “’with the passage of NCLB lurking on the horizon,’ Illinois placed hundreds of schools on a watch list and declared that future state testing would be high stakes.”
Neal and Schanzenbach were studying Chicago schools, however, and they concluded the opposite. They report that “ISAT performance played a small role in the CPS rules for school accountability over this time (1999 to 2001).” Neal and Schanzenbach explain that “in one year, the ISAT went from a relatively low-stakes state assessment to a decidedly high stakes exam.” But, “in the springs of 1999, 2000, and 2001, CPS took the ISAT with the expectation that the results would not have significant direct consequences in terms of the state accountability system.”
By the way, Neal’s and Schanzenbach’s title of “Left Behind by Design,” is not exactly a ringing endorsement of Schneider’s and Loveless’ spin in favor of No Child Left Behind.
Loveless cites two other papers in his footnotes, but a careful reading of provides evidence for both sides of Hocus Pocus and the History of High Stakes Tests - Living in Dialogue: