OH WHAT A TANGLED WEB WE WEAVE, WHEN WE MAKE TESTS PUNITIVE
BY JOHN THOMPSON
When I was in school in the 1960s, we occasionally took a standardized test, and we took ACT and SAT tests as a part of the college application system.
These were not the type of tests you could “teach to” or “study for.” Consequently, they provided a huge database of invaluable information about students and schools. These tests were a Consumer Reports on education, and they could have been an essential component of a data-informed – not data-driven – contemporary school reform movement.
I don’t want to get into the weeds of the social science, but the ACT and other tests were Norm Referenced Tests [NRTs] that assessed where students stood in comparison with others on the types of testable knowledge that students needed. They were not appropriate for holding individuals accountable. Back then, we showed a healthy respect for the proper use of test methodology, and ACTs and similar NRTs were not imposed for high-stakes purposes.
So, a new set of tests, Criterion Reference Tests [CRTs], were developed for measuring testable knowledge of Standards of Learning. Fifteen years ago, several outstanding experts in the Oklahoma City Public School System Planning Department explained, chapter and verse, how the seemingly arcane shift from NRTs to CRTs would corrupt tests and accountability systems.
With the passage of No Child Left Behind, the misuse of these new tests expanded further than we could have imagined. Then, Arne Duncan became the nominal secretary of education, and the test-loving Bill Gates became the real driver of school policies, and primitive bubble-in tests were misused in conjunction with not-ready-for-prime-time statistical models.
The entire testing system became hopelessly corrupted, undermining the quality of instruction in Oklahoma and the rest of the nation.
Now, Oklahoma is taking the lead in rescuing our schools from the test, sort, and punish mania known as school reform. A recurring suggestion, by good people who seek to reign in the testing frenzy, is the proposal that the ACT replace Oklahoma’s EOIs [see Rep. David Perryman’s HB 1497, for example].
While I don’t think that’s a good idea, I’d like to suggest a framework of discussing the potential value of using the ACT as a graduation examination.
The good news is that the ACT would help solve some of the problems that vex educators. The bad news is that the ACT could liberate adults, and many students, but dramatically increase the dropout rate.
Like most states, Oklahoma was coerced by the federal government into the use of “value-added” estimates of student test score growth in teachers’ evaluations. As an overwhelming majority of scholars explained, these models [known as VAMs] used inappropriate tests in an unreliable manner. They are biased against educators in high-poverty schools with large numbers of English Language Learners [ELLs] and special education students. If Oklahoma can’t find a way to abandon these invalid value-added evaluations, the exodus of our top teaching talent from poor schools will increase.
The courts have tended to conclude that value-added evaluations are improper and unfair, but that they cannot be struck down until they produce outcomes that are irrational. If systems tried to turn ACT test score into VAMs, that methodology would be so invalid that the entire evaluation system would likely collapse. That would be a great thing for educators and schools, and it would free Oklahomans from the single worst legacy of the education reform blame game.
Unfortunately, because of the way that testing has been misused over the last 12 years, the problem facing students is more complex. It took years to do so, but Oklahomans debated the type of End of Instruction [EOI] graduation tests that would be fair to all. We reached the reasonable conclusion that these exit exams should be minimum skills tests, and that the stakes attached to them should be Oh What A Tangled Web We Weave, When We Make Tests Punitive | Oklahoma Observer: