The Status Fallacy: New York State Edition
A recent New York Times story addresses directly New York Governor Andrew Cuomo’s suggestion, in his annual “State of the State” speech, that New York schools are in a state of crisis and "need dramatic reform." The article’s general conclusion is that the “data suggest otherwise.”
There are a bunch of important points raised in the article, but most of the piece is really just discussing student rather than school performance. Simple statistics about how highly students score on tests – i.e., “status measures” – tell you virtually nothing about the effectiveness of the schools those students attend, since, among other reasons, they don’t account for the fact that many students enter the system at low levels. How much students in a school know in a given year is very different from how much they learned over the course of that year.
I (and many others) have written about this “status fallacy” dozens of times (see our resources page), not because I enjoy repeating myself (I don’t), but rather because I am continually amazed just how insidious it is, and how much of an impact it has on education policy and debate in the U.S. And it feels like every time I see signs that things might be changing for the better, there is an incident, such as Governor Cuomo’s speech, that makes me question how much progress there really has been at the highest levels.
Before discussing the speech, however, I would like quickly to note that virtually none of what I am about to say actually pertains to whether or not Governor Cuomo’s general policy proposals are correct. This is not about the policies. It is about the interpretations of data used to craft and justify those policies.
Let’s start with the governor’s widely-reported reaction to the state’s teacher evaluation results. His “State of the State” speech included the following passage:
Now 38% of high schools students are college ready. 38%. 98.7% of high school teachers are rated effective. How can that be? How can 38% of the students be ready, but 98% of the teachers effective? 31% of third to eight graders are proficient in English, but 99% of the teachers are rated effective. 35% of third to eighth graders are proficient in math but 98% of the math teachers are rated effective. Who are we kidding, my friends? The problem is clear and the solution is clear. We need real, accurate, fair teacher evaluations.
Though not uncommon among advocates and policymakers, this comparison is, at best, highly misleading. In some respects, frankly, it is a little absurd.
For one thing, note that these student performance rates are based on the state’s new tests and higher standards, and that the proficiency rates were 75-85 percent just a few years ago (a point Aaron Pallas made, correctly, in the New York Times article mentioned above). Under the previous standards, theThe Status Fallacy: New York State Edition | Shanker Institute:
