A critical look at the annual high school rankings by U.S. News
U.S. News & World Report is famous for its college rankings — but it does others too, including an annual ranking of high schools. Are the schools declared the best really the best? Kevin Welner, the director of the National Education Policy Center at the University of Colorado Boulder, examines this issue in this post and the following post. Welner is an attorney and a professor education policy at the UC Boulder. He is also the co-founder of theSchools of Opportunity project, an effort to identify and highlight high schools that use research-based practices to ensure that all students have rich opportunities to succeed.
A version of this appeared on The Conversation, and I asked U.S. News to comment. The magazine took issue with Welner’s argument in a statement, which you can find at the end of this post.
By Kevin Welner
PART ONE
Rankings hold an odd sway over Americans. Assigning a numerical order somehow transforms a subjective opinion into an objective fact. It seems to matter little how the rankings are calculated or the care with which those calculations are undertaken.
I have recently been taking a closer look at the U.S. News and World Reportranking of “best high schools,” the nation’s most influential high school recognition program. In addition to surprising instances of carelessness, I discovered that the publication had deleted evidence concerning the earlier mistakes, and I found a troubling pattern whereby the rankings, which are supposed to recognize quality schooling, actually reward elite enrollment instead.
Given that this write-up is a bit lengthy, I will divide it into two separate parts. The first points out specific technical problems with the rankings. I briefly summarize the key problem that Carol Burris, Sean Feeney, and John Murphy identified and described here on The Answer Sheet, and I then provide additional evidence of how this problem plays out. I also describe how U.S. News apparently responded by removing evidence.
The second parts explores more fundamental problems with the rankings that would remain in place even if the technical problems were fixed. I will look at the results of the U.S. News ratings process, and I question whether it yields rankings that meaningfully identify “best” high schools.
A Big Problem, Now Concealed
The U.S. News rating process this year altered its past approach for determining whether a school’s students attained proficiency. To illustrate, imagine students A, B, and C entering high school in Grade 9. In the past,U.S. News used what is known as a “cohort” approach to evaluate school quality (e.g., a cohort is the group of entering ninth graders at a school). The question it would ask is whether these three students passed their state-level proficiency exams.
This year, the publication instead took a snapshot of test results in a school in the measured year. So the test scores of A, B, and C may or may not appear, depending on whether they took a proficiency exam that year. Further, the test scores of students outside their cohort might appear, if those non-cohort students took a test that U.S. News decided to include. Moreover, in high-performing districts, one or more of these students might be accelerated and A critical look at the annual high school rankings by U.S. News - The Washington Post: