NJ Teacher Evaluation: There's No Escaping The Problems With VAMs
Let's face it: it's been a rough couple of years for the supporters of test-based teacher evaluations.
Back in 2010, the Los Angeles Times published the "ratings" of the city's teachers based on test-scores, using a method called Value-Added Modeling (VAM). Researchers at the National Education Policy Center took at look at the LA Times's ratings and found them to be highly unreliable: when NEPC used a different (and arguably stronger) statistical model, more than half of the teachers fell into different categories of effectiveness.
In the next year, the Times tried to account for the bias and noise in their ratings. Their efforts, however, only made it more clear that VAMs are far too imprecise to use for high-stakes decisions.
In New York City, Gary Rubinstein published a series of brilliant posts on the VAM-based ratings of teachers last year. Not only did Rubinstein find there was little correlation between a teacher's score from one year to the next; there was little correlation for the same teacher's rating in two different subjects in the same year!
The truth is that the "confidence intervals" in the NYC ratings are very large; the ratings present a range of scores for a teacher, making it impossible to order rank teachers by their "effectiveness" with any reasonable
Back in 2010, the Los Angeles Times published the "ratings" of the city's teachers based on test-scores, using a method called Value-Added Modeling (VAM). Researchers at the National Education Policy Center took at look at the LA Times's ratings and found them to be highly unreliable: when NEPC used a different (and arguably stronger) statistical model, more than half of the teachers fell into different categories of effectiveness.
In the next year, the Times tried to account for the bias and noise in their ratings. Their efforts, however, only made it more clear that VAMs are far too imprecise to use for high-stakes decisions.
In New York City, Gary Rubinstein published a series of brilliant posts on the VAM-based ratings of teachers last year. Not only did Rubinstein find there was little correlation between a teacher's score from one year to the next; there was little correlation for the same teacher's rating in two different subjects in the same year!
The truth is that the "confidence intervals" in the NYC ratings are very large; the ratings present a range of scores for a teacher, making it impossible to order rank teachers by their "effectiveness" with any reasonable