The 50 million dollar lie
Last year I spent a lot of time making scatter plots of the released New York City teacher data reports to demonstrate how unreliable value-added measurements are. Over a series of six posts which you can read hereI showed that the same teacher can get completely different value-added rankings in two consecutive years, in the same year with two different subjects, and in the same year with the same subject, but in two different grades.Here is an example of such a scatter plot, this one showing the ‘raw’ score (for value added, this is a number between -1 and +1) for the same teachers in two consecutive years. Notice how it looks like someone fired a blue paint ball at the middle of the screen. This is known, mathematically, as a ‘weak correlation.’ If the value-added scores were truly stable from one year to the next, you would see a generally upward sloping line from the bottom left to the top right.