Beware Of Anecdotes In The Value-Added Debate
A recent New York Times “teacher diary” presents the compelling account of a New York City teacher whose value-added rating was 6th percentile in 2009 – one of the lowest scores in the city – and 96th percentile the following year, one of the highest. Similar articles – for example, about teachers with errors in their rosters or scores that conflict with their colleagues’/principals’ opinions – have been published since the release of the city’s teacher data reports (also see here). These accounts provoke a lot of outrage and disbelief, and that makes sense – they can sound absurd.
Stories like these can be useful as illustrations of larger trends and issues – in this case, of the unfairness of publishing the NYC scores, most of which are based on samples that are too small to provide meaningful information. But, in the debate over using these estimates in actual policy, we need to be careful not to focus too much on anecdotes. For every one NYC teacher whose value-added rank changed over 90 points between 2009
Stories like these can be useful as illustrations of larger trends and issues – in this case, of the unfairness of publishing the NYC scores, most of which are based on samples that are too small to provide meaningful information. But, in the debate over using these estimates in actual policy, we need to be careful not to focus too much on anecdotes. For every one NYC teacher whose value-added rank changed over 90 points between 2009
