A Few Additional Points About The IMPACT Study
The recently released study of IMPACT, the teacher evaluation system in the District of Columbia Public Schools (DCPS), has garnered a great deal of attention over the past couple of months (see our post here).
Much of the commentary from the system’s opponents was predictably (and unfairly) dismissive, but I’d like to quickly discuss the reaction from supporters. Some took the opportunity to make grand proclamations about how “IMPACT is working,” and there was a lot of back and forth about the need to ensure that various states’ evaluations are as “rigorous” as IMPACT (as well as skepticism as to whether this is the case).
The claim that this study shows that “IMPACT is working” is somewhat misleading, and the idea that states should now rush to replicate IMPACT is misguided. It also misses the important points about the study and what we can learn from its results.
First, to reiterate from our first post about the study, the analysis focuses solely on the teachers who are near the minimally effective (ME) and highly effective (HE) cutoff points. It is not an “overall” assessment of the system, as there is no way to know how teachers who are not close to these thresholds (i.e., the vast majority of teachers) are responding to the system. And improvement among all teachers is an extremely important outcome (as is how the system might affect the teacher labor supply).
(Side note: Given the very weak effects in the first year, I would really like to see this study replicated using 2-3 more years of data before drawing any strong conclusions.)
Moreover, this study does not really speak to the “quality” or “rigor” of the IMPACT scores and ratings. In a sense, it actually assumes that issue away insofar as teacher improvement is gauged in terms of IMPACT scores the