The Proportionality Principle In Teacher Evaluations
Our guest author today is Cory Koedel, Assistant Professor of Economics at the University of Missouri.
In a 2012 post on this blog, Dr. Di Carlo reviewed an article that I coauthored with colleagues Mark Ehlert, Eric Parsons and Michael Podgursky. The initial article (full version here, or for a shorter, less-technical version, see here) argues for the policy value of growth models that are designed to force comparisons to be between schools and teachers in observationally-similar circumstances.
The discussion is couched within the context of achieving three key policy objectives that we associate with the adoption of more-rigorous educational evaluation systems: (1) improving system-wide instruction by providing useful performance signals to schools and teachers; (2) eliciting optimal effort from school personnel; and (3) ensuring that current labor-market inequities between advantaged and disadvantaged schools are not exacerbated by the introduction of the new systems.
We argue that a model that forces comparisons to be between equally-circumstanced schools and teachers – which we describe as a “proportional” model – is best-suited to achieve these policy objectives. The conceptual appeal of the proportional approach is that it fully levels the playing field between high- and low-poverty schools. In contrast, some other growth models have been shown to produce estimates that are consistently associated with the characteristics of students being served (e.g., Student Growth Percentiles).
However, while the proportional approach appeals to an intuitive sense of “fairness” for some, others rightly point out that it might be unfair to enforce proportionality through the structure of a growth model, and that a proportional evaluation system could generate inaccurate teacher ratings. Based on empirical evidence from a number of studies (e.g., see here and here), the concern is that schools serving advantaged students, and teachers at those schools, are more effective than those serving disadvantaged students, a fact that would not be captured by the output from a proportional model. As noted in Dr. Di Carlo’s original blog post, this is a plausible hypothesis for a variety of reasons related to the differences between high- and low-poverty schools in terms of resources, labor-market access, etc. (although it is difficult to confirm this hypothesis based on available data and models).
My co-author Jiaxi Li and I tackle this concern head-on in a new paper. We empirically evaluate the relative merits of Shanker Blog » The Proportionality Principle In Teacher Evaluations: