Measuring Proficient Teachers Codifies Bad Teaching
Maja Wilson and Alfie Kohn have found themselves in a problematic minority during the accountability era dedicated to standards, high-stakes testing, and the ever-present rubric.
Rubrics, they argue, ultimately fail complex human behaviors such as writing. While rubrics facilitate statistical aspects of measuring human behaviors (such as teaching and learning), by doing so, they also tend to erode the quality of the very behaviors being measured.
As a writing teacher, I can confirm Wilson’s and Kohn’s critiques that student writing conforming to a rubric and thus deemed “proficient” or “excellent” can be and often is quite bad writing. Rubric-based labels such as “proficient” reflect compliance to the rubric, not writing quality.
Wilson, in fact, has demonstrated this by revising a professional and beautiful piece of writing by Sandra Cisneros so that is conforms to a computer-graded system’s criteria for high-quality writing. The result was more than disturbing with the revised work substantially worse but better correlated with what the Educational Testing Service (ETS) has deemed “good.”
While Wilson’s experiment focuses on computer-graded writing, the basis of that is having a generic rubric to determine writing quality, and thus, here we begin to investigate why rubric-driven evaluation of complex human behavior always fails:
- Rubrics reduce the unpredictable to the prescribed.
- To be practical, rubrics often attempt to be generic enough to cover huge categories—such as writing and teaching—and thus failing the reality that poetry writing is significantly distinct from journalism or that teaching second grade is significantly distinct from teaching high school physics.
- When rubrics use terminology that is broad enough to address those varieties, they are useless due to being too vague; when rubrics use terminology that is specific, they are useless because they are unduly prescriptive. If the learning objective is jumping rope, if proficiency is “students jump well,” we have no idea what “well” means, and if proficiency is “students jump 10 times without missing,” that 10 becomes all that Measuring Proficient Teachers Codifies Bad Teaching | the becoming radical: