Does Grading Bias Apply to Education Reports?
In my efforts to increase student interest in understanding probability, I often do a magic trick where I repeatedly shuffle a deck of cards and have three different students pick a card. Using the counting principle, students see that if we went through this process more than 132,000 times, theoretically I would guess the selected card of all three students only once. Of course, since I know how to trick my students into selecting the three cards I want them to select, I get the answers I want every time.
Last month, the U.S. Chamber of Commerce, the American Enterprise Institute, and the Center for American Progress published a damning report on education titled Leaders and Laggards that I believe uses similar sleight of hand. The report is presented as a fair, research-based, and data-supported assessment to determine how states are doing in innovation of education. What I see under the surface, however, is a strong bias for Teach for America and alternative-certification programs and against experienced teachers, teacher tenure, and teacher unions.
The report’s attention-getting strategy is to assign grades, with states receiving individual letter grades for seven broad educational categories. Each category comprises several indicators that are given numerical values and tabulated to produce a numerical score for the category, which is then converted to a letter grade.
But the data used in Leaders and Laggards’ seems highly selective. For example, one indicator under the “Staffing: Hiring and Evaluation” category is the state’s use of “national programs to recruit nontraditional teachers,” namely Teach for America, The New Teacher Project, and Troops to Teachers. A state’s score for this indicator is influenced heavily by whether or not it uses teachers from the TNTP and TFA. If it does, it gets 100 points (an A); if it doesn’t, it gets 59. (That score is then combined with a rating given for its relative number of Troops