Mistakes made with teacher evaluation scores, CPS admits
Citing a computer coding error, district officials have acknowledged that they miscalculated last year’s REACH performance task scores for one out of every five educators.
Only a tiny fraction of the 4,574 errors were significant enough to result in ratings changes, however. A total of 166 teachers were given corrected ratings earlier this year, and most moved up a category, CPS officials say. Teachers whose ratings dropped won’t be penalized.
The coding error involved matching student rosters with scores on performance tasks, the subject- and grade-specific assessments that were developed by committees of CPS teachers.
Though the problem was not extensive, the number of mistakes – and the possibility that there are still others – has renewed criticisms about the use of such a complex system to evaluate educators and put jobs on the line.
On the front end, the REACH (Recognizing Educators Advancing Chicago’s Students) evaluation system relies on teachers and principals to do classroom observations, administer tests and input data into a computer system. On the back end, REACH relies on a web of CPS departments, custom-designed software and third-party vendors to calculate student improvement on assessments, verify class rosters and tabulate final ratings.
“There needs to be an effort to reduce the complexity so teachers are getting fair, accurate, timely feedback,” says Jen Johnson, the CTU’s special coordinator for teacher evaluations. “This is serious stuff that leads to problems with potential job security.”
CPS Accountability Chief John Barker, whose office wrote the computer code that contained the error, says it’s now been corrected. He repeatedly stressed that the impact of the incorrect scores was minimal.
“Yes, that is something we are absolutely interested in as a district in getting it right. And getting it right means changing some scores,” Barker says. “What is important to know is that those are very, very minor issues. The number of changes in those scores that resulted in a teacher moving out of one category into another category is very, very small.”
It’s unclear, however, whether all of the errors have been corrected. Two teachers who spoke withCatalyst say their newly downgraded ratings no longer take all of their students’ growth on the performance task into account.
In one case, an elementary school teacher said her new score only counted half of the students who took the performance task at the beginning and end of the year. The other students -- who she says were counted the first time around -- are now missing from her data.
Similarly, an arts teacher at another school doesn't understand why her updated summative evaluation report now says that none of her students' performance task scores were counted toward her rating calculation. The "student growth" portion of her evaluation is now based entirely on her school's NWEA metrics – and not the performance task growth.
“Now we’re about to issue the second round of REACH Performance Tasks task for this school year,” she said. “And I ask myself, ‘Why am I even doing this? Who is to say it’s not going to be lost Mistakes made with teacher evaluation scores, CPS admits | Catalyst Chicago:
