An Education Hearing I’d Like To See
At the end of February, the District of Columbia Council’s Education Committee held its annual hearing on the performance of the District’s Public Schools (DCPS). The hearing (full video is available here) lasted over four hours, and included discussion on a variety of topics, but there was, inevitably, a block of time devoted to the discussion of DCPS testing results (and these questions were the focus of the news coverage).
These exchanges between Council members and DCPS Chancellor Kaya Henderson focused particularly on the low-stakes Trial Urban District Assessment (TUDA).* Though it was all very constructive and not even remotely hostile, it’s fair to say that Ms. Henderson was grilled quite a bit (as is often the case at these kinds of hearings). Unfortunately, the arguments from both sides of the dais were fraught with the typical misinterpretations of TUDA, and I could not get past how tragic it is to see legislators question the superintendent of a large urban school district based on a misinterpretation of what the data mean – and to hear that superintendent respond based on the same flawed premises.
But what I really kept thinking — as I have before in similar contexts — was how effective Chancellor Henderson could have been in answering the Council’s questions had she chosen to interpret the data properly (and I still hold out hope that this will become the norm some day). So, let’s take a quick look at a few major arguments that were raised during the hearing, and how they might have been answered.
The improvements in TUDA/NAEP scores are due to demographic changes in the district. Council Member David Catania made some (well-informed) arguments regarding the composition of the TUDA sample and how it might have been driving the 2009-2011 changes.
There is quite a bit of irony here: TUDA changes are being discussed based on the false premise that they represent “growth,” but the substance of the conversation is about demographic changes in the sample, which is the most salient manifestation of the fact that they’re not growth measures, but rather comparisons between two different groups of students.
So, instead of trying to fit a cross-sectional peg into a longitudinal hole, or asserting that painfully simplistic subgroup breakdowns can “prove” that the “growth” was “real” (they cannot), Ms. Henderson should simply have established the correct premises and made her point.
In other words, she should have said that student populations can shift quite bit, even over short periods of time, and raw NAEP/TUDA changes, which do not follow students over time, reflect these changes. That is not an opinion but a Shanker Blog » An Education Hearing I'd Like To See: