Research Study: VAM-Based Bias
Researchers from Indiana and Michigan State University, in a study released in the fall of 2012 but that recently came through my email again (thanks to Diane Ravitch), deserves a special post here as it relates to not only VAMs but also the extent to which all VAM models yield biased results.
In this study (albeit still not peer reviewed, so please interpret accordingly), researchers “investigate whether commonly used value-added estimation strategies produce accurate estimates of teacher effects under a variety of…student grouping and teacher assignment scenarios.” Researchers find that no VAM “accurately captures true teacher effects in all scenarios, and the potential for misclassifying teachers as high- or low-performing can be substantial [emphasis added].”
While these researchers suggest different statistical controls to yield less biased results (i.e., a dynamic ordinary least square [DOLS] estimator), the bottom line is that VAMs cannot “effectively isolate the ‘true’ contribution of teachers and schools to achievement growth” over time. Whether this will ever be possible given mainly the extraneous variables that are outside of the control of teachers and schools, but that continue to confound and complicate VAM-based estimates deeming them (still) unreliable and invalid, particularly for the high-stakes decision-making purposes for which VAMs are increasingly being tasked, is highly suspect.
The only way we might reach truer/more valid and less biased results is to randomly assign students and teachers to classrooms, which as evidenced in a recent article one of my doctoral students and I recently had published in the highly esteemed American
