VAMboozled! a blog about the issues surrounding teacher evaluation:
VAMboozled! is a blog about the issues surrounding teacher evaluation, teacher accountability, and value-added models (VAMs) in America’s public schools. VAMboozled! is also about the related issues surrounding America’s educational reform and accountability initiatives, and the federal and state policies being advanced, incentivized, adopted, and implemented across the nation. While other education blogs might focus on more general education topics, this blog is focused only on these issues, as current and controversial as they continuously are. The goal is to make more comprehensible and more accessible research-based information about these issues, and to better reach and inform teachers, administrators, policymakers, parents, students, and members of the general public, all of whom are stakeholders and who might ultimately be involved.
To stay current on these issues, to engage, to participate, to add to the professional dialogue, to offer solutions, and to add to our collective understandings about these topics, please subscribe.
Assumptions: Assumptions appeal to people’s emotions and, accordingly, are used to sway peoples’ thinking in desired ways. Assumptions are often made about intangible ideas or tangible products, but often without proof or data to support the legitimacy of the assumptions being made. Correspondingly, most governmental and private groups, in efforts to promote and/or protect sets of private and/or public interests, attempt to methodically sway or change public attitudes and behaviors oftentimes using assumptions. Such assumptions are often expressed via enthusiastic statements and bold claims, further engraining the assumptions and transforming them into accepted realities, while not always true or supported by evidence.
Bias: Bias is a huge threat to validity, as biasing factors (e.g., student risk factors) both distort the measurement of a variable and distort their interpretations, either increasing or decreasing, in this case, VAM-based estimates. This occurs even though the biasing factors are unrelated to what the test-based indicators (i.e., VAMs) are meant to represent (e.g., teacher effectiveness). Accordingly, if VAM estimates are highly correlated to biasing factors, then it becomes impossible to make valid interpretations about the causes of student achievement gains or losses as intended. Bias is most difficult to statistically “control for” because students are rarely if ever randomly assigned to classrooms (and teachers are rarely randomly assigned to classrooms as well).
Confidence Intervals: Confidence intervals are the statistical areas or ranges within which one can be confident that, in this case, a teacher’s true value-added estimate has been effectively and accurately captured. To contextualize, the typical confidence intervals used in statistics give or take about 5 percentage points from the reported estimate (i.e., given standard 95% confidence intervals). However, in New York, for example, the confidence intervals around 18,000 teachers’ value-added ratings spanned 35 percentile points in mathematics and 53 percentile points in reading/language arts. This meant that a mathematics teacher who ranked at the 50th percentile could have actually had a true score between the 33rd and 67th percentile ranks (rounding inwards). In reading/language arts, a teacher who ranked at the 50th percentile could have had an observed score any and everywhere between the 24th and 76th percentile rank (rounding inwards). While 95% confidence intervals are typically used, with VAMs these standard error ranges are typically much, much larger, to account for the relatively higher potential for measurement error.
Education Production Function: In the education production function, it is assumed that using VAMs will induce teachers to work harder, and using VAMs will incite teachers who do not work hard enough to improve out of fear of being penalized or terminated. Such
