Alice in PARCCland: Does ‘validity study’ really prove the Common Core test is valid?
The Massachusetts state Education Department last year commissioned Mathematica Policy Research to do a validity study comparing the Common Core test known as PARCC, created by a federally funded consortium of states, and the MCAS, or the Massachusetts Comprehensive Assessment System, which has been in use for years and is also aligned to the Common Core standards. At the time, state officials were considering whether to adopt the PARCC assessments or to keep using the MCAS, and allowed districts to choose which test to give to students for “accountability” purposes. The study was released in the fall, and shortly after that state education authorities voted to create a hybrid test that would incorporate elements of PARCC into the MCAS.
The study is titled “Predictive Validity of MCAS and PARCC: Comparing 10th Grade MCAS Tests to PARCC Integrated Math II, Algebra II, and 10th Grade English Language Arts Tests,” and the authors discussed their work in an article in the newest edition of the Education Nextjournal. The article says:
Ultimately, we found that the PARCC and MCAS 10th-grade exams do equally well at predicting students’ college success, as measured by first-year grades and by the probability that a student needs remediation after entering college. Scores on both tests, in both math and English language arts (ELA), are positively correlated with students’ college outcomes, and the differences between the predictive validity of PARCC and MCAS scores are modest. However, we found one important difference between the two exams: PARCC’s cutoff scores for college-and career-readiness in math are set at a higher level than the MCAS proficiency cutoff and are better aligned with what it takes to earn “B” grades in college math. That is, while more students fail to meet the PARCC cutoff, those who do meet PARCC’s college-readiness standard have better college grades than students who meet the MCAS proficiency standard.
Here’s a post challenging the results of the study, by William J. Mathis, managing director of the National Education Policy Center at the University of Colorado in Boulder, a member of the Vermont Board of Education and a former Vermont superintendent. The views expressed here are his own and do not reflect the views of any group with which he is associated. Following the post is a response from Mathematica.
By William J. Mathis
“When I use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just what I choose it to mean — neither more nor less.” — “Through the Looking Glass,” by Lewis Carroll
The PARCC tests have been criticized for being administered in high-stakes circumstancesbefore they were validated. PARCC’s rejoinder is they had content validity, meaning that the test was built according to their committee reviewed specifications. But what is missing isAlice in PARCCland: Does ‘validity study’ really prove the Common Core test is valid? - The Washington Post:
