Strolling through the PARCC (data)
THIS IS A FIRST CUT AT MY MUSINGS ON THE RELATIONSHIP BETWEEN PARCC AND NJASK SCORES ACROSS NEW JERSEY SCHOOLS. MORE REFINED BRIEF FORTHCOMING. BUT I WANTED TO GET SOMETHING OUT THERE ASAP.
A little background
During the spring of 2015, New Jersey schools implemented their first round of new assessments from the Partnership for Assessment of Readiness for College and Careers (PARCC). This test replaced the New Jersey Assessment of Skills and Knowledge (NJASK). Like NJASK, PARCC includes assessments of English Language Arts and Mathematics for children in grades 3 to 8. PARCC also includes a separate assessment of Algebra 1, administered to some 8th grade Algebra students, and other high school algebra students. PARCC also includes Geometry, Algebra 2, and a high school level language arts assessment.
Adoption of PARCC, and the name of the consortium itself are tied to a nationwide movement to adopt standards, curriculum and assessments which more accurately reflect whether or not students are “ready” for “college.”[1] Research on “college readiness” per se dates back for decades, as does policy interest in finding ways to increase standards in elementary and secondary education in order to reduce the remediation rates in public colleges and universities.[2]
Statistical evaluations of college readiness frequently define “readiness” in terms of successful completion of credit bearing (usually college level mathematics) courses during the first two years of undergraduate education.[3] Thus, when evaluating preparation for college, the goal is to identify measures or indicators that can be collected on students in elementary and secondary education that reasonably predict increased odds of “success” (as defined above). Detailed analyses of student transcript data dating back to the 1980s (with numerous subsequent similar studies in the following decades) point to such factors as highest level of mathematics courses successfully completed.[4]
Others have sought to identify specific tests and specific scores on those tests which might be associated with improved odds of undergraduate “success.”[5] One commonly cited benchmark for “college readiness” drawn from research on predicting success in college level coursework, is a combined SAT score of 1550.[6] Because of the availability of SAT data, others have evaluated their own state assessments, adjusting performance thresholds, to align with this SAT standard.[7] This SAT-linked standard is partial basis for the determination of cut scores in the PARCC exam.
While state officials in New Jersey and elsewhere have hyped the new generation of Common Core aligned assessments of “college readiness” as being more analytic, requiring deeper reasoning, problem solving and critical thinking, PARCC and its cousin SBAC (Smarter Balanced Assessment Consortium) are still rather typical standardized assessments of language arts and math. Cut scores applied to these assessments to determine who is or isn’t college ready are guided by other typical, highly correlated standardized tests previously determined to be predictive of a limited set of college academic outcomes.[8]
When state officials in New Jersey and elsewhere caution local district officials to avoid the desire to compare results of the new test with the old, they ignore that the statistical properties of the new tests are largely built on the design, and results (distributions) of other old tests, and dependent on the relatively high correlations which occur across any diverse sample of children taking nearly all standardized assessments of reading and math.
PARCC does offer some clear advantages over NJASK.
- Accepting the limitations of existing benchmark predictors of college readiness (like the SAT 1550 benchmark), PARCC cut-scores are, at least, based on some attempt to identify a college readiness standard. They are linked to some external criteria. By contrast, in all of the years that NJASK was implemented, department officials never once evaluated the predictive validity of the assessment, or the cut scores applied to that assessment (but for early studies which evaluated the extent to which ASK scores in one grade were predictive of ASK scores in the next). Other states, by contrast have conducted analyses of their pre-Common Core assessments,[9] and current assessments.[10]
- Use of PARCC will permit more fine grained comparisons of performance of New Jersey students, schools and districts to students, schools and districts in other states using PARCC assessments. While NAEP permits cross state comparisons, it does not sample children across all schools, districts and grades, nor are data made available at the district or school level.
- NJASK had a substantial ceiling effect on middle grades math assessments, reducing measured variations among students and schools. That is, in any middle to upper income school in New Jersey, significant numbers of children in grades 4 to 7 would achieve the maximum score of 300 on the NJASK math assessments. That is, children with high math ability achieved the same score, even if their abilities varied significantly. PARCC appears not appear to suffer this shortcoming.
But, while these obvious and relevant advantages exist, PARCC should not be oversold:Strolling through the PARCC (data) | School Finance 101: