The NAEP proficiency myth
On May 16, I got into a Twitter argument with Campbell Brown of The 74, an education website. She released a video on Slate giving advice to the next president. The video begins: “Without question, to me, the issue is education. Two out of three eighth graders in this country cannot read or do math at grade level.” I study student achievement and was curious. I know of no valid evidence to make the claim that two out of three eighth graders are below grade level in reading and math. No evidence was cited in the video. I asked Brown for the evidentiary basis of the assertion. She cited the National Assessment of Educational Progress (NAEP).
NAEP does not report the percentage of students performing at grade level. NAEP reports the percentage of students reaching a “proficient” level of performance. Here’s the problem. That’s not grade level.
In this post, I hope to convince readers of two things:
1. Proficient on NAEP does not mean grade level performance. It’s significantly above that.
2. Using NAEP’s proficient level as a basis for education policy is a bad idea.
Before going any further, let’s look at some history.
NAEP history
NAEP was launched nearly five decades ago. The first NAEP test was given in science in 1969, followed by a reading test in 1971 and math in 1973. For the first time, Americans were able to track the academic progress of the nation’s students. That set of assessments, which periodically tests students 9, 13, and 17 years old and was last given in 2012, is now known known as the Long Term Trend (LTT) NAEP.
It was joined by another set of NAEP tests in the 1990s. The Main NAEP assesses students by grade level (fourth, eighth, and twelfth) and, unlike the LTT, produces not only national but also state scores. The two tests, LTT and main, continue on parallel tracks today, and they are often confounded by casual NAEP observers. The main NAEP, which was last administered in 2015, is the test relevant to this post and will be the only one discussed hereafter. The NAEP governing board was concerned that the conventional metric for reporting results (scale scores) was meaningless to the public, so achievement standards (also known as performance standards) were introduced. The percentage of students scoring at advanced, proficient, basic, and below basic levels are reported each time the main NAEP is given.
Does NAEP proficient mean grade level?
The National Center for Education Statistics (NCES) states emphatically, “Proficient is not synonymous with grade level performance.” The National Assessment Governing Board has a brochure with information on NAEP, including a section devoted to myths and facts. There, you will find this:
Myth: The NAEP Proficient level is like being on grade level.
Fact: Proficient on NAEP means competency over challenging subject matter. This is not the same thing as being “on grade level,” which refers to performance on local curriculum and standards. NAEP is a general assessment of knowledge and skills in a particular subject.
Equating NAEP proficiency with grade level is bogus. Indeed, the validity of the achievement levels themselves is questionable. They immediately came under fire in reviews by the U.S. The NAEP proficiency myth | Brookings Institution:
Big Education Ape: If NAEP “Proficient” Means “Grade Level Proficiency,” Then America’s Private Schools Are in Trouble | deutsch29 - http://bigeducationape.blogspot.com/2016/05/if-naep-proficient-means-grade-level.html
Big Education Ape: The Arrogant Ignorance of Campbell Brown: Education Journalism in Decline | gadflyonthewallblog - http://bigeducationape.blogspot.com/2016/05/the-arrogant-ignorance-of-campbell.html
Big Education Ape: “Telegenic” Is Not a Sexist Term | deutsch29 - http://bigeducationape.blogspot.com/2016/05/telegenic-is-not-sexist-term-deutsch29.html
Big Education Ape: Campbell Brown’s Bizarre NAEP Response in the Washington Post | deutsch29 - http://bigeducationape.blogspot.com/2016/05/campbell-browns-bizarre-naep-response.html
Big Education Ape: Why a social media fight between Campbell Brown and her critics matters - The Washington Post - http://bigeducationape.blogspot.com/2016/05/why-social-media-fight-between-campbell.html
Big Education Ape: Education Expert Campbell Brown Offers Advice to the Next President | Diane Ravitch's blog - http://bigeducationape.blogspot.com/2016/05/education-expert-campbell-brown-offers.html