CREDO: Trying Hard to Shape Urban Charter Success from Shoddy Research
In December 2014, Margaret Raymond, founding director of the Center for Research on Education Outcomes, (CREDO), made the following remarks about charter performance at the City Club of Cleveland (Ohio):
I’ve studied competitive markets for much of my career. That’s my academic focus for my work. And it’s [education] the only industry/sector where the market mechanism just doesn’t work. I think it’s not helpful to expect parents to be the agents of quality assurance throughout the state.
As one might expect, these words coming from the CREDO director shocked some in attendance at this pro-charter-funded, Ohio event. As blogger and attendee Stephen Dyer reported:
Considering that the pro-market reform Thomas B. Fordham Foundation paid for this study and Raymond works at the Hoover Institution at Stanford — a free market bastion, I was frankly floored, as were most of the folks at my table.
Thus, it is with raised eyebrow that I learned of a new CREDO study released in spring 2015 and that supposedly demonstrates urban charter school superiority. The study also caught the attention of Baruch College researcher/journalist Andrea Gabor, who informed me of her painstaking invesrigation of the “new” CREDo study. As Gabor noted to me in an email:
…After years of researching schools on the ground in charter-heavy districts like New Orleans and New York City, I was skeptical about the [new CREDO] study’s approach and its findings. So, I hired Kaiser Fung, a respected statistician, to help me analyze the study. I then asked CREDO’s director, Macke (Margaret)Raymond, several questions via email; her answers raised more concerns. In the end, Fung and I found several major problems with the study; in the case of its analysis of New Orleans charter schools, the study even violated its own methodology. The problems go well beyond technical quibbles and suggest that any generalizations drawn from the study about the quality of traditional public schools relative to charter schools would be a big mistake.
Gabor wrote a post regarding her findings, New CREDO Study, New Credibility Problems: from New Orleans to Boston, which I feature here, in part:
Last month, Stanford University’s Center for Research on Education Outcomes (CREDO) released a new study on urban charter schools, which purports to show, for the first time, that charters outperform city public schools, at least on standardized-test scores. If true, the study’s findings are a potential bombshell since, thus far, studies have shown no meaningful difference between charter and public schools.The new study, Urban Charter School Study Report on 41 Regions claims to show that “urban charter schools in the aggregate provide significantly higher levels of annual growth in both math and reading compared to their TPS [public school] peers.”The years I’ve spent researching schools on the ground in charter-heavy districts like New Orleans and New York City made me skeptical of such an outcome. But because I am not an expert in research methodology, I decided to hire a respected statistician, Kaiser Fung, author of Numbersense and an adjunct professor of statistics at New York University who has no connection to the education-reform movement (and thus no axe to grind), to help me analyze the CREDO study.After combing through the study and its accompanying technical document, and after exchanging a series of emails with Macke Raymond, Director of CREDO, we found significant problems with the CREDO study. The problems go well beyond technical quibbles and suggest that any generalizations drawn from the study about the quality of traditional public schools relative to charterCREDO: Trying Hard to Shape Urban Charter Success from Shoddy Research | deutsch29: