Test-fraud detectives drawing scrutiny
When Caveon Test Security began its statistical detective work in Atlanta, it promised swift, enlightening results.
“I’m good. I’m fast,” Caveon’s then-President John Fremer told the business and civic leaders who hired the private, for-profit consulting firm in March 2010 to look into cheating in Atlanta schools.
Caveon delivered quickly, but it met spectacular failure. Gov. Sonny Perdue rejected the firm’s findings last year and publicly accused it of seeking to “confine and constrain the damage” from rampant cheating. This July, state investigators who conducted a deep, 10-month examination of Atlanta schools said Caveon vastly underrepresented the extent of test-tampering.
Now, Caveon is defending its work in Washington, D.C., too. Authorities are re-investigating the district’s 2009 test scores despite a Caveon examination of eight schools that proclaimed no proof of cheating.
For years, the small Utah firm has been the go-to contractor for test-fraud detection. But Caveon’s work in Atlanta and D.C. suggests the company is gaining a reputation more for clearing schools than catching cheaters, an Atlanta Journal-Constitution investigation has found.
Caveon officials reject the suggestion they would minimize cheating to please clients.
“Our results are very, very solid,” Fremer said recently.
Caveon’s secrecy about its methods has elicited complaints, too.
The firm has staunchly resisted disclosing the inner workings of its test-data analyses, making it impossible for others in the field to validate or critique the results. Because Caveon’s approach is proprietary, public agencies that hire it can’t fully grasp how it reaches conclusions about who is or isn’t cheating.
The questions raised by Caveon’s work underscore the pitfalls that await school districts and state education departments that seek to combat fraud in their standardized test scores.
Most school systems and states perform few checks on the scores, despite a rash of cheating scandals nationwide and the rising importance of test scores as a measure of student and teacher performance.
Agencies that want to do validity reviews find that standard approaches are scarce, the work is highly technical and the federal government, which requires the tests, Test-fraud detectives drawing scrutiny | www.ajc.com:
