How Does PISA Put the World at Risk (Part 3): Creating Illusory Models of Excellence
23 MARCH 2014 144 NO COMMENT
Few numbers in the world command as much power as PISA scores, not even the number of Olympic medals or Nobel Prize winners in the world today. It is utterly shocking and embarrassing to see some otherwise rational and well-educated people (or at least they should be) in powerful positions believe that three test scores show the quality of their education systems, the effectiveness of their teachers, the ability of their students, and the future prosperity of their society.
PISA has become the star-maker in the education universe because of its bold claim to assess “the extent to which students near the end of compulsory education have acquired key knowledge and skills that are essential for full participation in modern societies” [1, p. 15]. Moreover, PISA claims to find educational stars by identifying which education systems better prepare their children for “full participation in modern societies”—as measured by PISA scores. The goal is for educational systems to learn from “the highest-performing and most rapidly improving school systems” [1, p. 15].
The claims are as bold as they are illusory. In a June 2013 article in the Times Education Supplement (TES)magazine, William Stewart questioned if PISA is fundamentally flawed:
what if there are “serious problems” with the Pisa data? What if the statistical techniques used to compile it are “utterly wrong” and based on a “profound conceptual error”? Suppose the whole idea of being able to accurately rank such diverse education systems is “meaningless”, “madness”?What if you learned that Pisa’s comparisons are not based on a common test, but on different students answering different questions? And what if switching these questions around leads to huge variations in the all-important Pisa rankings, with the U.K. finishing anywhere between 14th and 30th and Denmark