In Research, What Does A “Significant Effect” Mean?
If you follow education research – or quantitative work in any field – you’ll often hear the term “significant effect.” For example, you will frequently read research papers saying that a given intervention, such as charter school attendance or participation in a tutoring program, had “significant effects,” positive or negative, on achievement outcomes.
This term by itself is usually sufficient to get people who support the policy in question extremely excited, and to compel them to announce boldly that their policy “works.” They’re often overinterpreting the results, but there’s a good reason for this. The problem is that “significant effect” is a statistical term, and it doesn’t always mean wha
