Did That Edtech Tool Really Cause That Growth? (Mary Jo Madda)
The quality of research on technology use in schools and classrooms leaves much to be desired. Yet academics and vendors crank out studies monthly. And they are often cited to justify using particular programs. How practitioners can make sense of research studies is an abiding issue. This post offers viewers some cautionary words in looking carefully at findings drawn from studies of software used in schools.
“Mary Jo Madda (@MJMadda) is Senior Editor at EdSurge, as well as a former STEM middle school teacher and administrator. In 2016, Mary Jo was named to the Forbes ’30 Under 30′ list in education.” This post appeared in EdSurge, August 10, 2016.
How do you know whether an edtech product is effective in delivering its intended outcomes? As the number of edtech products has ballooned in the past five years, educators—and parents—seek information to help them make the best decision. Companies, unsurprisingly, are happy to help “prove” their effectiveness by publishing their own studies, sometimes in partnership with third-party research groups, to validate the impact of a product or service.
But oftentimes, that research draws incorrect conclusions or is “complicated and messy,” as Alpha Public Schools’ Personalized Learning Manager Jin-Soo Huh describes it. With a new school year starting, and many kids about to try new tools for the first time, now is a timely moment for educators to look carefully at studies, scrutinizing marketing language and questioning the data for accuracy and causation vs. correlation. “[Educators] need to look beyond the flash of marketing language and bold claims, and dig into the methodology,” Huh says. But it’s also up to companies and startups to question their own commissioned research.
To help educators and companies alike become the best critics, here are a few pieces of advice from administrators and researchers to consider when reviewing efficacy studies—and deciding whether or not the products are worth your time or attention.
For Educators
#1: Look for the “caveat statements,” because they might discredit the study.
According to Erin Mote, co-founder of Brooklyn Lab Charter School in New York Did That Edtech Tool Really Cause That Growth? (Mary Jo Madda) | Larry Cuban on School Reform and Classroom Practice: