Mapping education research and judging influence
Executive Summary
Education research is a vast, multi-disciplinary field. In trying to understand it or make judgments about importance, influence, or where the action is, it can be helpful to see the big picture and not be swayed by where we happen to sit in the field. A map of education research derived from citation data can help us see the big picture. Responses to one of my recent Evidence Speaks postings and its relation to the annual Education Week Edu-Scholar Influence Rankings serve as an example of how a map might help.
Introduction
Education research is a large field involving, among others, psychologists, economists, sociologists, anthropologists, and political scientists, working on diverse problems and issues. When we make judgments about the field, we might tend do so from the vantage point of our own research community. A map of education research can help us appreciate other perspectives, allowing us to moderate and refine our judgments.
Responses to one of my own Evidence Speaks postings and the recent Education WeekRHSU Edu-Scholar Influence Rankings provide examples of how a map showing the big picture can broaden our understanding.[i]
In my earlier posting, based on a bibliometric analysis of the education literature, I reported that the three largest research communities within the field were Instructional Design (called Cognitive Load Theory in that posting), Motivation, and Science Education. Among the most highly cited, influential, and currently active scholars in these communities were John Sweller, Richard Mayer, Albert Bandura, Herbert Marsh, and Michelene Chi. This made perfect sense to me because for the past 35 years, I have been viewing education research from my own perspective grounded in psychology and cognitive science.
Some readers pointed out that many education researchers might never have heard of cognitive load theory or how mental representations affect solving physics problems. This is a valid criticism.
My claims looked particularly dubious when compared to the RHSU Edu-Scholar Public Influence Rankings. Each year Rick Hess ranks the 200 scholars (hereafter, the public influence scholars) who have most influenced the national discourse on education during the previous year. Instructional design and science education do not leap out from the names on that list. Heading the list for 2016 were Diane Ravitch, Linda Darling-Hammond, Howard Gardner, and Gary Orfield. None of the scholars I mentioned even appear on it.
A map of education research can help us understand this divergence in view. It also generates some interesting questions about the Influence Rankings. The rankings tend to over-represent scholars working in some research communities and under-represent others. Interestingly, it under-represents scholars working on instruction and subject-matter learning. Why might this be the case?
Analysis
A document co-citation map represents the semantic and intellectual structure of a research field by depicting co-citation relations among cited documents. The map used in this analysis is based on 2,327 articles published during 2014 in the top quartile (by impact factor) of Web of Science Education and Education Research journals (52 journals). These articles cite 90,000 other articles, i.e., there are 90,000 nodes in the co-citation map. Note that thesecited articles are not necessarily published in education research journals. They are published in journals representing numerous disciplines other than education: psychology, sociology, anthropology, policy studies, economics, linguistics, and neuroscience. The map contains 2,775 citations authored by 195 of the 200 public influence authors. A community detection algorithm applied to the co-citation network identifies over 110 research communities within it.[ii] The largest communities contain around 5–6 percent of the citations.
One can compare the distribution of citations to the work of public influence authors across the research communities to the distribution of all citations across the communities. This reveals communities within which public influence authors are over, under, or comparably represented compared to the field as a whole. One research community contains over 16 percent of the 2,775 public influence author citations.
Note that the citation data is used to map the field, to identify research communities, and to determine which scholars appear in which community. The analysis does not assume that highly cited researchers are necessarily influential outside of the academic literature. The number of times a scholar is cited or co-cited is not important in generating the map. No citation threshold is employed. We want to see where scholars appear on the map, even those cited only once. Identifying scholars who influence public discourse is a different task than identifying highly cited or co-cited scholars within the education research community; although, one might hope, or expect, that many of the public influence scholars are also among the academically prominent.
Table 1 shows the results. Community labels are derived from inspection of the cited references, (including book titles and reports) and the journals in which they were published. The labels are intended to capture the main theme of each community. The research foci of the communities are obvious from the labels with possibly two exceptions. Instructional Design focuses on research that attempts to design instructional interventions sensitive to limitations on human working memory (cognitive load). Politics and Education contains work on the political implications and political commitments of education and education policies, i.e., Common Core or state involvement in early childcare and parenting.Mapping education research and judging influence | Brookings Institution:
