Challenges to Assessing Program Performance and Recent Efforts to Address Them
The Department of Education provides billions of dollars in grants for programs to improve K-12 outcomes.
Do they work?
We found that Education faces challenges determining whether these programs work. Specifically, problems persist with:
Oversight. For example, some program grant files were missing descriptions of what grantees achieved.
Data quality. Education lacks assurance that some state-submitted data are accurate.
Capacity. A lack of qualified staff has impacted monitoring, and Education hasn’t updated hiring plans in years.
Study design. Measuring long-term outcomes can be difficult.
Oversight and monitoring. Weaknesses in Education's internal controls have hindered its oversight and monitoring of grantees and its assessments of K-12 program performance. In April 2017, GAO reported that Education's oversight of discretionary grants monitoring was limited. Some offices, including the Office of Elementary and Secondary Education, did not consistently document required monitoring activities in official grant files. GAO recommended that Education establish and implement written supervisory review procedures for official grant files. In response, Education officials said they are developing a standard operating procedure for maintaining official grant records, which they plan to issue in early 2019.
Data quality. Persistent quality issues with K-12 data that grantees submit to Education have limited Education's ability to use those data to assess performance. In April 2017, GAO reported that Education lacks reasonable assurance that data submitted by grantees for its 21st Century Community Learning Centers grant program are accurate, and that these data may not be useful for decision making and reporting. GAO recommended that Education check the accuracy of federal-level data submitted by grantees. In response, Education officials said they modified and improved the agency's data system to perform these types of checks and reduce errors.
Capacity. Education's ability to oversee and monitor grantees, collect and report quality data, and use performance assessment information in decision making is directly related to its capacity and organizational resources. According to Education officials, capacity has been and remains a challenge to assessing K-12 program performance. In its 2016 report on the Rural Education Achievement Program, Education's Office of Inspector General (OIG) concluded that Education's inadequate monitoring of program grantees was partly due to its limited human capital capacity. In response, Education requested additional staff and implemented a risk-based approach for selecting grantees to monitor, according to Education officials cited in the OIG report.
Methodological limitations. Education has faced methodological limitations assessing program performance, including difficulties assessing the benefits of flexible grant programs, isolating program impact, and measuring long-term outcomes. Education officials told GAO that these types of methodological challenges are difficult to address, although Education has taken steps to mitigate them. For example, GAO reported in May 2014 on the difficulty Education faces in evaluating the effectiveness of the Promise Neighborhoods program in part because the program provides certain flexibilities to grantees. Partly in response to GAO's recommendation that Education develop a plan to conduct a national evaluation of the program, Education awarded a contract in fiscal year 2018 to develop options for evaluating the program's effect on student outcomes. In November 2018, Education officials stated that they intend to award a new contract in late fiscal year 2019 to evaluate the effectiveness of the program nationwide.
Why GAO Did This Study
Education plays a key role in supporting educational opportunities for K-12 students, including awarding grants and overseeing compliance with federal education laws. However, questions have been raised about how Education assesses program performance. Both GAO and Education's OIG have reported on various management and oversight issues related to Educations program performance. GAO was asked to review Education's K-12 program performance assessment activities and related barriers. This report describes challenges Education faces in assessing the performance of its K-12 programs, as well as steps it has taken to address them.
To identify challenges to program performance assessment and learn about how Education addressed these challenges, GAO examined documents from Education, reviewed prior GAO work on government-wide performance assessment activities and Education's K-12 programs, and interviewed Education officials. To identify specific examples of K-12 programs that illustrate each of the challenges, GAO reviewed prior related GAO reports and relevant Education OIG reports and considered the views of Education officials. Although these examples are not generalizable, they reflect a variety of K-12 programs and show a range of Education's actions in response to the challenges. GAO also reviewed relevant federal laws and regulations.
For more information, contact Jacqueline M. Nowicki at (617) 788-0580 or nowickij@gao.gov.
U.S. GAO - K-12 Education: Challenges to Assessing Program Performance and Recent Efforts to Address Them
