K-12 Education:
Education Needs to Improve Oversight of Its 21st Century Program
GAO-17-400: Published: Apr 26, 2017. Publicly Released: Apr 26, 2017.
Highlights Page:
(PDF, 1 page)
Accessible Version:
(PDF, 62 pages)
Jacqueline M. Nowicki
nowickij@gao.gov
The Department of Education (Education) awards 21st Century Community Learning Centers (21st Century) grants to states, who in turn, competitively award funds to local organizations, which use them to offer academic enrichment and other activities to improve students' academic and behavioral outcomes. In their most recent grant competitions, states awarded 21st Century funds to nearly 2,400 organizations—including school districts and community-based organizations—based on a variety of criteria, such as the quality of their proposed program designs. Relevant research we reviewed that compared program participants to those of non-participants suggests that the 21st Century program is effective in improving students' behavioral outcomes, such as school-day attendance and reduced disciplinary incidents, more often than their academic outcomes. However, because Education's current 21st Century performance measures primarily focus on students' reading and math scores on state tests, Education lacks useful data about whether the program is achieving its objectives to improve students' behavioral outcomes such as attendance and discipline—the areas where the program most frequently has a positive effect. Education officials have not substantially revised the program's performance measures since 1998, in part because its authorization lapsed from fiscal years 2008 through 2016. Leading practices in performance measurement call for federal agencies to align performance measures with program objectives.
21st Century Community Learning Centers' Objectives and Performance Measures for Student Outcomes
Education's technical assistance to states does not adequately address challenges states face in evaluating their 21st Century programs and sustaining them when program funding ends. About a third of states reported in GAO's 50-state survey that they face challenges in evaluating program performance, such as difficulty designing evaluations that shed light on program effects. Further, the 21st Century program was reauthorized twice, which resulted in significant changes to state requirements for evaluating programs. However, Education has not provided states written guidance on developing and conducting high-quality evaluations since 1999. Federal standards for internal control state that when significant changes occur agencies should periodically review policies for continued relevance and effectiveness in achieving their objectives. Absent written guidance to states on conducting high-quality evaluations, Education may miss opportunities to help states improve their capacity to conduct such evaluations to assure the program is meeting its goals.
Why GAO Did This Study
Education's 21st Century program—funded about $1 billion annually since 2002—supports a broad array of activities outside the school day to improve student outcomes in high-poverty or low-performing K-12 schools. A statement accompanying the Consolidated and Further Continuing Appropriations Act of 2015 included a provision for GAO to review Education programs outside the regular school day.
GAO examined (1) how 21st Century funds are awarded and used, (2) what is known about the effectiveness of these programs, (3) how Education manages and uses program data to inform decision making, and (4) Education's technical assistance for evaluating and sustaining programs. GAO conducted a 50-state survey of program officials, obtaining a 100 percent response rate. GAO also reviewed selected state program evaluations and academic studies on student outcomes, and observed program activities and interviewed officials in four states representing a range of grant size and location.
What GAO Recommends
GAO is making four recommendations, including that Education expand its performance measures for behavioral outcomes and provide written guidance to states on conducting high-quality program evaluations. Education neither agreed nor disagreed with the recommendations, and outlined steps it is taking to address them.
For more information, contact Jacqueline M. Nowicki at (617) 788-0580 or nowickij@gao.gov.
Comments: Education neither agreed nor disagreed with our recommendations; rather, it generally noted that it will keep our recommendations in mind as it continues to implement changes in the program as a result of ESSA. Specifically, the department stated that it is in the process of re-examining whether additional or revised measures should be developed to align more significantly with the program's statutory objectives under ESSA. Education also expressed concern about collecting data on student attendance and disciplinary measures, noting that it will require effective collaboration between states, districts, and other eligible entities. However, we note that about half of states already collect data on at least one of these two measures; and that research has shown that 21st Century programs more often have positive effects on student attendance and reducing disciplinary incidents than on improving students' academic outcomes. Given these effects, we continue to believe that it is critical for Education to measure student attendance and disciplinary incidents to obtain more complete, accurate information on this program's effect on student outcomes.
Recommendation: The Secretary of Education should direct the Office of Academic Improvement to expand its performance measures for the 21st Century program to address all program objectives. Specifically, Education should establish performance measures related to key behavioral, including student attendance and disciplinary incidents, and socio-emotional outcomes.
Agency Affected: Department of Education
Comments: Education neither agreed nor disagreed with this recommendation. Education commented that it plans to build in additional data checks into the data system beyond its current checks on the data's completeness. Specifically, Education anticipates that new technology enhancements in the data system will be designed to flag for inconsistencies in data reporting. For example, the system may send a "flag" that participation data is significantly lower or higher than previously reported participation data. Further, Education indicated that it will consider whether auditors performing audits under the Single Audit Act can be asked and guided to do more checks on the accuracy and reliability of 21st Century program data.
Recommendation: The Secretary of Education should direct the Office of Academic Improvement to conduct federal-level data checks on the accuracy of 21st Century program data submitted by states. Such checks could test for logical relationships between fields. Education should also publicly disclose and address any data limitations it identifies, as appropriate.
Comments: Education neither agreed nor disagreed with this recommendation. Education outlined several steps it has taken to assist states in the past. For example, Education said that it provided six states with individualized technical assistance on strategies related to developing statewide evaluations and measures. Education also noted that, to date, it has conducted two webinars on state evaluations and is in the process of including presentations from those webinars on its online learning portal so that states will have easy access to the information. In addition, Education stated that it included presentations on evaluation strategies in the past during its Summer Institute. Education also said it would consider whether additional guidance for all states was needed. While these are important steps, we do not believe they are sufficient. We continue to believe that Education should prepare written guidance to assist all states in developing and conducting high-quality program evaluations.
Recommendation: The Secretary of Education should direct the Office of Academic Improvement to provide written, non-regulatory guidance to states on developing and conducting high-quality 21st Century state evaluations to help address the difficulties states face in measuring program performance and effectiveness.
Comments: Education neither agreed nor disagreed with this recommendation. Education noted that it hosts meetings twice a year for 21st Century state coordinators where strategies related to program sustainability are shared with states. These meetings covered topics such as reducing the amounts of 21st Century grant awards by a percentage each year. However, these meetings have not focused on topics on program sustainability for several years. We continue to believe that Education should take the lead in sharing information with states to help them address their sustainability challenges by sharing information on state policies and practices that have shown some success.
Recommendation: The Secretary of Education should direct the Office of Academic Improvement to use the information it collects from its monitoring visits and ongoing interactions with states to share effective practices across states for sustaining their 21st Century programs once program funding ends. This information could be shared using existing mechanisms such as Education's meetings with 21st Century state coordinators.
Find Recent Work on Education »
Certain Groups of Students Attend Alternative Schools in Greater Proportions Than They Do Other Schools
GAO-19-373: Published: Jun 13, 2019. Publicly Released: Jul 15, 2019.
Education Should Take Immediate Action to Address Inaccuracies in Federal Restraint and Seclusion Data [Reissued with revisions on July 11, 2019.]
GAO-19-551R: Published: Jun 18, 2019. Publicly Released: Jun 18, 2019.
Child Care and Early Education:
Most States Offer Preschool Programs and Rely on Multiple Funding Sources
GAO-19-375: Published: May 8, 2019. Publicly Released: Jun 7, 2019.
Special Education:
Varied State Criteria May Contribute to Differences in Percentages of Children Served
GAO-19-396SP: Published: Apr 3, 2019. Publicly Released: Apr 10, 2019.
Federal Student Aid:
Actions Needed to Evaluate Pell Grant Pilot for Incarcerated Students
GAO-19-130: Published: Mar 5, 2019. Publicly Released: Apr 4, 2019.
Opportunities to Strengthen Federal Accountability
GAO-19-484T: Published: Apr 3, 2019. Publicly Released: Apr 3, 2019.
Federal Data and Resources on Restraint and Seclusion
GAO-19-418T: Published: Feb 27, 2019. Publicly Released: Feb 27, 2019.
Public School Choice:
Limited Options Available for Many American Indian and Alaska Native Students
GAO-19-226: Published: Jan 24, 2019. Publicly Released: Feb 25, 2019.
Challenges to Assessing Program Performance and Recent Efforts to Address Them
GAO-19-266R: Published: Jan 31, 2019. Publicly Released: Jan 31, 2019.
Explore our Key Issues on Education
Federal Efforts to Prevent Drug Misuse
Improving Federal Programs that Serve Tribes and Their Members - High Risk Issue
Postsecondary Education Access and Affordability
Supports and Services for Transitioning Veterans