Environmental Protection: Additional Action Needed to Improve EPA Data on Informal Enforcement and Compliance Assistance Activities

Enforcing environmental laws and regulations, including those governing water, air, and hazardous waste, is a central part of EPA's mission. In partnership with states, EPA oversees compliance with these requirements for about 800,000 regulated entities, such as refineries and sewage treatment plants. OECA carries out much of EPA's compliance and enforcement responsibilities through the agency's 10 regional offices. OECA has a range of compliance assistance, compliance monitoring, and enforcement tools available to elicit compliance with laws and regulations from regulated entities. These tools include conducting on-site inspection, training staff and providing technical assistance, developing cases, and issuing warning letters.

GAO was asked to review EPA's enforcement efforts. This report examines, among other objectives, the types of information EPA collects on its compliance assistance, compliance monitoring, and enforcement actions. GAO analyzed written responses to its questions from all 10 regional offices, reviewed agency documents and databases, and interviewed EPA officials in headquarters and regional offices.

The Environmental Protection Agency (EPA) collects a range of information on compliance and enforcement such as data on inspections, violations, and enforcement actions. The agency uses these data to manage its efforts and assess progress in meeting the agency's strategic objectives. In an August 2018 memorandum, EPA's Office of Enforcement and Compliance Assurance (OECA) reported a key strategic change to increase compliance assistance activities (e.g., training) and informal enforcement actions (e.g., warning letters). However, the agency does not consistently collect or maintain data on either type of action (see figure). Specifically, OECA has not directed regional offices to collect or report data on compliance assistance activities since 2012 and, consequently, does not have guidance instructing regional offices to collect such data and specifying which mechanism offices should use to maintain these data. Also, the agency did not provide guidance to those offices defining informal enforcement actions or how to maintain data on them until September 30, 2019, but the guidance does not specify how to collect data on such actions. By clearly documenting in guidance how the offices should use the definition to collect data on such actions, EPA could more consistently collect these data.

As the figure shows, OECA does not require regional offices to collect data on compliance assistance or complete data on informal enforcement actions. Having complete information about its compliance assistance activities and informal enforcement is essential because EPA has elevated the role of such activities in its overall enforcement efforts. However, because EPA is not consistently collecting these data, the agency cannot be sure it is achieving its strategic objectives. EPA would have better assurance it has the information it needs by clearly documenting in guidance to the regional offices that they should:

collect data on compliance assistance activities and informal enforcement actions and

specify which mechanism to use to maintain compliance assistance data.

By doing so, EPA would have better assurance that the regional offices consistently collect and maintain these data in order to track progress toward the agency's strategic objective of increasing the use of such activities and actions.

GAO is making three recommendations to EPA, including that it should clearly document in guidance to its regional offices that they should collect data on compliance assistance activities and informal enforcement actions and specify which mechanism to use to maintain compliance assistance data. EPA agreed with GAO's recommendations and stated that the agency has either begun to or plans to implement them.

According to the Environmental Protection Agency (EPA), entities that  unlawfully release or expose communities or individuals to toxic  emissions and chemicals can damage the environment and cause  chronic illnesses and even deaths. A central part of EPA’s mission to  protect human health and the environment is enforcing environmental  laws and regulations, including those related to water, air, and hazardous  waste. In many instances, EPA has delegated authority to, or authorized,  states to implement and enforce federal environmental requirements.  EPA, in partnership with these states, oversees about 800,000 entities’  compliance with these requirements. According to EPA, these regulated  entities include petroleum refineries, factories, small businesses, sewage  treatment plants, and local governments. EPA decides how to promote  compliance with regulations, deter noncompliance by taking enforcement  actions, tailor efforts to encourage voluntary compliance, and inform  regulated entities of regulatory requirements. In its most recent strategic  plan, issued in 2018, EPA described a change in its approach to the  agency’s overall compliance and enforcement program that included,  among other things, a goal of increasing the agency’s use of compliance  assistance activitiesfor example, training and technical assistanceto  help regulated entities comply with laws and regulations.

EPA’s Office of Enforcement and Compliance Assurance (OECA) carries  out the agency’s environmental compliance and enforcement  responsibilities. OECA’s headquarters provides overall direction to the agency’s regional offices and authorized states on compliance monitoring  and enforcement policies and sometimes takes enforcement action.  OECA carries out much of its compliance monitoring, compliance  assistance, and enforcement responsibilities through its 10 regional  offices around the country. OECA staff in headquarters and in regional  offices are responsible for monitoring compliance by inspecting regulated  entities and tracking violations; taking enforcement actions, such as  sending notices of violation and developing administrative, civil, and  criminal cases against violators; providing compliance assistance, such  as training and technical assistance to regulated entities; and overseeing  the enforcement programs of authorized states, where applicable. OECA  collects data to monitor compliance and track enforcement actions and to  manage and assess the performance of its overall compliance and  enforcement program.

EPA recently made some changes to its compliance and enforcement  approach, including refocusing its efforts from enforcement to compliance  and working more closely with states in planning a range of compliance  and enforcement efforts to increase compliance among regulated  entities. Specifically, in EPA’s strategic plan for fiscal years 2018 through  2022, the agency stated that its priorities and goals include ensuring  compliance with the law and developing a more collaborative partnership  with authorized states. To meet the goals in EPA’s strategic plan, every  few years OECA has developed national initiatives to focus its  compliance and enforcement resources, including staff and funding, on its  strategic objectives.

In 2018, to help EPA achieve the strategic plan’s goal of ensuring  compliance with the law, OECA refocused how it describes its national  initiatives from national enforcement initiatives to national compliance  initiatives. In OECA’s 2019 policy memorandum announcing the national  compliance initiatives for fiscal years 2020 through 2023, OECA stated  that this change in focus conveys the overarching goal of increased  compliance and the use of not only formal enforcement actions (e.g., civil  actions), but also other compliance tools that include compliance  assistance, self-audits by regulated entities, and informal enforcement  actions (e.g., warning letters). OECA also stated in this 2019 policy  memorandum that while compliance assistance would be a feature in  increasing compliance, formal enforcement would remain important for  addressing serious noncompliance and creating general deterrence.  Furthermore, to address the goal in EPA’s strategic plan of developing a  more collaborative partnership between the agency and authorized  states, OECA in 2019 issued final guidance to regional offices,  superseding interim guidance it issued in 2018. This guidance included  expectations for regional offices to, among other things, enhance  communication and jointly plan the enforcement actions EPA or a state  will initiate.

We last reported on EPA’s overall enforcement efforts in July 2007 when  we reported on the EPA–state partnership to enforce environmental  laws. At that time, we found that EPA had improved its oversight of state  enforcement programs by implementing a consistent approach for  overseeing states. However, we also found that EPA could take additional  steps to ensure more consistent state performance and oversight of state  enforcement programs. We recommended that the agency take a number  of actions regarding its review of state programs. EPA generally agreed  with these recommendations and implemented them. We also  recommended that EPA conduct a comprehensive review of its regional  enforcement programs to address inconsistencies among the regional  offices in enforcement actions. EPA generally agreed with this  recommendation, but the agency has not yet taken actions to implement  it.

You asked us to review EPA’s enforcement efforts, and this report is the  first in response to your request. Our objectives were to examine (1) the  types of information EPA collects on its compliance monitoring,  compliance assistance, and enforcement actions; and (2) the ways in  which EPA communicates, to the public and Congress, the results of its  compliance activities and enforcement actions.

To address both objectives, we reviewed EPA guidance and policy  memoranda, past GAO and EPA Office of Inspector General reports,  EPA’s enforcement and compliance annual results reports, annual  performance reports, and other agency documents. We focused the scope of this report on the four primary laws OECA is responsible for  enforcing: Clean Water Act, Clean Air Act, Safe Drinking Water Act, and  Resource Conservation and Recovery Act.

To examine the types of information EPA collects on its compliance  activities and enforcement actions, we collected information about the  databases and websites EPA uses to maintain and present its  enforcement data. Specifically, we reviewed EPA documents and  interviewed EPA officials about the purpose, uses, and limitations of the  data in the Integrated Compliance Information System (ICIS) database,  which contains compliance and enforcement data, and the Enforcement  and Compliance History Online (ECHO) website, a public access website  that stores and integrates data from multiple EPA databases. We also  interviewed relevant EPA officials about statute- and regional office- specific databases. However, we did not make a determination about the  reliability, completeness, and accuracy of specific data in these  databases because we present an analysis of the types of data that EPA  collects rather than an analysis of the compliance or enforcement data  themselves. Although we did not make a determination about the  reliability of specific data in ICIS or on the ECHO website, EPA has  publicly reported on limitations to the data. Specifically, EPA’s ECHO  website includes a section that describes known data problems. For  example, the website states that EPA and the state agencies that report  data to EPA have identified some data issues that may impact the  completeness, timeliness, or accuracy of the data shown in ECHO.

We interviewed OECA headquarters staff about the type of data the  agency collects and the primary ways the agency uses these data. We  sent a list of questions to all 10 EPA regional offices about their collection  and use of compliance and enforcement data as well as limitations in the  use of these data, and we analyzed their written responses. We also  conducted telephone or in-person interviews with officials in all 10  regional offices to further explore their collection and use of compliance  and enforcement data, among other things, and to follow up on their  written responses to our questions.

We interviewed representatives of five organizations representing some  of the regulated entities to obtain their perspectives on a range of issues  related to enforcement. We selected these organizations based on  EPA’s public information about the types of entities it regulates, and used  the information to help develop topics for regional interviews. Because  this was a nonprobability sample of such organizations, the  representatives’ views are not generalizable to all such organizations but  provide examples of the views of regulated entities. Finally, we compared  EPA’s efforts for collecting data against federal standards for internal  control related to the design of control activities and the use of quality  information and against a selection of our leading practices for regulatory  approaches.

To examine the ways in which EPA communicates to the public and  Congress, the results of its compliance activities and enforcement  actions, we reviewed OECA’s annual reports and presentations on data  describing annual results. We compared EPA’s efforts in reporting data  against a selected leading practice for transparently reporting government  data and EPA guidance on communicating environmental data.

We conducted this performance audit from October 2018 to January 2020  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

OECA has a range of compliance monitoring, compliance assistance, and  enforcement tools available to elicit compliance with laws and regulations  from regulated entities, as shown in table 1.

Enforcement actions can result in, among other things, the imposition of  penalties, requirements to remedy the violation of law or regulation, or  both. OECA has developed policies and guidance for EPA staff that  describe the agency’s recommended responses to noncompliance based  on a number of factors and the escalation of enforcement responses to  continuing noncompliance. EPA guidance on informal and formal  enforcement actions provides an example related to the Resource  Conservation and Recovery Act. In that example, if a regulated entity  does not return to compliance or notify the state or EPA that it cannot  return to compliance within a certain number of days after an informal  enforcement action, the state or EPA may take a formal enforcement  action. Generally, according to this same 2010 EPA guidance, informal  enforcement actions address small or isolated problems, and formal  enforcement actions can address bigger problems.

OECA stores and manages a range of compliance monitoring and  enforcement data in ICIS. For example, ICIS includes descriptive  information about regulated entities, violations, and the outcome of  enforcement actions. ECHO, the public access website that integrates  data from multiple agency databases, has an internal component for staff  and other federal agencies and publicly available components. Staff in  EPA’s 10 regional offices, OECA headquarters staff, and states input data  into ICIS, which feeds data into ECHO. Regional office staff and OECA  headquarters staff also use statute-specific databases to maintain data on  compliance with a particular law or office-specific databases built to  maintain data, according to the preferences of a particular regional or  headquarters office.

EPA Collects a Range of Compliance Monitoring and Enforcement Data, but Does Not Maintain Data on Informal Enforcement Actions and Compliance Assistance

EPA requires regional offices to collect and enter a range of information  on its compliance monitoring and enforcement activities—such as permit,  inspection, and violations data—into the agency’s national databases.  The agency uses these data to manage its oversight efforts and assess  how well the efforts are meeting the agency’s strategic objectives. In  addition, EPA is piloting an effort to collect data on coordination with  states. However, EPA regional offices do not consistently collect or  maintain data on informal enforcement actions. In addition, EPA does not  require regional offices to collect and maintain data on their compliance  assistance activities; therefore, it has no requirements for regional offices  to enter data into the agency’s national databases.

EPA Requires Regional Offices to Collect Some Data to Manage and Assess Its Oversight Efforts

EPA requires regional offices to collect information from various data  sources and enter it into national databases to monitor regulated entities’  compliance with environmental laws and track the agency’s enforcement  actions. The information generally includes permit data on limits on  emissions or for discharge of pollutants into waters, inspection or other  evaluation data, violations data (e.g., failure to take or submit results for  drinking water samples); informal enforcement actions, and formal  enforcement actions, as shown in figure 1.

OECA uses the data in its databases to manage the overall enforcement  and compliance program and assess how well its efforts are meeting the  objectives outlined in the agency’s strategic plan, according to EPA  officials. For example, officials in one regional office told us that regional  managers typically review ICIS data (for example, the number of  inspections conducted) to monitor their progress toward meeting strategic  objectives at the regional level. These regional officials said that staff in  their office conduct monthly reviews of ICIS data to understand how their  current efforts on certain indicators compare to prior years.

OECA headquarters officials told us that the agency had begun to pilot a  mechanism to collect data that can help measure agency progress in  coordinating with states, one of the agency’s strategic objectives.  Specifically, OECA officials told us that in 2018 the agency began a pilot  effort to track instances in which regional office staff provide assistance  with state enforcement actions, also characterized as “state assists.”  According to agency guidance issued in June 2019, a state assist is  defined as any instance in which the state could not or would not take the  action without OECA’s help or any instance in which a state explicitly  requests that OECA take over a case after OECA has identified a  violation. During the pilot effort, state assists are documented as such  when a regional office has expended substantial resources to identify a  violation, develop the injunctive relief, or help the state take an action to  obtain a remedy for the violation.

According to OECA guidance, the pilot effort, which OECA officials expect  to continue through 2021, will help the agency better track its efforts in  this area. As of June 2019, according to our analysis of written  responses, officials in eight of the 10 regional offices described having  documented a state assist as defined by OECA. For example, officials in  one regional office stated in their written response to our questions that  one specific case against a company located in three different states  would have been handled by the regional office. Instead, the regional  office agreed to let two of the states take the lead for the cases in those  states, and the regional office handled the case in the third state and  documented this as two state assists.

EPA’s Regional Offices Do Not Consistently Collect or Maintain Data on Informal Enforcement Actions

OECA collects data on some informal enforcement actions, such as the  number of warning letters sent to regulated entities, but EPA regions do  not always collect data about these actions, according to EPA  headquarters officials. As a result, the data do not tell the full story of  OECA’s enforcement efforts, according to OECA’s Assistant  Administrator in testimony during a February 26, 2019, congressional  hearing.

Furthermore, OECA headquarters officials we interviewed said that data  on EPA and state informal enforcement actions are incomplete in EPA’s  ECHO website in part because EPA policy and related guidance for each  of the various programs defines informal enforcement differently and  these definitions can differ from the definitions in ECHO. In a 2010  document, EPA explained how the various agency policy guidance and  ECHO define formal and informal enforcement actions differently. For  example, the document states that policy guidance for the Clean Air Act  defines notices of violation as formal enforcement actions, but that policy  guidance for the Clean Water Act and the Resource Conservation and  Recovery Act defines notices of violation as informal enforcement  actions. Similarly, this same 2010 document states that administrative  penalty orders of field citations are considered informal enforcement  actions in the policy guidance for the Clean Water Act, but formal  enforcement actions in the policy guidance for the Clean Air Act and  Resource Conservation and Recovery Act. In addition, the document  states that ECHO characterizes notices of violations under the Clean Air  Act as informal enforcement actions even though the policy guidance  defines them as formal enforcement actions.

OECA headquarters officials highlighted two issues that affect the  agency’s ability to consistently maintain data on informal enforcement  actions: (1) using different definitions of informal enforcement actions  across programs and (2) maintaining data on such actions inconsistently.  OECA headquarters officials said that they were addressing the first issue  of not having one clear definition of informal enforcement actions that  applies across all of the air, water, and hazardous waste programs. In  September 2019, OECA headquarters officials said EPA was finalizing a  single definition of informal enforcement actions for the purpose of  collecting more consistent information. In January 2020, EPA provided us  with a September 30, 2019, memorandum that defines enforcement  response tools, including a definition of informal enforcement action  across all programs.

Regarding maintaining data inconsistently, while most of the regional  offices collect data on some informal enforcement actions, they use  different mechanisms to maintain these data. According to our analysis of  written responses, officials in nine of the 10 regional offices stated that  their offices collect data on some informal enforcement actions such as  warning letters, notices of noncompliance, notices of violation, and  notices of determination. However, the officials described using different  mechanisms for maintaining the data they collect on informal  enforcement. For example, officials in five of the nine regional offices that  collect data on some informal enforcement actions stated that they  maintain the data in ICIS. As we described, ICIS data feeds into ECHO,  which has components available to the public. In three of the nine  regional offices that collect data on some informal enforcement actions,  staff collect data on such actions in a database other than ICIS, such as a  statute- or office-specific database, according to our analysis of written  responses. Finally, one of the nine regional offices that collect data on  some informal enforcement actions maintains those data in paper  records, according to an official in that office.

In our October 2017 report on key considerations for agency enforcement  decisions, we reported that transparency and availability of data are  important to promoting compliance and achieving regulatory objectives.  As described earlier, EPA changed the focus of its national priorities from  enforcement to compliance and increased its use of informal enforcement  actions to achieve its regulatory objectives. Having complete information  about informal enforcement actions is essential because EPA has  elevated the role of such activities in its overall enforcement efforts. EPA  often works informally with regulated entities to help them comply with  environmental laws and regulations, according to its 2018 EPA  Enforcement Annual Results report. However, the agency does not  have complete information on those actions for evaluating its compliance  monitoring and enforcement performance. Moreover, more complete and  consistent information about OECA’s informal enforcement actions would  provide a fuller picture of EPA’s overall enforcement efforts. This, in turn,  would better enable EPA and OECA to assess whether they are  achieving the agency’s regulatory objectives and improve the  transparency of OECA’s informal enforcement actions for Congress and  the public.

Guidance can help agencies communicate expectations and ensure  consistency with a standard. While EPA has issued guidance on how  various agency policies and ECHO define formal and informal  enforcement actions, the agency has not provided guidance to regional  offices on how they should collect or maintain data on informal  enforcement actions. According to federal standards for internal control,  management should design control activities to achieve objectives and  respond to risks, such as by clearly documenting internal control in  management directives, administrative policies, or operating manuals.  On September 30, 2019, EPA issued a memorandum that provides  definitions for enforcement response tools, including informal  enforcement actions, and instructions on how to report such actions. Now  that the agency has finalized its definition of informal enforcement actions  and specified which mechanisms to use to maintain data on such actions,  by clearly documenting in guidance to the regional offices how they  should use the definition to collect data on these actions, EPA would have  better assurance that the regional offices consistently collect and maintain  these data.

EPA Does Not Require Its Regional Offices to Collect and Maintain Data on Compliance Assistance Activities

According to EPA headquarters officials, OECA stopped requiring  regional offices to collect data and report on their compliance assistance  activities around 2012. Prior to that time, each regional office had a full- time staff member dedicated to coordinating compliance assistance  activities, according to these officials. However, the staff member’s  activities were the only compliance assistance data that regional offices  collected and maintained. EPA officials stated that the regional offices  stopped collecting the compliance assistance activities associated with  this position when the agency redirected the funding for the full-time staff  position to compliance monitoring and other enforcement efforts. As a  result, EPA officials told us that the agency does not have consistent data  about its compliance assistance activities.

EPA officials told us that the agency made a policy decision to stop  dedicating funding to compliance assistance but encouraged staff to  continue conducting compliance assistance activities as part of the  agency’s outreach for other programs. EPA headquarters officials said  that as of September 2019, the agency had no plans to require regional  offices to collect and report data on compliance assistance. However,  according to these officials, although the agency stopped funding the  compliance assistance coordinator position, regional staff continue to  conduct a range of compliance assistance activities as part of their  regular enforcement duties. Figure 2 shows the types of compliance and  enforcement data that EPA collects, including that the agency does not  require regional offices to collect information about compliance  assistance.

According to our analysis of written responses, officials in nine of the 10  regional offices reported that they collect some data on the compliance  assistance activities their offices conduct. Officials in one office said that  they do not collect data on compliance assistance activities because it is  not required. The types of data on compliance assistance that the nine  regional offices collect and the methods those offices use for maintaining  the data differ, according to our analysis of written responses. For  example, some regional officials described collecting data on compliance  assistance provided over the telephone, and other officials described  collecting data on on-site compliance assistance provided during  inspections. According to our analysis of written responses, officials in  two regional offices described providing on-site compliance assistance for  minor issues during inspections and tracking the number of times such  assistance was provided. Officials in the nine regional offices that still  collect data on some compliance assistance activities described storing  the data differently, either in region-specific databases or in paper files.  Officials in two of these regional offices said that regional staff decide how  to document telephone calls from regulated entities for assistance.  Officials in one region stated that they no longer conduct large-scale  compliance assistance activities such as conducting workshops or  developing informational materials because EPA eliminated the reporting  requirement.

Having complete information about its compliance assistance activities is  essential because EPA has elevated the role of such activities in its  overall enforcement efforts. However, EPA does not have complete  information on its compliance monitoring and enforcement activities,  partly because the agency does not require the collection of data on  compliance assistance activities. EPA’s lack of complete information on  its compliance assistance activities is inconsistent with its change in  policy. In addition, in our October 2017 report on key considerations for  agency enforcement decisions, we reported that transparency and  availability of data are important to promoting compliance and achieving  regulatory objectives. Having complete information about its compliance  assistance activities may provide more complete information on those  activities for evaluating its compliance monitoring and enforcement  performance.

As discussed earlier, most of the regional offices continue to collect some  information on compliance assistance even though they are not required  to do so and use varying mechanisms to maintain the information.  Because EPA does not direct the regional offices to collect data on  compliance assistance activities, the agency would not have issued  guidance instructing regional offices to collect such data and specifying  which mechanism to use to maintain them. However, according to federal  standards for internal control, management should design control  activities to achieve objectives and respond to risks, such as by clearly  documenting internal control in management directives, administrative  policies, or operating manuals. Without clearly documenting in guidance  to the regional offices that they should collect data on compliance  assistance activities and specifying which mechanism to use to maintain  the data, such as ICIS, EPA will not have the information it needs to track  progress toward its strategic objective of increasing the agency’s use of  compliance assistance activities to help regulated entities comply with  laws and regulations.

EPA Communicates the Results of Its Compliance Monitoring Activities and Enforcement Actions to the Public and Congress through a Website and Annual Reports

EPA communicates the results of its compliance monitoring activities and  enforcement actions by making data available to the public and Congress  through its website and annual reports. EPA’s ECHO website allows the  public to view data over time, such as the number of facilities inspected  by an authorized state or EPA from fiscal years 2011 to 2019. To help  the public understand the data presented on its ECHO and other  websites, EPA websites list a number of national and state-specific  known data limitations concerning the data collected for its environmental  programs. For example, the ECHO website identifies whether certain  years of data are not appropriate for analyzing trends, such as its data on  penalties under the Clean Water Act prior to 2015.

EPA issues annual performance reports that include data on compliance  monitoring and enforcement to fulfill requirements under the Government  Performance and Results Act and other requirements. These reports  describe progress toward the three strategic goals and related objectives  in EPA’s Fiscal Year 2018-2022 Strategic Plan. In addition, since 2017,  EPA has published a Year in Review report that outlines the agency’s  accomplishments, including in the area of enforcement, using data on its  compliance and enforcement actions to present the results of its efforts.  In addition, at the end of each fiscal year, OECA publishes a Fiscal Year  EPA Enforcement and Compliance Annual Results report and companion  data graphs that provide enforcement data over a selected time period on  such topics as the number of EPA inspections conducted, cases initiated,  and value of fines and penalties collected. Environmental groups and  media outlets have used EPA’s data to develop analyses, conclusions,  and inferences about changes in EPA’s enforcement results.

In December 2018, we reported that providing information about a  dataset—for example, known limitations of the data in that dataset— allows users to determine whether the database is suitable for their  intended purpose and make informed decisions about whether and how  to use it. For example, EPA’s 2000 EPA Quality Manual for  Environmental Programs states that published reports with environmental  data shall be accompanied by a readily identifiable section or appendix  that discusses the quality of the data and any limitations on the use of the  data with respect to their original intended application. It also states that  the agency’s reports should include applicable statements about possible  misuse of the data for other purposes.

EPA’s Fiscal Year 2018 Annual Performance Report includes a link to  companion reports on its website that describe, among other things, the  sources of the data used in the report and the known limitations of those  data. Specifically, the companion reports include information such as  the definition of terms used, units of measurement, data sources, method  for analyzing the data, and the known limitations of the data.

However, neither of EPA’s other 2018 annual reports we reviewed fully  disclosed known limitations to the data the agency included in each  report:

Year in Review 2018. OECA’s Year in Review 2018 report, the most recent report available at the time of our review, includes a range of data—such as number of actions taken, monetary results, the reduction of emissions in tons, and data over selected time periods— to accompany its statements about the agency’s accomplishments. However, the report does not include any information about data sources or known limitations of the data.

Fiscal Year 2018 EPA Enforcement and Compliance Annual Results. EPA’s Fiscal Year 2018 EPA Enforcement and Compliance Annual Results report, also the most recent at the time of our review, includes data sources and some known limitations of the data. For example, the report states that the data on results do not include state and local inspections or enforcement actions. Additionally, the report includes statements about changes in how the agency stores data that may prevent the data from being comparable across years. The report lists the various sources of the data used to create the report’s charts and graphs. EPA has published known limitations of these data on its ECHO website and indicated that broad data issues may affect the completeness, timeliness, or accuracy of the data in its various systems. However, based on our review of the report, it does not include information about known limitations of all of the data in the report.

In addition, neither the Year in Review 2018 report nor the Fiscal Year  2018 EPA Enforcement and Compliance Annual Results report includes a  readily identifiable section or appendix that discusses the known  limitations of the data, as called for by leading practices for transparently  reporting government data and as exemplified in EPA’s manual governing  environmental data quality. In commenting on our assessment of the  annual reports, EPA officials did not provide a reason why the reports do  not discuss known data limitations but told us in a prior meeting that the  current documentation on the ECHO website includes the current known  data limitations.

Furthermore, EPA’s Fiscal Year 2018 EPA Enforcement and Compliance  Annual Results report does not fully describe how the data in the report  should be interpreted given the known data limitations the report contains.  For example, the 2018 annual results report provides a partial picture of  overall enforcement of environmental laws because the data exclude  state enforcement actions. In addition, for the yearly data across years  (2008 through 2018 or 2012 through 2018), EPA does not fully provide  information on any limitations in how the data should be analyzed; for  example, whether the data are appropriate for the purpose of identifying  trends or providing a snapshot of an activity for a single year. EPA does,  however, include information on the impact of one or two large cases on  the data presented for some data in the report such as the volume of  contaminated soil and water to be cleaned up or the treatment and  disposal of hazardous and nonhazardous waste.

In our November 2019 report on data transparency, we concluded that  without the transparent disclosure of known data limitations, users may  view or analyze data without full knowledge of the extent to which the  data are timely, complete, accurate, or comparable over time. Our  November 2019 report also concluded that this could lead users to  inadvertently draw inaccurate information or conclusions from the data.  OECA’s Assistant Administrator has discussed the known limitations of  EPA’s data in the annual reports. In a February 26, 2019, testimony  before Congress, OECA’s Assistant Administrator stated that the  averages for some of the metrics used in EPA’s annual results report  cannot be interpreted to represent a statistical trend. OECA’s Assistant  Administrator also stated that changes in the number of enforcement  actions may be a function of changes in programmatic decisions and may  not be reflective of changes in the underlying compliance of regulated  entities with environmental statutes. By including the known limitations of  data in its annual reports and providing information on the intended use of  EPA’s data, as called for by leading practices for transparently reporting  government data and as exemplified in existing EPA guidance for  environmental data, EPA would have better assurance that Congress and  the public are informed about the data presented and how the data  should be interpreted.

EPA collects a range of information and uses the information to manage  its enforcement and compliance program and assess how well its efforts  are meeting the objectives outlined in the agency’s strategic plan and  other documents. However, while most of the regional offices collect data  on some informal enforcement actions, they use different mechanisms to  maintain these data, and the agency has not provided guidance to  regional offices on how they should collect or maintain the data. Without  documenting in guidance to the regional offices how they should collect  data on informal enforcement actions and specifying which mechanism to  use to maintain the data, EPA lacks assurance that the regional offices  will consistently collect and maintain these data. On September 30, 2019,  EPA issued a memorandum that provides definitions for enforcement  response tools, including informal enforcement actions and instructions  on how to report such actions. We view this as a step in the right  direction. Now that the agency has finalized its definition of informal  enforcement actions and provided instructions on how regional offices  should report such actions, by clearly documenting in guidance on how  regional offices should use the definition to collect data on these actions,  EPA would have better assurance that the regional offices consistently  collect and maintain these data.

Similarly, EPA does not have complete information on its compliance  monitoring and enforcement activities because the agency does not  require the collection of data on compliance assistance activities. As a  result, the agency has not issued guidance instructing regional offices to  collect such data and specifying which mechanism to use to maintain  them. Without clearly documenting in guidance to the regional offices that  they should collect data on compliance assistance activities and  specifying which mechanism to use to maintain the data, such as ICIS,  EPA will lack key information. Such information is needed to track  progress toward its strategic objective of increasing the agency’s use of  compliance assistance activities to help regulated entities comply with  laws and regulations.

While EPA communicates the results of its compliance monitoring  activities and enforcement actions through its website and annual reports,  neither of its 2018 annual reports includes a readily identifiable section or  appendix that discusses the known limitations of the data. The 2018  annual results report also does not fully describe how the data in the  report should be interpreted, given the known data limitations the report  contains. By including the known limitations of the data in its annual  reports and providing information on the intended use of EPA’s data, EPA  would have better assurance that Congress and the public are informed  about the data presented and how the data should be interpreted.

We are making the following three recommendations to EPA:  The Assistant Administrator for EPA’s Office of Enforcement and  Compliance Assurance should clearly document in guidance to the  regional offices how they should use the definition of informal  enforcement actions to collect data on these actions.   (Recommendation 1)

The Assistant Administrator for EPA’s Office of Enforcement and  Compliance Assurance should clearly document in guidance to the  regional offices that they should collect data on compliance assistance  activities and specify which mechanism to use to maintain the data, such  as ICIS. (Recommendation 2)

The Assistant Administrator for EPA’s Office of Enforcement and  Compliance Assurance should include the known limitations of data in its  annual reports and provide information on the intended use of EPA’s  data. (Recommendation 3)

We provided a draft of this report to EPA for review and comment. In its  written comments, reproduced in appendix I, EPA stated that it agreed  with all three of our recommendations and many of our findings and  conclusions. EPA also provided technical comments, which we  incorporated into the report, as appropriate.

In response to our first recommendation to clearly document in guidance  how regional offices should use the definition of informal enforcement to  collect data on these actions and specify a mechanism to maintain the  data, EPA said that the agency issued a September 30, 2019,  memorandum for headquarters and regional enforcement offices to  implement. This memorandum provides guidance on EPA definitions for  enforcement response tools, to promote consistency and clarity in the use  of enforcement terms, according to EPA. EPA also said that the guidance  defines “informal enforcement action.” The guidance includes instructions  on how to report such actions. The guidance states that, with two  exceptions, headquarters and regional offices are expected to report, in  ICIS, all informal enforcement actions across all programs that meet the  new definition. In addition, the guidance states that because it is only a  definitional document and does not include guidance on appropriate use  of the enforcement response policy tools, the agency will work to identify  the specific changes in practice needed (i.e., changes in use and  reporting). The guidance states that EPA anticipates that informal  enforcement actions meeting the new definition will be included in the  agency’s certified annual enforcement results beginning in fiscal year  2020. We view EPA’s guidance as a step in the right direction, and the  guidance states that EPA will provide training and additional guidance for  enforcement staff to ensure consistent implementation across regional  offices and headquarters. Additional guidance will provide EPA with an  opportunity to specify how regional offices are to use the definition of  informal enforcement to collect data on these actions. We modified our  recommendation because EPA’s recent guidance specifies mechanisms  for EPA employees to maintain data on informal enforcement actions.

In response to our second recommendation to clearly document in  guidance that regional offices should collect data on compliance  assistance activities and specify a mechanism to maintain the data, EPA  said that it would collect data on compliance assistance for each of the  National Compliance Initiatives and maintain those data in ICIS. In  response to our third recommendation to include known data limitations in  annual reports and provide information on intended use of its data, EPA  stated that it acknowledges the importance of providing information about  a dataset to facilitate proper interpretation. For that reason, EPA said that,  in time for its fiscal year 2020 report, the agency will create a webpage to  describe how best to interpret the data presented in the agency’s Fiscal  Year EPA Enforcement and Compliance Annual Results report and  include a reference to that webpage in the report itself as well as the Year  in Review report.

In technical comments related to our third recommendation, EPA stated  that several of the limitations we identified in the report do not affect the  data included in its Fiscal Year EPA Enforcement and Compliance Annual  Results report. In considering EPA’s technical comments, we modified the  text of the report concerning examples of the annual report’s data  limitations, as appropriate.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  appropriate congressional committees, the Administrator of EPA, and  other interested parties. In addition, the report is available at no charge on  the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or gomezj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix II.

Appendix I:Comments from the Environmental Protection Agency

In addition to the contact named above, Chad M. Gorman (Assistant  Director); Tahra Nichols (Analyst in Charge); Mark Braza; Courtney  Carroux; Tara Congdon; Jazzmin Cooper; Matthew Hunter; Caroline  Prado; Dan Royer; Jeanette Soares; Kiki Theodoropoulos, Sonya  Vartivarian, and Michelle R. Wong made key contributions to this report.

EPA’s mission includes ensuring factories, local governments, sewage treatment plants, and others follow environmental laws and regulations, but it doesn’t consistently track all of its efforts. EPA regional offices informally help entities comply (e.g., training), and they conduct informal enforcement actions after violations (e.g., warning letters).

Although EPA’s strategic goals call for increasing use of these two types of informal activities, it hasn’t given regional offices clear and complete guidance on how to track and monitor them. We recommended EPA create guidelines to ensure it has data it needs to track these activities.