Data Act: Quality of Data Submissions Has Improved but Further Action Is Needed to Disclose Known Data Limitations

The DATA Act requires federal agencies to disclose roughly $4 trillion in annual federal spending and link this spending information to federal program activities so that policymakers and the public can more effectively track federal spending through its life cycle. The act also requires OMB and Treasury to establish data standards to enable consistent reporting of agency spending. The DATA Act includes a provision for GAO to report on the quality of the data collected and made available through USAspending.gov.

Specifically, this report addresses: (1) the timeliness, completeness, and accuracy of the data, and the implementation and use of data standards; and (2) progress made in developing a data governance structure consistent with key practices, and how it affects data quality. GAO examined a projectable government-wide sample of Q4 FY2018 spending data from a Treasury database that populates data on USAspending.gov by comparing them to agency source records and other sources. GAO also compared the results of Q4 2018 with results from its previous review of Q2 FY2017 data.

The Digital Accountability and Transparency Act of 2014 (DATA Act) requires federal agencies to report spending data to USAspending.gov, a public-facing website. A total of 96 federal agencies submitted required spending data for quarter four of fiscal year 2018 (Q4 FY2018). GAO examined the quality of these data and compared the results with the results of its prior review of quarter two of fiscal year 2017 (Q2 FY2017) data, as appropriate. GAO identified improvements in overall data quality, but challenges remain for completeness, accuracy, use of data standards, disclosure of data limitations, and overall data governance.

Completeness. The number of agencies, agency components, and programs that submitted data increased compared to Q2 FY2017. For example, 11 agencies did not submit data in Q4 FY2018, compared to 28 in Q2 FY2017. Awards for 39 financial assistance programs were omitted from the data in Q4 FY2018, compared to 160 financial assistance programs in Q2 FY2017.

Accuracy. Based on a projectable governmentwide sample, GAO found that data accuracy for Q4 FY2018—measured as consistency between reported data and agency source records or other authoritative sources and applicable laws and reporting standards—improved for both budgetary and award transactions. GAO estimates with 95 percent confidence that between 84 a 96 percent of the budgetary transactions and between 24 and 34 percent of the award transactions were fully consistent for all applicable data elements. In Q2 FY2017, GAO estimated that 56 to 75 percent of budget transactions and 0 to 1 percent of award transactions were fully consistent.

Use of data standards. GAO continued to identify challenges related to the implementation and use of two data elements— Award Description and Primary Place of Performance Address— that are particularly important to achieving the DATA Act's transparency goals. GAO found that agencies continue to differ in how they interpret and apply The Office of Management and Budget's (OMB) standard definitions for these data elements. As a result, data on USAspending.gov are not always comparable, and in some cases it is difficult for users to understand the purpose of an award or to identify the location where the performance of the award occurred.

USAspending.gov presentation. GAO identified known data limitations that were not fully disclosed on USAspending.gov. For example, the 90-day delay for inclusion of Department of Defense procurement data is not clearly communicated. In addition, although the website provides a total figure for unreported spending it is unclear whether it includes the 11 agencies that did not submit data. Not knowing this information could lead users of USAspending.gov to inadvertently draw inaccurate conclusions from the data.

Data governance. OMB and the Department of the Treasury (Treasury) have established some procedures for governing the data standards established under the DATA Act, but procedures for enforcing the consistent use of established data standards have yet to be developed. Persistent challenges related to how agencies interpret and apply data standards underscore GAO's prior recommendations on establishing a governance structure that ensures the integrity of these standards.

GAO maintains that OMB and Treasury should address prior recommendations on monitoring agency submissions, implementing data standards, disclosing data limitations, and developing a robust data governance structure. In addition, GAO makes two new recommendations to Treasury regarding disclosing on USAspending.gov specific known data limitations. Treasury agreed with GAO's recommendations.

In the 5 years since the enactment of the Digital Accountability and  Transparency Act of 2014 (DATA Act), much progress has been made to  improve the transparency of federal spending data, which was roughly  $4.45 trillion in fiscal year 2019. The Office of Management and Budget  (OMB) and the Department of the Treasury (Treasury) established a set  of data standards to enable the reporting and tracking of federal spending  data displayed on USAspending.gov. Treasury, in collaboration with  OMB, issued additional guidance and improved the technical architecture  used by federal agencies to facilitate their efforts to report spending data.  With these improvements and improvements made in reporting data at  the agency level, more agencies are reporting more data to Treasury and  thus making more information available to the public.

The ongoing implementation of the DATA Act is one of several  government-wide initiatives under way focused on improving the  transparency and quality of federal data assets. Recent initiatives that  extend beyond the DATA Act include the cross-agency priority (CAP)  Goal Leveraging Data as a Strategic Asset under the 2019 President’s  Management Agenda. This CAP Goal includes the development of a  federal data strategy. In addition, the Foundations for Evidence-Based  Policy Making Act of 2018 (Evidence Act), enacted in January 2019,  requires, among other things, that agencies designate a Chief Data  Officer to help improve data quality across government.

While these more recent initiatives provide opportunities for continued  improvement, our prior work examining the quality of the data made  available under the DATA Act has found significant data quality  challenges that limit their usefulness. The DATA Act contains a provision  requiring us and agency inspectors general (IG) to report on the  completeness, timeliness, quality, and accuracy of the data—in 2017,  2019, and 2021. This is our second assessment of the quality—defined  as encompassing the concepts of timeliness, completeness, and  accuracy—of data agencies were required to report pursuant to the DATA  Act. More specifically, this report addresses: (1) the timeliness,  completeness, and accuracy of the data, and the implementation and use  of data standards; and (2) progress made to develop a data governance  structure consistent with leading practices, and how it affects data quality.  We also update the status of our previous recommendations related to  implementation of the DATA Act and data transparency.

To assess the timeliness, completeness, and accuracy of the data  submitted and the implementation and use of data standards, we  analyzed agency submission files for the fourth quarter of fiscal year 2018  (Q4 FY2018) on USAspending.gov and reviewed a representative  stratified random sample of transactions selected from the  USAspending.gov database containing spending data for Q4 FY2018.

We designed our stratified random sample to estimate rates within each  of the three data files: (1) procurement award transactions, (2) assistance  award transactions, and (3) budgetary records. Estimates for the results  of the procurement, assistance, and budgetary samples have sampling  errors of +/-7.8, 8, and 10 percentage points or less, respectively, at the  95 percent level of confidence. See table 1 for a listing of the six  budgetary data elements and the 38 procurement and financial  assistance award data elements and subelements that we tested in our  review.

We compared the results of our review of Q4 FY2018 data to those of our  second quarter of fiscal year 2017 (Q2 FY2017) data that we reviewed in  our first mandated assessment of data quality. For both reviews, we  examined a projectable sample of budgetary and award transactions from  a database that according to Treasury is partly used to display data on  USAspending.gov. However, there were the following differences: (1) our  2017 sampling frame was confined to the 24 Chief Financial Officers Act  of 1990 (CFO Act) agencies (which represented 99 percent of obligations in our data set at that time), while our sampling frame for this  review included all agencies that submitted Q4 FY2018 data files as of  February 11, 2019; (2) more agencies and their components reported  data in Q4 FY2018 than in Q2 FY2017; (3) in 2017, our estimated error  rate calculations included elements of certain sampled transactions that  were determined to be not applicable to the transaction, and were  classified as consistent with agency sources in both the numerator and  denominator, while in this review, we excluded not-applicable elements  from both the numerator and denominator of the estimated rate  calculations; (4) our sampling frame for this review included more data  elements and subelements than were in our Q2 FY2017 sampling frame;  (5) in this review, since three data elements we reviewed were derived by  Federal Procurement Data System-Next Generation (FPDS-NG) and  Financial Assistance Broker Submission (FABS) rather than provided by  agencies, we compared the information in the sample to other sources  rather than agency documents and therefore did not compare those  results to Q2 FY2017; (6) agencies’ Q4 FY2018 data were submitted  under policies and procedures outlined in the DATA Act Information  Model Schema (DAIMS) v1.3 which reflects changes in validation rules  and reporting requirements from the DAIMS v1.0 that was in effect in  2017; (7) OMB issued additional guidance on DATA Act reporting since  we reported in 2017; and (8) changes were made to the Treasury  broker—the system that collects and validates agency data—since our  last report.

To evaluate how the current data governance structure affects data  quality, we compared data quality challenges we identified during our  review to key practices for data governance identified in our prior work.  To assess progress made to develop a data governance structure  consistent with key practices, we reviewed policy and other  documentation related to ongoing efforts to develop a government-wide  structure for governing the standards established under the act, and  interviewed OMB staff about these efforts. For the agencies selected in  our sample, we also reviewed agency data quality plans and agency  guidance intended to facilitate agency efforts to establish data  governance programs, and interviewed agency officials on their data  governance efforts.

To update the status of our prior recommendations related to the  implementation of the DATA Act, we reviewed new guidance and other  related documentation, and interviewed OMB staff and Treasury officials.  See app. IV for an update on our open recommendations related to data  transparency and DATA Act implementation. Additional details regarding  our objectives, scope, and methodology along with information about data  reliability are provided in app. II.

We conducted this performance audit from November 2018 to November  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Signed into law on May 9, 2014, the DATA Act expands on previous  federal transparency legislation. It requires a greater variety of data  related to federal spending by agencies, such as budget and financial  information, to be disclosed and agency spending information to be linked  to federal program activities so that policymakers and the public can more  effectively track federal spending through its life cycle.

The act gives OMB and Treasury responsibility for establishing  government-wide financial data standards for any federal funds made  available to, or expended by, federal agencies. As Treasury and OMB  implemented the DATA Act’s requirement to create and apply data  standards, the overall data standardization effort has been divided into  two distinct, but related, components: (1) establishing definitions which  describe what is included in each data element with the aim of ensuring  that information will be consistent and comparable and (2) creating a data  exchange standard with technical specifications that describe the format,  structure, tagging, and transmission of each data element.

Accordingly, OMB took principal responsibility for developing policies and  defining data standards. Treasury took principal responsibility for the  technical standards that express these definitions, which federal agencies  use to report spending data for publication on USAspending.gov. Under  the act, agencies are required to submit complete and accurate data to  USAspending.gov, and agency-reported award and financial information  is required to comply with the data standards established by OMB and  Treasury. See app. V for more information on the sources of data and  process for submitting data under the DATA Act.

GAO Reports on Data Quality and Data Governance

Since the DATA Act’s enactment in 2014, we have issued a series of  reports and made recommendations based on our ongoing monitoring of  DATA Act implementation. In November 2017, we issued our first report  on data quality, which identified issues with, and made related  recommendations about, the completeness and accuracy of the Q2  FY2017 data that agencies submitted, agencies’ use of data elements,  and Treasury’s presentation of the data on Beta.USAspending.gov. In  addition, as part of our ongoing monitoring of DATA Act implementation,  and in response to provisions in the DATA Act that call for us to review IG  reports and issue reports assessing and comparing the quality of agency  data submitted under the act and agencies’ implementation and use of  data standards, we issued a report in July 2018, based on our review of  the IG reports of the quality of agencies’ data for Q2 FY2017.

Our prior reports identified significant data quality issues and challenges  that may limit the usefulness of the data for Congress and the public.  These data quality challenges underscore the need for OMB and  Treasury to make further progress on addressing our 2015  recommendation that they establish clear policies and processes for  developing and maintaining data standards that are consistent with key  practices for data governance. Such policies and processes are needed  to promote data quality and ensure that the integrity of data standards is  maintained over time.

In March 2019, we reported on the status of OMB’s and Treasury’s efforts  to establish policies and procedures for governing data standards. We  found that OMB and Treasury have established some procedures for  governing the data standards established under the DATA Act, but a  formal governance structure has yet to be fully developed. Therefore, we  made recommendations to OMB to clarify and document its procedure for  changing data definition standards, and to ensure that related policy  changes are clearly identified and explained.

Data Quality Has Improved, but Challenges with Completeness, Accuracy, and the Implementation and Use of Data Standards Remain

For Q4 FY2018, 107 agencies, including all 24 CFO Act agencies and 83  non-CFO Act agencies, determined they were required to submit data, or  they would voluntarily submit data, under the DATA Act. Of these 107  agencies, 96 submitted data for Q4 FY2018. This is an increase over the  initial submissions for Q2 FY2017 when 78 agencies submitted data that  covered 91 federal entities. This represents an improvement in the  number of agencies reporting. However, not all the required files  submitted by agencies were complete, and the data submitted were not  always accurate (i.e., consistent with agency source records and other  authoritative sources and applicable laws and reporting standards). In  addition, we found that some CFO Act agencies did not include certain  financial assistance programs that made awards during fiscal year 2018  in their submissions. Finally, some agencies continued to have  challenges in reporting some data elements in accordance with  standards.

Agencies That Submitted Data Were Generally Timely, but Several Agencies Failed to Report All or Some of Their Data

While the total number of agencies that submitted data for Q4 FY2018  increased compared to Q2 FY2017, more agencies submitted their data  for Q4 FY2018 after the due date compared to Q2 FY2017. In addition,  the data for Q4 FY2018 available on USAspending.gov are not complete  because some agencies failed to submit data or submitted partial data.

Fourteen agencies submitted late. Agencies were required to submit their  DATA Act files for Q4 FY2018 by November 14, 2018—45 days after the  end of the quarter. Eighty-two agencies submitted their data on time.  These 82 agencies represented about 84 percent of the total obligations  government-wide reported to Treasury on the SF 133 for Q4 FY2018.  Fourteen agencies submitted their data after the November 14, 2018 due  date. Our prior review of data submitted for Q2 FY2017 found that one  agency submitted data after the due date.

Eleven agencies did not submit data. Eleven non-CFO Act agencies did  not submit any DATA Act files for Q4 FY2018. By contrast, in reviewing  Q2 FY2017 data, we identified 28 agencies that determined they should  have reported data under the DATA Act, but did not. Agencies told us  that they did not submit data for Q4 FY2018 because (1) there was  confusion or miscommunication between the agency and its shared  service provider about who was responsible for reporting the data;   (2) their officials had determined the agency was not required to report;  (3) new staff were unfamiliar with DATA Act requirements; and   (4) technical or systems issues, such as a financial system upgrade in  process, prevented them from reporting their data.

Multiple agencies submitted blank files. Of the 96 agencies that submitted  DATA Act files for Q4 FY2018, 35 non-CFO Act agencies submitted the  file that links budget and award information (i.e., File C) or the file  containing procurement data (i.e., File D1) that did not contain any data  (i.e., files were blank).

Specifically, 34 non-CFO Act agencies submitted a blank File D1, which  contains procurement data, and 16 of those 34 also submitted a blank  File C. Another non-CFO Act agency submitted a blank File C only. File C  data are particularly important to oversight and transparency because  they link budget and award information, as required by the DATA Act.  Without this linkage, policymakers and the public may be unable to  effectively track federal spending because they would be unable to see  obligations at the award and object class level.

Agencies told us they submitted files without data for reasons including:  (1) their data was submitted by and comingled with their shared service  provider’s DATA Act submissions; (2) they did not have award activity to  report or award activity was below the micro-purchase threshold for  reporting; and (3) they do not use the Federal Procurement Data System- Next Generation or their systems were unable to produce the data  necessary to create the files.

We did not assess the completeness of File D1 in 2017, but we found that  13 agencies submitted a blank File C in Q2 FY2017. Of these 13  agencies, two were CFO Act agencies with large amounts of award  activity —the Departments of Defense (DOD) and Agriculture (USDA)— both of which did submit a File C with data for Q4 FY2018.

Two agencies submitted incomplete files. DOD and Treasury submitted  all seven required DATA Act files for Q4 FY2018, but the data in some of  those files were not complete. According to DOD officials, its File C  submission for Q4 FY2018 included data from six of its 18 accounting  systems. DOD officials said they are working to report data from all 18  systems in File C by the fourth quarter of fiscal year 2019. They said  prior to Q4 FY2018, OMB granted DOD extensions for reporting financial  and payment information in File C, as permitted by the act. DOD  officials said the extensions allowed DOD to focus on financial statement  audit readiness, build a single source tool from which File C obligation  data could be aligned with procurement and grant data, and coordinate  with the intelligence community on concerns over increased  transparency.

According to Treasury officials, the agency’s data submission did not  include the spending of one of its component organizations—the Treasury  Executive Office for Asset Forfeiture, Equitable Sharing Program— because OMB guidance does not allow for reporting aggregate  transactions when Primary Place of Performance, a required data  element, is at a multistate or nationwide level. According to Treasury  officials, Treasury is working with OMB and the Treasury DATA Act  Program Management Office to allow for these types of transactions to be  reported.

In our 2017 review, we identified similar challenges with the  completeness of agencies’ DATA Act submissions for Q2 FY2017 and  made recommendations to Treasury and OMB to improve the  completeness of data on USAspending.gov. We recommended that  Treasury reasonably assure that ongoing monitoring controls to help  ensure the completeness and accuracy of agency submissions are  designed, implemented, and operating as intended. Treasury agreed with  this recommendation. In September 2019, Treasury officials told us that  they are working to formalize a process for monitoring agency  submissions that will include emailing reminders to agencies prior to  submission deadlines, following up with agencies that do not submit  required data on time, and forwarding a list of non-compliant agencies to  OMB.

We also recommended that OMB continue to provide ongoing technical  assistance that significantly contributes to agencies making their own  determinations about their DATA Act reporting requirements and that it  monitor agency submissions. While OMB generally agreed with our  recommendation, it has not yet taken steps to monitor agency  submissions to help ensure their completeness. In October 2019, OMB  staff told us that they believe monitoring agency submissions is not their  responsibility.

During this review we asked agencies why they did not submit data for  Q4 FY2018. Subsequently, five of them submitted their data late (out of  the initial 18 agencies that had not submitted data), demonstrating that  simple monitoring tasks such as a follow up call or email can result in  actions taken by the agencies. To address ongoing challenges with the  completeness of agencies’ DATA Act submissions, we continue to  maintain that Treasury and OMB should monitor agencies’ submissions to  help ensure the completeness and accuracy of those data submissions.  See app. IV for more information on the status of these  recommendations.

Agencies did not report awards made to 39 financial assistance  programs. Seven of the 24 CFO Act agencies did not report spending for  at least one financial assistance program that made awards during fiscal  year 2018. File D2 contains detailed information about individual financial  assistance awards. We compared the spending data reported by the 24  CFO Act agencies in File D2 against the Assistance Listings, formerly  known as the Catalog of Federal Domestic Assistance (CFDA), a  government-wide compendium of federal programs, projects, services,  and activities that provide assistance or benefits to the American public.

As of March 2019, the Assistance Listings website contained 2,926  programs for the CFO Act agencies. Of these, 39 programs  (approximately 1 percent) were not included in the Q4 FY2018 DATA Act  submissions, even though these agencies stated that they made  reportable awards during fiscal year 2018. In comparison, in July 2017,  the CFDA listed 2,219 programs for the CFO Act agencies. Of these  2,219 programs, 160 programs (approximately 7 percent) were not  included in the Q2 FY2017 DATA Act submissions even though they  made reportable awards. The remaining programs either reported at  least one award or did not make awards that were subject to reporting.

To provide a sense of magnitude of the underreporting, we obtained  estimates of the total projected annual spending for these programs for  fiscal year 2018 from the Assistance Listings website and applicable  agencies. Based on the estimated obligations, the 39 programs account  for approximately $11.5 billion in estimated annual obligations in fiscal  year 2018. The omitted amounts largely resulted from USDA’s failure to  report 27 programs representing more than 99 percent of the estimated  annual obligations. According to USDA officials, USDA did not submit  awards for some of these programs because it maintains that the  information in legacy reporting systems is incompatible with the Treasury  broker. USDA is working on solutions to resolve identified reporting  challenges with its financial and awards systems.

Treasury took steps to address findings on completeness issues for  financial assistance programs we reported in 2017. At Treasury’s request, we provided details regarding the programs that were omitted from the  USAspending.gov database for fiscal year 2017, which Treasury shared  with the appropriate agencies. In our review of fiscal year 2018 data, we  found that only nine of these programs did not report.

Budgetary and Award Data Accuracy Has Improved

Based on the results of testing performed on a sample of budgetary and  award transactions, we found that the overall completeness within  individual transactions and accuracy of the reported data was high. We  selected a projectable government-wide sample of 405 transactions and  tested 41 data elements and subelements associated with them for  completeness and accuracy. We determined data completeness within  the transaction based on whether the element included a value and  whether the value was appropriate. We determined accuracy of data  elements by determining consistency with agency source records as well  as applicable laws and reporting standards.

Specifically, based upon our sample we estimate with a 95 percent  confidence level that all the data in the population were between 99 and  100 percent complete and between 90 and 93 percent accurate. We  further analyzed accuracy at the transaction and individual data element  levels as follows:  1.  Transaction level, which describes the extent to which all applicable  data elements within an individual transaction are complete and  consistent with agency source records, and applicable laws and  reporting standards.  2.  Data element level, which describes the extent to which the data  elements and subelements used for reporting budgetary and award  information were consistent with agency source records and  applicable laws and reporting standards.

Consistency of transactions. For data submitted in Q4 FY2018, we found  that the level of consistency differed between budgetary and award  transactions, but both improved compared to the data we sampled for our  review of Q2 FY2017 data. Based on our projectable government-wide  sample of Q4 FY2018 data, we estimate with 95 percent confidence that  between 84 and 96 percent of the budgetary transactions and between 24  and 34 percent of the award transactions in the USAspending.gov  database were fully consistent with agency sources. We considered a  transaction to be “fully consistent” if the information contained in the  transaction was consistent with agency records for every applicable data  element. This result represents an increase in consistency from what we  reported in 2017, when we estimated that between 56 and 75 percent of  budgetary transactions were fully consistent, and between 0 and 1  percent of award transactions were fully consistent.

In addition to the transactions that were fully consistent, we estimate that  94 to 100 percent of budgetary transactions and 62 to 72 percent of  award transactions in the population were significantly consistent. We  considered a transaction significantly consistent if 90 percent or more of  the data elements and subelements in the transaction were consistent  with agency source records and applicable laws and reporting standards.

Consistency of data elements. We also found improvements in the  consistency of budgetary and award data elements with agency records,  and applicable laws and reporting standards. As shown in figure 1, more  data elements were significantly consistent and fewer were significantly  inconsistent in Q4 FY2018 than Q2 FY2017.

We considered a data element to be “significantly consistent” if the  estimated consistency rate was at least 90 percent. Five of six of the  budgetary data elements were significantly consistent in Q4 FY2018,  compared to four of seven data elements in our 2017 review. We also  found improvements in the consistency of award data elements and  subelements compared to our 2017 review. Eighteen of the 35 award  data elements and subelements in our sample were significantly  consistent in Q4 FY2018, compared to only one of 26 data elements and  subelements we tested in our 2017 review. See figure 2 for the data  elements and subelements in our sample that were significantly  consistent.

We considered a data element “significantly inconsistent” if it was either  not consistent with agency records or incomplete at least 10 percent of  the time. We found that no budgetary data elements were significantly  inconsistent, which is an improvement from our 2017 review where we  found one budgetary data element—Obligation—significantly  inconsistent. Similarly, we found fewer significantly inconsistent award  data elements compared to our 2017 review. Specifically, we found five of  35 award data elements and subelements significantly inconsistent in Q4  FY2018, compared to 11 of 26 in our 2017 review. See figure 3 for the  data elements and subelements in our sample that were significantly  inconsistent.

Unverifiable data elements. We found no data elements that exhibited a  significant amount of unverifiable information—incomplete or inadequate  agency source records that prevented us from determining whether the  data element was significantly consistent or inconsistent. See app. III for  details.

While we tested the consistency of agency records and applicable laws  and reporting standards for the 41 data elements and subelements  previously discussed, we performed a different test for three other data  elements that contained a value derived by FPDS-NG and FABS. These  data elements and subelements—Legal Entity County Name, Primary  Place of Performance County Name, and Primary Place of Performance  Congressional District—were assessed against the other sources from  which they were derived, such as data from the U.S. Census Bureau and  house.gov, rather than agency records. We found that each were neither  significantly consistent nor significantly inconsistent with their sources.  See appendix III, table 5 for details.

Overall Data Quality Is Limited by Challenges in the Implementation and Use of Some Data Standards

The DATA Act requires OMB and Treasury to establish data standards to  produce consistent and comparable reporting of federal spending data.  While we found improvements in the overall completeness and accuracy  of the data when compared with the results of our 2017 review, we  identified persistent challenges with the implementation and use of two  award data elements—Award Description and Primary Place of  Performance Address that limit the usefulness of these data. We  previously reported that these data elements are particularly important to  achieving the transparency goals envisioned by the DATA Act because  they inform the public what the federal government spends money on and  where it is spent.

In our sample results, we found agencies reported values for Award  Description that were significantly inconsistent with agency sources and  with the established standard for reporting this data element which is  defined by the DATA Act data standard as a “brief description of the  purpose of the award.” Based on our testing of a representative sample of  Q4 FY2018 transactions, we estimate that the Award Description data  element was inconsistent with agency source records or contained  information that was inconsistent with the established standard in 24 to 35  percent of awards. While this represents an improvement over the results  we reported for this data element in 2017, we found in our testing that agencies continue to face challenges in reporting Award Description  consistent with the established standard. See figure 4 for several  examples of the Award Description data submitted by agencies in our  sample, which illustrates the range of agency interpretations of this data  element from understandable to incomprehensible.

Lengthy, technical description. For example, the National Aeronautics and  Space Administration (NASA) included several paragraphs for the  description of procurement and financial assistance award transactions in  our sample that were long and highly technical. These descriptions did  not meet the data standard because they contained acronyms, jargon,  and other technical terminology that might be challenging for others  outside the agency to understand. NASA officials said they use the Award  Description field internally to search for vendors when making awards for  similar services. Thus, they instructed contract officers to include as much  information as possible to maximize the Award Description field for later  use.

As of June 2019, the General Services Administration decreased the  character limit for reporting Award Description in FPDS-NG for  procurement awards from 4,000 characters to 250 characters to  discourage agencies from copying and pasting sizeable portions of a  contract’s contents rather than thoughtfully including a brief description of  what is being procured. NASA officials said that the new maximum will  limit the flexibility to search for contractors. They are seeking alternatives  for these searches.

No description provided. The Department of Education reported  “unknown title” for the Award Description for the majority of the financial  assistance award transactions in our sample. This does not meet the data  standard because it does not provide any information about the award.  Agency officials said the Award Description is provided by the applicant  and if one is not provided, their system automatically will populate it as  “unknown title.”

Geographic information. DOD reported location information for the Award  Description in several transactions in our sample. The locations reported  in the description field were not understandable except to agency officials.  For example, one field contained the text “4542874050!TRBO REGION  1.” DOD officials explained that this description includes the part number  for a medical supply item and the region of the country and is auto  populated by an agency system. While the description is consistent with  agency sources, it is not easily understood by the public. The Defense  Federal Acquisition Regulation Supplement Procedures, Guidance, and  Information provides instructions to use plain English as much as  possible, and to explain numbers and acronyms. DOD officials said the  agency is investigating methods to improve how similar transactions are  auto-populated.

Description of modification. The Department of Homeland Security (DHS)  used the Award Description field to describe modifications to contracts  instead of the good or service being procured. Specifically, DHS reported  “de-obligate excess funds and closeout” for a modification to a contract  that procured information technology products and services. DHS officials  said reporting the nature of the modification, rather than the original  purpose of the award, is consistent with practices used in contract writing  systems across the federal government and is intended to inform the  public of changes made to the contract by the modification. DHS is  working with Treasury to clarify how this information is displayed on  USAspending.gov and suggested that additional information on how  award descriptions for modifications are to be reported would be  beneficial and should be provided in the DAIMS.

We found that some individual agencies have taken steps to provide  additional guidance on Award Description to ensure agency personnel  are providing information that is consistent with the standard. Four  agencies in our sample had additional guidance for their contracting  officers. For example, officials from the Department of Veterans Affairs  (VA) said that in June 2019, VA trained hundreds of members of its  contracting workforce with curriculum that included an interactive game to  illustrate how to provide a brief description of an award that meets the  standard for reporting this information. Officials from 11 agencies said  additional guidance on Award Description could help ensure those  entering the data understand the standard definition and report  appropriate information, for example, by providing examples of award  definitions that meet the standard. In the absence of government-wide  guidance, agencies have reported values that are inconsistent with the  data standard and not comparable between agencies.

Agencies also reported several challenges with reporting Primary Place of  Performance Address for nonroutine locations, which OMB and Treasury  defined as “where the predominant performance of the award will be  accomplished.” Taking into account each of its subelements, we found  the information regarding Primary Place of Performance Address had  higher rates of inconsistency than the majority of the data elements in our  review.

Multiple subrecipients. Agency officials reported challenges with  identifying Primary Place of Performance Address in cases where an  award is made to a recipient that further distributes the funding to  subrecipients. For example, the U.S. Agency for Global Media  (USAGM) awards Radio Free Europe/Radio Liberty a grant that funds  work globally. Officials from USAGM said that as a U.S. not-for-profit  organization, Radio Free Europe/Radio Liberty, maintains corporate  headquarters in Washington, D.C., but, as an international media  organization, maintains many offices abroad. USAGM reports the  Primary Place of Performance Address as Washington, D.C. because  it is where the organization maintains its corporate office, but much of  the performance takes place in other locations.

In another example, the Department of Health and Human Services’  (HHS) Centers for Medicare and Medicaid Services (CMS) reports the  Primary Place of Performance Address for Medicare payment data as  the county of its payment processing centers, even though each  processing center makes payments to recipients in multiple states and  counties. CMS contracts with Medicare Administrative Contractors  (MAC) to process and pay Medicare fee-for-service claims. For each  type of Medicare claim, the number of jurisdictions and the number of  MACs that handle that type of claim vary. At the time of our review,  there were 12 jurisdictions for Medicare Part A and B claims handled  by MACs. As shown in figure 5, the jurisdictions are made up of  multiple states.

In addition to the MAC jurisdictions for Medicare Part A and Part B  claims, there were four home health and hospice jurisdictions and four  durable medical equipment jurisdictions. Thus, there are 20 MAC  jurisdictions, almost all of which covered multiple states. As a result,  the spending for Medicare payments is reported in a small number of  counties instead of where the beneficiaries of Medicare services are  located.

Software. Officials from three agencies in our review said that it is  challenging to determine Primary Place of Performance Address for  software licenses when purchased as a service. For example, there  could be multiple performance locations, but none of these locations  are predominant.

Large or undefined locations. Officials from the agencies in our  review reported challenges in meeting the standard for reporting large  or undefined performance locations. For example, officials from the  Delta Regional Authority said that it was difficult, at times, to  determine the Primary Place of Performance Address for watersheds  because they can cover a large area and cross multiple jurisdictions.  Officials from the National Science Foundation (NSF) said that for  projects that may not have a single location, they report the location  that corresponds to the research asset’s physical location or the  primary site. For example, for a research vessel, NSF officials report  the awardee’s address, which is generally the vessel’s homeport as  the Primary Place of Performance Address. In another example,  NASA officials said that when they let contracts for services  performed on the International Space Center, they report the  command center in Houston as the Primary Place of Performance  Address.

For some of these non-routine locations, the FPDS-NG data dictionary  provides guidance for procurement transactions. For example, for  services being performed in oceans and seas, it directs agencies to report  the closest U.S. city. For services being performed in the atmosphere or  space, the FPDS-NG Data dictionary directs agencies to report the  location from which the equipment conducting the services was launched.  However, the DATA Act Information Model Schema (DAIMS) Data  Dictionary does not include the same level of detailed guidance for  reporting financial assistance awards and directs agency officials to report  the location where the predominant performance of the award will be  accomplished.

Officials from several agencies said it would be helpful for OMB and  Treasury to issue guidance on Primary Place of Performance Address for  financial assistance awards to help agencies report this information  consistent with the established standard. In the absence of more specific  guidance, agencies are using different decision rules to identify the  Primary Place of Performance Address for financial assistance awards  which could limit the usefulness of this information to the public.

We previously identified similar issues with Award Description and  Primary Place of Performance Address on USAspending.gov. We  recommended that OMB and Treasury provide agencies with additional  guidance to address potential clarity, consistency, or quality issues with  the definitions for specific data elements including Award Description and  Primary Place of Performance Address and that they clearly document  and communicate these actions to agencies providing these data as well  as to end-users. OMB issued guidance in June 2018 which provides  clarification on reporting requirements for some data element  definitions.

However, additional guidance is needed to clarify how agencies are to  report spending data using standardized data element definitions that  may be open to more than one interpretation, and then broadly  communicate this information to agencies and the public. We continue to  believe additional guidance is needed to facilitate agency implementation  of certain data definitions to produce consistent and comparable  information. Given the challenges we identified in this report and in  previous reports with Award Description and Primary Place of  Performance Address, we have concerns about whether the guidance  OMB issued provides sufficient detail for agencies to consistently interpret  and implement the definitions. See app. IV for more information on the  status of this recommendation.

Known Data Limitations Are Not Transparent to Users of USAspending.gov

Treasury does not fully disclose all known data limitations on  USAspending.gov. According to OMB guidance, federal agencies should  be transparent about the quality of information and identify the limitations  of the data they disseminate to the public. Further, Treasury’s  Information Quality Guidelines state that, when disseminating information  to the public, information should be presented within the proper context to  disseminate information in an accurate, clear, complete, and unbiased  manner.

In November 2017, we identified data quality limitations that were not  disclosed on USAspending.gov. We recommended that Treasury disclose  known data quality issues and limitations on USAspending.gov.  Treasury agreed with this recommendation and has taken steps to better  disclose some of these limitations, but many of the issues we identified in  2017 continue to present challenges. Some of these challenges apply  widely, while others were specific to particular agencies. They include the  following:

Data not submitted or incomplete. One step taken by Treasury to  improve disclosure was to create a webpage in USAspending.gov that  provides information on unreported data. However, it is unclear  exactly what this information covers. For example, it is unclear  whether the information on unreported data includes financing  accounts, agencies that should have reported but did not submit data,  missing data for agencies that did submit, or spending that was not  reported because obligation amounts fell below $25,000 and was  therefore not required to be reported. As a result, users do not  clearly know what data are unreported or the amount that was  required to be reported.

Optional data elements and subelements. Another issue we  identified in 2017 and found again in our current review was that key  information about the reporting requirements for some data elements  and subelements was not adequately disclosed to the public.  Specifically, for Q4 FY2018 certain data elements were listed in  guidance as optional for agencies to report. According to Treasury  officials, agencies were not required to report these data elements  because the data standard was not fully implemented. For example,  prior to fiscal year 2019, the data element Funding Office Name was  optional for financial assistance awards. Additionally, as of September  2019, Period of Performance Start Date and Period of Performance  Current End Date remained optional for reporting pending  government-wide agreement on the standard.

USAspending.gov does offer some information regarding optional  data elements by providing a link to the DAIMS Reporting Submission  Specifications document. However, this document is not labeled in a  way that would make it clear to the user what information can be  found there. Moreover, some agencies may voluntarily submit data for  optional fields so only partial information for optional data elements  may be displayed on USAspending.gov. Because data limitations  related to optional data elements are not prominently displayed on  USAspending.gov, users may not know which data elements or  subelements are potentially incomplete.

A more systematic approach for identifying and disclosing known data  limitations on USAspending.gov—including procedures for addressing  wide ranging issues such as communicating changes in the reporting  requirements for certain data elements and information about data that  may be unreported or incomplete—could help users of the data better  understand potential quality issues with particular data elements and  sources, and how to appropriately interpret the data. While Treasury  has taken steps to better disclose data limitations, it needs to take further  action to implement a more systematic approach, in line with our 2017  recommendation.

In addition to such broader challenges, we identified two specific data  limitations involving DOD and HHS:

Delay in availability of DOD procurement data. A third issue we  identified in our 2017 review, and again in our current review,  concerns how information on DOD procurement data is presented on  USAspending.gov. Specifically, information related to a 90-day delay  in data availability for DOD procurement awards is not posted on  USAspending.gov. FPDS-NG—which collects information on contract  actions for display on USAspending.gov—releases DOD-reported  procurement data to the public after a 90-day waiting period to help  ensure the security of these data before they are released to the  public. This also results in a 90-day delay in reporting these data to  USAspending.gov. FPDS-NG clearly states that DOD data are subject  to a 90-day delay as seen in figure 6.

While DOD reports this data limitation in its senior accountable official  certification statement, it is not presented prominently to users who are  viewing DOD’s spending data. For example, DOD’s delay in data  availability is not presented on DOD’s agency profile page or with queries  on specific transactions associated with DOD. Until such information is  transparently communicated, users of USAspending.gov who access  DOD procurement data directly or as a result of broader government-wide  searches are likely unaware that the information may be incomplete or  not comparable.

Medicare payment data. Additionally, in this review we found  limitations in how Medicare payment data are made available to the  public. According to HHS officials, CMS reports the Primary Place of  Performance Address for Medicare payment data as the county for  the applicable Medicare Administrative Contractor (MAC) because the  MAC is the direct recipient of the agency’s contract award. As a result,  Medicare spending data on USAspending.gov are not reported in the  county where the Medicare beneficiaries are located. There are more  than 3,200 counties and county equivalents in the United States and  Puerto Rico, but only 20 Medicare MAC jurisdictions. Although  Medicare payments may reach every county in the country, the users  of USAspending.gov will only see this spending in the counties in  which a MAC is located. We found that this information is not  described on USAspending.gov. HHS officials said that they identified  this limitation to the transparency of Medicare payment data to  Treasury in 2016. They suggested that Treasury add information  about how Medicare payments are reported on USAspending.gov to  avoid confusion for users of the data. However, at that time, Treasury  determined that it was unnecessary to provide this additional  information on USAspending.gov. Until such information is  transparently communicated, it will be unclear to the user that  Medicare payments are consolidated in the counties where MACs are  located.

Fully Implementing Data Governance Consistent with Key Practices Would Improve Data Quality

Enforcing the Consistent Application of Data Standards across the Federal Government Would Improve Data Quality

One of the purposes of the DATA Act is to establish government-wide  data standards to provide consistent and comparable data that are  displayed accurately for taxpayers and policymakers on  USAspending.gov. As we have reported previously, establishing a data  governance structure—an institutionalized set of policies and procedures  for providing data governance throughout the life cycle of developing and  implementing data standards—is critical for ensuring that the integrity of  data standards is maintained over time. Such a structure, if properly  implemented, would greatly increase the likelihood that the data made  available to the public will be accurate.

Accordingly, in 2015, we recommended that OMB, in collaboration with  Treasury, establish a set of clear policies and procedures for developing  and maintaining data standards that are consistent with leading practices  for data governance. This recommendation has not been implemented.  Having formalized policies and procedures in place for one of these key  practices—managing, controlling, monitoring, and enforcing the  consistent application of data standards once they are established—could  help address some of the data quality challenges we identified in this and  previous reviews.

As described earlier, agencies experience challenges reporting Award  Description and Primary Place of Performance Address. We continue to  believe that having a robust data governance structure that includes  policies and procedures for enforcing the consistent application of the  established standards would lead to greater consistency and  comparability of reporting for data elements, such as Award Description  and Primary Place of Performance Address.

Efforts Continue to Develop a Robust Data Governance Structure to Ensure the Integrity of Data Standards

OMB and Treasury have established some procedures for governing the  data standards established under the DATA Act, but a robust governance  structure has yet to be fully developed and operational. Since the  enactment of the DATA Act in 2014, OMB has relied on a shifting array of  advisory bodies to obtain input on data standards. In March 2019, we  reported that the governing bodies involved in initial implementation  efforts had been disbanded, and that their data governance functions  were to be accomplished within the broader context of the cross-agency  priority (CAP) goals established under the 2018 President’s Management  Agenda (PMA). Since we issued our report, OMB has taken additional  steps to develop a government-wide data structure and to establish data  governance programs at each agency. OMB staff told us that they  envision agencies as incubators of data governance where they can learn  lessons on data governance. Toward that end, OMB, in collaboration with  other interagency groups, has taken a number of steps to further develop  data governance at both the agency and government-wide levels:  In October 2019, OMB issued a set of grants management data  standards under the Results Oriented Accountability for Grants CAP  Goal. According to OMB staff, they received more than 1,100 public  comments on draft standard data elements which were released for  public comment in November 2018.

OMB issued a memorandum in April 2019 that outlines approaches to  shared services and the governance structure established to support  shared services used for data reporting.

In June 2019, as part of the CAP Goal Leveraging Data as a Strategic  Asset, OMB issued the draft 2019-2020 Federal Data Strategy Action  Plan (Action Plan). This document identifies both government-wide  and agency-level action steps for improving data governance. To  address government-wide data governance, the Action Plan calls for  improvement in the standards for financial management data and  geospatial data. The Action Plan directs agencies to establish a body  of internal stakeholders responsible for data governance. These  bodies will be made up of senior level staff and be responsible for  assessing agency capability and ensuring monitoring and compliance  with policies and standards related to data. Agencies are also  instructed to assess data and related infrastructure maturity, identify  opportunities to increase staff data skills, and identify data needs to  answer key agency questions.

OMB also issued initial guidance in July 2019 to support agency  efforts to implement the first phase of the Evidence Act. For  example, the Evidence Act requires, among other things, agencies to  designate a Chief Data Officer by July 13, 2019. OMB also guidance  directs agencies to establish a data governance body, chaired by the  Chief Data Officer, with participation from relevant senior-level staff  from agency business units, data functions, and financial  management by September 30, 2019.

In July 2019, the Federal Data Strategy Team issued a data  governance playbook. According to OMB officials, this playbook is  not guidance, but is meant to be a framework for agency-level data  governance accompanied by forthcoming resources. OMB staff told  us that updates to the playbook would come relatively quickly, but  also said they had no planned time frames for doing so.

Agencies Have Taken Initial Steps to Implement Data Governance Programs and Data Quality Plans

Agencies have taken initial steps to establish data governance programs  and develop data quality plans. As of September 2019, seven of the 30  agencies included in our review reported that they have taken steps to  designate a Chief Data Officer as required by the Evidence Act. Twenty  reported establishing internal bodies similar to the data governance  bodies as directed by OMB guidance. The make-up and function of data  governance bodies varies across agencies. The Department of Labor  reported its Data Board was formalized and that the acting Chief Data  Officer had become the official Chief Data Officer. The U.S. Agency for  International Development reported establishing a DATA Act Governance  Council to facilitate the effective implementation of the DATA Act. Other  agencies reported similarly structured bodies referred to as working  groups, steering committees, and consortiums.

As of September 2019, 19 agencies reported that they have completed a  data quality plan as required by OMB Memorandum, M-18-16. Nine  agencies that do not have a data quality plan will have one completed by  September 30, 2019. The data quality plans from the agencies in our  sample varied in scope and content. Features of data quality plans we  reviewed included a description of a data governance board, an  assessment of existing and planned internal controls for data quality, and  determination of priority data elements based on assessments of risk of  data quality issues.

For example, the Departments of Commerce and the Interior each  conducted a risk assessment on the likelihood and consequence of  improper reporting for assistance and procurement data. They will employ  strategies or controls to mitigate risks related to the highest risk elements.  Similarly, Treasury named targeted data elements based on their  relevancy and further assessed the risk of improper reporting of each  element based on existing internal controls.

Agencies in our review reported using a variety of sources of guidance in  developing their data quality plans, including the Data Quality Playbook  issued by the Leveraging Data as a Strategic Asset Working Group in  November 2018, OMB Circular M-18-16, and guidance on conducting  required reviews under the DATA Act from the Council of inspector  general for Integrity and Efficiency. While some agencies in our review  reported that the information from these sources was helpful, they also  noted the need for additional guidance, including help understanding the  reporting requirements for certain data elements.

In the 5 years since enactment, OMB, Treasury, and federal agencies  have made significant strides to address many of the policy, technical,  and reporting challenges presented by the DATA Act’s requirements. We  found improvements in the overall quality of the data on  USAspending.gov compared to our 2017 review of data quality. To  continue moving forward with this progress and to fully realize the DATA  Act’s promise of helping to improve data accuracy and transparency,  more needs to be done to address continued challenges with the  completeness and accuracy of key data elements. For example, OMB  and Treasury have not fully addressed our recommendations to monitor  agency submissions and ensure agencies are accountable for the  completeness and accuracy of their data submissions.

In addition, without the transparent disclosure of known data limitations,  users may view, download, or analyze data made available on the  website without full knowledge of the extent to which the data are timely,  complete, accurate, or comparable over time. This could lead users to  inadvertently draw inaccurate information or conclusions from the data.  We have previously recommended that Treasury disclose known data  limitations on USAspending.gov. The agency has taken some steps  toward this goal. However, as we have shown, work remains for Treasury  to develop a more systematic approach for disclosing known data  limitations on its website. In the meantime, we believe it is important to  address the specific data limitations we identify in this report. These  include the need to provide users with information about the delay in the  availability of DOD procurement data, and how Medicare payment data  are reported.

Finally, the challenges we have found with data completeness and  accuracy, and the transparency around data limitations also demonstrate  the importance of continued progress by OMB and Treasury in  addressing our previous open recommendations to develop a robust and  transparent data governance structure, and implement controls for  monitoring agency compliance with DATA Act requirements.

We maintain that OMB and Treasury should address our prior  recommendations on DATA Act implementation, including  recommendations on monitoring agency submissions, providing  additional guidance on reporting established data standards,  implementing a systematic approach to facilitate the disclosure of known  data limitations on USAspending.gov, and developing a robust and  transparent governance structure. We are making a total of two new  recommendations to Treasury regarding the disclosure on  USAspending.gov of specific known data limitations:  The Secretary of the Treasury should ensure that information about the  90-day delay for displaying DOD procurement data on USAspending.gov  is transparently communicated to users of the site. Approaches for doing  this could include prominently displaying this information on the DOD  agency profile page, in the unreported data section, and in search results  that include DOD data. (Recommendation 1)

The Secretary of the Treasury should ensure that information regarding  how the Primary Place of Performance Address for Medicare payment  data are reported is transparently communicated to the users of  USAspending.gov. (Recommendation 2)

We provided a draft of this report to the Departments of Agriculture  (USDA), Defense (DOD), Commerce, Education, Health and Human  Services (HHS), Homeland Security, the Interior (DOI), Labor (DOL), the  Treasury, and Veterans Affairs (VA); the Office of Management and  Budget (OMB); the National Science Foundation (NSF); the National  Aeronautics and Space Administration (NASA); the Small Business  Administration (SBA); the U.S. Agency for International Development  (USAID); the U.S. Agency for Global Media (USAGM); and the Delta  Regional Authority (DRA) for review and comment. USAID and Treasury  provided written responses, which are summarized below and reproduced  in appendixes VII and VIII, respectively. DHS and OMB provided  technical comments, which we incorporated as appropriate. USDA, DOD,  Commerce, Education, HHS, DOI, DOL, VA, NSF, NASA, SBA, USAGM,  and DRA had no comments on the draft report.

In its written comments, USAID stated that it is committed to DATA Act  reporting and the accessibility and transparency of its spending data. In  its written comments, Treasury stated its commitment to fully realizing the  DATA Act’s promise of helping to improve data accuracy and  transparency. Treasury agreed with our two recommendations on the  disclosure of specific known data limitations and stated that it will work  with HHS and DOD to implement them in the coming months. Treasury  also stated that it remains committed to fully implementing our prior  recommendations on DATA Act implementation.

We are sending copies of this report to the relevant congressional  committees; the Secretaries of Agriculture, Defense, Commerce,  Education, Homeland Security, the Interior, Labor, the Treasury, and  Veterans Affairs; the Directors of the Office of Management and Budget  and the National Science Foundation; the Administrators of National  Aeronautics and Space Administration, the Small Business  Administration, and U.S. Agency for International Development; the Chief  Executive Officer of the U.S. Agency for Global Media; the Chairman of  the Delta Regional Authority; and other interested parties. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff has any questions about this report, please contact  Michelle Sager at (202) 512-6806 or sagerm@gao.gov or Paula M.  Rascona at (202) 512-9816 or rasconap@gao.gov. Contact points for our   Offices of Congressional Relations and Public Affairs may be found on  the last page of our report. Key contributors to this report are listed in app.  IX.

Appendix I: List of Agencies and Number of Transactions in Our Sample

National Science Foundation  Nuclear Regulatory Commission (NRC)

The Broadcasting Board of Governors changed its name to the U.S. Agency for Global Media in  August 2018.

Appendix II: Objectives, Scope, and Methodology

The Digital Accountability and Transparency Act of 2014 (DATA Act)  requires that we report on the timeliness, completeness, accuracy, and  quality of the data submitted under the act and the implementation and  use of data standards. This review responds to the act’s requirement by  addressing the following: (1) the timeliness, completeness, accuracy, and  quality of the data and the implementation and use of data standards; and  (2) the extent to which progress has been made to develop a data  governance structure consistent with key practices, and how it affects  data quality. We also update the status of select implementation issues  and our previous recommendations related to implementing the DATA Act  and data transparency.

To assess the timeliness, completeness, accuracy, and quality of the data  submitted and the implementation and use of data standards, we  analyzed agency submission files for the fourth quarter of fiscal year 2018  (Q4 FY2018) on USAspending.gov and reviewed a representative  stratified random sample from the Department of the Treasury’s  (Treasury) USAspending.gov database download for Q4 FY2018.

Specifically, to assess timeliness, we accessed agency submission files  on USAspending.gov for Q4 FY2018 and determined whether agencies  submitted their data by the established deadline—45 days after the end of  the quarter or November 14, 2018—based on the date agencies certified  their submissions. To help understand the proportion of spending that  agencies reported by the due date, we obtained and analyzed a file from  Treasury containing SF 133 Report on Budget Execution and Budgetary  Resources (SF 133) data—which includes unaudited balances reported  by agencies—for Q4 FY2018. These obligation balances are only used  for illustrative purposes in our report. They include financing accounts,  among other things, which are not required to be reported under the  DATA Act.

To assess completeness, we determined whether (1) all agencies that  determined they are required to or would voluntarily submit DATA Act  files did so, (2) the transactions reported in the files submitted by  agencies contained all required data for that transaction, and (3) the  database contained required assistance award data from the 24 Chief  Financial Officers Act of 1990 (CFO Act) agencies. To determine  whether all agencies that should have reported Q4 FY2018 data did so,  we compared Treasury’s list of agencies that determined they were  required to or would voluntarily report data to the agency file submissions  on USAspending.gov for Q4 FY2018. We followed up with agencies that  had not reported to find out the reasons for not reporting, but we did not  verify the accuracy of their responses.

To assess the completeness of files submitted by agencies, we accessed  the agency submission files for Q4 FY2018 available on  USAspending.gov and determined whether all files for each agency  contained data (i.e., were not blank). We followed up with agencies that  submitted a blank File C and/or File D1 that did not contain any data to  find out why the files were blank, but we did not verify the accuracy of  their responses. We also made inquiries of agencies to determine  whether any agency components or systems did not submit data. Finally,  we tested completeness of agency submissions through our sample  testing, described in detail below.

To assess the completeness of assistance data in the USAspending.gov  database, we determined the extent to which federal agencies were  reporting required award data based on a list of potential award-making  agencies/programs from Assistance Listings on beta.SAM.gov, formerly  the Catalog of Federal Domestic Assistance. We identified all programs  listed in the Assistance Listings, as of September 2018. For the 24 CFO  Act agencies only, we compared programs listed in the Assistance  Listings to data in the USAspending.gov database to determine which  programs reported information on at least one assistance award for fiscal  year 2018. For any program reporting no assistance award information for  the year, we asked agency officials why information was not reported. For  all programs that agency officials determined either made an award but  did not report it, or reported awards late to USAspending.gov, we  extracted the agencies’ obligation estimates for fiscal year 2018 as  reported in the Assistance Listings.

To further assess completeness of the data and to assess accuracy of the  data and the implementation and use of data standards, we extracted all  records included in the scope of our review from a database used to  display data on USAspending.gov. The records covered activity during  Q4 FY2018 (July through September 2018). To extract all records from  the database, we mapped the database fields to the data elements within  the scope of our audit.

Once we had the data within the scope of our audit for Q4 FY2018, we  performed the following steps:

Sampling data to determine completeness and accuracy: From  the database we extracted, we selected a stratified random probability  sample of 405 records for Q4 FY2018. Data records were stratified  into procurement award transactions, assistance award transactions,  and budgetary records. We randomly selected 158 procurement  awards, 150 financial awards, and 97 budgetary records. Estimates  for the results of the procurement, assistance, and budgetary samples  have sampling errors of +/- 7.8, 8, and 10 percentage points or less,  respectively, at the 95 percent level of confidence. The probability  sample was designed to estimate the overall rate of reporting errors  for a data element with a sampling error of no greater than plus or  minus 5.3 percentage points at the 95 percent level of confidence.  Because we followed a probability procedure based on random  selections, our sample is only one of a large number of samples that  we might have drawn. Since each sample could have provided  different estimates, we express our confidence in the precision of our  particular sample’s results as a 95 percent confidence interval (e.g.,  +/- 7 percentage points). This is the interval that would contain the  actual population value for 95 percent of the samples we could have  drawn. For 41 data elements and subelements required by FFATA or  the DATA Act, we first assessed the extent to which a data element  was complete—whether there was a value and if that value was  appropriate. If the data element was not complete, then we also  considered that data element to not be accurate. For those elements  that were complete, we then assessed the extent to which the data  were accurate by comparing the information in our sample to the  information contained in the originating agency’s underlying source  documents, where available, and determining whether the data were  consistent with applicable laws and reporting standards, as  applicable. Therefore we determined an element was inconsistent if it  was either inconsistent with the agency documents, applicable laws or  reporting standards, or incomplete. For three data elements that  contained values derived by Federal Procurement Data System-Next  Generation (FPDS-NG) and Financial Assistance Broker Submission  (FABS) based on other values provided by agencies, we compared  the information in the sample to other sources, such as data from the  U.S. Census Bureau and house.gov. This allowed us to verify  whether the values in our sample were consistent with the systems  from which they were derived. We then interviewed agency officials to  discuss differences between the information in our sample and  information in agency or other sources.

Data element and subelement testing: Table 3 shows the 44 data  elements and subelements tested in the statistical sample—including  six budgetary data elements and 38 award data elements and  subelements. Individual data elements may vary with their  representation in the sample (e.g. Legal Entity Address Lines 1 and 2)  because the data element was not required for all of the sampled data  records. Specific error rates by category can be found in app. III.

The government-wide results are a weighted total of the three strata of  our sample: (1) procurement award transactions, (2) assistance award  transactions, and (3) budgetary records. For reporting purposes, we  combined some of the results for the award strata because some data  elements appear in both Files D1 (procurement) and D2 (financial  assistance). See app. I for the list of agencies and number of records  randomly selected and tested in each strata.

If we determined, after reviewing agency source documents, that a data  element was not applicable to the sampled record, we did not factor the  data element into our evaluation of completeness and accuracy. We  determined an element to be unverifiable if no agency source records  were provided or the records provided did not meet our audit standards.

To test the controls over the reliability of agency data, we obtained  supporting documentation to confirm that the agency provided only official  agency source documents, such as a system of records notice. When  such a supporting document was unavailable, we reviewed agency  transparency policy documentation, data verification and validation plans  or procedures, or system source code information to ensure the reliability  of the data. We did not assess the accuracy of the data contained in  sources provided by agencies. For the purposes of our review, we  defined data quality as encompassing the concepts of timeliness,  completeness, and accuracy. Therefore, our assessment of overall data  quality is reflected in our specific assessments of these components.

We also reviewed OMB, Treasury, and agency documents related to  DATA Act implementation. We interviewed OMB and Treasury officials on  their role in DATA Act implementation and interviewed officials from the  agencies in our sample to discuss their test results and efforts to submit  data under the DATA Act.

To describe changes in data quality since our prior work, we compared  the results of our review of Q4 FY2018 data to the results of our review of  quarter two fiscal year 2017 (Q2 FY2017) data performed in our first  assessment of data quality. For both reviews, we examined a projectable  sample of budgetary and award transactions from a database that,  according to Treasury, is partly used to display data on  USAspending.gov. However, there were the following differences: (1) our  2017 sampling frame was confined to the 24 CFO Act agencies (which  represented 99 percent of obligations in our data set at that time), while  our sampling frame for this review included all agencies that submitted  Q4 FY2018 data files as of February 11, 2019; (2) more agencies and  their components reported data in Q4 FY2018 than in Q2 FY2017; (3) in  2017 our estimated error rate calculations included elements of certain  sampled transactions that were determined to be not applicable to the  transaction and were classified as consistent with agency sources in both  the numerator and denominator while in this review, we excluded not  applicable elements from both the numerator and denominator of the  estimated rate calculations; (4) our sampling frame for this review  included more data elements and subelements than were in our Q2  FY2017 sampling frame; (5) in this review, since three data elements we  reviewed were derived by FPDS-NG and FABS rather than provided by  agencies, we compared the information in the sample to other sources  rather than agency documents and therefore did not include those results  in our comparisons to Q2 FY2017; (6) agencies’ Q4 FY2018 data were  submitted under policies and procedures outlined in DAIMS v1.3 which  reflects changes in validation rules and reporting requirements from the  DAIMS v1.0 that was in effect in 2017; (7) OMB issued additional  guidance on DATA Act reporting since we reported in 2017; and (8)  changes were made to the Treasury broker since our last report.

To evaluate how the current data governance structure affects data  quality, we compared data quality challenges we identified during our  review to key practices for data governance identified in our prior work to  underscore the need for a more robust structure consistent with key  practices. To assess progress made to develop a data governance  structure consistent with key practices, we reviewed policy and other  documentation related to ongoing efforts to develop a government-wide  structure for governing the standards established under the act and  interviewed OMB staff about these efforts. We also reviewed agency data  quality plans—guidance intended to facilitate agency efforts to establish  data governance programs—and interviewed agency officials on their  data governance efforts.

To update the status of our recommendations related to the  implementation of the DATA Act, we reviewed new guidance and other  related documentation, and interviewed OMB staff and Treasury officials.  See app. IV for an update on our recommendations related to DATA Act  implementation.

We conducted this performance audit from November 2018 to November  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Appendix III: Estimates of Consistency Rates for Award Transactions and Budgetary Accounts/Balances

Appendix III: Estimates of Consistency Rates  for Award Transactions and Budgetary  Accounts/Balances Accurate/consistent (%)  Q4 FY2018  Q2 FY2017  Q4 FY2018  Q2 FY2017  Q4 FY2018  Q2 FY2017  3-8 97-100 83-91 0-1 5-11 0-3  Inconsistent (%)

Catalog of Federal Domestic Assistance  Number (CFDA)

Data element  Primary Place of Performance Address (all  subelements)

Legal Entity Address City Name refers to two subelements under DAIMS v.1.3 (Legal Entity Address  City Name and Foreign City Name), which we combined for reporting purposes.  Legal Entity Address State Name refers to three subelements under DAIMS v.1.3 (Legal Entity  Address State Description for procurement awards and Legal Entity Address State Name and Foreign  Province Name for financial assistance awards), which we combined for reporting purposes.  Legal Entity Address Zip Code refers to four subelements under DAIMS v.1.3 (Legal Entity Address  Zip+4 for procurement awards, Legal Entity Address Zip 5 and Last 4 for financial assistance awards,  and Legal Entity Address Foreign Postal Code for foreign financial assistance awards), which we  combined for reporting purposes.  Primary Place of Performance Address Zip Code is one subelement under DAIMS v.1.3 (Primary  Place of Performance Address Zip+4), which contains both the first five digits from the zip code and  the last 4. However, the USAspending.gov database we obtained our sample from contained the zip  code information for this element in two parts: 5 digit zip code and +4. Therefore, we present these  subelements separately for reporting purposes.  Element was optional for fourth quarter of fiscal year 2018.

Unverifiable includes data elements rates as inaccurate because agency records were insufficient to  complete the test or because the agency did not provide supporting documentation.

In our prior Digital Accountability and Transparency Act of 2014 (DATA  Act) reports, we have made recommendations to both the Department of  the Treasury (Treasury) and the Office of Management and Budget  (OMB) on a range of topics. Treasury and OMB have collectively taken  action that resulted in closure of nine prior recommendations on the data  transparency and implementation of the DATA Act. Table 7 provides a  listing of open DATA Act recommendations at the time this report was  issued as well as a short discussion of their status. Full and effective  implementation of the open recommendations listed below will contribute  to more reliable and consistent federal data to measure the cost and  magnitude of federal investments as well as facilitate efforts to share data  across agencies to improve transparency, accountability, decision- making, and oversight.

Appendix V: Sources of Data and Process Overview on USAspending.gov

The Digital Accountability and Transparency Act of 2014 (DATA Act)  requires the Office of Management and Budget (OMB) and the  Department of the Treasury (Treasury) to establish government-wide data  standards that to the extent reasonable and practicable produce  consistent, comparable, and searchable spending data for any federal  funds made available to or expended by federal agencies. These  standards specify the data elements to be reported under the DATA Act  and define and describe what is to be included in each data element, with  the aim of ensuring that data will be consistent and comparable. The  DATA Act requires OMB and Treasury to ensure that the standards are  applied to the data made available on USAspending.gov which has many  sources of data. Some data are from agency systems, while other data  are pulled or derived from government-wide reporting systems. Key  award systems that generate data files that are linked to agency  submitted files include  the Federal Procurement Data System-Next Generation (FPDS-NG),  which collects information on contract actions;  the Financial Assistance Broker Submission (FABS) which collects  information on financial assistance awards;  the System for Award Management which is the primary database for  information on entities that do business with the federal government  (i.e., contractors and grantees), and in which such entities must  register; and  the Federal Funding Accountability and Transparency Act of 2006  (FFATA) Subaward Reporting System (FSRS), which provides data  on first-tier subawards reported by prime award recipients.

Agencies submit procurement award information to FPDS-NG daily and  financial assistance award information (grants, loans, insurance and other  financial assistance) to FABS at least twice monthly. These award data  are reflected in USAspending.gov daily. As depicted in figure 7, agencies  are expected to submit financial data linked to award data and certified on  a quarterly basis, 45 days after the close of the quarter. They submit  three data files with specific details and data elements to Treasury’s  DATA Act Broker (broker) from their financial management systems  quarterly (Files A, B, C). In February 2019, to reduce agency burden,  Treasury made updates including an optional new broker feature that  agencies can use to generate a provisional File A which agencies can  choose to upload and submit as their File A in the regular submission  process. The new feature produces an agency’s provisional File A based  on budget and financial information reported by the agency to the  Government-wide Treasury Account Symbol Adjusted Trial Balance  System for the creation of the SF 133 Report on Budget Execution and  Budgetary Resources. The broker then extracts award and subaward  information from existing government-wide reporting systems to build four  files that include procurement information, information on federal  assistance awards such as grants and loans, and recipient information  (Files D1, D2, E, and F).

Each agency’s data must pass a series of validations in the broker and  then be certified by the agency’s senior accountable official (SAO) before  they are submitted for display on USAspending.gov. According to OMB  guidance, the purpose of the SAO certification is to provide reasonable  assurance that the agency’s internal controls support the reliability and  validity of the data submitted to Treasury for publication on the website.  The SAO assurance means that, at a minimum, the data reported are  based on appropriate controls and risk management strategies as  described in OMB Circular A-123, Management’s Responsibility for  Enterprise Risk Management and Internal Control. In addition, agencies  should include information about any data limitations in their SAO  certification statements.

Appendix VI: Agencies That Submitted Data for Quarter Four of Fiscal Year 2018

Committee for Purchase from People Who  Are Blind or Severely Disabled (AbilityOne  Commission)

Appendix VII: Comments from the U.S. Agency for International Development

Appendix VIII: Comments from the Department of the Treasury

Appendix IX: GAO Contacts and Staff Acknowledgments

In addition to the above contacts, Peter Del Toro (Assistant Director),  Kathleen Drennan (Assistant Director), Michael LaForge (Assistant  Director), Maria C. Belaval (Auditor-in-Charge), Barbara Lancaster  (Analyst-in-Charge), Diane Morris (Auditor-in-Charge), Carl Barden,  Daniel Berg, Mark Canter, Jenny Chanley, Shelby Clark, Tracy Davis  Ross, Tabitha Fitzgibbon, Valerie Freeman, Jamaika Hawthorne, Michael  Kany, Roy Kilgore, Peter Kramer, Seraé LaFache-Brazier, Krista Loose,  Tonyita Muschette, Quang Nguyen, Kristine Papa, Joseph Raymond, Lisa  Rowland, Susan Sato, John A. Schaefer, Sara Shore, James Skornicki,  Andrew J. Stephens, James Sweetman, Jr., Silvia Symber, and Lisa Zhao  made key contributions to this report. Additional members of GAO’s  DATA Act Internal Working Group also contributed to the development of  this report.

DATA Act: Customer Agencies’ Experiences Working with Shared  Service Providers for Data Submissions. GAO-19-537. Washington, D.C.:  July 18, 2019.

DATA Act: Pilot Effectively Tested Approaches for Reducing Reporting  Burden for Grants but Not for Contracts. GAO-19-299. Washington, D.C.:  April 30, 2019.

DATA Act: OMB Needs to Formalize Data Governance for Reporting  Federal Spending. GAO-19-284. Washington, D.C.: March 22, 2019.

Open Data: Treasury Could Better Align USAspending.gov with Key  Practices and Search Requirements. GAO-19-72. Washington, D.C.:  December 13, 2018.

DATA Act: Reported Quality of Agencies’ Spending Data Reviewed by  OIGs Varied Because of Government-wide and Agency Issues.  GAO-18-546. Washington, D.C.: July 23, 2018.

DATA Act: OMB, Treasury, and Agencies Need to Improve Completeness  and Accuracy of Spending Data and Disclose Limitations. GAO-18-138.  Washington, D.C.: November 8, 2017.

DATA Act: As Reporting Deadline Nears, Challenges Remain That Will  Affect Data Quality. GAO-17-496. Washington, D.C.: April 28, 2017.

DATA Act: Office of Inspector General Reports Help Identify Agencies’  Implementation Challenges. GAO-17-460. Washington, D.C.: April 26,  2017.

DATA Act: Implementation Progresses but Challenges Remain.  GAO-17-282T. Washington, D.C.: December 8, 2016.

DATA Act: OMB and Treasury Have Issued Additional Guidance and  Have Improved Pilot Design but Implementation Challenges Remain.  GAO-17-156. Washington, D.C.: December 8, 2016.

DATA Act: Initial Observations on Technical Implementation.  GAO-16-824R. Washington, D.C.: August 3, 2016.

DATA Act: Improvements Needed in Reviewing Agency Implementation  Plans and Monitoring Progress. GAO-16-698. Washington, D.C.: July 29,  2016.

DATA Act: Section 5 Pilot Design Issues Need to Be Addressed to Meet  Goal of Reducing Recipient Reporting Burden. GAO-16-438. Washington,  D.C.: April 19, 2016.

DATA Act: Progress Made but Significant Challenges Must Be Addressed  to Ensure Full and Effective Implementation. GAO-16-556T. Washington,  D.C.: April 19, 2016.

DATA Act: Data Standards Established, but More Complete and Timely  Guidance Is Needed to Ensure Effective Implementation. GAO-16-261.  Washington, D.C.: January 29, 2016.

Federal Spending Accountability: Preserving Capabilities of Recovery  Operations Center Could Help Sustain Oversight of Federal  Expenditures. GAO-15-814. Washington, D.C.: September 14, 2015.

DATA Act: Progress Made in Initial Implementation but Challenges Must  be Addressed as Efforts Proceed. GAO-15-752T. Washington, D.C.: July  29, 2015.

Federal Data Transparency: Effective Implementation of the DATA Act  Would Help Address Government-wide Management Challenges and  Improve Oversight. GAO-15-241T. Washington, D.C.: December 3, 2014.

Government Efficiency and Effectiveness: Inconsistent Definitions and  Information Limit the Usefulness of Federal Program Inventories.  GAO-15-83. Washington, D.C.: October 31, 2014.

Data Transparency: Oversight Needed to Address Underreporting and  Inconsistencies on Federal Award Website. GAO-14-476. Washington,  D.C.: June 30, 2014.

Federal Data Transparency: Opportunities Remain to Incorporate  Lessons Learned as Availability of Spending Data Increases.  GAO-13-758. Washington, D.C.: September 12, 2013.

Government Transparency: Efforts to Improve Information on Federal  Spending. GAO-12-913T. Washington, D.C.: July 18, 2012.

Electronic Government: Implementation of the Federal Funding  Accountability and Transparency Act of 2006. GAO-10-365. Washington,  D.C.: March 12, 2010.

The DATA Act seeks to improve the quality and comparability of federal spending data available to Congress and the public, and requires it to be posted on USAspending.gov.

We found that data quality on USAspending.gov has improved since our 2017 review. However, varying interpretations of data standards lead to persistent data quality issues. Additionally, known data limitations are not fully disclosed.

To help ensure that users have the best quality data possible and can understand what they're working with, we restated our prior recommendations on these issues and made 2 new ones to Treasury on disclosing data limitations.