Small Business Administration Contracting Programs: Additional Efforts Needed to Implement GAO Recommendations

Federal agencies conduct a variety of procurements that are reserved for small business participation through small business set-asides. These set-asides can be for small businesses in general, or they can be specific to small businesses that meet additional eligibility requirements in programs such as those for WOSB or HUBZone. SBA administers both the WOSB and HUBZone programs. SBA also produces an annual Small Business Procurement Scorecard to measure how much contracted spending federal agencies allocate to small businesses and whether the federal government is meeting its goals for awarding contracts to small businesses.

GAO issued three reports between September 2018 and March 2019 on SBA contracting programs (see GAO-18-666 , GAO-18-672 , and GAO-19-168). This testimony is primarily based on these three reports and discusses prior GAO findings and SBA's progress on implementing GAO's recommendations on (1) the WOSB program, (2) the HUBZone program, and (3) SBA's procurement scorecard.

To update the status of prior recommendations, GAO reviewed updates from SBA and interviewed officials.

The Small Business Administration (SBA) has not fully implemented GAO's prior recommendations to address oversight deficiencies in the Women-Owned Small Business (WOSB) and Historically Underutilized Business Zone (HUBZone) programs and to improve evaluation of its procurement scorecard. GAO maintains that its recommendations should be addressed.

Women-Owned Small Business Program. In its March 2019 report, GAO found that SBA had not addressed WOSB program oversight deficiencies identified in GAO's 2014 report (GAO-15-54). For example, GAO had found that SBA did not have procedures related to reviewing the performance of the four third-party certifers—private entities approved by SBA to certify the eligibility of WOSB firms—as well as information the certifiiers submitted to SBA. GAO recommended that SBA establish procedures to assess the performance of the certifiers and the information they submitted. While SBA conducted a compliance review of the certifiers in 2016, SBA said in June 2018 that it had no plans to conduct further compliance reviews until a final rule implementing a new certification process was completed. SBA officials said that they expected the rule to be implemented by June 2021. By waiting to improve its oversight of the WOSB program, SBA cannot provide reasonable assurance that certifiers are complying with program requirements and cannot improve its efforts to identify ineligible firms or potential fraud.

HUBZone Program. In September 2018, GAO reported that it had reviewed case files for a nongeneralizable sample of 12 firms in Puerto Rico that received HUBZone certification between March 2017 and March 2018 and found that SBA did not consistently document or follow its policies and procedures for certification reviews. For example, SBA did not have complete documentation in nine of 12 cases and did not follow its policy to conduct three levels of review when determining whether to approve or deny a firm in four of 12 cases. As a result, SBA did not have reasonable assurance that firms meet HUBZone criteria. SBA said that it planned to implement GAO's recommendations that SBA (1) update internal policy manuals for certification and recertification and (2) conduct and document reviews of staff compliance with relevant procedures. However, as of May 2019, SBA had not provided documentation showing that it had completed these planned actions.

Small Business Procurement Scorecard. For fiscal year 2017, SBA revised the methodology for its Small Business Procurement Scorecard, which assesses the efforts of federal agencies to support contracting with small businesses. For example, one revision reduced the share of the total scorecard grade devoted to prime contracting achievement (the dollar amount of contracts awarded directly to small businesses). GAO recommended in September 2018 that SBA design and implement a comprehensive evaluation to assess the scorecard revisions. Since that report was issued, SBA has proposed but not yet implemented a two-phase evaluation of the scorecard to include an evaluation of the scorecard's effect on federal agencies achieving small business contracting goals. SBA said that it expects to complete phase one by September 2019 and has not provided a time frame for phase two.

I am pleased to be here today to discuss our recent work on the Small  Business Administration’s (SBA) contracting programs and SBA’s  progress in implementing our recommendations. Federal agencies  conduct a variety of procurements that are reserved for small business  participation through small business set-asides. These set-asides can be  for small businesses in general, or they can be specific to small  businesses that meet additional eligibility requirements, such as those for  SBA’s Women-Owned Small Business (WOSB) or Historically  Underutilized Business Zones (HUBZone) programs. SBA also produces  an annual Small Business Procurement Scorecard (scorecard) to  measure how much contracted spending federal agencies allocate to  small businesses and whether the federal government is meeting its  goals for awarding contracts to small businesses.

My testimony today is based primarily on three reports we issued  between September 2018 and March 2019, as well as a report we issued  in October 2014. I will discuss SBA’s progress on implementing our prior  recommendations on (1) the WOSB program, (2) the HUBZone program  in Puerto Rico, and (3) SBA’s procurement scorecard.

To conduct the work on the WOSB program, we reviewed relevant laws,  regulations, and program documents; analyzed federal contracting data  from April 2011 through June 2018; and interviewed SBA officials,  officials from contracting agencies selected to obtain a range of  experience with the WOSB program, and three of the four private third- party certifiers. For our HUBZone report, we analyzed SBA documents  and reviewed files of a nongeneralizable sample of 12 firms located in  Puerto Rico that received certification between March 2017 and March  2018. We also interviewed SBA officials, representatives from HUBZone- certified firms in Puerto Rico, and local economic development agencies  in Puerto Rico. To conduct the work on SBA’s procurement scorecard, we  analyzed SBA’s prior and revised scorecard methodology and results and  interviewed officials from SBA, four other federal agencies selected based  on small business procurement volume and other attributes, and three  groups representing the interests of small businesses. To update the  status of recommendations from these reports, we reviewed updates from  SBA and interviewed officials. We conducted the work on which this  testimony is based in accordance with generally accepted government  auditing standards. More details on our methodology can be found in the  three reports on which this testimony is based.

SBA Has Not Fully Addressed Deficiencies in Oversight and Implementation for the WOSB Program

SBA has not fully addressed deficiencies we have previously identified for  the WOSB program, and these deficiencies are affected by SBA’s  ongoing implementation of changes to the program authorized by the  National Defense Authorization Act of 2015 (2015 NDAA). As of early  June 2019, SBA had implemented one of the three changes to the  program authorized in the 2015 NDAA. Specifically, in September 2015  SBA published a final rule to implement sole-source authority (to award  contracts without competition), effective October 2015. The two other  changes—authorizing SBA to implement its own certification process for  WOSBs and requiring SBA to eliminate the option for firms to self-certify  that they are eligible for the WOSB program—had not been  implemented. On May 14, 2019, SBA published in the Federal Register a  proposed rule that eliminates the self-certification option and describes a  potential certification process to be administered by SBA. SBA officials  have stated that the agency will not eliminate self-certification until the  new certification process for the WOSB program is in place, which they  expect to implement by June 2021.

In addition, SBA has not fully addressed WOSB program oversight  deficiencies described in our March 2019 report and first identified in our  2014 report. We reported that SBA did not have formal policies for  reviewing the performance of its four approved third-party certifiers  (private entities approved by SBA to certify the eligibility of WOSB firms),  including their compliance with their agreements with SBA. Further, we  found that SBA had not developed formal policies and procedures for,  among other things, reviewing the monthly reports that certifiers submit to  SBA. We recommended that the Administrator of SBA establish  comprehensive procedures to monitor and assess the performance of the  third-party certifiers in accordance with their agreements with SBA and  program regulations. While SBA has taken some steps to address our  recommendation, including conducting a compliance review of the  certifiers in 2016, SBA officials said in June 2018 that SBA had no plans  to conduct further compliance reviews until the final rule implementing the  new certification process was completed. By waiting to improve its  oversight of the WOSB program, SBA cannot provide reasonable  assurance that certifiers are complying with program requirements and  cannot improve its efforts to identify ineligible firms or potential fraud. In  addition, the implementation of sole-source authority in light of these  continued oversight deficiencies can increase program risk.  Consequently, we maintain that our recommendation should be  addressed.

SBA also has not fully addressed deficiencies related to eligibility  examinations that we described in our March 2019 report and first  identified in our October 2014 report. We found that SBA lacked  formalized guidance for its eligibility examination processes and that the  examinations identified high rates of potentially ineligible businesses. As  a result, we recommended that SBA enhance its examination of  businesses that register for the WOSB program to ensure that only  eligible businesses obtain WOSB set-asides. Specifically, we suggested  that SBA should take actions such as (1) completing the development of  procedures to conduct annual eligibility examinations and implementing  such procedures; (2) analyzing examination results and individual  businesses found to be ineligible to better understand the cause of the  high rate of ineligibility in annual reviews and determine what actions are  needed to address the causes, and (3) implementing ongoing reviews of  a sample of all businesses that have represented their eligibility to  participate in the program.

SBA has taken some steps to implement our recommendation, such as  including written policies and procedures for WOSB program eligibility  examinations in a standard operating procedure and a Desk Guide.  However, SBA does not collect reliable information on the results of its  annual eligibility examinations. In addition, SBA continues to have no  mechanism to look across examinations for common eligibility issues to  inform the WOSB program. As we noted in 2014, by not analyzing  examination results broadly, the agency is missing opportunities to obtain  meaningful insights into the program, such as the reasons many  businesses are deemed ineligible.

Further, SBA still conducts eligibility examinations only of firms that have  already received a WOSB award. Restricting the samples in this way  limits SBA’s ability to better understand the eligibility of businesses before  they apply for and are awarded contracts, as well as its ability to detect  and prevent potential fraud.

We recognize that SBA has made some effort to address our  recommendation by documenting procedures for conducting annual  eligibility examinations of WOSB firms. However, without maintaining  reliable information on the results of eligibility examinations, developing  procedures for analyzing results, and expanding the sample of  businesses to be examined to include those that did not receive  contracts, SBA limits the value of its eligibility examinations and its ability  to reduce ineligibility among businesses registered to participate in the  WOSB program. Leading fraud risk management practices state that  federal program managers should design control activities that focus on  fraud prevention over detection and response, to the extent possible.  The deficiencies in SBA’s oversight of the WOSB program limit SBA’s  ability to identify potential fraud risks and develop any additional control  activities needed to address these risks. As a result, the program may  continue to be exposed to the risk of ineligible businesses receiving set- aside contracts. In addition, in light of these continued deficiencies, the  implementation of sole-source authority without addressing the other  changes made by the 2015 NDAA could increase program risk. For these  reasons, we maintain that our previous recommendation that SBA  enhance its WOSB eligibility examination procedures should be  addressed.

In addition, similar to previous findings from SBA’s Office of Inspector  General, our March 2019 report found that about 3.5 percent of contracts  using a WOSB set-aside were awarded for ineligible goods or services  from April 2011 through June 2018. At that time, SBA was not reviewing  contracting data that could identify this problem and inform SBA which  agencies making awards may need targeted outreach or training. As a  result, we found that SBA could not provide reasonable assurance that  WOSB program requirements were being met and that the program was  meeting its goals. We recommended that SBA develop a process for  periodically reviewing the extent to which WOSB program set-asides are  awarded for ineligible goods or services and use the results to address  identified issues, such as through targeted outreach or training to  agencies making awards under the ineligible codes. In early May 2019,  SBA said that it had initiated such efforts.

SBA Has Not Yet Implemented Recommendations to Improve the HUBZone Certification Process

In September 2018, we found that although SBA had adopted criteria and  guidance for a risk-based approach to certifying and recertifying firms for  the HUBZone program in March 2017, the extent to which it conducted a  risk assessment to inform its approach was unclear. In 2015, we found  that SBA lacked key controls for its recertification process and  recommended that SBA assess the process. In 2009, SBA increased  documentation requirements for certification but not recertification (which  determines continued program eligibility every 3 years). In March 2017,  SBA changed its recertification criteria and guidance to require firms with  $1 million or more in HUBZone contract awards to provide documentation  to support continuing eligibility. During our work for the September 2018  report, SBA officials stated they had completed a risk assessment of the  HUBZone recertification process, but did not provide us with  documentation on when they performed the risk assessment, which risks  were identified and considered, or what analysis established the $1  million threshold. As of May 2019, SBA had not provided documentation  showing that it had performed the risk assessment, but we maintain that  an assessment of the recertification process would help inform a risk- based approach to reviewing and verifying information from firms that  appear to pose the most risk to the program. In addition, SBA had not  provided documentation showing that a technology-based solution  designed to address some of the ongoing challenges with the  recertification process had been implemented. SBA officials had  previously estimated this solution would be available first in spring 2017  and then by the end of calendar year 2017.

We also found in our September 2018 report that, based on our review of  case files for a nongeneralizable sample of 12 firms in Puerto Rico that  received HUBZone certification between March 2017 and March 2018,  SBA did not consistently document or follow its policies and procedures  for certification reviews:

SBA did not have complete documentation in nine of 12 cases. SBA  officials described alternative procedures they used to determine  firms’ eligibility, but SBA had not updated its internal policy manuals to  reflect these procedures, and analysts did not document use of such  procedures in the files we reviewed. As a result, SBA did not have  reasonable assurance that firms met HUBZone criteria.

In four of 12 cases, SBA did not follow its policy to conduct three  levels of review (by an analyst, a senior analyst, and the program  director or deputy) when determining whether to approve or deny a  firm.

It was unclear to what extent SBA reviewed staff compliance with  certification and recertification review procedures. SBA provided an  assurance letter stating that it evaluated the Office of HUBZone’s  internal controls and concluded the controls were effective, but the  letter did not specify what steps SBA took for the evaluation.

We recommended that SBA (1) update its internal policy manuals for  certification and recertification reviews to reflect existing policies and  procedures not currently in written guidance and (2) conduct and  document reviews of staff compliance with procedures associated with  HUBZone certification and recertification. In response to our report, SBA  said that it planned to update its internal policies on certification and  recertification by issuing a procedural notice and to begin reviewing and  documenting staff compliance with the updated procedures outlined in the  notice. However, as of May 2019, SBA had not provided documentation  showing that it had completed these planned actions.

SBA Has Taken Some Steps to Address Recommendations about the Procurement Scorecard

In September 2018, we found that for fiscal year 2017, SBA revised the  methodology for its Small Business Procurement Scorecard, which is  used to assess federal agencies’ progress toward small business  procurement goals. SBA made revisions to address requirements  specified in the National Defense Authorization Act for Fiscal Year 2016.  SBA (1) reduced the share of the total scorecard grade devoted to prime  contracting achievement, which is the dollar amount of contracts awarded  directly to small businesses, and (2) added an element calculating  changes in the number of small businesses receiving prime contracts.  SBA made two additional revisions—with input from other agencies’  representatives—to increase the share of subcontracting achievement  results and the share of the peer review of required activities designed to  facilitate small business procurement. In July 2018, officials said they had  begun developing a plan to evaluate the effects of the revised scorecard  methodology but did not provide a draft plan. Conducting a well-designed  and comprehensive evaluation could aid SBA in determining whether the  scorecard is an effective tool for helping to achieve the agency’s strategic  goals.

In our September 2018 report, we also found that the published fiscal  year 2017 scorecards originally contained errors, including an incorrect  grade and numeric score for one agency, and SBA does not have a  process to ensure that scorecard results are published accurately.  Although SBA later corrected the errors, it did not initially document that  scorecards had been changed, which is inconsistent with SBA’s policy on  information quality. SBA officials said that errors occurred in the process  of formatting scorecards for publication. Errors in the published  scorecards—and the initial lack of disclosure about corrections—weaken  data reliability and may undermine confidence in scorecard data.

We recommended that SBA (1) design and implement a comprehensive  evaluation to assess scorecard revisions and (2) institute a process for  reviewing scorecards for accuracy prior to publication and a mechanism  for disclosing corrected information. Since our report, SBA has proposed  a two-phase program evaluation of the scorecard. SBA officials said that  they plan for phase one to include a report to Congress on the impact of  the small business procurement goal program for Chief Financial Officers  Act agencies and to provide a recommendation on continuing, modifying,  expanding, or terminating the scorecard program. SBA plans to provide  the phase one report in September 2019. In phase two, SBA plans to  conduct a program evaluation that investigates the effectiveness of the  small business contracting scorecard on federal agency small business  contracting goal achievement. SBA has not provided a time frame for  phase two. With respect to the second recommendation, SBA officials  said that SBA has developed a procedure that includes a prepublication  review process for procurement scorecards. The officials said the  procedure identifies responsibilities, provides for an independent peer  review, and includes supervisory review. Officials said the procedure also  includes measures for post-publication review and corrections. We will  review supporting documentation for this new procedure to assess  whether this recommendation can be closed as implemented.

Chairman Rubio, Ranking Member Cardin, and Members of the  Committee, this completes my prepared statement. I would be pleased to  respond to any questions that you may have at this time.

If you or your staff have any questions about this testimony, please  contact William Shear, Director, Financial Markets and Community  Investment at (202) 512-8678 or shearw@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this statement. GAO staff who made key contributions to  this testimony are Andrew Pauline (Assistant Director), Paige Smith  (Assistant Director), Winnie Tsen (Assistant Director), and Jennifer  Schwartz.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

We testified that the Small Business Administration could do more to address recommendations in 3 of our prior reports.

We recommended that SBA improve its oversight of 2 programs intended to increase contract opportunities for two groups: women-owned businesses and businesses operating in "underutilized" areas. SBA could better review the vetting of applicants and participants to ensure awards go to eligible businesses.

In 2018, we found errors and recommended an evaluation of SBA's annual Small Business Procurement Scorecard, which measures the federal government's contracting efforts. SBA proposed an approach for evaluating the scorecard.


Commissaries and Exchanges: DOD and Congress Need More Reliable Information on Expected Savings and Costs of Consolidating the Defense Resale Organizations

DOD operates about 240 commissaries and 2,500 exchanges that sell groceries and retail goods and services to servicemembers, their families, and retirees. Commissaries and exchanges are operated by four resale organizations, and in November 2018 a DOD task force completed a business case analysis on consolidating those organizations.

The National Defense Authorization Act for Fiscal Year 2020 included a provision for GAO to review DOD's business case analysis. This report evaluates the extent to which (1) DOD's business case analysis for consolidating the four resale organizations provided reliable savings and cost estimates and (2) the military departments concurred with the business case analysis and DOD shared their accompanying comments with Congress.

GAO evaluated the business case analysis against DOD- and GAO-identified key elements of economic analyses; reviewed comments on the business case analysis; and interviewed DOD officials.

A Department of Defense (DOD) task force's business case analysis for consolidating the defense resale organizations—the Defense Commissary Agency (DeCA), the Army and Air Force Exchange Service, the Navy Exchange Service Command, and Marine Corps Community Services—may not provide reliable savings and cost estimates. These organizations sell groceries and retail goods to servicemembers, their families, and retirees. The task force recommended consolidating the four resale organizations into a single organization, estimating “net savings” (i.e., savings minus costs) of about $690 million to $1.3 billion during the first 5 years. However, the task force may have overestimated savings and underestimated costs.

Savings from reducing the cost of goods sold. The task force estimated that DOD would save several hundred million dollars annually by reducing the cost of purchasing goods that are resold in stores. Specifically, the task force multiplied the fiscal year 2017 total cost of goods sold for all four resale organizations by industry benchmarks, reasoning that mergers lead to more savings when merging organizations sell a high amount of identical products. However, task force data show that DeCA and the exchange organizations have limited identical products; the overlap between DeCA products and those of at least one exchange organization amounts to less than one-third of the total cost of goods sold. Thus, multiplying the benchmarks by the total cost of goods sold for all four organizations may not have been appropriate.

Information technology (IT) costs. The task force estimated the costs of developing new, common IT systems to operate a consolidated resale organization to be between $326 million and $401 million, about 50 percent of estimated consolidation costs. The task force stated that it based IT cost estimates on data resale organizations provided for major upgrades or system replacements. But GAO found that about 40 percent of the IT cost estimate was based on minor upgrades or partial replacements, not major upgrades or system replacements. Thus, the estimate may be understated.

Headquarters relocation costs. According to the task force, there will be costs if DOD decides to relocate the four defense resale organizations to a new headquarters location. However, the task force did not include cost estimates for relocation in its business case analysis.

According to federal law, the operation of the commissary and exchange systems may not be consolidated unless authorized by Congress. Until the task force reassesses and updates, as necessary, its savings and costs estimates, DOD and Congress will not have reliable information to consider resale consolidation.

The military departments officially concurred with the business case analysis, but provided written comments detailing fundamental concerns with the analysis, such as the use of proprietary industry benchmarks and the estimated savings and costs. In April 2019, DOD reported to Congress that the military departments agreed with consolidation, but did not disclose the accompanying comments. Without more complete reporting of those comments, Congress has limited visibility of the views of the organizations involved in a potential consolidation.

GAO is making four recommendations, including that DOD reassess and update as necessary its estimates for consolidation savings and costs, and provide additional information to Congress on the military departments' comments on the November 2018 business case analysis. DOD concurred with three recommendations and provided updated estimates. DOD did not concur with the last recommendation. GAO continues to believe providing such information is beneficial, as discussed in the report.

The Department of Defense (DOD) operates about 240 commissaries  and 2,500 exchange facilities worldwide to enhance the quality of life of  uniformed servicemembers, their families, and retirees by providing  reduced-priced groceries and retail goods and services. Commissaries  provide groceries and household goods at reduced prices to eligible  customers. To pay for operating costs that exceed the commissaries’  sales revenue, Congress directed that approximately $1.3 billion annually  from fiscal years 2015 through 2019 be made available from amounts  appropriated to the Defense Working Capital Fund for commissary use.  Exchanges provide goods and services similar to department or retail  stores and also operate other stores, such as gas stations and convenience stores. In fiscal year 2018, the exchanges generated about  $12 billion in sales revenue. Unlike commissaries, exchanges rely on  non-appropriated funding, including sales and other revenue, to cover  operating expenses. Revenue generated by the exchanges also helps  fund certain morale, welfare, and recreation activities.

Commissaries and exchanges are operated by four separate  organizations, referred to in this report as the defense resale  organizations. The Defense Commissary Agency (DeCA) has operated all  DOD commissaries since 1991; prior to that, the military services  operated their own commissaries. The Army and Air Force Exchange  Service (AAFES), the Navy Exchange Service Command (NEXCOM),  and Marine Corps Community Services (MCCS) operate their own  service-specific exchanges. Each of these four defense resale  organizations has its own headquarters, chief executive officer or  equivalent, and board of directors. According to federal law, the defense  commissary system (DeCA) and the exchange stores system (AAFES,  NEXCOM, and MCCS) currently must be operated as separate systems  of DOD, and the operation and administration of these defense retail  systems may not be consolidated or otherwise merged unless specifically  authorized by Congress.

Prior studies have recommended consolidating the existing defense  resale organizations, citing the potential to eliminate redundancies and  achieve cost savings. For example, in 1990, a DOD study group  recommended that the three exchange organizations be consolidated into  a single exchange organization, similar to DeCA. In 2015, the Military  Compensation and Retirement Modernization Commission went further  and recommended that all four resale organizations be consolidated into  a single organization.

No consolidation has taken place since DeCA was created in 1991, and  DOD stated in a 2016 report that it did not recommend creating a single  resale organization at that time due to the differences in how DeCA and  the exchange organizations fulfill their specific missions and deliver their  respective benefits (we assessed this report in November 2016; see app.  I for information on our prior work related to commissaries and  exchanges). However, in May 2018, the Deputy Secretary of Defense,  citing challenges generally faced by grocers and retailers, stated that  consolidating the four defense resale organizations offered the greatest  potential to achieve efficiencies needed for the survivability of the defense  resale enterprise.

The Deputy Secretary of Defense also directed DOD’s Chief  Management Officer to establish a task force to perform a business case  analysis for consolidating the resale organizations. The task force—which  comprised DOD officials supported by contractors, including from Boston  Consulting Group—completed the business case analysis in November  2018. In its business case analysis, the task force recommended  consolidating the four defense resale organizations into a single  organization, and estimated that consolidation would result in “net  savings” (i.e., estimated savings minus estimated costs) ranging from  $690 million to $1.3 billion during the first 5 years, followed by annual net  savings of approximately $390 million to $670 million.

The National Defense Authorization Act for Fiscal Year 2020 included a  provision that we review DOD’s business case analysis. This report  evaluates the extent to which: (1) DOD’s business case analysis for  consolidating the four defense resale organizations provided reliable  savings and cost estimates and (2) the military departments concurred  with the business case analysis and DOD shared their accompanying  comments with Congress.

For our first objective, we reviewed the task force’s business case  analysis to identify the savings and cost estimates for consolidating the  defense resale organizations. We assessed the extent to which the task  force developed reliable savings and cost estimates in its business case  analysis by evaluating it against the key elements of an economic  analysis, as identified in DOD Instruction 7041.03, Economic Analysis for  Decision Making, the GAO Cost Estimating and Assessment Guide, and  our Assessment Methodology for Economic Analysis. We used our  Assessment Methodology for Economic Analysis to assess the business  case analysis against five key methodological elements that are  necessary for an economic analysis: objective and scope, methodology,  analysis of effects, transparency, and documentation. Each key element  consists of economic concepts that represent best practices. These key  methodological elements are not intended to be exhaustive or to  supersede or alter relevant federal and agency requirements for  economic analysis. We determined whether the business case analysis  considered and properly adhered to each of these key elements. We use  “fully met” to indicate that the business case analysis considered and  followed the best practices for a key element and “partly met” to indicate  that the business case analysis partly considered and followed the best  practices for a key element.

We also obtained data and met with officials from the task force (including  contractors from Boston Consulting Group), the Office of the Secretary of  Defense, the military departments, and the four defense resale  organizations—DeCA, AAFES, NEXCOM, and MCCS—to understand  and evaluate the task force’s savings and cost estimates. Additionally, we reviewed prior reports on defense resale reform, including reports from  DOD, the Military Compensation and Retirement Modernization  Commission, and Boston Consulting Group.

For our second objective, we reviewed the written comments on the  business case analysis from the military departments and the defense  resale organizations. We also reviewed DOD’s April 2019 report to  Congress on the task force’s business case analysis and evaluated the  extent to which it included information on the recommendations of the  Secretaries of the military departments, as required by the John S.  McCain National Defense Authorization Act for Fiscal Year 2019.  Finally, we met with officials from the task force, the military departments,  and the four defense resale organizations to discuss the comments and  concerns submitted on the business case analysis.

We conducted this performance audit from June 2019 to April 2020 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

In June 2018, DOD’s Chief Management Officer established the  Community Services Reform Task Force to perform a business case  analysis to determine whether consolidating the defense resale  organizations would result in efficiencies. The task force conducted its  work from July 2018 through November 2018, during which it collected  financial and other data from the four resale organizations and conducted  workshops with subject matter experts from the resale organizations. In  November 2018, the military departments were given an opportunity to  review the business case analysis, provide comments, and indicate  whether they concurred with the analysis.

In its business case analysis, the task force recommended consolidating  the four defense resale organizations into a single organization. The  task force stated that consolidation would eliminate duplication that  currently exists across the resale organizations and increase the  competitiveness, or financial viability, of defense resale, which has seen  sales declines in recent years. Specifically, the task force recommended  that a single chief executive officer or director be responsible for leading  the organization and report to a single board of directors. The task force  also recommended that separate leadership positions for commissary  operations and exchange operations be established, and that a chief  administrative officer manage the business functions that are common to  the current resale organizations, such as information technology (IT),  human resources, marketing, and finances. Figure 1 shows the task  force’s recommended organizational chart for the consolidated resale  organizations.

The task force estimated that the time frame for consolidating the four  defense resale organizations would be 5 years for implementation, and  that consolidation would result in “net savings” (i.e., estimated savings  minus estimated costs) ranging from about $690 million to $1.3 billion  during the first 5 years, followed by annual net savings of approximately  $390 million to $670 million every year thereafter. Specifically:

Estimated savings: The task force estimated that consolidating the  four defense resale organizations would result in savings in three  areas: (1) reduction of the cost of goods sold in the commissaries and  the exchanges; (2) reduction of the cost of goods and services that  are not sold but are necessary for operating stores (e.g., plastic  shopping bags and custodial services); and (3) reduction of payroll  costs by eliminating redundant personnel. According to the task  force, consolidation would result in estimated savings of $1.4 billion to  $2.1 billion over the first 5 years, followed by annual savings of $470  million to $750 million.

Estimated costs: The task force estimated that consolidating the  defense resale organizations would result in costs from four areas: (1)  development of new, common IT systems; (2) severance pay for  separating employees and retention bonuses to incentivize  employees to remain; (3) operation of a transformation management  office, supported by private contractors, to implement the  consolidation; and (4) costs to convert DeCA to a non-appropriated  fund organization. According to the task force, consolidation would  result in estimated costs of $700 million to $810 million over the first 5  years, followed by annual costs of $80 million.

DOD’s Chief Management Officer in March 2019 and the Deputy  Secretary of Defense in August 2019 both approved the results of the  business case analysis and directed that plans be made for consolidation,  pending congressional action to remove the statutory prohibition on  consolidating the commissary and exchange systems.

DOD’s Business Case Analysis Supporting Defense Resale Consolidation May Not Provide Reliable Savings and Cost Estimates

The task force may have overestimated the expected savings from  reducing the “cost of goods sold” (i.e., the cost of purchasing products  that are resold in commissaries or exchanges) and underestimated the  expected costs from IT consolidation and headquarters relocation.

DOD’s Task Force May Have Overestimated the Expected Savings to Be Achieved from Reducing the Cost of Goods Sold

The task force estimated that most of the savings (i.e., about 70 percent  annually) to be achieved from consolidating the four defense resale  organizations would result from reducing the cost of goods sold.  According to the task force’s business case analysis, retailers often pay  different costs for identical products, and mergers are an opportunity for  retailers to compare costs across a larger combined organization and  make decisions that maximize savings. In the case of a consolidated  defense resale organization, the task force stated in its business case  analysis that savings could be achieved by implementing what the task  force called category management reforms and by obtaining the lowest  cost for identical products sold by both commissaries and exchanges.  Task force officials added that one board of directors and one chief  executive officer overseeing the consolidated resale organization would  be more likely to achieve savings than the current, individual boards of  directors and chief executive officers of the resale organizations.

DeCA and one or more exchange organization, or

Two or more of the exchange organizations (and not DeCA).

In addition to overlap in identical products sold, task force officials told us  that savings from category management reforms are dependent, in part,  on the amount of overlap in vendors that sell products to the resale  organizations. Specifically, task force officials stated that there are  opportunities to reduce cost of goods sold through negotiations with  vendors that sell items to both DeCA and the exchange organizations.  For example, task force officials stated that the consolidated organization  could negotiate better prices with a vendor that sells family-size items to  DeCA and single-size items to the exchange organizations, even though  those items are not identical.

Industry Benchmarks  The industry benchmarks used by the task  force are based on proprietary data gathered  and owned by Boston Consulting Group  based on its experience working with mergers  and category management reforms in the  private sector retail industry. These  benchmarks were presented as a percentage  of cost of goods sold; specifically, the task  force estimated that savings from obtaining  the lowest cost for identical items were from 1  to 1.5 percent of the cost of goods sold, and  savings from category management efforts  were from 2.5 to 4 percent of the cost of  goods sold. We did not review and evaluate  the underlying data that were used to develop  the proprietary benchmarks.

Based on this information, the task force calculated the estimated savings  that would result from reducing the cost of goods sold by multiplying the  total cost of goods sold for all four resale organizations in fiscal year 2017  ($9.5 billion) by industry benchmarks developed by Boston Consulting  Group (see sidebar for more information on these benchmarks). This  calculation showed an estimated annual savings of $329 million to $517  million from reducing the cost of goods sold.

However, additional information from the task force suggests this savings  estimate may be overstated because there is limited overlap in the  products DeCA sells (i.e., groceries and household goods) and the  products the exchange organizations sell (i.e., goods and services similar  to retail stores). According to the task force, about $2.2 billion of DeCA’s  cost of goods sold in fiscal year 2017 were for products also sold by at  least one of the exchange organizations, which is equivalent to about 23  percent of the total cost of goods sold for the four resale  organizations. This differs from the data provided in the business case  analysis, which stated that 62 percent of the total cost of goods sold was  for identical products sold by two or more resale organizations; however,  that figure also includes products sold by two or more exchange  organizations and not DeCA (we further discuss product overlap among  the exchange organizations below). Given the more limited product  overlap between DeCA and the exchange organizations, it is unclear  whether using the total cost of goods sold for all four resale organizations  as the basis for estimating savings was appropriate.

Additionally, the business case analysis did not fully identify the amount  of vendor overlap that exists between DeCA and the three exchange  organizations, but the data that were provided in the business case  analysis suggest that limited vendor overlap exists. Specifically, the  business case analysis provided data for 10 vendors that sell to the  defense resale organizations, but those vendors represent less than 20  percent of the cost of goods sold to DeCA and the exchange  organizations in fiscal year 2017. Further, only 5 of the 10 vendors  identified in the business case analysis sold goods to both DeCA and the  exchange organizations, and their cost of goods sold accounted for about  10 percent ($972 million) of the total cost of goods sold for the four resale  organizations ($9.5 billion). Based on these data, the extent of vendor  overlap between DeCA and the exchange organizations—and, as a  result, how much can be saved through category management reforms by  consolidating DeCA and the exchange organizations—is unclear.

Although the task force stressed the importance of a conservative  estimate in both its business case analysis and in meetings with us, our  assessment of the assumptions and methodology for estimating savings  from the cost of goods sold found that a more conservative approach  could have been used to better ensure estimated savings were not  overstated. For example, one method could have been to multiply the  benchmarks by the cost of goods sold for just the three exchange  organizations (about $5.5 billion in fiscal year 2017, per the task force), as  data provided by the task force indicate that about 67 percent of the cost  of goods sold for the exchange organizations in fiscal year 2017 were for  identical products sold by at least two exchange organizations. Another  method, which task force officials suggested after we shared our  concerns about their methodology, could have been to multiply the benchmarks by the cost of goods sold for the exchange organizations,  plus the portion of DeCA’s cost of goods sold that overlaps with at least  one exchange organization (about $2.2 billion in fiscal year 2017, per the  task force). Either method would be more conservative than the one  adopted in the business case analysis and would yield a savings estimate  that is about 20 to 40 percent lower, but would be more consistent with  the task force’s assertion that consolidation savings are dependent on the  amount of overlap among the merging organizations.

DOD policy states that an economic analysis should base its analysis of  benefits on facts and data whenever possible. Additionally, our  Assessment Methodology for Economic Analysis states that an economic  analysis should examine the effects of an action by considering relevant  alternatives and justifying what the world would be like under each  alternative; describe and justify the analytical choices, assumptions, and  data used; and assess how plausible adjustments to each important  analytical choice and assumption affect the estimates of savings.

Ensuring that the estimates for cost of goods savings are accurate is  particularly important, as they account for approximately 70 percent of the  task force’s overall savings estimate from consolidation. However, the  task force did not fully identify and analyze in its business case analysis  how many identical products are sold by both DeCA and the exchange  organizations or how many vendors sell products to both DeCA and the  exchange organizations. According to task force officials, they did not  provide data on product overlap between DeCA and the exchange  organizations because it would not change their savings methodology or  estimate, and they did not provide more information on vendor overlap  because of the proprietary nature of that data. However, the amount of  product and vendor overlap that exists across the four resale  organizations will have a direct effect on the amount of savings to be  achieved from consolidation, as acknowledged by the task force.

Without the task force reassessing the approach it used to estimate  savings from the cost of goods sold and, if necessary, making  adjustments to those estimates, decision makers in DOD and Congress  may lack confidence in the reliability of the task force’s savings estimates  in the business case analysis and will not have complete information as  they consider defense resale consolidation.

DOD’s Task Force May Have Underestimated the Expected Costs of Consolidating the Four Defense Resale Organizations

Based on our analysis of the business case analysis, we found that  DOD’s task force may have underestimated the expected costs of  consolidating the four defense resale organizations in two areas: (1) the  development of new, common IT systems and (2) the location of a new  headquarters for the consolidated organization.

Task Force May Have Underestimated the Cost of Consolidating Defense Resale Organizations’ IT Systems

The task force estimated in its business case analysis that most of the  costs (i.e., about 50 percent annually) of consolidating the four defense  resale organizations will result from developing new, common IT systems  to support the consolidated organization. In the business case analysis,  the task force stated that it worked with the four resale organizations to  calculate a cost estimate of $292 million to $352 million for developing  five types of IT systems that are needed for the consolidated  organization: merchandising, store inventory management, financial  management and general ledger, transportation and logistics, and  ecommerce.

According to the business case analysis, the task force’s cost estimates  for developing new, common IT systems for the consolidated organization  were to be based on data provided by the resale organizations on recent  or projected costs for replacing similar systems or performing major  upgrades to existing systems, when available. For example, the task  force’s estimate for the merchandising system was approximately $115  million, which the business case analysis stated is based on a $35 million  estimate provided by AAFES, a $23.5 million estimate provided by  NEXCOM, a $15 million estimate provided by MCCS, and a $41 million  estimate provided by DeCA.

However, the task force’s cost estimate for IT consolidation may be  understated because it is based, in part, on less expensive minor IT  system upgrades and partial replacements, according to the resale  organizations. Based on our analysis of information provided to us by the  resale organizations, about $140 million (about 40 percent) of the overall  IT cost estimate was based on what the resale organizations described  as minor upgrades or partial replacements.

Specifically, while MCCS confirmed the cost estimates attributed to them  in the business case analysis were for total IT system replacement costs,  the other three resale organizations—AAFES, NEXCOM, and DeCA— disagreed with the task force’s characterization that all the data used to  calculate IT system estimates represented costs for replacements or  major upgrades. AAFES told us that the cost estimates cited in the  business case analysis for its merchandising, financial management and  general ledger, and transportation and logistics systems reflected minor  upgrades of specific modules within the overall systems, and the cost to  replace or upgrade the entire system would be significantly higher.  NEXCOM stated that the cost estimates for upgrading its merchandising,  store inventory management, and financial management and general  ledger systems were for minor upgrades, not replacements or major  upgrades, as stated by the task force. DeCA told us that the estimate for  replacing its store inventory management system only represented 1 year  of costs, even though DeCA plans to incur replacement costs through at  least 2022.

According to the task force, the task force and the resale organizations  agreed on the methodology for estimating IT costs, and the subject matter  experts from the resale organizations provided the cost data used in the  business case analysis. However, based on information provided by the  resale organizations, it appears that the task force may not have always  based its cost estimate on replacement or major upgrade costs— consistent with the key assumption that new IT systems would be  developed for the consolidated resale organization—but, rather, used  minor upgrade or partial replacement costs in some cases. Specifically,  task force officials told us they believed the estimates provided by the  resale organizations were too high to be minor upgrades or partial  replacements, based on their understanding of IT requirements for resale  operations.

Further, task force officials stated that their overall IT cost estimate was  likely overstated, not understated. For example, they stated that their  estimate is higher than what is typically spent for a private sector  consolidation of similar size. However, the task force stated that it did  not use private sector IT cost estimates in its business case analysis  because it determined that public sector IT costs would likely be higher  than private sector IT costs. Additionally, task force officials told us that  some planned spending on existing IT systems by the four resale  organizations would not be necessary as a result of consolidation.  However, the business case analysis does not quantify how much future  spending could be reduced or factor those reductions into the IT cost  estimate.

According to the GAO Cost Estimating and Assessment Guide, cost  estimates are developed based on assumptions that are defined to  establish the baseline conditions the estimate will be built from.  Additionally, our Assessment Methodology for Economic Analysis states  that an economic analysis should define an appropriate baseline that  represents the best assessment of what the world would be like under  that alternative. Thus, estimating costs that reflect the baseline  conditions is a key step in developing a sound cost estimate.

Additionally, we have previously reported that federal IT investments  frequently fail or incur cost overruns and schedule slippages. As such, high-quality data are imperative for ensuring proper management and  oversight of IT investments. The task force’s IT cost estimate is  particularly important, as it represents about 50 percent of the total  estimated costs for defense resale consolidation. Until the task force  consults with the resale organizations to reassess the methodology for  estimating IT costs, decision makers in DOD and Congress may not have  a reliable and complete understanding of the estimated costs for the  implementation of new, common IT systems, which is information DOD  and Congress need as they consider defense resale consolidation.

DOD’s Task Force Did Not Provide a Cost Estimate for Relocating the Four Defense Resale Organizations to a New Headquarters Location

According to the task force, there would be costs associated with  relocating AAFES, NEXCOM, MCCS, and DeCA to a new headquarters  location, to include relocating existing personnel, hiring new personnel,  and obtaining real estate. Although no relocation options were presented  in the business case analysis, task force officials told us there are multiple  options for where to locate the headquarters of a consolidated resale  organization. One option cited by the task force would be to create a new  headquarters in the Washington, DC, area, which would be the most  expensive option, as it would likely involve acquiring new real estate and  hiring personnel in a high-cost region. Another option cited by the task  force would be to locate all exchange operations and staff at the existing  AAFES headquarters in Dallas, TX, and maintain commissary operations  and staff at the existing DeCA headquarters at Fort Lee, VA. This option  would likely be less expensive, as personnel and available real estate are  already present at both locations. In January 2020, task force officials  also told us that an even less expensive option they might consider is  maintaining commissary and exchange headquarters staff at their current  locations, but having personnel work for the consolidated organization,  rather than for DeCA or the exchange organizations.

Despite the potential for relocation costs, the task force did not include a  range of cost estimates for different relocation options in its business  case analysis. According to task force officials, relocation cost estimates  were not included because the headquarters location has not been  chosen, and costs will vary widely depending on the chosen location.  While actual relocation costs will depend on the chosen headquarters  location, this fact does not prevent the task force from presenting a range  of cost estimates in advance of that decision being made. Task force  officials also said that including relocation cost estimates would not have  changed the conclusion of the business case analysis. However, without  a range of relocation cost estimates, we were unable to assess the effect  of relocation costs on the conclusion of the business case analysis.

DOD policy states that an economic analysis should quantify the costs  associated with each alternative under consideration whenever possible  so that they may be included in the economic analysis calculation.  Additionally, our Assessment Methodology for Economic Analysis states  that an economic analysis should quantify the important costs, where  feasible, to inform decision makers about the economic effects of a  proposed action. Without developing and providing a range of relocation  cost estimates from the least expensive option to the most expensive,  decision makers in DOD and Congress will not be fully informed about the  costs of consolidation, which is necessary information for deciding  whether to consolidate the four defense resale organizations.

Military Departments Officially Concurred with the Business Case Analysis, but DOD Did Not Share Their Accompanying Comments with Congress

The military departments officially concurred with the task force’s  business case analysis for consolidating the four defense resale  organizations. However, the military departments also provided written  comments that detailed concerns with fundamental aspects of the  business case analysis, to include: the use of proprietary industry  benchmarks; estimated savings, costs, and timeline of the consolidation;  and the proposed governance structure for the new resale organization.  In an April 2019 report to Congress that summarized the business case  analysis, DOD stated that the military departments agreed with the  consolidation. However, the report did not disclose the military  departments’ comments and concerns on the business case analysis,  which are relevant as Congress considers defense resale consolidation.

In their written comments, the military departments either stated concerns  about the consolidation or included critical comments from the exchange  organizations—all of which opposed the consolidation. Specifically:

The Army concurred with the business case analysis but noted that  funding for morale, welfare, and recreation programs must be  preserved or increased as a result of the consolidation. In addition,  the Army’s comment letter included as an attachment written  comments from AAFES, which expressed opposition to the  consolidation and detailed concerns with the business case analysis.  For example, AAFES stated that the business case analysis relied on  unverifiable, proprietary industry benchmarks that overstated the  benefits of consolidation, underestimated the costs and time to  consolidate, and did not account for recent efforts by the resale  organizations to reduce costs by collaborating on a purchasing  alliance.

The Air Force also concurred with the business case analysis, but  noted in its comments that mergers and acquisitions have historically  cost more, taken longer, and saved less than originally expected. As a  result, the Air Force recommended that a phased implementation plan  be followed to guard against financial risk. The Air Force also stated  that morale, welfare, and recreation funding currently provided by the  exchanges should be maintained while opportunities are examined to  reduce the need for appropriated funding.

The Navy initially non-concurred with the business case analysis in  December 2018. In its comment letter, the Navy stated that the task  force’s analysis was flawed beyond repair and included comments  from NEXCOM and MCCS that also opposed consolidation. For  example, the exchanges’ comments stated that the expected cost  savings were overstated, that potential inefficiencies from  consolidation were not discussed, and that the resale organizations  could achieve cost savings through greater collaboration without the  need for consolidation. NEXCOM and MCCS also stated concern that  the task force’s savings projection relied heavily on unverifiable  industry benchmarks. In addition, MCCS expressed concern that  consolidation could result in unexpected costs from separating  exchange operations from the rest of MCCS operations, which also  include the Marine Corps’ morale, welfare, and recreation and family  programs. In January 2019, the Navy changed its position to concur  subject to several significant comments and clarifications, and  attached a letter detailing comments and concerns similar to those it  submitted with its original non-concurrence in December 2018.

Officials from the resale organizations further articulated their concerns  about the business case analysis when they met with us. For example,  resale officials told us they are concerned that savings are overstated,  that costs are understated, and that the proprietary benchmarks used by  the task force are unverifiable or may not be applicable to the public  sector. Exchange officials also stated that they are worried about the  effect of consolidation on morale, welfare, and recreation funding  generated by the exchanges. Specifically, exchange officials are  concerned that exchange revenue currently used for morale, welfare, and  recreation programs could be used to pay for consolidation expenses or  to reduce the amount of appropriated funds allocated to the  commissaries.

Despite the concerns detailed in the comments from the military  departments and resale organizations, DOD did not include them in its  April 2019 report to Congress summarizing the results of the business  case analysis. The National Defense Authorization Act for Fiscal Year  2019 required DOD to include in its report the recommendations of the  Secretaries of the military departments regarding the plan to consolidate  the defense resale organizations. When we asked the task force why  DOD did not provide Congress with the comments and concerns cited by  the military departments and the resale organizations, officials stated that  they were advised by DOD’s Office of General Counsel not to include the  comments because they contained information that may have disclosed  DOD’s deliberative process.

Task force officials also stated that the savings, cost, and timeline  estimates in the business case analysis were conservative, and that the  proprietary industry benchmarks are based on years of experience by  Boston Consulting Group and similar to those cited by prior studies.  Regarding the purchasing alliance formed by the resale organizations to  reduce their cost of goods sold, task force officials stated they do not  believe such efforts to reduce costs will be sustained without a single  chief executive officer and board of directors to ensure those efforts  continued, as recommended in the business case analysis. Finally, the  task force stated in the business case analysis that any savings achieved  from consolidation could be used to increase morale, welfare, and  recreation funding or reduce appropriations used to fund DeCA, and that decisions on how to allocate savings will be made by the proposed board  of directors.

According to task force officials, some of the concerns articulated by the  military departments and the exchanges could be motivated by a general  opposition to consolidation. However, without a more complete reporting  of the military departments’ perspectives on consolidation and the task  force’s response to those comments, Congress may be unaware of the  views various organizations within DOD have regarding the business  case analysis, which is relevant information as Congress considers  defense resale consolidation. Moreover, fully reporting the comments and  concerns could strengthen trust and collaboration among the task force,  military departments, and resale organizations on any future resale  reforms.

Four defense resale organizations currently operate about 240  commissaries (operated by DeCA) and 2,500 exchange facilities  (operated by AAFES, NEXCOM, and MCCS) worldwide to provide  reduced-priced groceries and retail goods and services to DOD  servicemembers, their families, and retirees. DeCA operations are funded  in part by appropriations, which have totaled approximately $1.3 billion in  recent years. By law the commissary and exchange organizations must  be operated separately. In November 2018, a DOD task force completed  a business case analysis and concluded that consolidating the four  defense resale organizations into a single organization would result in  several hundred million dollars in annual cost savings. However, we  found that the task force’s projected savings from reducing the cost of  goods sold may be overestimated, and that projected costs for IT  development and headquarters relocation may be underestimated.  Further, while the military departments concurred with the task force’s  recommendation to consolidate, DOD did not fully share their comments  and concerns about the business case analysis with Congress. DOD’s  proposed consolidation will cost several hundred million dollars, take  years to implement, and involve multiple DOD organizations. Given the  cost and complexity of the proposed defense resale consolidation, DOD  can ensure that Congress has the reliable information it needs to consider  consolidation by reviewing and updating savings and cost estimates and  sharing comments and concerns from the military departments.

We are making the following four recommendations to DOD.

The Secretary of Defense should ensure that the DOD Chief  Management Officer direct the task force to reassess its approach to  estimating savings from cost of goods sold—to include reassessing its  use of the cost of goods sold for all four defense resale organizations  rather than, for example, just for the three exchange organizations—and  make any necessary adjustments to its savings estimates for  consolidation and provide that updated information to Congress.  (Recommendation 1)

The Secretary of Defense should ensure that the DOD Chief  Management Officer direct the task force, in consultation with the resale  organizations, to reassess its methodology for estimating IT costs of  consolidation, and make any necessary adjustments to its range of IT  cost estimates and provide that updated information to Congress.  (Recommendation 2)

The Secretary of Defense should ensure that the DOD Chief  Management Officer direct the task force to develop a range of cost  estimates for relocating the defense resale organizations, and adjust its  range of cost estimates for consolidation and provide that updated  information to Congress. (Recommendation 3)

The Secretary of Defense should ensure that the DOD Chief  Management Officer provide additional written information to Congress on  the comments and concerns from the military departments and resale  organizations on the task force’s November 2018 business case analysis,  as well as the task force’s response to those comments and concerns.  (Recommendation 4)

We provided a draft of this product to DOD for review and comment. In its  comments, reproduced in appendix III and summarized below, DOD  concurred with the first three recommendations and did not concur with  the fourth recommendation. DOD also provided technical comments,  which we incorporated as appropriate.

DOD stated in its letter that it continues to firmly believe that consolidation  of above-store operations of DeCA and the military exchanges is the right  path forward and that it intends to move forward with this effort. DOD also  requested that we consider the first three recommendations as  implemented, based on information provided in the letter and as detailed  below.

Regarding the first recommendation for the task force to reassess  its approach to estimating savings from cost of goods sold—to  include reassessing its use of cost of goods sold by all four  defense resale organizations rather than, for example, just the  three exchange organizations—and make any necessary  adjustments to its savings estimates for consolidation and provide  that updated information to Congress, DOD stated that it had  reassessed its approach and found that there is significant overlap  and, therefore, savings opportunity in products sold by DeCA and  the exchanges. DOD also provided revised savings estimates that  exclude DeCA’s cost of goods sold from its methodology that  show net savings ranging from $309 million to $739 million in the  first 5 years of consolidation, followed by $255 million to $457  million per year thereafter. These figures are about 44 percent to  55 percent lower than the business case analysis’s estimate for  the first 5 years and about 32 to 35 percent lower per year  thereafter. By providing these revised savings estimates, we  believe that DOD has addressed the intent of the  recommendation.

With regard to the second recommendation for the task force to  reassess its methodology for estimating IT costs of consolidation,  and make any necessary adjustments to its range of IT cost  estimates and provide that updated information to Congress, DOD  stated that the task force followed up with AAFES, NEXCOM, and  DeCA to get an update on the cost estimates these entities  expressed concern about to us. However, according to DOD,  those resale organizations were unable to provide alternate data  to use in place of the numbers in the business case analysis. DOD  further stated in its letter that because no alternative data were  provided, the department will continue to use the estimate in the  business case analysis and will reengage with the resale  organizations to develop more detailed IT design plans and make  any necessary updates to the IT cost estimates as integration  planning moves forward. As DOD develops its more detailed IT  design plans and associated cost estimates, we will follow up with  the department, including the resale organizations, to determine  whether this recommendation has been addressed.

In commenting on the third recommendation for the task force to  develop a range of cost estimates for relocating the defense  resale organizations, and adjust its range of cost estimates for  consolidation and provide that updated information to Congress,  DOD provided three possible courses of action, along with  corresponding cost estimates. These possible courses of action,  from least expensive to most expensive, are: (1) maintain  operations at all four existing locations (no cost); (2) maintain  commissary operations at DeCA headquarters, perform all  exchange functions at AAFES headquarters, and close the  NEXCOM and MCCS headquarters (one-time costs of $5.5 million  and recurring annual costs of $1.3 million); and (3) create a new  headquarters to perform all commissary and exchange operations  near Washington, D.C. (one-time costs of $19.6 million and  recurring annual costs of $19.7 million). DOD stated that  consolidation would still result in financial benefits, even if the  department chooses the most costly of these courses of action. By  providing these cost estimates, we believe that DOD has  addressed the intent of the recommendation.

While we have determined that DOD has met the intent of the first and  third recommendations, we also note that, in its comment letter, the  department questioned some aspects of our analysis and conclusions  regarding the first three recommendations. We stand by our analysis and  conclusions and offer the following response:

DOD stated in its comment letter that modifying the business case  analysis’s approach to cost of goods savings would result in an  incorrect use of benchmarks and go against industry best practice.  For example, DOD stated that estimating savings by using the  cost of goods sold for just the three exchange organizations, or for  the three exchange organizations plus a portion of DeCA, would  be flawed. However, as noted in our report, the latter method was  recommended to us by task force officials when we raised  concerns about the accuracy of the task force’s savings estimates  in the business case analysis. In addition, multiplying the  benchmarks by the cost of goods sold for the exchange  organizations, as opposed to for all four resale organizations,  would be more consistent with the assertion in the business case  analysis that consolidation savings are dependent on the overlap  among the merging organizations.

DOD questioned the accuracy of some of our figures in the report  by providing different data on product and vendor overlap between  DeCA and the exchange organizations. However, this information  was not included in the task force’s business case analysis or  offered to us during the course of our audit. In addition, when we  asked for supporting documentation that would allow us to  validate the new figures, DOD did not provide any.

DOD stated in its comments that excluding all or including only a  portion of DeCA’s cost of goods sold implies that there is no  opportunity to achieve savings between DeCA and the  exchanges. Our report does not make this assertion, but rather  offers a methodology that would result in a more conservative  savings estimate, consistent with the data presented in the  business case analysis and provided by task force officials, to  better ensure that estimated savings were not overstated.

As noted above, DOD did not concur with the fourth recommendation for  the department’s Chief Management Officer to provide additional written  information to Congress on the comments and concerns from the military  departments and resale organizations on the task force’s November 2018  business case analysis, as well as on the task force’s response to those  comments and concerns. DOD stated in its written response to our report  that the department considered all the comments submitted in its  decision-making process and that all of the military department  secretariats agreed with above-store consolidation, despite their  comments on the business case analysis. DOD further stated that the  military department comments regarding the business case analysis were  shared with congressional committee professional staff, and DOD  suggested in its letter that this recommendation be closed.

However, DOD’s written response did not provide information on which  comments were shared, whether those comments were communicated in  writing or orally, or which committee or committees received information  on the comments. In their written comments on the business case  analysis, the military departments detailed concerns with fundamental  aspects of the analysis, to include: the use of proprietary industry  benchmarks; estimated savings, costs, and timeline of consolidation; and  the proposed governance structure for the new resale organization. We  continue to believe that implementing this recommendation would help  ensure that Congress has the full information it needs as it considers  defense resale consolidation and would also help strengthen trust and  collaboration among the various DOD stakeholders involved in defense  resale, particularly given their role in any consolidation, should one occur.  We will follow up with DOD as part of our regular recommendation follow- up process.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; and the Secretaries of the Army,  Air Force, and Navy. In addition, the report is available at no charge on  our website at https://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-2775 or FieldE1@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.

Appendix I: Prior GAO Work on Commissaries and Exchanges

The National Defense Authorization Act for Fiscal Year 2016 mandated  that the Department of Defense (DOD) report on DOD’s plan to achieve  budget neutrality for commissaries and exchanges—which DOD  interpreted as ending the use of appropriated funding—and included a  provision for us to assess DOD’s report. In November 2016, we found  that DOD’s May 2016 report did not provide a plan for achieving budget  neutrality. DOD reported that it would not be able to eliminate fully the  use of appropriated funds for defense resale, but the department did not  provide detailed information supporting that conclusion. Instead, the  report stated that DOD expected to achieve $2 billion in cost savings over  a 5-year period from fiscal year 2017 through fiscal year 2021. However,  we found that the report did not include any assumptions, methodology,  or specific time frames related to initiatives that would lead to these  savings. We recommended that DOD provide information to Congress to  support its conclusion about budget neutrality and develop a plan for  achieving reductions to defense resale appropriations. DOD concurred  with our recommendations, but as of February 2020 had not addressed  them.

In March 2017, we reported on DOD’s commissary operations, including  the extent to which the Defense Commissary Agency (DeCA) had  assurance that it was maintaining the desired savings rate for its  customers. DeCA’s desired savings rate—which at the time of our March  2017 report was 30 percent and is now 23.7 percent—shows how much a  customer can expect to save on grocery purchases at a commissary in  comparison to purchases at other local grocery stores. We found that  DeCA lacked reasonable assurance that it was maintaining its desired  savings rate for commissary customers because of weaknesses in its  methodology for calculating the savings rate. For example, the  methodology did not use a random sample of overseas commissaries or  account for seasonal and geographic variations in item prices. We also  found that DeCA’s business model departed from practices generally  employed by commercial grocery stores. For example, DeCA did not  assess the contribution of the sale of each product to a given store’s total  sales in determining which products to sell, and it had not conducted cost- benefit analyses for its use of stocking and custodial service contracts or  product distribution options across all commissaries. We recommended  that DOD (1) address limitations identified in its savings rate  methodology; (2) develop a plan with objectives, goals, and time frames  to improve efficiency in product management; and (3) conduct  comprehensive cost-benefit analyses for service contracts and  distribution options. As of February 2020, DOD had addressed the first  two recommendations but had not addressed the third recommendation.

Appendix II: Assessment of the Department of Defense’s (DOD) Business Case Analysis on Defense Resale Consolidation

We assessed DOD’s business case analysis on consolidating the four  defense resale organizations against the five key elements of an  economic analysis, as described in our Assessment Methodology for  Economic Analysis (see table 1).

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

Elizabeth A. Field, (202) 512-2775 or fielde1@gao.gov  In addition to the contact named above, Suzanne Perkins (Assistant  Director), Geoffrey Peck (Analyst-in-Charge), Pedro Almoguera, Noah  Gerber, Mae Jones, Matthew Kienzle, Amie Lesser, Felicia Lopez, and  Jeanne Sung made key contributions to this report.

DOD operates commissaries (grocery stores) and exchanges (retail stores) that sell products to servicemembers, their families, and retirees.

DOD wants to consolidate the four organizations that operate commissaries and exchanges into a single organization, to eliminate redundancies and save money. However, DOD may have overestimated the savings and underestimated the costs of consolidation in its business case analysis. DOD also did not provide Congress with comments and concerns from the military services on consolidation.

We recommended that DOD reassess and update its savings and cost estimates and provide additional information to Congress.