DOD Acquisition Reform: Leadership Attention Needed to Effectively Implement Changes to Acquisition Oversight

Amid concerns about the ability of DOD's acquisition process to keep pace with evolving threats, Congress included numerous reforms in recent National Defense Authorization Acts that could help to streamline acquisition oversight and field capabilities faster.

GAO was asked to examine DOD's efforts to implement these reforms. This report addresses (1) the progress DOD has made implementing selected oversight reforms related to major defense acquisition programs; (2) how DOD has used middle-tier acquisition pathways; and (3) challenges DOD faces related to reform implementation. GAO reviewed five reforms: milestone decision authority designation; cost, fielding, and performance goals; independent technical risk assessments; restructuring of acquisition oversight offices; and middle-tier acquisition. GAO analyzed applicable statutes and implementing guidance, collected information from DOD about the number and types of middle-tier acquisition programs, reviewed relevant documentation, and interviewed DOD officials.

The Department of Defense (DOD) has made progress in implementing reforms to restructure the oversight of major defense acquisition programs. As a result of one of these reforms, decision-making authority for many programs shifted from the Office of the Secretary of Defense to the military departments (see figure).

Questions remain about how some reforms GAO reviewed will be carried out. For example, no programs have been required to have cost and fielding goals set under DOD's new process yet, and DOD has formed a working group to determine when to delegate risk assessments to the military departments.

DOD also began using new pathways referred to as middle-tier acquisition to rapidly prototype and field new weapon systems. Middle-tier programs are expected to field capabilities within 2 to 5 years. As of March 2019, military departments were using this authority for 35 unclassified programs (see table).

Source: GAO analysis of Department of Defense data. | GAO-19-439

DOD has yet to fully determine how it will oversee middle-tier acquisition programs, including what information should be required to ensure informed decisions about program selection and how to measure program performance. Without consistent oversight, DOD is not well positioned to ensure that these programs—some of which are multibillion dollar acquisitions—are likely to meet expectations for delivering prototypes or capability to the warfighter quickly.

DOD also continues to face implementation challenges, including one related to disagreements about oversight roles and responsibilities between the Office of the Secretary of Defense and the military departments. Senior DOD leadership has not fully addressed these disagreements. As a result, DOD is at risk of not achieving an effective balance between oversight and accountability and efficient program management.

GAO is making four recommendations, including that DOD should identify the types of information needed to select and oversee middle-tier acquisition programs consistently, and clarify the roles and responsibilities of the Office of the Secretary of Defense and the military departments for acquisition oversight. DOD concurred with GAO's recommendations and described actions planned to address them.

Many of the Department of Defense’s (DOD) major defense acquisition  programs face challenges delivering innovative technologies to the  warfighter to keep pace with evolving threats. In the National Defense  Authorization Acts for Fiscal Years 2016 and 2017, Congress included  numerous reforms that could help to streamline acquisition oversight and  field capabilities faster. In congressional hearings in 2017 and 2018, the  Chairmen of the Senate and House Armed Services Committees  emphasized the importance of these reforms and expressed concerns  that without improving the speed of and increasing the amount of  innovation in the DOD acquisition process, the U.S. military would lose its  technological advantage. Collectively, the reforms Congress put forth  fundamentally altered roles and responsibilities for major defense  acquisition program oversight to give significantly more authority for  managing acquisition programs to the military departments. Further, the  reforms included an alternative acquisition process for programs intended  to field capabilities within 2 to 5 years—a process referred to as middle- tier acquisition. Middle-tier acquisition programs are generally exempt  from DOD’s traditional acquisition and requirements development  policies.

You asked us to examine DOD’s efforts to implement provisions to  streamline acquisition oversight included in the National Defense  Authorization Acts for Fiscal Years 2016 and 2017. This report addresses  (1) the progress DOD has made to implement selected oversight reforms  related to major defense acquisition programs; (2) how DOD has used  middle-tier acquisition pathways and the extent to which DOD has  developed guidance on middle-tier program oversight; and (3) challenges  DOD faces related to reform implementation.

This report also includes an assessment of DOD’s efforts to implement  our previous portfolio management recommendations and identifies  opportunities and challenges related to portfolio management that DOD  may face as it continues to implement acquisition reforms. This  information is included pursuant to provisions in the conference report for  the National Defense Authorization Act for Fiscal Year 2018 and the  Senate Armed Services Committee report accompanying a bill for the  National Defense Authorization Act for Fiscal Year 2018 for us to review  project, program, and portfolio management standards within DOD.

We focused our review on five selected reforms from the National  Defense Authorization Acts for Fiscal Years 2016 and 2017 that we  determined substantially affected DOD’s oversight of acquisition  programs. Our selections were informed by our analysis of acquisition- related provisions in those Acts, our past work on factors affecting the  oversight of major defense acquisition programs, and interviews with  DOD officials on their perspectives. We also reviewed related  amendments to the reforms we selected that were included in National  Defense Authorization Acts for subsequent years.

To assess the progress DOD has made to implement selected oversight  reforms for major defense acquisition programs, we analyzed three  selected reforms that affect processes related to DOD’s oversight of  major defense acquisition programs: designating military departments to be the milestone decision  conducting independent technical risk assessments; and establishing cost, fielding, and performance goals.

We also analyzed one reform that restructured acquisition oversight  functions in the Office of the Secretary of Defense. For each of these  reforms, we analyzed DOD and military department policies and guidance  as of March 2019 to determine steps DOD and the military departments  had taken to implement the reforms. Further, we requested and analyzed  data provided by DOD about the milestone decision authority levels for  the major defense acquisition program portfolio. To obtain additional  information about the restructuring of acquisition oversight functions  within the Office of the Secretary of Defense, we also reviewed  organizational charts and staffing and vacancy data. To assess the  reliability of the data on major defense acquisition programs and staffing,  we discussed the data and sources used to compile the data with DOD  officials, reviewed the data for logical inconsistencies, and compared the  data when possible to other sources, such as publicly available lists of  major defense acquisition programs. On the basis of these steps, we  determined that the data we used were sufficiently reliable for the  purposes of assessing the status of DOD’s implementation of the reforms  we reviewed.

To determine how the military departments have used middle-tier  acquisition pathways, we reviewed the relevant statute and DOD and  military department guidance with regard to the selection and oversight of  programs. We compared the guidance with our past work on elements of  business cases that should be completed at program initiation to  determine what elements were addressed by DOD guidance. We also  reviewed the extent to which the guidance addressed relevant internal  controls related to consistent measurement of program performance.  Further, we obtained information from the military departments about the  number and types of middle-tier acquisition programs that they were  executing as of March 2019. For each of the military departments, we  selected three middle-tier programs to review in additional detail. We  selected programs to obtain a range of program costs and types of  programs being executed under the middle-tier acquisition pathway (such  as space, artillery, and ground vehicle programs). For these programs,  we collected and analyzed additional information such as acquisition  decision memorandums, acquisition strategies, program cost and  schedule estimates, and risk assessments to determine what business  case documents the programs had developed prior to program initiation.

To assess the challenges DOD faces with regard to reform  implementation, we reviewed policy and guidance issued by DOD that  outlined roles and responsibilities for the Office of the Secretary of  Defense and the military departments with regard to acquisition oversight  and compared them to leading practices for leadership involvement in  agency transformations that we had identified in prior work. We also  collected and analyzed information about DOD’s actions taken to  implement prior recommendations we have made to improve portfolio  management at DOD and analyzed how the acquisition reforms we  reviewed had affected portfolio management at DOD. Lastly, we reviewed  DOD’s plans and ongoing efforts to develop performance measures and  collect data to assess the effects of acquisition reforms and compared  these efforts with related success factors for reform implementation  identified in our past work. See appendix I for more information on our  objectives, scope, and methodology.

We conducted this performance audit from March 2018 to June 2019 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

DOD’s acquisition of weapon system programs has been on our High  Risk List since 1990 because DOD programs consistently fall short of  cost, schedule, and performance expectations. Congress and DOD have  long explored ways to curtail these cost, schedule, and performance  problems, and both took related actions about a decade ago, with  Congress passing the Weapon Systems Acquisition Reform Act of 2009  and DOD implementing its “Better Buying Power” initiatives. The  Weapon Systems Acquisition Reform Act of 2009 aimed to improve the  organization and procedures of DOD for the acquisition of major weapon  systems, for example by revising the certifications that programs were  expected to complete before approval for system development start. The  new certifications included the need to conduct trade-offs among cost,  schedule, and performance objectives and for independent verification of  technology maturity. In 2010, DOD started its own acquisition reform  initiatives through “Better Buying Power.” These reforms required DOD  programs to conduct analyses of program affordability and set cost  targets, among other things, which placed cost constraints on programs  and encouraged programs to find cost improvements during program  execution. These and other reforms championed sound management  practices, such as realistic cost estimating, increased use of prototyping,  and systems engineering. In 2016, we found that DOD was beginning to  decrease the amount of cost growth in its major defense acquisition  program portfolio.

Despite DOD’s improvements in cost control, however, members of  Congress remained concerned that the DOD acquisition process was  overly bureaucratic and too slow to deliver capability to the warfighter.  Congress enacted numerous additional acquisition-related provisions in  the National Defense Authorization Acts for Fiscal Year 2016 and  subsequent years that addressed the processes with which DOD and the  military departments acquire goods and services and encourage  innovation. These provisions addressed a wide range of acquisition  issues, such as the:  creation of new processes for oversight of major defense acquisition  programs; development of streamlined alternative acquisition paths; and  changes to DOD’s other transaction authority, which allows DOD to  enter into agreements that generally do not follow a standard format  or include terms and conditions required in traditional mechanisms,  such as contracts or grants.

Congress also required that DOD establish a panel in the National  Defense Authorization Act for Fiscal Year 2016, referred to as the  “Section 809 Panel,” to identify ways to streamline and improve the  defense acquisition system. The panel issued its final report in January  2019, which, together with its earlier reports, included a wide range of  recommendations aimed at changing the overall structure and operations  of defense acquisition.

DOD Acquisition Programs and Authorities

DOD acquisition policy defines an acquisition program as a directed,  funded effort that provides a new, improved, or continuing materiel,  weapon, or information system, or a service capability in response to an  approved need. DOD Directive 5000.01, The Defense Acquisition  System, provides management principles and mandatory policies and  procedures for managing all acquisition programs. Oversight levels and  procedures for DOD’s acquisition programs are outlined in DOD  Instruction 5000.02, Operation of the Defense Acquisition System.

Traditionally, defense acquisition programs are classified into acquisition  categories based on the value and type of acquisition. DOD’s most costly  programs have historically been referred to as major defense acquisition  or Acquisition Category I programs. Programs with lower costs are  categorized as Acquisition Category II or III programs. The acquisition  category of a program can affect oversight levels and procedures, such  as what program information and documents are required and who is  designated as the milestone decision authority. Among other  responsibilities, the milestone decision authority approves entry of an  acquisition program into the next phase of the acquisition process and is  accountable for cost, schedule, and performance reporting.

Overview of DOD Weapon System Decision-Making Processes

DOD’s acquisition process includes three major milestones at which  program offices provide information to or receive a waiver from the  milestone decision authority. The milestone decision authority then makes  a decision on whether the program is ready to transition to the next  acquisition phase.

The milestones normally represent transition points in programs at which  there is a marked increase in the funding required for the program.

Milestone A is the decision for an acquisition program to enter into the  technology maturation and risk reduction phase.

Milestone B is the decision to enter the engineering and  manufacturing development phase.

Milestone C is the decision to enter the production and deployment  phase.

Programs may start at different milestones depending on the  circumstances of the particular program, such as whether the  technologies the program plans to use are mature. Some major defense  acquisition programs, such as the Marine Corps’ Amphibious Combat  Vehicle program and the Navy’s Next Generation Jammer-Mid Band  program, entered the acquisition system at milestone A. Other programs,  such as the Air Force’s Combat Rescue Helicopter program and the  Army’s Armored Multi-Purpose Vehicle program, entered directly at  milestone B without having a milestone A because technologies were  considered mature by the Office of the Secretary of Defense and an  independent review team, respectively.

Figure 1 illustrates the key milestones associated with the defense  acquisition system.

DOD’s acquisition policy encourages tailoring the acquisition process,  including tailoring of documentation or information requirements. In  previous work, we identified opportunities for DOD to tailor the  documentation and oversight needed for major defense acquisition  programs. In 2015, we found that 24 acquisition programs we surveyed  spent, on average, over 2 years completing up to 49 information  requirements for their most recent milestone decision. We found that  DOD’s review process was a key factor that influenced the time needed  to complete the information requirements. In total, the requirements  averaged 5,600 staff days to document, yet acquisition officials  considered only about half of the requirements as high value. We  recommended that DOD eliminate reviews and information requirements  that do not add value or are no longer needed. DOD agreed with both  recommendations and took some actions through its Better Buying Power  initiatives to streamline documentation and staff reviews.

Among the information requirements that acquisition officials considered  most valuable were those that support a sound business case. A solid,  executable business case provides credible evidence that (1) the  warfighter’s needs are valid and that they can best be met with the  chosen concept, and (2) the chosen concept can be developed and  produced within existing resources—such as technologies, design  knowledge, funding, and time. Establishing a sound business case for  individual programs depends on disciplined requirements and funding  processes, and calls for a realistic assessment of risks and costs; doing  otherwise undermines the intent of the business case and increases the  risk of cost and schedule overruns and performance shortfalls. The  program’s business case typically includes documentation of the  capabilities required of the weapon system, the strategy for acquiring the  weapon system, sound cost estimates based on independent  assessments, and a realistic assessment of technical and schedule risks.

DOD Weapon System Acquisition Program Oversight Roles and Responsibilities

Several entities at the enterprise level (meaning the Office of the  Secretary of Defense, Joint Chiefs of Staff, and Joint Staff) and the  military department level play a role in the oversight and budgeting for  DOD weapon system acquisition programs. In general, at the enterprise  level, the acquisition and budgeting processes are managed by  subordinate offices within the Office of the Secretary of Defense. More  specifically:

The Under Secretary of Defense for Research and Engineering is  responsible for establishing policies on and supervising all aspects of  defense research and engineering, technology development,  technology transition, prototyping, experimentation, and  developmental testing activities and programs, including the allocation  of resources for defense research and engineering. This organization  has a significant role in activities prior to milestone B, but also  interacts with major defense acquisition programs throughout their life  cycles with regard to technical risks. For major defense acquisition  programs, the Under Secretary conducts assessments in areas such  as technology maturity, interoperability, and cyber security.

The Under Secretary of Defense for Acquisition and Sustainment  is responsible for establishing policies on and supervising all matters  relating to acquisition (including (1) system design, development, and  production; and (2) procurement of goods and services) and  sustainment (including logistics, maintenance, and materiel  readiness). This organization has certain oversight responsibilities for  major defense acquisition programs throughout the acquisition  process, such as collecting and distributing performance data. The  Under Secretary is the Defense Acquisition Executive and serves as  the milestone decision authority for certain major defense acquisition  programs.

The Director, Cost Assessment and Program Evaluation and the  Under Secretary of Defense (Comptroller) manage the annual  budget preparation process for acquisition programs. These  organizations have cost assessment and budgetary responsibilities,  respectively, for major defense acquisition programs leading up to  each milestone and once these programs have been fielded.

At the military department level, the service acquisition executive, also  known as the component acquisition executive, is a civilian official within  a military department who is responsible for all acquisition functions within  the department and can serve as the milestone decision authority. The  following officials serve as the service acquisition executive for the  military departments:  the Assistant Secretary of the Air Force (Acquisition, Technology, and  Logistics) for the Air Force;  the Assistant Secretary of the Army (Acquisition, Logistics and  Technology) for the Army; and  the Assistant Secretary of the Navy (Research, Development and  Acquisition) for the Navy and the Marine Corps.

We focused our review on five selected reforms from the National  Defense Authorization Acts for Fiscal Years 2016 and 2017. Three of the  reforms affect the processes related to DOD’s oversight of major defense  acquisition programs, the fourth restructured acquisition oversight  functions in the Office of the Secretary of Defense, and the fifth provides  alternative acquisition pathways for programs that are not considered  major defense acquisition programs and have an objective of being  completed within 5 years. Table 1 identifies the source of the five reforms  that we reviewed and provides a brief summary of each reform.

For additional detail on the statute, amendments, and related DOD  guidance we reviewed, see appendix II.

DOD Has Made Progress in Implementing Acquisition Oversight Reforms and Efforts to Reorganize Are Ongoing

We found that DOD has made progress implementing reforms that have  affected the oversight of major defense acquisition programs. Decision- making authority for these programs has been realigned between the  Office of the Secretary of Defense and the military departments. In  addition, new processes are in place to improve DOD’s consideration of  program cost, fielding, and performance goals and assessment of  technical risk although questions remain about how they will be  implemented. The Office of the Secretary of Defense has also  restructured in an effort to increase innovation in the earlier stages of the  acquisition process and reduce cost, schedule, and performance risks in  later stages. While the restructure has begun to take shape, additional  steps remain to be completed, including developing charters and fully  staffing new offices. These steps are important to determining how  acquisition oversight roles within the Office of the Secretary of Defense— which had been executed by a single office for decades—will be divided  and how new offices will be structured to effectively carry out their work.

DOD Has Implemented Reforms That Affect the Oversight of Major Defense Acquisition Programs

Milestone decision authority for most major defense acquisition programs  now resides with the military departments, a reform generally required for  programs starting after October 1, 2016 by section 825 of the National  Defense Authorization Act for Fiscal Year 2016. According to data from  DOD’s Defense Acquisition Visibility Environment system, as of March  2019, milestone decision authority was at the military department level for  80 of 89 major defense acquisition programs. The 80 programs include all   six programs that started at milestone B or an equivalent milestone since  this reform became effective on October 1, 2016, and 74 other programs  that started before the reform became effective.

The nine programs retained by the Office of the Secretary of Defense all  began prior to the reform becoming effective and include programs that  are high risk, joint, or have had significant cost or schedule growth, such  as the F-35 Joint Strike Fighter program and the Army’s Integrated Air  and Missile Defense program. See appendix III for more information  about milestone decision authority, including a list of the major defense  acquisition programs as of March 2019 and the milestone decision  authority for each.

Prior to this reform going into effect, the Under Secretary of Defense for  Acquisition, Technology and Logistics within the Office of the Secretary of  Defense typically was the milestone decision authority for major defense  acquisition programs until they entered the production and deployment  phase—that is, for the milestone A, B, and C decisions. The Under  Secretary then typically delegated milestone decision authority to the  military departments after the milestone C decision. Under the new  reform, the Secretary of Defense may designate an alternate milestone  decision authority under certain circumstances. For example, the  Secretary may determine that the program meets one of several  characteristics outlined in statute, such as addressing a joint requirement  or the program being critical to a major interagency requirement. There  are now substantially more major defense acquisition programs with  decision authority at the military department level. This change resulted  from both the statutory reform for newly started programs and changes to  milestone decision authority for existing programs resulting from a  separate review conducted by the Office of the Secretary of Defense after  the reform became effective, wherein the military department was  designated the milestone decision authority for approximately 20  programs. See figure 2 for trends in the level of milestone decision  authority from 2012 to 2019. major defense acquisition program will  provide an options matrix to stakeholders  including the Under Secretaries of  Defense for Research and Engineering  and Acquisition and Sustainment, Cost  Assessment and Program Evaluation,  and the Joint Staff, which must include at  least three options that represent differing  assumptions about possible solutions,  technical risks, cost, schedule, and  affordability. gathering and distributing data and lessons learned, and  conducting or approving independent cost estimates.

These stakeholders must be granted the  access necessary to complete  independent analysis in their area of  responsibility. This analysis will consider  aggregated risk regarding technical  feasibility, cost, schedule, and  affordability, and will be submitted to the  milestone decision authority.

A goal establishment meeting will be held  within 30 days of the program’s analysis of  alternatives outbrief and will be co-chaired by  the milestone decision authority and Vice  Chief of the pertinent military service(s) and  supported by the stakeholders identified  above.  As of March 2019, no programs have held a  milestone A since the reform became  effective, and no programs have had goals  established under the new process. establish a process to develop program cost, fielding, and  performance goals for major defense acquisition programs that reach  milestone A after October 1, 2017. The statute described the goals as  follows: (1) the cost goal is for both procurement unit cost and  sustainment cost, (2) the fielding goal is the date for initial operational  capability, and (3) the performance goal is for technology maturation,  prototyping, and a modular open system approach. DOD issued a  policy for the process in November 2018, stating that stakeholders will  complete independent analyses in their areas of responsibility to  consider the aggregated risk regarding technical feasibility, cost,  schedule, and affordability, which will be submitted to the milestone  decision authority (typically at the military department level). The  policy stated that it applies to all major defense acquisition programs  that enter the acquisition process after October 1, 2017, without  regard to what milestone initiates the program. The policy also stated  that the Office of the Secretary of Defense will have the opportunity to  consult with the milestone decision authority on revised goals if the  program exceeds its initial cost or fielding goals prior to the next  milestone or production decision.

DOD acquisition policy already required programs to document  objectives for system cost, schedule, and performance in an  acquisition program baseline at milestone B and affordability cost  goals were to be set at milestone A. Under the new process, fielding  and performance goals are established earlier and all three goals  (cost, fielding, and performance) are required to be established before  funds are obligated for technology development, systems  development, or production, rather than being set at specific program  milestones. The new process also adds a meeting to review and  discuss the goals before they are approved by the milestone decision  authority. Officials from the Office of the Under Secretary of Defense  for Acquisition and Sustainment told us that this new process is  intended to consolidate existing information to inform earlier decisions  on which investments the department wants to make.

As of March 2019, no major defense acquisition programs have held a  milestone A since the statutory requirement became effective, and no  major defense acquisition programs have had goals established under  the new process. According to officials from the Office of the Under  Secretary of Defense for Acquisition and Sustainment, no new programs  have been required to have goals established since DOD’s policy for the  process was issued in November 2018. These officials told us they rely  on the milestone decision authority to notify them when goals need to be  established and that the first programs expected to have goals  established under the new policy are the Army’s Gator Landmine  Replacement Program and the Air Force’s Mk21A Reentry Vehicle. Both  programs are slated to go through the process in mid-2019.

Independent technical risk assessments. The Under Secretary of  Defense for Research and Engineering is now responsible for  conducting or approving independent technical risk assessments for  major defense acquisition programs prior to milestones A and B and  before production decisions. According to DOD’s December 2018  independent technical risk assessment policy, the assessments will  consider the full spectrum of technology, engineering, and integration  risk, including critical technologies and manufacturing processes, and  the potential impacts to cost, schedule, and performance.

The reform required the assessments for major defense acquisition  programs reaching milestone A after October 1, 2017; no major  defense acquisition programs have held a milestone A since that date.  DOD policy issued in December 2018 implementing the statute states  that the assessments will be conducted for all major defense  acquisition programs at each upcoming milestone throughout the  acquisition process, effective December 3, 2018. As a result, the  assessments will be conducted regardless of whether the program  reached milestone A after October 1, 2017. As of March 2019, the  Office of the Under Secretary of Defense for Research and  Engineering had conducted eight independent technical risk  assessments on major defense acquisition programs. One additional  assessment on the Infrared Search and Track Block II program was  delegated to the Navy to conduct, although the Office of the Under  Secretary of Defense for Research and Engineering still approved the  assessment.

While DOD acquisition guidance previously provided for similar types  of assessments, they were not always required to be conducted or  approved at the Office of the Secretary of Defense level for all major  defense acquisition programs. DOD acquisition guidance previously  provided for the Office of the Secretary of Defense to request broad  program assessments related to systems engineering, including risk  areas, at all milestones for major defense acquisition programs with  milestone decision authority at the Office of the Secretary of Defense  level. Additionally, all major defense acquisition programs were  required to have a separate assessment of critical technology  elements prior to entering the system development phase or the  production and deployment phase if the system enters the acquisition  life cycle after system development. DOD’s December 2018 policy  requires that independent technical risk assessments be conducted or  approved at the Office of the Secretary of Defense level by the Office  of the Under Secretary of Defense for Research and Engineering  unless this responsibility is delegated, regardless of the level of the  milestone decision authority.

Office of the Secretary of Defense Reorganization Is Ongoing and Many Key Leadership Positions Are Not Filled

The Office of the Secretary of Defense officially reorganized its  acquisition organization on January 31, 2018, in response to Section 901  of the Fiscal Year 2017 National Defense Authorization Act. Under the  reorganization, responsibilities of the former Under Secretary of Defense  for Acquisition, Technology and Logistics were divided between two new  offices—the Under Secretary of Defense for Research and Engineering  and the Under Secretary of Defense for Acquisition and Sustainment (see  fig. 3 and app. IV for organizational charts).

According to the conference report accompanying the legislation, the  priorities framing the conference discussions on reorganization included  elevating the mission of advancing technology and innovation within  DOD, and fostering distinct technology and acquisition cultures. The  report further states that the conferees expect that the Under Secretary of  Defense for Research and Engineering would take risks, test, and  experiment, and have the latitude to fail, as appropriate. Additionally, the  report states that the conferees expect the Under Secretary of Defense  for Acquisition and Sustainment to focus on timely, cost-effective delivery  and sustainment of products and services, and seek to minimize any risks  to that objective. It is too early to say whether the goals of the  reorganization have been realized.

In July 2018, the Deputy Secretary of Defense issued a memorandum  outlining the overall organizational structures, roles, and responsibilities of  the two new Under Secretary offices. Responsibilities of many prior  subordinate offices were realigned to one of the two new Under Secretary  offices as part of the reorganization. For example, systems engineering  falls under the Under Secretary of Defense for Research and Engineering  and contracting policy and oversight falls under the Under Secretary of  Defense for Acquisition and Sustainment.

New offices or positions were also created during the reorganization. For  example, the Office of the Under Secretary of Defense for Research and  Engineering created eight assistant director positions to serve as resident  experts in strategic technology areas, such as cyber, quantum science,  and hypersonics. Similarly, the Office of the Under Secretary of Defense  for Acquisition and Sustainment created an Assistant Secretary of  Defense for Sustainment. Previously, sustainment activities were spread  across several organizations headed by two Assistant Secretaries of  Defense.

While foundational steps to stand up the two new Under Secretary offices  have been taken, as of March 2019, reorganization actions were ongoing  in two major areas: completing chartering directives that define the scope  of responsibilities for the two new offices and hiring additional people for  the new offices, including for several senior leadership positions.

Chartering directives: Officials from the Office of the Chief  Management Officer originally expected charters for the two offices to  be completed by January 2019, but progress has been delayed.  According to DOD policy, chartering directives are required to define  the scope of functional responsibilities and identify all delegated  authorities for the chartered organizations. According to a July 2018  memorandum issued by the Deputy Secretary of Defense, the Chief  Management Officer is to oversee the development of the charters.  Officials from the Office of the Chief Management Officer stated that  they are doing so with significant input from the Under Secretaries of  Defense for Research and Engineering and Acquisition and  Sustainment. These officials told us that the development of the  charters has taken longer than expected because redistributing the  responsibilities of a single office into two new offices was complicated  due to the number of shared or partially overlapping interests.

Officials from the Offices of the Under Secretaries of Defense for  Research and Engineering and Acquisition and Sustainment now  estimate the charters will be completed in July 2019 after department- wide coordination, though they said that time frame may be optimistic  given the challenges to date. These officials also told us they expect  that they will need to make additional changes to other existing  acquisition policies and guidance to incorporate the new content of  the chartering directives once complete.

Hiring additional employees: In order to stand up the two newly- created organizations, on February 1, 2018, 516 civilian and military  positions from the former Office of the Under Secretary of Defense for  Acquisition, Technology and Logistics were divided between the two  new Under Secretary offices. Finalizing staffing for both offices has  been a gradual process that will not be completed until at least fiscal  year 2020 because of the need to: (1) reduce positions to meet  statutorily-directed cost-savings objectives; (2) realign positions  between the two offices; and (3) hire additional staff. Table 2  provides additional detail on past and expected changes to authorized  positions.

Both Under Secretaries are still working to staff their offices, with  approximately 30 percent of current positions vacant in the Office of the  Under Secretary of Defense for Research and Engineering, and 8 percent  of current positions vacant in the Office of the Under Secretary of  Defense for Acquisition and Sustainment. See figure 4 for the current  status of staffing within both offices.

Both Under Secretaries have experienced challenges while staffing their  offices. For example:

The Office of the Under Secretary of Defense for Acquisition and  Sustainment has experienced challenges stemming from needing to  meet required personnel reductions while also hiring staff to align with  the revised priorities from the reorganization. As part of the  restructuring, the office will absorb all of the 57 remaining civilian and  military position reductions that were originally assigned to the former  Office of the Under Secretary of Defense for Acquisition, Technology  and Logistics. These reductions will occur during both fiscal years  2019 and 2020. At the same time, officials said they are still working  to hire staff with skills in needed areas such as data analytics.  Officials said they are leveraging existing authorities such as voluntary  early retirement authority and voluntary separation incentive  payments to meet their targeted number of authorized positions by the  end of fiscal year 2020.

Officials from the Office of the Under Secretary of Defense for  Research and Engineering said their challenges have primarily been  negotiating the appropriate number of positions for the organization  and staffing the organization in a timely manner. For example, 13  positions are not currently available to be filled because they will not  be transferred from the Office of the Under Secretary of Defense for  Acquisition and Sustainment until fiscal year 2020. The officials also  stated that there have been delays related to developing new position  descriptions, revalidating existing position descriptions, and finding  individuals with the right skill sets for positions.

Both offices have been delayed in filling key leadership positions.

According to officials from these offices, vacant positions include the  Deputy Director of Mission Engineering and Integration, the Director  of Systems Engineering, and the Principal Director of Defense Pricing  and Contracting. Senior officials from both offices told us that they  have been unable to fill some vacant senior executive positions since  the most recent Secretary of Defense resigned on December 31,  2018. The inability to fill these positions is due to the Office of  Personnel Management’s general policy to suspend processing for  senior executive service career appointments when an agency head  leaves, until a successor is appointed at the agency. As of March  2019, a new Secretary of Defense had yet to be confirmed. Senior  level officials also told us that some decisions about structure and  staffing may be held up until after these executive positions are filled,  but that in the interim, they are moving forward with daily operations  and in some instances have other employees acting in those roles.

Military Departments Are Using Middle-Tier Acquisition Pathways, but DOD Has Yet to Determine How Certain Aspects of Program Oversight Will Work Military Departments Are Using Middle-Tier Acquisition Pathways to Execute Programs of Varying Costs and Complexity

As of March 2019, the military departments had begun using middle-tier  acquisition pathways for over 35 rapid prototyping and rapid fielding  programs under interim guidance issued by the Under Secretary of  Defense for Acquisition and Sustainment and the military departments.  However, DOD has yet to determine certain aspects of program  oversight, including what information military departments should  consider in selecting programs and what metrics and data the Office of  the Secretary of Defense and military department leaders should use to  assess performance.

The Departments of the Air Force, Army, and Navy have begun to  execute over 35 unclassified and classified acquisition programs using  new acquisition pathways distinct from the traditional DOD acquisition  process. Section 804 of the National Defense Authorization Act for  Fiscal Year 2016 required DOD to issue guidance establishing two new  streamlined acquisition pathways for DOD—rapid prototyping and rapid  fielding—under the broader term “middle tier of acquisitions.” According to  the Joint Explanatory Statement accompanying the National Defense  Authorization Act, the guidance was to create an expedited and  streamlined “middle tier” of acquisition programs intended to be  completed within 5 years. The Joint Explanatory Statement noted that  middle-tier programs would be distinctive from rapid acquisitions that are  generally completed within 6 months to 2 years and traditional  acquisitions that last much longer than 5 years. Statute lays out more  specific intended time frames and expectations for programs using these  two pathways:

The rapid prototyping pathway is to provide for the use of  innovative technologies to rapidly develop fieldable prototypes to  demonstrate new capabilities and meet emerging needs. The  objective of a rapid prototyping program is to field a prototype that can  be demonstrated in an operational environment and provide for a  residual operational capability within 5 years of the development of an  approved requirement.

The rapid fielding pathway is to provide for the use of proven  technologies to field production quantities of new or upgraded  systems with minimal development required. The objective of a rapid  fielding program is to begin production within 6 months and complete  fielding within 5 years of the development of an approved  requirement.

Middle-tier acquisition pathways are distinct from the traditional  acquisition system for major defense acquisition programs. These  pathways allow for programs to be exempted from the acquisition and  requirements processes defined by DOD Directive 5000.01 and the  Manual for the Operation of the Joint Capabilities Integration and  Development System. The statute does not identify a dollar limit for  programs using middle-tier acquisition pathways.

Middle-tier programs are typically approved for initiation by the service  acquisition executive, although Air Force policy also allows for smaller  programs to be initiated by the program executive officer. Table 3 shows  the number of unclassified programs initiated by the military departments  as of March 2019.

The middle-tier programs initiated to date represent a range of products,  dollar amounts, and complexity. For example, one of the smaller dollar  value programs is an approximately $30 million Navy effort to develop a  prototype rocket motor that would support extended ranges for an existing  missile. One of the larger dollar value programs is a multibillion dollar  Army effort to develop the next generation combat vehicle. The military  departments generally require funding these programs through the  traditional budget process, using DOD’s existing planning, programming,  budgeting, and execution process. Based on estimated program costs  reported by the military departments, we found that approximately half of  the programs initiated to date would be categorized as major defense  acquisition programs if they were not being pursued under a middle-tier  pathway. In some cases, such as the Army’s Lower Tier Air and Missile  Defense Sensor program, an existing program planned as a major  defense acquisition program shifted to a middle-tier acquisition pathway.  Appendix V includes a list of middle-tier acquisition programs started by  the military departments as of March 2019.

DOD Has Issued Interim Guidance, but Has Yet to Determine Certain Aspects of Middle-Tier Program Oversight

Although DOD and the military departments have issued interim guidance  for using middle-tier acquisition pathways, we found that DOD has not  provided department-wide guidance on how certain aspects of program  oversight will be conducted. DOD has yet to determine what types of  business case information should be submitted to decision makers to help  ensure well-informed decisions about program initiation and how program  performance will be measured consistently.

DOD and the Military Departments Have Each Issued Interim Guidance

Section 804 of the National Defense Authorization Act for Fiscal Year  2016 required the Under Secretary of Defense for Acquisition,  Technology and Logistics to establish guidance for middle-tier  acquisitions. In response, the Under Secretary of Defense for Acquisition  and Sustainment issued interim guidance in April 2018 that provided the  military departments and other DOD components with the authority to  implement middle-tier acquisition programs on an interim basis through  September 30, 2019. The guidance laid out the broad purposes and  requirements of middle-tier acquisition authorities, and encouraged the  military departments and other DOD components using middle-tier  acquisition pathways to develop specific implementation processes and  procedures to implement the interim authority. Between April 2018 and  September 2018, the military departments each issued their own  implementing guidance, which provided additional details on how middle- tier programs would be selected and overseen within their department  during the period of the interim authority.

Subsequently, the Under Secretary of Defense for Acquisition and  Sustainment issued two additional interim guidance memorandums: the  first in October 2018, which described how the Office of the Secretary of  Defense and the Joint Staff would conduct oversight of the military  departments’ use of middle-tier acquisition pathways, and the second in  March 2019, which addressed sustainment planning considerations for  programs using the rapid fielding pathway. The Director, Cost  Assessment and Program Evaluation, also issued guidance in April 2019  that included a life-cycle cost estimating policy for programs using the  rapid fielding pathway.

DOD Guidance Does Not Consistently Identify Business Case Elements to Be Developed and Considered for Program Selection

Statute requires that the guidance from the Office of the Secretary of  Defense include a “merit-based process” for considering potential middle- tier programs, although the interim guidance does not describe what the  process should include or what information should be considered by  decision makers to assess merit other than meeting the needs  communicated by the Joint Chiefs of Staff and the combatant  commanders. Guidance from each of the military departments provides  additional detail on the program selection process, to include describing  generally the type of information decision makers should consider when  selecting programs. Neither the Office of the Secretary of Defense’s  guidance nor the military departments’ guidance fully identifies key  elements of a business case to be provided as part of the program  initiation process.

Our past work has shown that in order to make sound decisions about  initiating acquisition programs, it is important that decision makers have  the information they need to assess the business case, including that (1)  the warfighter need exists and that it can best be met with the chosen  concept and (2) the concept can be developed and produced within  existing resources. Information needed to establish a business case for  a traditional acquisition program typically includes a requirements  document (which provides information on the capabilities required of the  weapon system); the strategy for acquiring the weapon system; sound  cost estimates based on independent assessments; and a realistic  assessment of risks, including those risks related to technology and  schedule. For a middle-tier acquisition program, business case  information would help decision makers make well-informed decisions, to  include assessing whether the program is likely to meet objectives  established in statute to complete a prototype with a residual operational  capability (in the case of a rapid prototyping program) or complete fielding  (in the case of a rapid fielding program) within 5 years of an approved  requirement.

Programs using a middle-tier pathway are intended to be completed  within 5 years, and guidance may provide for expedited and streamlined  procedures. As a result, the appropriate documents to provide business  case information for a middle-tier acquisition program may not need to be  as detailed as those for a major defense acquisition program. These  documents may also vary to some extent depending on whether a  program is a rapid prototyping or a rapid fielding program. However,  having this type of information available in some form at program initiation  can help decision makers to assess the soundness of a program’s  business case at the time a decision is made to start a new program.  Oversight at this time is critical because, as we have previously reported,  program initiation presents the greatest point of leverage in the program  life cycle for decision makers. Table 4 provides additional detail about  certain types of business case documentation that are to be considered at  program initiation for middle-tier acquisition programs according to the  Office of the Secretary of Defense and the military departments’  guidance.

Section 804 of the National Defense Authorization Act for Fiscal Year  2016 directed the Under Secretary of Defense for Acquisition, Technology  and Logistics to establish guidance for middle-tier acquisitions within 180  days of enactment of the statute (which would have been May 2016), but  guidance was not issued until April 2018. According to officials who were  involved in efforts to develop the guidance, the Office of the Secretary of  Defense circulated multiple iterations of draft guidance, but was unable to  reach agreement with the military departments because of concerns that  the guidance was too burdensome. These officials told us that as a result,  the Under Secretary of Defense for Acquisition and Sustainment decided  instead to issue broad interim guidance and allow each of the military  departments and other DOD components to develop processes and  procedures to implement the interim authority. As stated in the Under  Secretary of Defense for Acquisition and Sustainment’s April 2018 interim  guidance, the Under Secretary of Defense for Acquisition and  Sustainment would develop final guidance for the department in 2019  based on lessons learned from the military departments and other DOD  components. According to officials from the Office of the Under Secretary  of Defense for Acquisition and Sustainment, the Under Secretary began  the process in February 2019 to develop this final guidance. The process  was in its initial stages as of March 2019 and officials involved told us  they hope to complete the final guidance by September 2019.

The business case information programs provided to decision makers at  initiation varied widely for nine middle-tier acquisition programs we  reviewed. We found that certain types of business case information, such  as an assessment of schedule risk that would indicate whether a program  could realistically be expected to be completed within time frame  objectives in statute, were often not completed at the time of program  initiation. For example:

Six programs had approved requirements at program initiation. Three  of these programs had requirements validated through DOD’s  traditional requirements process prior to the decision to start under a  middle-tier pathway. Two of these programs, both of which were Air  Force programs, had previously planned to start as major defense  acquisition programs. The third program, an Army program, had  requirements based on those approved for an existing major defense  acquisition program. The other three of these programs, all of which  were Navy programs, had high-level requirements that described, for  example, what environments the system should be tested in or what  quantity should be fielded. These requirements were approved by the  Navy’s Accelerated Acquisition Board of Directors, which includes the  Chief of Naval Operations and the Assistant Secretary of the Navy for  Research, Development and Acquisition, among other officials. Three  programs were still in the process of developing requirements at the  time of program initiation.

Only one of the nine programs had an approved acquisition strategy  at the time of program initiation. Officials from the other programs told  us they planned to develop an acquisition strategy or were in the  process of developing or updating one.

While all nine of the programs had developed at least a draft cost  estimate at program initiation, only one of the nine programs had an  assessment of its program cost estimate completed by the military  department cost agency at the time of program initiation. Officials  from three other programs said that an assessment by the military  department cost agency was in progress or planned. Officials from the  other five programs told us they had developed a draft cost estimate  at program initiation that in some cases was still expected to change  and that they did not plan for an assessment by the military  department cost agency.

The programs varied in the extent to which they assessed risk at  program initiation. Four programs had risk assessments that  addressed schedule and technology risks, which are types of risks we  have identified in our previous work as important to understanding a  program’s business case. Two other programs had risk assessments  that included either schedule or technology risks but not both. Officials  from the other three programs stated that they were still in the process  of assessing risks and had yet to assess risks related to meeting  statutory schedule objectives at the time of program initiation.

Without the Office of the Under Secretary of Defense for Acquisition and  Sustainment identifying in its final guidance the minimum program  information needed to help decision makers evaluate the program’s  business case, DOD cannot ensure that the military departments are  consistently considering these types of information. As a result, DOD is  not well positioned to ensure that approved middle-tier acquisition  programs represent sound investments and are likely to meet the  objective of delivering prototypes or capability to the warfighter within 5  years.

DOD Has Yet to Identify Metrics to Monitor Program Performance in a Consistent Manner

The Office of the Secretary of Defense and the military departments  generally collect program data but we found that neither the Office of the  Secretary of Defense nor the military departments has identified metrics  that would allow them to use that data to measure and report on program  performance in a consistent manner. Developing such metrics would  allow senior leaders in the Office of the Secretary of Defense and the  military departments to monitor and assess performance across the  portfolio of middle-tier programs during program execution, including  whether programs are on track to meet statutory objectives for rapid  prototyping and rapid fielding. Table 5 provides additional detail on the  extent to which guidance addresses the collection of program data and  identification of metrics to measure program performance.

The Office of the Secretary of Defense began collecting middle-tier  program data from the military departments in November 2018 as part of  an effort to ensure that middle-tier authority was being used appropriately  within the department. However, the office has yet to determine what  metrics it will use to measure program performance consistently across  the portfolio. Officials within the Office of the Secretary of Defense who  are involved with collecting the data told us that they are still refining what  data should be collected, determining how to standardize definitions to  improve the consistency of data, and considering how to use the data  collected to monitor program execution. For example, they are still trying  to determine the appropriate triggers that would allow them to know that a  middle-tier program may be experiencing cost or schedule challenges.

Similarly, guidance from two of the three military departments requires the  collection of program data, but the military departments also have not  identified metrics to consistently measure performance across programs.  The Navy’s guidance does not require the collection of program data or  identify metrics to measure program performance. Interim guidance from  the Air Force and the Army requires the collection of program data and  also requires programs to develop metrics to measure performance, but  these metrics are not required to be consistent across programs.  Decisions about specific metrics to be reported are left to the discretion of  the decision authority for each program, who is typically the service  acquisition executive or a program executive officer. As a result, these  metrics may not allow consistent measurement of performance across  programs because, for example, programs may have a different starting  point for reporting data, or may change the metrics that are being  assessed at different points within the life of a program.

According to federal internal control standards, the ability of agency  management to compare actual performance to planned or expected  results throughout the organization and analyze significant differences is  important to help ensure that the agency is meeting objectives and  addressing risks appropriately. These standards also state that agency  management should define objectives in quantitative or qualitative terms  to permit reasonably consistent measurement of performance toward  those objectives. For middle-tier acquisition programs, statute includes  objectives related to fielding time frames for both rapid prototyping and  rapid fielding programs. Additionally, for rapid prototyping, part of the  objective is that the prototype fielded can be demonstrated in an  operational environment and provide for residual operational capability.

Middle-tier acquisition programs are to be provided streamlined  processes, including for program oversight. Decisions about how to  measure program performance therefore should be considered in light of  how to facilitate oversight without losing the benefits of the flexibilities  offered by middle-tier pathways. However, without the Office of the Under  Secretary of Defense for Acquisition and Sustainment identifying in its  final guidance a minimum set of metrics that can be used to measure  performance of programs across the military departments, DOD risks not  knowing how the department’s portfolio of middle-tier programs is  progressing, including whether programs are on track to meet statutory  objectives for rapid prototyping and rapid fielding. As a result, senior  leaders in the Office of the Secretary of Defense and the military  departments may lack insight needed to identify and address emerging  challenges in a timely manner. This is particularly important given that the  portfolio includes complex, costly programs that address important  capability gaps for the department.

DOD Faces Challenges in Addressing Disagreements about Oversight Roles and Responsibilities, Improving Portfolio Management, and Assessing Effectiveness of Reforms

While DOD has made progress implementing individual reforms, it  continues to face challenges that affect the implementation of the reforms  we reviewed. First, we found that senior DOD leadership has not fully  addressed disagreements about the division of acquisition oversight roles  and responsibilities between the Office of the Secretary of Defense and  the military departments. As a result, there have been continuing  differences of opinion about how to implement specific reforms. Second,  DOD has yet to address persistent portfolio management challenges that  affect its ability to effectively manage its portfolio of weapon system  investments. Lastly, DOD has yet to develop processes to assess the  effectiveness of recent reforms. Without developing such processes,  DOD officials will not be well positioned to assess whether reforms are  having the intended effects, such as improving innovation and delivering  capability to the warfighter more quickly, or if additional changes are  necessary to achieve such outcomes.

Top DOD Leadership Has Not Fully Addressed Continuing Disagreements over the Division of Roles and Responsibilities for Acquisition Oversight

Top DOD leadership has not fully addressed disagreements that remain  about the division of acquisition oversight responsibilities between the  Office of the Secretary of Defense and the military departments. Our past  work has shown that in times of significant organizational transformation,  top leadership must set the direction, pace, and tone for the  transformation. Personal involvement of these leaders in driving change,  including the Secretary and Deputy Secretary, helps provide stability.  Internal control standards for federal agencies also emphasize the  importance of management communicating information down and across  organizational levels in order to enable personnel to perform key roles in  achieving objectives and addressing risks. The Deputy Secretary of  Defense has weighed in on the division of acquisition oversight  responsibilities within the Office of the Secretary of Defense and has  addressed specific roles and responsibilities for certain reforms. However,  despite continuing disagreements about the division of oversight roles  and responsibilities between the Office of the Secretary of Defense and  the military departments, DOD’s top leadership has not provided a  detailed framework addressing the appropriate roles of each party for  acquisition oversight.

Officials from the Office of the Secretary of Defense and the military  departments we met with expressed different opinions on the appropriate  oversight role of the Office of the Secretary of Defense. For example, the  Under Secretaries of Defense for Research and Engineering and  Acquisition and Sustainment both stated that in cases where the  milestone decision authority is at the military department level, the military  departments do not see the value in having the Office of the Secretary of  Defense involved. This is consistent with concerns that officials from all  three military departments have raised in speaking with us. Specifically,  officials from all three departments raised concerns that the Office of the  Secretary of Defense is overreaching on its oversight responsibilities in  some cases, and creating new oversight processes that contradict the  intent of recent reforms to speed up the acquisition process.

Implementation of several of the reforms we reviewed has resulted in  disagreements between the Office of the Secretary of Defense and the  military departments that have yet to be resolved. For example:

Cost, fielding, and performance goals. Despite the issuance of  policy by the Deputy Secretary of Defense in November 2018 on the  establishment of cost, fielding, and performance goals, military  department officials have continued to express concerns that the  process is too burdensome and involves too many stakeholders from  the Office of the Secretary of Defense. These officials stated that  Office of the Secretary of Defense involvement in programs with  decision authority at the military departments, such as participation in  meetings with the milestone decision authority to provide advice on  cost, fielding, and performance goals, would slow down programs that  other reforms were intended to accelerate. They added that they had  expressed these concerns to the Office of the Secretary of Defense  during the drafting of the policy, but they did not feel that their input  was appropriately considered in the final policy. Officials from the  Office of the Under Secretary of Defense for Acquisition and  Sustainment stated that the analysis and meetings that involve the  Office of the Secretary of Defense are ways for stakeholders to advise  the milestone decision authority on program decisions based on  information from existing oversight mechanisms, such as independent  cost estimates and analyses of alternatives. Previously this type of  oversight was conducted via multiple meetings leading up to program  milestones. The policy states that the policy procedures will be  revisited in 6 months and lessons learned incorporated where  needed.

Independent technical risk assessments. Debates about who  should conduct independent technical risk assessments were  elevated to the Deputy Secretary of Defense. Subsequently, the  Deputy Secretary issued guidance in December 2018 to reiterate that  the Under Secretary of Defense for Research and Engineering would  conduct or approve these assessments for all major defense  acquisition programs, although that responsibility may be delegated.  However, despite the issuance of new guidance, there continue to be  ongoing debates about when assessments will be delegated to the  military departments. The December 2018 guidance does not include  criteria for when responsibility for the assessments may be delegated.  Officials from the Office of the Under Secretary of Defense for  Research and Engineering said that decisions about whether to  delegate assessments should be based primarily on the risk level of  the program, but officials from military departments stated that these  assessments should be conducted within the military department.  Officials from the Office of the Under Secretary of Defense for  Research and Engineering told us that they had convened a joint  working group with the military departments in February 2019 to  address this and other implementation issues related to independent  technical risk assessments. In the meantime, nearly all assessments  continue to be conducted by the Office of the Under Secretary of  Defense for Research and Engineering.

Middle-tier acquisition. Office of the Secretary of Defense and  military department officials also disagree on the extent to which the  Office of the Secretary of Defense should weigh in on the  appropriateness of a program using a middle-tier pathway. DOD’s  October 2018 interim governance guidance provided that the Office of  the Secretary of Defense may determine that specific programs were  not appropriate for a middle-tier pathway. However, officials from the  Air Force and the Army expressed concerns to us about whether that  determination was appropriate to be made by the Office of the  Secretary of Defense since, from their perspective, programs should  be selected at the military department level. Office of the Secretary of  Defense officials also told us that there are differences of opinion  between them and the military departments on the appropriate  amount of information that programs should report to the Office of the  Secretary of Defense, including whether the same information should  be provided by all middle-tier programs, regardless of expected  program cost. As stated earlier, DOD is in the process of finalizing  guidance for middle-tier acquisition programs, which could address  these issues.

Documents that could outline roles and responsibilities of the various  parties for acquisition oversight are still being developed. For example, as  discussed earlier, officials from the Offices of the Under Secretaries of  Defense for Research and Engineering and Acquisition and Sustainment  told us that chartering directives for these offices, which are expected to  be completed in July 2019, may address to some extent how the offices  should work together and with the military departments and other external  organizations. In addition, officials from the Office of the Under Secretary  of Defense for Acquisition and Sustainment told us that while reforms are  currently being implemented under multiple different polices and guidance  documents, DOD Instruction 5000.02 will be substantially revised,  including to reflect the latest reforms. When completed, the instruction is  expected to provide further detail on how oversight activities will be  carried out by various acquisition entities. Officials stated that they hoped  to complete a version of the revision of DOD Instruction 5000.02 by the  end of 2019, but they acknowledged that this estimate was optimistic and  that it might take longer than expected to come to agreement on this  policy.

Without a comprehensive framework from top leadership in the near term  that addresses acquisition oversight roles and responsibilities in detail,  DOD’s ability to continue with reform implementation, including its ability  to finalize policies that could clarify roles and responsibilities, may be  slowed by ongoing disagreements. In the longer term, without resolving  these issues, DOD cannot ensure that it is achieving the balance between  oversight and accountability and efficient program management that  senior leadership expects as an outcome of acquisition reform. With too  little oversight, acquisition programs may not be properly scrutinized  before they are started, which could lead to poor program cost and  schedule outcomes. Alternatively, if new oversight processes are too  burdensome, DOD may not achieve the expected benefits of streamlining  its acquisition processes.

DOD Has Yet to Address Persistent Weapon System Portfolio Management Challenges

As part of this review, we also assessed DOD’s efforts to implement our  previous portfolio management recommendations and identified  opportunities and challenges related to portfolio management that DOD  may face as it continues to implement acquisition reforms. Our past work  has shown that when investments are not managed as a portfolio at the  enterprise level (meaning at the level of the Office of the Secretary of  Defense, Joint Chiefs of Staff, and Joint Staff), the military departments  plan to acquire more weapons than DOD can afford, sometimes develop  potentially duplicative solutions to common needs, and do not always  choose an optimal mix of investments to ensure the department can  maintain its technological edge in the future.

Realigning roles and responsibilities for decisions related to weapon  system programs between the Office of the Secretary of Defense and the  military departments could lead to further questions about who is  ultimately responsible and accountable for portfolio management  decisions if leadership roles are not clearly defined. Officials we met with  from the Office of the Secretary of Defense told us that questions remain  about the division of responsibilities between the Office of the Secretary  of Defense and the military departments for making these types of  portfolio management decisions. They told us that concerns we had  previously identified about the division of decision-making authority for  portfolio management had yet to be addressed during the implementation  of recent acquisition reforms, and that in some cases, the reforms had led  to additional questions. For example, the Under Secretary of Defense for  Research and Engineering told us that while the statute that created his  position as part of the restructuring of the former Office of the Under  Secretary of Defense for Acquisition, Technology and Logistics assigns  him responsibility for allocating resources for defense research and  engineering, because he does not have control over the research and  engineering budget, in actuality the military departments decide how to  prioritize their investments.

We found in August 2015 that DOD has had difficulty implementing  portfolio management at the enterprise level in part due to diffuse  decision-making responsibilities that make it difficult to determine who is  empowered to make enterprise-level weapon system investment  decisions. At that time, we recommended that DOD revise its portfolio  management directive in accordance with portfolio management best  practices. We also recommended that the Secretary of Defense  designate the Deputy Secretary of Defense or some appropriate delegate  responsible for the directive’s implementation, among other  recommendations. DOD partially concurred with the recommendations,  but the planned actions DOD identified at the time of our report did not  fully address the issues we identified. For example, DOD stated that it did  not plan to revise its portfolio management directive as we recommended,  but instead planned to rescind it and direct stakeholders to participate in  portfolio management through the requirements, acquisition, and budget  processes. In response, we expressed concern that this approach could  reinforce the stove-piped governance structure that we found to be an  impediment to integrated portfolio management.

As of March 2019, DOD had yet to implement our recommendations. An  official from the Office of the Under Secretary of Defense for Acquisition  and Sustainment told us that DOD is revising its portfolio management  directive, but that there was not yet an estimated completion date. We  are not making new recommendations on portfolio management in this  report, but we continue to believe that DOD should implement our prior  recommendations in order to improve its portfolio management  capabilities. See appendix VI for additional details on our assessment of  DOD’s progress in this area.

DOD Has Yet to Develop Processes to Assess the Effectiveness of Acquisition Reforms

DOD is beginning to monitor the implementation of certain reforms, but  has yet to establish processes to assess the overall effectiveness of its  reform efforts. Collectively, the reforms offer the potential for DOD to  significantly reduce the time needed to approve and field acquisition  programs by allowing the military departments additional opportunities to  tailor documents needed for approval and limiting oversight by the Office  of the Secretary of Defense. Ultimately, DOD anticipates that this  opportunity will improve the speed at which new capabilities are delivered  to the warfighter.

Our prior work has identified steps that agencies, such as DOD, can take  to help ensure successful implementation of reform efforts, including establishing clear outcome-oriented goals and performance measures putting in place processes to collect the needed data and evidence to  effectively measure the reforms’ outcome-oriented goals.

The Office of the Secretary of Defense has taken some initial steps to  collect data that may help to measure the outcomes of a few reforms, but  has yet to determine goals or processes for assessing the overall  effectiveness of the reforms. For example, as previously discussed, the  Office of the Under Secretary of Defense for Acquisition and Sustainment  began initial efforts to collect middle-tier acquisition program data, such  as cost and schedule data, from the military departments in November  2018. Officials from that office told us that once they address reliability  concerns with the data they are receiving, such as ensuring that  programs report schedule data in a consistent fashion, they anticipate  that they will be able to use the data to better understand the military  departments’ use of middle-tier acquisition pathways. However, according  to officials we spoke with from the offices of the Under Secretaries of  Defense for Research and Engineering and Acquisition and Sustainment,  DOD has not determined how it will assess whether the reforms are  collectively resulting in an acquisition process that is more efficient or how  it will measure their effect on cost and schedule outcomes. These officials  told us that it is important to have data to assess the effect of recent  acquisition policy and organizational changes, but they have not  determined specifically who will do the assessment, how it will be done,  and what data will be needed. They told us that as a part of the  reorganization of the Office of the Under Secretary of Defense for  Acquisition, Technology and Logistics, they are still in the process of  assessing data gaps and needs within the newly-formed organizations  and that this type of analysis needs to be completed before they  determine how they will assess recent reforms.

We recognize that assessing the cumulative effect of recent acquisition  reforms on the acquisition process and on the cost and schedule  performance of the major defense acquisition program portfolio could take  several years because a critical mass of programs will need to go through  the new acquisition processes. In the interim, however, determining how  an assessment of reforms will be conducted is an important first step in  determining whether the reforms are having their intended effect. If DOD  officials wait too long to plan for how the department will assess the effect  of recent acquisition reforms, including identifying who will be responsible  for the assessment and what data will be needed, they may miss the  opportunity to collect data from the beginning of implementation needed  to measure progress. As a result, they may not be informed about early  indications of improvements or problems in the cost, schedule, and  performance of programs.

Recent acquisition reforms have given DOD significant opportunity to  focus on delivering innovative capability to the warfighter more quickly  and reduce bureaucratic processes that had built up over time. While  DOD has made progress in implementing these reforms, continued  attention from top leadership would help ensure that the progress the  department has made is not unnecessarily slowed or halted. Middle-tier  acquisition will require careful consideration as the department proceeds  with the development of final guidance. Middle-tier programs are  generally exempt from traditional acquisition and requirements processes,  but they may still be large, expensive programs critical to the  department’s ability to meet its mission. Identifying the types of business  case elements decision makers should consider when initiating programs  would improve the department’s ability to ensure that the programs the  military departments select are sound investments and likely to succeed  using a middle-tier acquisition pathway. Identifying metrics to track  performance consistently across the portfolio of middle-tier programs will  provide necessary information to senior leaders once programs have  been started to assess the performance of middle-tier acquisition  programs, including whether they are well positioned to meet statutory  objectives.

The department also faces challenges that affect the implementation of  the reforms we reviewed. These sweeping changes have resulted in  some disagreements about oversight roles and responsibilities between  the Office of the Secretary of Defense and the military departments that  have not been fully resolved. Clear communication from top leadership of  a framework for oversight roles and responsibilities that is detailed  enough to address areas of continued disagreement would help the  department to move forward with effective implementation of the reforms.  Developing an approach to assess the effects of recent acquisition  reforms is also critical so that DOD can monitor whether reforms are  collectively having the effect of speeding up the acquisition process  without unintended negative consequences on cost and performance of  acquisition programs.

We also continue to believe that DOD should address our past  recommendations to clarify and strengthen roles and responsibilities at  the enterprise level for making portfolio management decisions to make  sure that its investments are strategy-driven, affordable, and balance  near- and long-term needs. In fact, these recommendations may take on  more importance for DOD in light of the implementation of acquisition  reforms that will further diffuse responsibility for initiating and overseeing  acquisition programs.

We are making the following four recommendations to DOD:  The Secretary of Defense should direct the Under Secretary of Defense  for Acquisition and Sustainment to identify in final guidance the types of  business case elements potential middle-tier acquisition programs should  develop and decision makers should consider at program initiation to  assess the soundness of programs’ business cases, including whether  programs are well positioned to meet statutory objectives.  (Recommendation 1)

The Secretary of Defense should direct the Under Secretary of Defense  for Acquisition and Sustainment to determine and identify in final  guidance for middle-tier acquisition programs the metrics that will be used  to assess the performance of middle-tier acquisition programs across the  military departments, including whether programs are meeting statutory  objectives. (Recommendation 2)

The Secretary of Defense should ensure that a comprehensive  framework that clarifies the roles and responsibilities of the Office of the  Secretary of Defense and the military departments for acquisition  oversight is communicated by senior leadership. This framework should  be detailed enough to address areas of continued disagreement among  key stakeholders and serve to inform the department’s revisions of other  acquisition policies such as DOD Instruction 5000.02. (Recommendation  3)

The Secretary of Defense should develop a plan for how the department  will assess the effect of recent acquisition reforms, including identifying  who will be responsible for the assessment and what data will be needed.  (Recommendation 4)

We provided a draft of this product to DOD for comment. In its comments,  reproduced in appendix VII, DOD concurred with our four  recommendations. DOD also provided technical comments with regard to  improving the clarity of the discussion of certain reforms and providing  additional context about military departments’ oversight practices for  middle-tier acquisition programs, among other issues. We incorporated  DOD’s technical comments as appropriate.

In its written comments, DOD described planned actions to address our  recommendations. Specifically, in response to our first recommendation,  to identify the types of business case elements that should be considered  by decision-makers for middle-tier programs at program initiation, DOD  stated that it expects to identify these business case elements in its final  guidance on middle-tier programs, which it expects to complete in  September 2019. In response to our second recommendation, to identify  metrics that will be used to assess the performance of middle-tier  programs, DOD stated that it plans to determine performance metrics in  coordination with its release of its final guidance on middle-tier programs.  DOD expects to release this guidance in late 2019. In response to our  third recommendation, for senior leadership to clarify acquisition oversight  roles and responsibilities, DOD stated that these roles and responsibilities  will be finalized through the issuance of chartering directives and updated  acquisition policy; issuance is expected by the end of 2019. Finally, in  response to our fourth recommendation, to plan for assessing the effects  of acquisition reforms, DOD stated that it has included a division in the  Office of the Assistant Secretary of Defense for Acquisition to analyze  and assess this and other high-level oversight and policy issues.

DOD’s planned actions to address our first, second, and fourth  recommendation, if implemented effectively, should address the intent of  our recommendations. With regard to our third recommendation,  however, we do not believe that the steps outlined in DOD’s written  comments are likely to fully address the disagreements about acquisition  oversight roles and responsibilities that we identified in the report. We  acknowledge in the report that DOD plans to issue chartering directives  and re-issue DOD Instruction 5000.02 as part of its efforts to outline the  roles and responsibilities of various parties for acquisition oversight, as  DOD reiterated in its written comments. However, without a  comprehensive framework to inform the revisions of acquisition policies,  such as DOD Instruction 5000.02, DOD’s ability to finalize these policies  may be hindered by the disagreements between the Office of the  Secretary of Defense and the military departments that we identified in  our report. These disagreements are persistent and focused on  fundamental acquisition oversight issues. Simply issuing chartering  directives and finalizing policy as planned may not be enough to ensure  that areas of disagreement are resolved and that officials within the Office  of the Secretary of Defense and the military departments have a shared  understanding of an acquisition oversight framework for the entire  Department that will serve as the basis for any policy. Furthermore,  without senior leadership within DOD communicating this framework to  the Office of the Secretary of Defense and the military departments in  sufficient detail to address areas of disagreement among key  stakeholders, disagreement will likely persist and the intended impacts of  reforms could be stymied.

We are sending copies of this report to the appropriate congressional  committees and the Acting Secretary of Defense. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or oakleys@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix VIII.

Appendix I: Objectives, Scope, and Methodology

This report addresses (1) the progress the Department of Defense (DOD)  has made to implement selected oversight reforms for major defense  acquisition programs; (2) how DOD has used middle-tier acquisition  pathways and the extent to which DOD has developed guidance on  middle-tier program oversight; and (3) challenges DOD faces related to  reform implementation. The conference report for the National Defense  Authorization Act for Fiscal Year 2018 and the Senate Armed Services  Committee report accompanying a bill for the National Defense  Authorization Act for Fiscal Year 2018 also contained provisions for GAO  to review project, program, and portfolio management standards within  DOD. Appendix VI of this report includes our assessment of DOD’s  efforts to implement our previous portfolio management  recommendations and identifies opportunities and challenges related to  portfolio management that DOD may face as it continues to implement  acquisition reforms.

We focused our review on five selected reforms from the National  Defense Authorization Acts for Fiscal Years 2016 and 2017 that we  determined substantially affected DOD’s oversight of acquisition  programs. Our selections were informed by our analysis of the National  Defense Authorization Acts for Fiscal Years 2016 and 2017 and our past  work on factors affecting the oversight of major defense acquisition  programs. We also interviewed officials from the Office of the Secretary  of Defense and the military departments to obtain their perspectives on  the most significant reforms to acquisition oversight and considered those  perspectives when we made our selections. For the purposes of our  report, when we refer to a reform, we are referring to a specific change to  DOD’s acquisition oversight processes or roles and responsibilities. Two  of the reforms we reviewed align with sections of the National Defense  Authorization Act for Fiscal Year 2016, and the other three align with one  or more sections from the National Defense Authorization Act for Fiscal  Year 2017. Table 6 identifies the specific sections or subsections that we  reviewed for each reform.

We also reviewed related amendments to these sections from National  Defense Authorization Acts for subsequent years to determine whether  the National Defense Authorization Act sections we reviewed, or sections  of the U.S. Code that were added by sections we reviewed, had been  modified since being signed into law. When we identified amendments,  we assessed DOD’s progress in implementing the statute as amended.  Appendix II provides additional details about the original legislative  requirements and amendments, if any, to each of the reforms we  selected.

To identify the progress DOD has made to implement selected oversight  reforms for major defense acquisition programs, we analyzed three  selected reforms that affect processes related to DOD’s oversight of  major defense acquisition programs: designating military departments to be the milestone decision performing independent technical risk assessments; and establishing cost, fielding, and performance goals.

We also analyzed one reform that restructured acquisition oversight  functions in the Office of the Secretary of Defense. We analyzed the  associated National Defense Authorization Act sections and reviewed  related acquisition policies and guidance from the Office of the Secretary  of Defense and the military departments (see app. II for a list of key  guidance we reviewed for each reform). For each reform, we analyzed  DOD and military department policies and guidance to determine steps  DOD and the military departments had taken to implement the reforms.  We also compared new or updated policies and guidance, when  available, with prior policies and guidance to determine how oversight  roles, responsibilities, and processes had changed for DOD’s major  defense acquisition programs.

To obtain additional insight into how designation of milestone decision  authority had changed as a result of recent reforms, we requested and  analyzed data provided by DOD about the milestone decision authority  levels for the major defense acquisition program portfolio. To assess the  reliability of these data, we discussed the data and sources used to  compile them with DOD officials, reviewed the data for errors, reviewed  related documentation on programs with milestone decision authority at  the military department level, and compared the data when possible to  other sources, such as publicly available lists of major defense acquisition  programs. On the basis of these steps, we determined that the data we  used were sufficiently reliable to identify changes in the level of milestone  decision authority over time for major defense acquisition programs.

To assess changes resulting from the reorganization of Office of the  Under Secretary of Defense for Acquisition, Technology and Logistics, we  also reviewed updated organizational charts and staffing and vacancy  data for the successor offices (the Office of the Under Secretary of  Defense for Research and Engineering and the Office of the Under  Secretary of Defense for Acquisition and Sustainment) and compared  these to past organizational charts and staffing data for the Office of the  Under Secretary of Defense for Acquisition, Technology and Logistics. To  determine the current percentage of positions vacant in each office, we  compared actual data for filled positions as of March 2019 to the total  number of vacant positions as of the same point in time. The vacancy  numbers do not include vacant positions that are slotted for future  reduction or transfer. To assess the reliability of these data, we requested  and reviewed written responses from DOD officials on the reliability of the  data and sources used to compile it, reviewed the data for logical  inconsistencies, and compared the data when possible to other sources,  such as related data provided for other time frames. On the basis of these  steps, we determined that the data we used were sufficiently reliable to  identify the current staffing status for the two new Under Secretary  offices.

To determine how DOD has used middle-tier acquisition pathways, we  reviewed the relevant statute and guidance, and obtained information  from the military departments about the number and types of programs  using middle-tier acquisition pathways as of March 2019. We analyzed  the guidance from the Office of the Under Secretary of Defense for  Acquisition and Sustainment and the military departments to determine  how they were implementing the statute with regard to selection of  programs and program oversight. We also compared the guidance with  our past work on elements of business cases that should be completed at  program initiation to determine what elements were addressed by DOD  guidance.

At each of the military departments, we judgmentally selected three  middle-tier programs to review in additional detail. We selected programs  to obtain a range of program costs (including programs that were above  the equivalent threshold cost for designation as a major defense  acquisition program if the program were not using a middle-tier  acquisition pathway, as well as those below that threshold) and types of  programs being executed under middle-tier acquisition pathways (such as  space, artillery, software, missile, and ground vehicle programs).

Air Force: Hypersonic Conventional Strike Weapon; Next Generation  Overhead Persistent Infrared Space; Protected Tactical Enterprise  Service;

Army: Extended Range Cannon Artillery; Optionally Manned Fighting  Vehicle; Rapid Opioid Countermeasures System; and

Navy: STANDARD Missile-2 Block IIIC; STANDARD Missile-6 Block  IB Phase IA Rocket Motor; STANDARD Missile-6 Block IB Phase IB  All Up Round.

For these programs, we collected and analyzed additional information  such as acquisition decision memorandums, acquisition strategies,  program cost and schedule estimates, and risk assessments. We also  interviewed or received detailed written responses from program officials  that addressed issues such as how decisions were made to execute  programs under middle-tier acquisition pathways and how oversight for  programs was being conducted.

Further, we reviewed interim guidance from the Office of the Under  Secretary of Defense for Acquisition and Sustainment and the military  departments to determine how DOD planned to measure middle-tier  program performance. We compared DOD and the military departments’  guidance on developing metrics and collecting data to assess middle-tier  program performance to relevant internal controls related to consistent  measurement of program performance.

To assess the challenges DOD faces with regard to reform  implementation, we reviewed policy and guidance issued by top DOD  leadership that outlined roles and responsibilities for the Office of the  Secretary of Defense and the military departments with regard to  acquisition oversight and compared them to leading practices for  leadership involvement in agency transformations that we had identified  in prior work. We also collected and analyzed information about DOD’s  actions taken to implement prior recommendations we have made to  improve portfolio management at DOD and analyzed the acquisition  oversight reforms we included in this review to identify opportunities and  challenges related to portfolio management that DOD may face as it  continues to implement acquisition reforms. Lastly, we reviewed DOD’s  plans and ongoing efforts to develop performance measures and collect  data to assess the effects of acquisition reforms and compared these  efforts with success factors for reform implementation identified in our  past work.

For all objectives, we also conducted interviews with officials from the  Office of the Secretary of Defense, the Joint Staff, and the military  departments to obtain additional insight into implementation status,  implementation challenges, and future plans, including:

Office of the Secretary of Defense: The Office of the Under  Secretary of Defense for Research and Engineering, the Office of the  Under Secretary of Defense for Acquisition and Sustainment, the  Office of the Under Secretary of Defense (Comptroller), the Office of  the Chief Management Officer, the Office of the Director of  Operational Test and Evaluation, the Office of the Director of Cost  Assessment and Program Evaluation, and the Office of the General  Counsel.

Joint Staff: Force Structure, Resource and Assessment Directorate,  J-8.

Military departments: For each of the three military departments (Air  Force, Army, and Navy) we interviewed acquisition officials from the  Service Acquisition Executive’s office, requirements officials  supporting the Chief of Staff of the respective armed force, and  officials from the military department cost agencies. At the Air Force  we also interviewed officials from the Office of the General Counsel.

We conducted this performance audit from March 2018 to June 2019 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Summary of Original Legislation and Amendments to Acquisition Reforms Reviewed by GAO

Appendix III: Milestone Decision Authority for Major Defense Acquisition Programs as of March 2019

Appendix III: Milestone Decision Authority for  Major Defense Acquisition Programs as of  March 2019 Programs with milestone decision authority at the Air Force level (26)  Advanced Extremely High Frequency Satellite  AIM-120 Advanced Medium Range Air-to-Air Missile  Air Force Intercontinental Ballistic Missile Fuze Modernization  Airborne Warning and Control System Block 40/45 Upgrade  B61 Mod 12 Life Extension Program Tailkit Assembly  F-15 Eagle Passive Active Warning Survivability System  Family of Advanced Beyond Line-of-Sight Terminals  Global Positioning System III Follow-On Production  Military Global Positioning System User Equipment Increment 1  MQ- 9 Reaper Unmanned Aircraft System  Small Diameter Bomb Increment II  Space Based Infrared System High  Space Fence Ground-Based Radar System Increment 1  Wideband Global SATCOM  Programs with milestone decision authority at the Army level (18)  AH-64E Apache New Build  Airborne & Maritime/Fixed Station Joint Tactical Radio System  AN/TPQ-53 Counterfire Target Acquisition Radar  Common Infrared Countermeasure  Programs with milestone decision authority at the Army level (18)  Guided Multiple Launch Rocket System/Guided Multiple Launch Rocket System  Alternative Warhead  Handheld, Manpack, and Small Form Fit Radios  M88A2 Heavy Equipment Recovery Combat Utility Lift Evacuation System  MQ-1C Gray Eagle Unmanned Aircraft System  Patriot Advanced Capability-3 Missile Segment Enhancement  RQ-7B Shadow Tactical Unmanned Aircraft System  Warfighter Information Network-Tactical Increment 2  Programs with milestone decision authority at the Navy level (36)  Advanced Arresting Gear  AGM-88E Advanced Anti-Radiation Guided Missile  Air and Missile Defense Radar  Amphibious Combat Vehicle Phase 1 Increment 1  CVN 78 Gerald R. Ford Class Nuclear Aircraft Carrier  DDG 1000 Zumwalt Class Destroyer  DDG 51 Arleigh Burke Class Guided Missile Destroyer  H-1 Upgrades (4BW/4BN)

Joint Precision Approach and Landing System  LHA 6 America Class Amphibious Assault Ship  Littoral Combat Ship Mission Modules  LPD 17 San Antonio Class Amphibious Transport Dock  MQ-4C Triton Unmanned Aircraft System  Programs with milestone decision authority at the Navy level (36)  MQ-8 Fire Scout Unmanned Aircraft System  Offensive Anti-Surface Warfare Increment 1 (Long Range Anti-Ship Missile)

Appendix IV: Organizational Charts from Before and After the Reorganization of the Office of the Secretary of Defense

Appendix V: Programs Using Middle-Tier Acquisition Pathways as of March 2019

Appendix VI: Department of Defense (DOD) Efforts to Implement Portfolio Management Best Practices

The conference report for the National Defense Authorization Act for  Fiscal Year 2018 and the Senate Armed Services Committee report  accompanying a bill for the National Defense Authorization Act for Fiscal  Year 2018 included provisions for GAO to review project, program, and  portfolio management standards within DOD. This appendix includes our  assessment of DOD’s efforts to implement our previous portfolio  management recommendations and identifies opportunities and  challenges related to portfolio management that DOD may face as it  continues to implement acquisition reforms.

Portfolio management is used by leading commercial companies to help  ensure their investments are optimized to meet customer needs within  available resources. Portfolio management focuses on products  collectively at an enterprise level and involves evaluating, selecting,  prioritizing, and allocating limited resources to projects that best  accomplish strategic or organizational goals. It is also a vehicle to make a  wide variety of decisions, including capability and funding trade-offs, to  achieve the optimal capability mix for a given level of investment.

For DOD, effective portfolio management can help to ensure that weapon  system investments are strategy-driven and affordable and balance near-  and long-term needs. Take a hypothetical example in which DOD starts  with 10 programs and $50 billion to invest. Without portfolio management,  program managers may seek to get the most that they can out of each of  the 10 programs, without assessing their aggregate contributions to  defense. Using portfolio management, DOD executives would look at  different combinations of and approaches to the 10 programs to  determine what, collectively, would provide the best capabilities for $50  billion. This would enable executives to decide, for example, whether it is  better to concentrate more investment in seven programs rather than fund  all 10 as best as possible. In another example, if a program began to  have cost or performance problems, portfolio management would  consider whether the other programs in the portfolio could address the  requirements of the problematic program rather than just putting more  money into it.

Portfolio management activities at DOD are carried out at both the  enterprise level and the military department level and responsibilities are  divided among the requirements community, the acquisition community,  and the budget community. At the enterprise level, the primary offices  responsible for portfolio management are the Joint Staff (representing the  requirements community), the Under Secretary of Defense for Research  and Engineering and the Under Secretary of Defense for Acquisition and  Sustainment (representing the acquisition community), and the Director of  Cost Assessment and Program Evaluation (representing the budget  community).

In 2007, we identified several best practices for portfolio management,  which we revalidated in 2015. These leading practices encourage  organizations to assess product investments collectively from an enterprise level,  rather than as independent and unrelated initiatives;  continually make go/no-go decisions through a gated review process  to rebalance portfolios based on investments that add the most value; use an integrated approach to prioritize needs and allocate resources  in accordance with strategic goals;  rank and select investments using a disciplined process to assess the  costs, benefits, and risks of alternative products; empower leadership to make investment decisions and hold  leadership accountable for investment outcomes; and provide sustained leadership for portfolio management.

Portfolio management best practices and the Project Management  Institute’s portfolio management standards also state that organizations  should conduct regular reviews to adjust to strategic changes or changes  in the mix of products within a portfolio, among other reasons. From a  DOD perspective, portfolio reviews can help increase return on taxpayers’  investments in weapon systems in a number of ways, such as: helping to  ensure investments align with national security and military strategies;  prioritizing the most important investments; selecting the optimum mix of  investments; identifying and eliminating unwarranted duplication;  monitoring programs’ health to determine whether changes to the  portfolio are warranted; and determining whether investments are  affordable.

Previous GAO Findings and Recommendations on Portfolio Management at DOD

We have previously reported that DOD was not effectively using portfolio  management to optimize its weapon system investments. In 2015, we  identified several factors that inhibited DOD’s ability to do so, including  fragmented governance, a lack of sustained leadership, and a perceived  lack of decision-making authority at the enterprise level. We also found  that DOD’s portfolio management policy was dated, not fully consistent  with best practices, and was not being implemented by the Department,  in part due to changes in leadership priorities. Further, DOD’s enterprise- level requirements, acquisition, and budgeting communities—meaning  those at the Office of the Secretary of Defense, Joint Chiefs of Staff, and  Joint Staff level—were not consistently conducting portfolio reviews or  collaborating to integrate key information. As a result, we reported that  DOD may have been missing opportunities to better leverage its  resources and identify investment priorities that best reflect DOD-wide  needs.

We recommended that DOD update its portfolio management policy;  designate a senior official responsible for its implementation; conduct  annual portfolio reviews that integrate key information from the  requirements, acquisition, and budget processes; and invest in analytical  tools to support its portfolio management efforts. DOD partially concurred  with the recommendations, but the planned actions DOD identified at the  time of our report did not fully address the issues we identified. As of  March 2019, DOD had yet to implement our recommendations from 2015  (see table 16 for details of implementation status).

Recent Acquisition Reforms Offer Opportunities to Improve Portfolio Management at DOD but Could Also Exacerbate Existing Challenges

It is too soon to assess the effect of the acquisition reforms we reviewed  on DOD’s portfolio management efforts because a critical mass of  programs has not yet gone through the new acquisition processes.  Depending on how the department implements these reforms, some  aspects of these reforms could help to address previously-identified  deficiencies in portfolio management in the department. For example:

Officials in the Office of the Under Secretary of Defense for  Acquisition and Sustainment told us that now that milestone decision  authority for major defense acquisition programs has largely shifted  from the Office of the Secretary of Defense to the military  departments, they expect that they will have more time to focus on  portfolio-level issues such as identifying how systems need to work  together to fill capability gaps since they are less involved in the  details of individual programs. We previously reported that DOD’s  processes were too focused on optimizing individual investments  rather than considering investments across the department.

The process developed by DOD to establish cost, fielding, and  performance goals brings together officials from DOD’s acquisition,  requirements, and budget communities, the three key entities with  responsibility for portfolio management, to provide advice on the  establishment of program goals. We previously reported that DOD’s  enterprise-level processes, organizations, and decision makers  oversee weapon system investments generally as stove-pipes and not  as an integrated whole. While the process assesses programs on an  individual basis rather than collectively from an enterprise level as  called for by portfolio management best practices, it may still provide  additional shared insight across the acquisition, requirements, and  budget communities to help assess portfolios in a more integrated  fashion at an enterprise level.

However, other aspects of certain reforms have the potential to  exacerbate challenges we have previously identified with DOD’s portfolio  management approach if not actively managed. For example:

Realigning roles and responsibilities for decisions related to weapon  systems programs could lead to further questions about who is  ultimately responsible and accountable for portfolio management  decisions if leadership roles are not clearly defined. We previously  reported that DOD’s governance structure for portfolio management  was fragmented in part as a result of widely-dispersed decision  making responsibilities for weapon system investments. We found  that this dispersion of responsibility made it difficult to determine who  was empowered to make enterprise-level weapon system investment  decisions and who can be considered portfolio managers. According  to portfolio management best practices, leadership should be clearly  defined and held accountable for outcomes.

Programs under middle-tier acquisition pathways have fewer  requirements to report program information to offices within the Office  of the Secretary of Defense and the Joint Staff than major defense  acquisition programs. For example, middle-tier acquisition programs  are generally exempted from the Joint Capabilities Integration and  Development System for requirements development. Therefore Joint  Staff officials may have less information about program requirements  than for a major defense acquisition program. Office of the Secretary  of Defense officials told us they are working with the military  departments and other stakeholders to determine what information is  needed for oversight and portfolio management for middle-tier  acquisition programs. Office of the Secretary of Defense and Joint  Staff officials told us that guidance issued by the Under Secretary of  Defense for Acquisition and Sustainment in October 2018 that gives  the Office of the Secretary of Defense and Joint Staff formal roles in a  governance process may help to ensure sufficient insight.

DOD’s ability to develop a common set of portfolios to facilitate  integrated portfolio analysis may be more difficult. We previously  reported that the requirements, acquisition, and budget communities  at DOD were using different portfolio constructs, meaning that they  defined their portfolios differently and did not use a standard approach  to group investments into portfolios. We identified the use of different  approaches as a barrier to taking an integrated approach to prioritize  needs and allocate resources in accordance with strategic goals, as  called for by portfolio management best practices. For example, the  requirements community uses eight joint capability areas for  examining warfighter needs, acquisition portfolios vary by military  department, and budget data are organized into 11 major force  programs. In our prior work, many officials at DOD said that using a  wide variety of portfolio constructs is necessary and sometimes  beneficial given the different roles and perspectives of the  organizations involved. However, as notionally illustrated in figure 8,  the different communities need to go through an extensive mapping  exercise when they want to analyze their portfolios from another  perspective—for example, examining funding associated with joint  capability areas.

With the reorganization of the Office of the Under Secretary of Defense  for Acquisition, Technology and Logistics, officials from the Offices of the  Under Secretaries of Defense for Research and Engineering and  Acquisition and Sustainment told us that portfolio management activities  that used to be conducted by the Office of the Under Secretary of  Defense for Acquisition, Technology and Logistics are now split between  their offices. Officials from the Office of the Under Secretary of Defense  for Research and Engineering told us that they were still in the process of  determining what portfolio construct they would use to group investments  for portfolio management purposes. If that office decides to use a different  portfolio construct than other entities, that decision will increase the  already complex process of mapping together portfolios in order to  perform an integrated portfolio analysis. Officials from both offices told us  that they were working on a pilot effort to conduct portfolio management  by focusing on DOD’s missions rather than programs, which could help to  standardize the portfolio constructs if the approach is accepted on a wider  scale.

Appendix VII: Comments from the Department of Defense

Appendix VIII: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Cheryl Andrew (Assistant  Director), Marie Ahearn, Peter W. Anderson, David Dornisch, Anne  McDonough, Melissa Pope, Scott Purdy, Juli Steinhouse, Sara Sullivan,  Anne Louise Taylor, Alyssa Weir, and David Wishard made key  contributions to this report.

Members of Congress have been concerned that DOD's weapons acquisition process is too bureaucratic and slow to deliver innovations to the field. (DOD acquisitions are on our High Risk List.) Recent legislation included reforms to try to speed up the process.

DOD has begun to carry out these reforms, including shifting more oversight decisions from the Office of the Secretary of Defense to the military departments and using a more streamlined process in some cases. However, questions remain about oversight roles and how some reforms will be implemented.

Our recommendations include clarifying acquisition oversight roles and responsibilities.