Columbia Class Submarine: Overly Optimistic Cost Estimate Will Likely Lead to Budget Increases

The Navy has identified the Columbia class submarine program as its top acquisition priority. It plans to invest over $100 billion to develop and purchase 12 nuclear-powered ballistic missile submarines to replace aging Ohio class submarines by 2031.

The National Defense Authorization Act for Fiscal Year 2018 and House Report 115-200 included provisions that GAO review the status of the program. This report examines (1) the Navy's progress and challenges, if any, in meeting design goals and preparing for lead submarine construction; (2) the reliability of the Navy's cost estimate; and (3) how the Navy is implementing a special fund and associated authorities to construct Columbia class submarines.

GAO reviewed Navy and shipbuilder progress reports, program schedules, and construction plans. GAO assessed the Navy's cost estimate and compared it to best practices for cost estimating. GAO also reviewed certain Navy funding and acquisition authorities and interviewed program officials.

This is a public version of a sensitive report that GAO issued in March 2019. Information that the Department of Defense (DOD) deemed sensitive has been omitted.

The Navy's goal is to complete a significant amount of the Columbia class submarine's design—83 percent—before lead submarine construction begins in October 2020. The Navy established this goal based on lessons learned from another submarine program in an effort to help mitigate its aggressive construction schedule. Achieving this goal may prove to be challenging as the shipbuilder has to use a new design tool to complete an increasingly higher volume of complex design products (see figure). The shipbuilder has hired additional designers to improve its design progress. The Navy also plans to start advance construction of components in each major section of the submarine, beginning in fiscal year 2019, when less of the design will be complete.

The Navy's $115 billion procurement cost estimate is not reliable partly because it is based on overly optimistic assumptions about the labor hours needed to construct the submarines. While the Navy analyzed cost risks, it did not include margin in its estimate for likely cost overruns. The Navy told us it will continue to update its lead submarine cost estimate, but an independent assessment of the estimate may not be complete in time to inform the Navy's 2021 budget request to Congress to purchase the lead submarine. Without these reviews, the cost estimate—and, consequently, the budget—may be unrealistic. A reliable cost estimate is especially important for a program of this size and complexity to help ensure that its budget is sufficient to execute the program as planned.

The Navy is using the congressionally-authorized National Sea-Based Deterrence Fund to construct the Columbia class. The Fund allows the Navy to purchase material and start construction early on multiple submarines prior to receiving congressional authorization and funding for submarine construction. The Navy anticipates achieving savings through use of the Fund, such as buying certain components early and in bulk, but did not include the savings in its cost estimate. The Navy may have overestimated its savings as higher than those historically achieved by other such programs. Without an updated cost estimate and cost risk analysis, including a realistic estimate of savings, the fiscal year 2021 budget request may not reflect funding needed to construct the submarine.

GAO is making three recommendations: that the Navy update the lead submarine cost estimate with cost risk analysis using current cost data, develop a realistic estimate of savings from use of the Fund's authorities, and use this updated cost estimate to inform its budget request for lead submarine construction. DOD concurred with GAO's recommendations.

The Navy plans to invest approximately $128 billion to research, develop,  and purchase the replacement for 14 Ohio class nuclear-powered ballistic  missile submarines—the current sea-based leg of the nation’s strategic  nuclear deterrent. According to the Navy, the lead Columbia class  submarine will need to make its first patrol in fiscal year 2031 in order to  avoid a deterrence gap; the Ohio class submarines begin to retire in  2027. The Navy has identified the 12-submarine Columbia class program  as its top acquisition priority and has set an aggressive schedule to  deliver the lead submarine in fiscal year 2027, followed by a period of  testing before the first patrol occurs. The Navy is continuing its design  efforts and plans to begin advance construction of some of the  submarine’s components in fiscal year 2019. In 2014, Congress created  the National Sea-Based Deterrence Fund (the Fund), the use of which  provides the Navy with several acquisition authorities. One of the  purposes of these authorities is to reduce material and equipment costs  for Columbia class submarines.

In light of the criticality of the deterrence mission and the cost and  schedule pressures facing the Columbia class program, the House Armed  Services Committee report accompanying the National Defense  Authorization Act for Fiscal Year 2018 included a provision that we  examine the program to include, among other things, technology  development, design progress, and program cost estimates. The act also  included a requirement for the Navy to prepare and submit matrices on  the Columbia program’s design and construction goals and progress and  included a provision that we assess these matrices. This report, which  addresses both provisions, examines (1) the Navy’s progress and  challenges, if any, associated with meeting design goals and preparing  for lead submarine construction; (2) the reliability of the Navy’s cost  estimate for the Columbia class submarine program; and (3) how the  Navy is implementing the Fund and associated authorities to construct  Columbia class submarines. This is our second public report examining  the Columbia class program.

This report is a public version of a sensitive report that we issued in  March 2019. The Department of Defense (DOD) deemed some of the  information in our March report to be sensitive, which must be protected  from public disclosure. Therefore, this report omits sensitive information  about the Navy’s development of critical technologies for the Columbia  class program, including specific details about the technologies. Although  the information provided in this report is more limited, the report  addresses the same objectives as the sensitive report and uses the same  methodology.

To assess the Navy’s progress and any challenges associated with  meeting design goals and preparing for lead submarine construction, we  reviewed Navy and General Dynamics Electric Boat’s (the lead  shipbuilder) documents, including program briefings, schedules, and  contract status reports to assess the program’s progress against its  planned schedule. We reviewed the Navy’s and the shipbuilder’s plans for  design management and completion and compared the plans with  progress reports to identify any delays. We also reviewed ongoing  development efforts and schedules for the Columbia program’s critical  technologies to determine risks to their development and integration. We  also reviewed the matrices submitted by the Navy to Congress in  February and October 2018, to determine the status of the program and  identify any changes to the Navy’s design, construction, and cost goals  for the program since our December 2017 report.

To assess the reliability of the Navy’s cost estimate, we determined the  extent to which the estimate was consistent with cost estimating best  practices as identified in our Cost Estimating and Assessment Guide.

We also reviewed supporting documents, such as the program life-cycle  cost estimate, briefs, memoranda, and relevant DOD and Navy policies.  We compared the program’s cost estimate against independent estimates  and assessments from the Office of Cost Assessment and Program  Evaluation (CAPE); the Naval Center for Cost Analysis (NCCA); and the  Congressional Budget Office (CBO).

To assess how the Navy is implementing the Fund and associated  authorities to construct Columbia class submarines, we reviewed the  legislation establishing and modifying the Fund, as well as budget  documents and DOD reprogramming approvals. To corroborate  information for each of these objectives, we interviewed DOD and Navy  officials and shipbuilder representatives responsible for the Columbia  class program. Appendix I provides additional information on the scope  and methodology of our review.

The performance audit upon which this report is based was conducted  from December 2017 to March 2019 in accordance with generally  accepted government auditing standards. Those standards require that  we plan and perform the audit to obtain sufficient, appropriate evidence to  provide a reasonable basis for our findings and conclusions based on our  audit objectives. We believe that the evidence obtained provides a  reasonable basis for our findings and conclusions based on our audit  objectives. We subsequently worked with DOD from February 2019 to  April 2019 to prepare this unclassified version of the original sensitive  report for public release. This public version was also prepared in  accordance with these standards.

Over the next 10 years, the Navy plans to continue developing critical  technologies, complete detail design, and begin construction of the lead  Columbia class submarine. In December 2017, we found that the  schedule to deliver the lead submarine was aggressive, with extensive  overlap—or concurrency—between development, design, and  construction, as shown in figure 1.

Our prior work reviewing shipbuilding programs has shown that the  programs with the greatest amount of overlap between shipbuilding  phases often have the highest cost and schedule growth, as well as  quality and performance issues.

The National Defense Authorization Act for Fiscal Year 2018 included  reporting requirements for the Columbia class program. As part of these  annual reporting requirements, the Navy must submit to Congress  matrices that identify (1) key milestones, events, and performance goals  for the design and construction of the Columbia class program; and (2)  costs associated with the design and construction period of the Columbia  class program. The Navy submitted its initial matrices to Congress in  February 2018 and an update to the matrices in October 2018. The next  matrices update is due in March 2019 and annually, thereafter, until the  lead Columbia submarine is delivered.

The Navy is developing a number of new technologies related to  submarine propulsion, missile tubes, and survivability that are planned to  ensure that the Columbia class will remain operationally relevant  throughout its planned 42.5-year service life, as shown in figure 2.

In 2015, as part of its technology readiness assessment, the Navy  identified two technologies—the advanced carbon dioxide removal unit  and the stern area system—as critical technology elements. However, as  we found in 2017, several Columbia class technologies that met GAO’s  definition of a critical technology element were not identified by the Navy  as critical technologies. In addition, several of these were immature,  with technology readiness levels (TRL)—used to describe the maturity of  critical technologies—of less than 7. See appendix II for a description of  TRLs.

As part of its matrices to Congress, the Navy is required to report on the  TRLs of the integrated power system, nuclear reactor, propulsor,  coordinated stern features, stern area system, and common missile  compartment—which are the critical technologies we identified in our prior  report. Table 1 lists each GAO-identified critical technology and its TRL  as of October 2018, as reported by the Navy.

Columbia Design and Construction Approach

Two shipbuilders—General Dynamics Electric Boat (Electric Boat) and  Huntington Ingalls Industries Newport News (Newport News)—design  and build nuclear submarines. Electric Boat is the prime contractor for  both design and construction of the Columbia class program, with  Newport News serving as a subcontractor. Similar to the Virginia class  program, each shipbuilder will construct segments of the submarine, but  Electric Boat will complete final outfitting and deliver the submarines to  the Navy. The Navy awarded a detail design contract in September 2017  to Electric Boat for work including completion of the submarine’s design,  component and technology development, and prototyping efforts. The  detail design process for the Columbia class program encompasses three  activities, which began after the Navy set the technical requirements for  the submarine in 2016:

Arrangements outline the steel structure and routes distributive  systems—such as electrical or piping systems—throughout the  submarine. At this time, the shipbuilder generates a three-dimensional  computer-aided design model for the area.

Disclosures complete the design work for even the lowest-level items  of the submarine, including material information. After these are  completed, the shipbuilder can begin ordering material and long lead  items for the submarine.

Work instructions are three-dimensional electronic products that  shipyard workers use to construct the submarine.

Figure 3 illustrates the design phases for the Columbia class program.

The shipbuilder will design and construct Columbia class submarines in  six large hull segments, referred to as super modules, a method also  used to construct most of the Virginia class submarines. During  construction, the modules will largely be outfitted with systems and  connections prior to being attached together during final assembly.  According to the shipbuilder, this method is more efficient than outfitting  the hull after it is constructed because more workspace is available to  install equipment. Figure 4 illustrates the super modules within the  submarine.

A reliable cost estimate is critical to program success. It provides the  basis for informed investment decision making, realistic budget  formulation and program funding, meaningful progress measurement,  proactive course correction when warranted, and accountability for  results. GAO’s Cost Estimating and Assessment Guide states that  reliable cost estimates reflect four characteristics, which encompass 19  best practices. These characteristics—comprehensive, well  documented, accurate, and credible—are shown in table 2.

For Navy shipbuilding programs, including the Columbia class, several  different entities are involved in cost estimating:

The Naval Sea Systems Command (NAVSEA) Cost Engineering and  Industrial Analysis Group develops the program life-cycle cost  estimate, which is an estimate accounting for the total cost to the  government of acquisition and ownership of a system over its full life.

NCCA develops an independent cost assessment for certain Navy  programs, such as the Columbia class program, at milestone events  in the defense acquisition system. This assessment is not a separate  estimate, but rather a review of the NAVSEA program life-cycle cost  estimate.

A cost review board, comprised of multiple Navy offices, establishes a  service cost position based on their review of the program life-cycle  cost estimate and the independent cost assessment.

The Office of the Secretary of Defense’s CAPE conducts or approves  independent cost estimates for major defense acquisition programs.  Independent cost estimates are statutorily required for major defense  acquisition programs at milestone events.

The milestone decision authority, which in the case of the Columbia  class program is the Under Secretary of Defense for Acquisition and  Sustainment, reviews the service cost position and independent cost  estimate and selects the cost estimate to baseline and fund the  program.

The most recent milestone event for the Columbia program was the  Milestone B decision in January 2017, where the program received  approval to proceed to the next acquisition phase—engineering and  manufacturing development, which includes detail design of the lead  submarine. In a memo documenting that decision, the milestone decision  authority noted that significant development risks remain for the Columbia  program and cost control must remain a priority. To limit program cost  growth, the milestone decision authority established an affordability cap:  the average submarine procurement cost should not exceed $8.0 billion  in constant year 2017 dollars. Figure 5 summarizes the cost estimating  process for the Columbia class program’s Milestone B review.

Navy Is Managing an Aggressive Build Schedule, but Early Design and Construction Challenges Signal Schedule Risk

The Navy is attempting to mitigate an aggressive schedule for lead  submarine construction by (1) setting a goal to mature a significant  amount of the submarine’s design prior to the start of construction and (2)  beginning advance construction of submarine modules prior to October  2020. The shipbuilder is working to improve design performance and  would have to maintain this increased pace to achieve its design goal,  which is necessary to mitigate schedule risk associated with constructing  the lead submarine. This may prove challenging as it must complete an  increasingly higher volume and complexity of design products. At the  same time, the Navy is continuing to develop several critical technologies  and recent manufacturing defects with the integrated power system and  missile tubes are among the challenges that the Navy is facing in  ensuring timely delivery of critical components to the shipyard. Finally, to  achieve Columbia’s aggressive construction schedule, while  simultaneously building Virginia class submarines, the shipbuilder is  working to ensure that it has sufficient shipyard capacity—including new  facilities, additional suppliers, and an increased workforce.

Shipbuilder Would Have to Maintain Its Increased Pace to Meet Its Design Maturity Goal and Reduce Schedule Risk

The shipbuilder has failed to achieve its planned rates for completing  design arrangements and disclosures to meet its design maturity goal in  recent months—hampered by implementation of a new design software  tool and an insufficient number of designers to meet monthly design  completion rates. As we reported in December 2017, the Navy’s priority is  to complete a high level of design—specifically, 100 percent of design  arrangements and 83 percent of design disclosures—by the start of lead  submarine construction in October 2020. By maturing the design before  beginning construction on the lead submarine, the Navy is attempting to  mitigate the risk of costly rework from design changes and subsequent  delays to the Columbia class program’s 84-month construction schedule,  which the Navy has acknowledged is aggressive. The Navy established  the design maturity goal for Columbia based on lessons learned from the  Virginia class program, when the shipbuilder began constructing the lead  submarine with only 76 percent of arrangements and 43 percent of  disclosures completed and, subsequently, realized 21 percent cost  growth.

Since the shipbuilder began work on the detail design, it has generally  met its overall goal of completing the arrangements on schedule. As  detail design continues, however, the shipbuilder is transitioning from  relatively simple designs for the hull to the more complex designs for the  submarine’s internal systems, increasing the pace needed to complete  the remaining designs, as shown in figure 6.

Navy officials stated that design disclosures are generally considered the  most challenging phase of design work, where the shipbuilder specifies  the lowest-level items and defines all aspects of the submarine. The  shipbuilder has to maintain this increased pace in order to achieve the  design maturity goal by the start of lead submarine construction.

However, the shipbuilder’s design progress in completing disclosure  products has fallen short of its plan in recent months as the planned pace  and complexity of the design has increased. Using data from the  program’s cost performance reports, we analyzed the shipbuilder’s  monthly design progress according to a schedule performance index that  measures the value of the work completed against the work scheduled.  For example, if the schedule performance index is less than 1.00, then  the shipbuilder has completed less than a dollar’s worth of work for each  dollar that was scheduled. As shown in figure 7, since January 2018,  schedule performance has consistently fallen below 1.00.

Both DOD and Navy officials attributed the shipbuilder’s design delays to  challenges adapting to a new design software tool. Beginning with the  Columbia class program, the shipbuilder transitioned to a new customized  software tool for design and construction because its prior software was  no longer supported by the original developer. However, the shipbuilder  has experienced problems developing the tool, which has resulted in  slower progress to complete both design arrangements and disclosures,  as certain aspects of the software’s functionality were delayed. Navy  officials stated that, as of June 2018, they believe that design software  functionality was performing at a level that no longer impeded design  progress. While the designers have gained proficiency with the new  design tool to complete arrangements and disclosures, according to Navy  officials, the shipbuilder is now facing similar challenges using the tool to  generate work instructions. Navy program officials also stated that the  shipbuilder has not delivered some of the software functionality needed to  produce work instructions as scheduled. Further, Navy officials noted that  the process to create work instructions from completed disclosures takes  longer with the new design software so the shipbuilder has begun  generating work instructions earlier.

According to Navy officials and shipbuilder representatives, the  shipbuilder hired 150 additional designers in an effort to recover its design  schedule and meet future monthly design goals. However, adding  designers to recover and maintain the shipbuilder’s design schedule  ultimately increases the program’s design costs. Similar to the schedule  analysis above, we used data from cost performance reports to analyze  the shipbuilder’s monthly design progress according to a cost  performance index that measures the budgeted value of the work  completed against what it actually costs to complete it. For example, if the  cost performance index is less than 1.00, then less than a dollar’s worth  of work has been completed for each dollar spent. As shown in figure 8,  the shipbuilder’s cost performance has consistently fallen below 1.00  since December 2017.

If the shipbuilder cannot address challenges associated with using the  software tool to generate work instructions discussed above, it will likely  need additional design hours in the future, resulting in higher costs in  order to mature the design on schedule.

Navy’s Use of Advance Construction to Mitigate Aggressive Schedule Is Not without Risk

Navy officials and shipbuilder representatives expect to mitigate risks  associated with the Columbia construction schedule by accelerating the  building of certain components more than a year in advance of the formal  start of construction. They anticipate that this advance construction  strategy will allow them to gain 2 months of schedule margin for final  assembly and testing prior to delivery of the lead submarine. Starting in  December 2018, the shipbuilder will begin constructing modules of the  submarine as part of its advance construction effort. In 2017, we reported  that the Navy had planned to begin advance construction for four of the  submarine’s six super modules, but since our report was issued, it now  plans to begin construction on all six super modules including building  components like the stabilizers, impulse tanks, and others. Figure 9  shows the start of advance construction for each super module.

Navy officials estimate that the current advance construction efforts will  require approximately 631,000 labor hours. In addition, advance  construction efforts would require that the Navy accelerate delivery of  equipment provided to the shipbuilder for installation on the submarine,  such as pumps and valves.

Shipbuilder representatives stated that a lesson learned from the Virginia  class program was that construction of certain complex components  should begin as early as possible if capability requirements and designs  are stable. However, based on its plan, the shipbuilder will begin advance  construction having completed less than 40 percent of the total design  disclosures for the Columbia class submarine, as shown in figure 10.

The number of disclosures completed at the start of advance construction  is less than half of those the shipbuilder plans to complete by the start of  lead submarine construction in October 2020. Navy officials stated that  they believe the risk associated with beginning construction with a less  mature overall design is mitigated because the program selected  components for advance construction that are well understood and  unlikely to be affected by design changes, like ballast tanks, decking, and  hull segments. In addition, Navy officials stated that they will not begin  construction on the component or hull unless the arrangements  associated with the structure of that area of the submarine are complete.

However, based on the shipbuilder’s design plans, the arrangements and  disclosures of adjoining areas of the super module may not be complete,  which could negatively affect construction. Specifically, the shipbuilder’s  design plans indicate that it will have completed 100 percent of  disclosures for only one super module at the start of advance  construction. As we have found in our prior work, proceeding with  construction despite having completed fewer designs than planned  increases the likelihood of design changes later that may, in turn, require  costly and time-intensive re-work to change components that have  already been built. Shipbuilder representatives acknowledged that there  is risk in starting construction of some components prior to completing the  design for individual super modules or the entire submarine. However,  shipbuilder representatives stated that they believe this risk is reduced by  only starting construction on components for which the disclosures are  complete.

Recent Challenges with Critical Technologies Have Reduced Available Schedule Margin

While ship design is underway, the Navy is continuing to develop and  mature the critical technologies related to the Columbia class program.  While these critical technologies are not required at the shipyard for  several years, recent challenges have eroded available schedule margin,  as illustrated below:  Integrated Power System: In 2017, we reported that the Navy  experienced manufacturing problems associated with the integrated  power system. We found that the Navy continues to experience  problems with the electric drive of the integrated power system that  could potentially affect construction of the lead submarine. A  manufacturing defect that affected the system’s first production- representative propulsion motor required extensive repair that  consumed 9 months of schedule margin at the land-based test facility.  The Navy now plans to test the motor at the same time it had  originally scheduled to make any final design changes before starting  production. This could constrain opportunities to implement timely,  corrective actions if problems are discovered during testing.

Common Missile Compartment: Navy officials stated that, in July  2018, the shipbuilder identified substantial weld defects in missile  tubes from one of three tube suppliers and resulted in investigations  of the missile tubes from all suppliers. These defects were discovered  after seven tubes in various stages of outfitting had already been  delivered to the shipyard and five additional tubes under production  have been affected. Navy program officials stated defects occurred  because inexperienced welders performed the complex work and  inspectors at the supplier’s facility subsequently failed to identify the  defects. While the Navy and shipbuilder are still determining the cost  and schedule impacts of the weld defects, program officials estimated  that addressing this issue will consume up to 15 of the 23-month  schedule margin for these components. In addition, program officials  stated that the Navy likely will be responsible for some of the cost  associated with investigating the root cause of the defects and risk  mitigation efforts going forward.

Given the erosion of available schedule margin, there is less time  available to address issues without resulting in schedule delays. For  example, the shipbuilder’s construction plans for two super modules do  not include schedule margin to accommodate any delays that may occur  as the technologies are matured and detail design is completed. One of  these, the stern super module contains three technologies that are not  fully mature—the integrated power system, stern area system, and  advanced propulsor bearing. The integrated power system is not  expected to reach full maturity until October 2019 and the remaining two  technologies will not be mature until after the shipbuilder begins  construction on the lead submarine, not including those components that  begin advance construction years earlier. Without schedule margin to  accommodate any changes or issues, any delays in delivering equipment  to the shipyard on time could disrupt the shipbuilder’s construction  sequence for the lead submarine.

Shipbuilder Is Facing Oversight and Capacity Challenges in Preparation for Columbia Construction

To meet the Navy’s aggressive construction schedule for the lead  submarine, the shipbuilder has to ensure that it has the capacity to meet  a substantially higher workload and effectively balance Columbia and  Virginia class construction. At the same time as construction on Columbia  begins in 2020, the shipbuilder will also have begun constructing two  modified Virginia class attack submarines per year. To accommodate  the construction of both submarine classes, the shipbuilder is planning an  extensive expansion of its facilities, including new buildings, a pier, an  ocean transport barge, and a floating dry dock. The anticipated increases  in workload at the shipyard will also require the shipbuilder to manage a  higher volume of build materials and an expansion of its workforce. While  construction of new facilities is progressing on schedule, according to  shipbuilder representatives, it faces other challenges preparing for  Columbia class construction.

Achieving the planned construction schedule will require the Navy and  shipbuilder to ensure that materials arrive on time and meet quality  expectations, but according to Navy officials, supplier oversight has been  a challenge for this shipbuilder in the past. Both Navy officials and  shipbuilder representatives stated that they are concerned about the  capacity of its suppliers to meet the demand for high-quality components  given an industrial base that has diminished significantly since previous  major submarine construction efforts in the 1980s. Many of the parts and  equipment on Columbia class are common with those used on Virginia  class submarines but, in other instances, suppliers are producing  components for the first time after a considerable break, such as missile  tubes that have not been produced since the early 1990s.

Navy program officials and shipbuilder representatives stated that they  monitor supplier capacity and quality—among other areas—and they  have several methods to intervene if a supplier is not able to perform as  needed. The shipbuilder and the Navy have formed a group to assess the  three primary areas of supplier performance:

Capability: includes the uniqueness of the supplier’s product on the  market, challenges in shifting to a different supplier due to intellectual  property rights or technical knowledge, and the ability for the supplier  to sustain their own supply base.

Capacity: includes the supplier’s ability to increase production without  decreasing quality, maintain that capacity over the program’s  production, their financial dependence on Navy programs for revenue,  lead time needed to meet new orders, and the capacity of their own  suppliers.

Cost: includes the costs of increasing production spread out across  demand from Navy programs.

In 2017, the shipbuilder assessed its supplier base using these areas,  identified the criticality and risk of each supplier based on their potential  impact to the program and potential alternate suppliers, and conducted a  gap analysis comparing the supplier’s current performance to the  program’s desired performance. Based on the results of the analysis, the  shipbuilder identified and is monitoring at-risk suppliers in coordination  with the Navy to determine if immediate intervention is needed, such as  investing in new facilities for the supplier, improving manufacturing  workflow, or finding new sources of material from that supplier.

Despite these efforts, supplier oversight remains an issue, because—in  the instance of the missile tube welds mentioned above—the shipbuilder  focused on managing certain anticipated risks, as opposed to actively  managing the supplier’s quality and performance with on-site independent  inspections, according to Navy officials. In response to the missile tube  issues, the shipbuilder has proposed additional supplier oversight by  assessing the need for on-site inspection teams depending on the risk  each supplier poses to the program. Navy officials stated that they have  begun some assessments but, as of March 2019, had yet to determine  who will pay for this additional oversight. We plan to more fully assess the  Navy and shipbuilder’s oversight of its suppliers for the Columbia class  program in future work.

Building Workforce Capacity and Capability

According to shipbuilder representatives, the start of lead submarine  construction for the Columbia class, combined with expanding Virginia  class construction, increases the demand for hiring and retaining skilled  workers at levels not seen at this shipyard since the 1980s. Navy officials  expressed concerns about the risk of adding large numbers of new  workers, including an influx of inexperienced welders and inspectors— issues that also contributed to the defects in missile tubes discussed  above. To support growing workload from both the Columbia and Virginia  submarine programs, the shipbuilder plans to increase workforce at its  two facilities over the next decade: by 66 percent at Quonset Point,  Rhode Island—where the components and individual submarine modules  will be constructed—and 174 percent at Groton, Connecticut—where the  super modules will undergo final outfitting and assembly. To meet this  increased demand in a skilled workforce, the shipbuilder assessed future  demographic trends in the area surrounding its facilities and found that,  while sufficient labor will likely be available, more training will be  necessary. Consequently, the shipbuilder established internal and  external training programs and partnerships with educational institutions  in the area to grow the qualified workforce in time to begin lead  submarine construction in October 2020.

The influx of inexperienced workers can temporarily decrease  construction efficiency as compared to a current, more experienced  workforce. For example, when the Virginia class program expanded its  workforce to build a second submarine each year, the addition of new  staff contributed to an 8 percent decrease in cost efficiency for the  program. Shipbuilder representatives at one production facility have  already reported reduced efficiency following increased hiring of new  workers. The shipbuilder’s goal is to maintain an average of 8 years of  experience for workers in core trades, such as welding. However, the  shipbuilder’s projections show that the new workforce ramp-up at the  Groton facility will reduce workers’ average experience from 13.1 years to  a low of 5.6 years in 2028—just after the shipbuilder plans to deliver the  lead Columbia class submarine. If workforce growth or efficiency  assumptions are not met, the shipbuilder may resort to scheduling  overtime work or outsourcing some activities to meet the program’s  construction schedule, which would have cost impacts for the program.

Columbia Class Cost Estimate Is Not Reliable and Does Not Reflect Program Risks

The Navy’s procurement cost estimate of $115 billion to construct  Columbia class submarines is not reliable because it does not reflect  likely program costs and risks. We assessed the Columbia class cost  estimate by comparing it with the best practices identified in GAO’s Cost  Estimating and Assessment Guide. We found that it substantially met  the criteria for the comprehensive characteristic of a reliable cost  estimate, and partially met the criteria for the remaining characteristics,  including accurate and credible. In particular, we found that the cost  estimate does not accurately reflect program costs because it is based on  overly optimistic labor hour assumptions, and  is not fully credible because while the Navy conducted risk and  sensitivity analyses to test the likelihood of achieving its assumptions,  it selected a specific cost estimate that informs the program’s budget  which does not include any margin in case those assumptions are not  achieved. In addition, the cost estimates and assessments conducted  by other entities produced a range of results, indicating that there is a  high degree of uncertainty regarding program costs.

See appendix III for the full results of our assessment of the Navy’s cost  estimate.

Navy officials stated they plan to update the Columbia class cost estimate  in support of DOD’s decision to authorize construction of the lead  submarine and this decision is expected to occur in summer 2020. Navy  officials also stated that they expect that the cost estimate will be  complete by the end of fiscal year 2019, followed by an independent cost  assessment to support the authorization decision. However, this  timeframe does not provide assurance that both the update and the  independent assessment will be complete before the Navy requests  funding from Congress for lead submarine construction, as part of its  fiscal year 2021 budget request, which could be submitted as early as  February 2020. If so, decision makers may be basing their decisions on  outdated or incomplete information.

Columbia’s Cost Estimate Is Not Accurate Because It Relies on Overly Optimistic Labor Hour Assumptions

The Columbia class cost estimate relies on optimistic program  assumptions and does not reflect the likely labor hour costs that the Navy  will incur to construct the submarines. As part of our assessment of the  Columbia program cost estimate, we found that it did not fully meet the  best practices for an accurate estimate. A cost estimate is considered  accurate when it is based on an assessment of the most likely costs—that  is, it is neither overly conservative nor overly optimistic. The Navy  estimates that it will need $115 billion to design and construct 12  submarines and NAVSEA cost estimators identified labor costs as a  primary source of cost risk. As discussed below, if the program’s  optimistic assumptions are not realized, the program may require more  funding than originally planned to construct the Columbia class.

The Navy anticipates that it will need 12 million labor hours to directly  construct the lead submarine—referred to as touch labor. This  represents 17 percent fewer labor hours than what was needed for the  lead Virginia class submarine, when adjusted for weight differences. To  develop this estimate, NAVSEA estimators relied heavily on historical  touch labor hour data from the construction of the lead Virginia class  submarine and cost data from the Ohio class submarine program for  unique ballistic submarine components, such as missiles. NAVSEA  estimators took the following steps to develop the Columbia lead  submarine estimate:  In general, heavier ships cost more to construct, so NAVSEA cost  estimators calculated a weight-adjusted estimate based on Virginia  class labor hours to account for the heavier weight of the Columbia  class. This resulted in an initial estimate of 14.5 million touch labor  hours for the lead submarine.

NAVSEA cost estimators then made numerous adjustments in the  cost estimate that reduced the expected number of labor hours based  on multiple assumptions that differences in the design and  construction process would lead to more efficient construction of  Columbia class submarines than previous submarine classes. These  adjustments subsequently decreased the estimate to 12 million touch  labor hours for the lead submarine.

NAVSEA cost estimators then used the lead Columbia submarine  estimate as the basis to calculate labor hours for follow-on  submarines, estimating an average of 8.9 million touch labor hours.

Figure 11 illustrates NAVSEA’s touch labor hour calculation for the lead  submarine.

However, the touch labor hour estimate is overly optimistic—with  assumptions on construction efficiencies that are either unsubstantiated  or unprecedented compared to Virginia class and other shipbuilding  historical data. Compared to the Navy’s estimate, Columbia’s estimated  touch labor hours, as calculated by other organizations, are more  conservative. For example, CBO questioned the Navy’s assumption that  ballistic submarines are less expensive to build than attack submarines,  after accounting for weight differences and estimated that for the overall  class, including the lead and follow-on submarines, the Navy would more  likely realize an 8 percent reduction rather than the 19 percent reduction  estimated by the Navy.

While the shipbuilder will likely realize some efficiencies from initiatives to  improve design and construction processes, our analysis of the Navy’s  assumptions used to develop the cost estimate indicates that they likely  overstate the labor hour reduction the shipbuilder can realistically  achieve. These assumptions include that the program (1) achieves its  design goals at the start of construction; (2) is constructed more efficiently  than Virginia class submarines; and (3) successively reduces the number  of hours needed to construct follow-on submarines. If these assumptions  are not realized, overall program costs could be higher than the Navy’s  procurement estimate of $115 billion. Navy officials stated that they  believe that these assumptions are valid and that the cost estimate is  achievable. However, our assessment indicates that the assumptions for  the cost estimate are overly optimistic, as discussed below.

The Navy’s cost estimate does not reflect the risk that the shipbuilder may  not achieve its planned design completion goals. As we reported above,  design performance to date has slowed and the shipbuilder has had to  hire additional designers in an effort to mature its design on schedule.  NAVSEA cost estimators stated that they recognize that an incomplete  design at the start of ship construction was a significant driver of cost  growth on other shipbuilding programs. For the Columbia class, NAVSEA  cost estimators assumed that achieving the design maturity goal would  eliminate 2 million labor hours by reducing costs associated with rework  and out of sequence work. In October 2018, NCCA officials stated that  they recently reviewed shipbuilder data and the expected design  completion at construction start continues to range between 55 and 75  percent complete—the same range that they estimated in their  independent assessment. While this lower rate would be an improvement  over the Virginia class program, it would still fall short of the 84 percent  assumption built into the cost estimate. If the shipbuilder does not  complete the design at its planned rate and begins construction with a  less mature design, it may need additional labor hours to construct the  ship, resulting in increased program costs.

Overly Optimistic Estimate of Efficiencies

The Navy’s cost estimate includes assumptions that reduce Columbia’s  estimated touch labor hours due to efficiencies from constructing  Columbia and Virginia class submarines concurrently, an assumption with  which the shipbuilder does not agree. NAVSEA cost estimators calculated  a 1.1 million-labor hour reduction, attributing the decrease to efficiencies  gained from constructing multiple submarines at the same time, basing  their assessment on shipbuilder estimates of the Virginia class. However,  it is unclear how increased shipyard production would result in fewer labor  hours to construct each submarine. Shipbuilder representatives stated  that rather than a reduction in touch labor hours, they expect to realize  efficiencies from increased production primarily from reduced overhead  rates and material costs.

Further, the Navy’s independent assessment analyzed labor hour data for  Virginia class construction and found that there was no correlation  between the number of submarines constructed at a time and the total  number of labor hours. However, increasing shipyard production to  include both Virginia and Columbia class construction may increase  schedule risk for the shipbuilder, which could result in additional costs if  the shipbuilder does not achieve planned increases in its workforce and  facility upgrades. When the number of Virginia class submarines under  construction increased, both shipyards experienced inefficiencies due to  poorly planned ramp-up requirements. In addition, DOD officials stated  that problems encountered on one program could affect the other as the  shipbuilder is relying on the same workforce and vendor base for both  programs.

The Navy’s cost estimate also assumed construction efficiencies— because the Columbia class submarine will be less dense than the  Virginia class submarine—another assumption with which the shipbuilder  does not agree. Navy officials stated that less dense submarines are less  costly to construct as the additional space within the hull allows for faster  and more efficient work. However, the shipbuilder conducted analysis to  compare the density of various areas of the Virginia class and Columbia  class submarines and found that areas had very similar density.  Specifically, there was only a 1 percent and 3 percent difference, between  the forward compartments and aft compartments, respectively—some of  the more complex areas of the submarine. If the shipbuilder does not  realize these construction efficiencies, more total labor hours would be  required to construct the submarine, resulting in increased cost.

The Navy’s cost estimate assumes that the costs for follow-on Columbia  class submarines will decrease at a rate that may overstate the  improvements the shipbuilder can realistically achieve. The Navy expects  the number of labor hours to construct Columbia class follow-on  submarines to decrease based on an assumed learning curve rate.  Learning occurs when construction is consistent and continuous and the  shipbuilder learns how to do repetitive tasks more efficiently. The  decrease in the number of expected labor hours is expressed as a  learning curve rate, where a lower percentage indicates that less labor is  required for follow-on units. NAVSEA cost estimators calculated a  learning curve of 88.9 percent for Columbia class submarines. A learning  curve indicates that as the number of units doubles, unit cost decreases  by a constant percentage. In this case, the cost estimate assumed that  the fourth submarine would require only 88.9 percent the amount of labor  to build the second submarine.

NAVSEA’s assumption may overstate the potential learning rate that  Columbia can expect to achieve. The first four Virginia class submarines,  hull numbers SSN 774 through 777, incorporated modular construction  techniques where submarines were built in 10 modules. The next six  Virginia class submarines, hull numbers SSN 778 through 783, were  constructed using four modules. As a result of the improvements in the  modular construction process, construction across the first ten  submarines was not consistent, which is a condition that is necessary to  determine the learning curve rate. Therefore, there is no way to determine  what share of the labor hour reduction on later submarines was due to  learning or process improvements. Rather, SSN 778, the first Virginia  class submarine to use the four modular construction approach is a better  starting point to determine the shipbuilder’s capacity for learning. The  Navy’s independent assessment included a separate learning curve  analysis for Virginia class submarine hulls SSN 778 through 791 and  calculated a potential learning curve rate of 93.9 percent. A learning curve  assumption applies to all follow-on submarines and has a cumulative  effect on the number of labor hours and, ultimately, the cost of these  submarines. In the case of the Columbia program, the rate will apply to  the second through twelfth submarines. Figure 12 shows how the  difference in the learning curve rate can affect the estimated labor hours  for follow-on submarines.

Therefore, a small change in the assumed learning curve rate can have a  significant effect on the cost estimate for follow-on submarines. For  example, the Navy’s independent assessment of the cost estimate  calculated that production costs could increase by $3.59 billion in  constant year 2010 dollars if a learning curve of 93.9 percent was  realized, rather than the 88.9 percent rate estimate. Our previous work on  Navy shipbuilding performance has shown that the Navy has consistently  underestimated the costs for follow-on ships, with costs for Virginia class  submarines underestimated by close to 40 percent.

Columbia Cost Estimate Is Not Credible Because It Does Not Adequately Account for Program Risks

The Columbia program cost estimate did not fully meet the best practice  criteria to be considered credible because, in part, Navy program  management did not sufficiently account for program risks when it  selected the final estimate. To determine the estimate’s credibility, we  examined the extent to which

NAVSEA cost estimators tested, among other things, the sensitivity of  key cost elements such as labor hours and conducted uncertainty  analyses to quantify risks; and an independent cost estimate and assessment were conducted by  groups outside the acquiring organization (specifically, CAPE and  NCCA) to determine whether other estimating methods produced  similar results.

We found that while the Navy program management’s $115 billion  procurement cost estimate for the Columbia class is overly optimistic in  some of its assumptions, the estimate does not reflect any contingency to  offset the likely effects of not meeting the assumptions, which is a best  practice. In addition, the independent cost estimates and assessments  conducted by other organizations had varying results, indicating the high  level of uncertainty regarding Columbia program costs.

Sensitivity and Risk Analysis Indicate Insufficient Cost Risk Coverage

Navy leadership’s decision to select $115 billion as the program cost  estimate means that there is no margin in the program budget to cover  likely program costs if risks are realized. The best practices identified in  GAO’s cost estimating guide state that the results of a risk analysis  should be used to select a cost estimate that is sufficient to manage  program risks. NAVSEA cost estimators conducted a risk analysis to  identify and quantify program risks, and determined the effects of  changing key cost driver assumptions and factors—important steps in  creating a high quality estimate. However, while NAVSEA cost  estimators identified 54 risk parameters for construction costs, we found  that some of the inputs for these ranges resulted in a cost estimate that  understates the potential impact of program cost risks. For example, the  risk ranges do not sufficiently account for the issues we identified above,  including that  increased shipyard construction could result in similar inefficiencies  that occurred in the production of the Virginia class, requiring more  labor hours than estimated; and  shipbuilder workforce ramp-up could result in decreased efficiency  and quality due to the influx of new workers even greater than the  issues observed on the Virginia class when shipyard construction  increased.

For other risk parameters, such as cost of material provided by the  shipbuilder, the cost estimate documentation was not sufficient for us to  analyze whether the risk ranges included in the estimate were reasonable  (i.e., not overly optimistic or pessimistic). As a result, we could not  determine whether the risk analysis sufficiently captures the risk of  program cost growth, or what the probability is of achieving the $115  billion procurement cost estimate.

Further, Columbia’s program management and the milestone decision  authority selected $115 billion as the program’s procurement cost  estimate, without adjusting for the likelihood of cost growth in the design  or construction of Columbia class submarines identified in the risk  analysis. As we reported in December 2017, the risk analysis developed  by NAVSEA indicated that there is only a 45 percent probability that the  overall program cost estimate will be sufficient to cover program costs.  The cost estimating best practices identified in our cost estimating guide  state that a risk-adjusted cost estimate helps ensure that sufficient  funding will be available for the expected program costs. Additionally, a risk-adjusted cost estimate is consistent with federal internal control  standards, which indicate that risk mitigation efforts should be selected to  sufficiently respond to risks. However, Columbia program officials stated  that they believe program risks can be managed within the current cost  estimate—which they consider to be conservative—as it does not account  for all of the program’s potential cost savings. Specifically, the Navy  anticipates that the program will realize up to $1.9 billion in additional cost  savings from use of authorities associated with the National Sea-Based  Deterrence Fund (the Fund), such as the authority to purchase  components for multiple submarines—which we discuss later in this  report. As a result, the program office estimate represents the program  manager’s cost goal for the Columbia program, rather than the risk- adjusted estimate. Even if the Navy were to achieve the full anticipated  $1.9 billion savings, these savings represent only 1.5 percent of program  costs. Such cost savings are unlikely to cover program cost overruns for a  high-risk program, such as Columbia, given that historically shipbuilding  programs experience 27 percent cost growth. As the current estimate  does not include any reserves for cost overruns, program management is  relying on these potential savings to help mitigate likely cost growth.

Independent Cost Reviews Indicate Varying Results

Several entities have conducted independent reviews of the Columbia  program cost estimate, with varying results. CAPE conducted an  independent cost estimate and NCCA conducted an independent cost  assessment of the program estimate in support of the Columbia class  program’s Milestone B review. CAPE’s independent cost estimate was 3  percent lower than the Navy’s service cost position, which it stated was  due to CAPE’s use of lower shipyard labor rates. However, NCCA’s  assessment did not produce similar results as the program cost estimate  and concluded that the program is at risk of up to $6.14 billion in cost  growth. The program manager reviewed the recommendations in the  independent cost assessment and determined that the program office  estimate appropriately weighs program risks. Navy leadership selected  the program office estimate to serve as the Navy’s service cost position  because program officials stated that they believe program risks can be  managed within the program cost estimate. CBO also conducted a cost  estimate and projected that procurement of 12 submarines would be 6  percent higher than the program estimated. The results of these cost  estimates and NCCA’s assessment are summarized in table 3.

As part of the Milestone B review, the milestone decision authority  reviewed the service cost position and CAPE’s independent cost  estimate. The independent cost assessment was reviewed by Navy  leadership as part of the service cost position process and, therefore, was  not briefed as part of the milestone review. The milestone decision  authority accepted the Navy service cost position and directed the Navy  to use this estimate as the basis of its fiscal year 2018 budget request. It  also established an $8 billion affordability cap for the average  procurement cost of all 12 submarines to control future program costs.

Congress May Not Have Up-to-Date Cost Information When Considering Columbia Class Budget Request for Lead Submarine Funding

Navy officials stated that they plan to update the cost estimate for the  lead submarine in support of a planned Defense Acquisition Board  review, in the third quarter of fiscal year 2020. At that point, the Navy will  be seeking approval from the milestone decision authority to award the  contract for construction of the lead submarine. However, the Navy and  DOD’s general timeframes do not provide assurance that the planned  update of the cost estimate would be completed prior to the fiscal year  2021 budget request, which will include funding for lead submarine  construction, as shown in figure 13 below.

The milestone decision authority has directed CAPE, with assistance from  NCCA, to assess the lead submarine cost estimate to support the  decision to authorize the Navy to award the contract for lead submarine  construction. Since this assessment will occur after the Navy has updated  the lead submarine cost estimate, it is even less likely that the program  budget request will reflect the results from the independent cost  assessment. Additionally, the current program cost estimate the Navy  developed for the Milestone B review does not reflect the program’s  current strategy to use authorities associated with the Fund to achieve  cost savings, as discussed further below. The best practices identified in  GAO’s cost estimating guide state that cost estimates should be regularly  updated and reflect the program acquisition baseline. Updating the cost  estimate and risk analysis to include these anticipated savings and  current program data would improve its reliability and help ensure that  budget requests are sufficient to execute the Columbia program as  planned. After we provided our draft report to DOD for comment, Navy  officials briefed us on the changes they had made to the program’s  estimate to date, stating that they updated the cost risk analysis as part of  an internal program review. While the Navy plans to update the lead  submarine cost estimate again by the end of fiscal year 2019 to support  the Defense Acquisition Board review in the summer of 2020, it has yet to  provide specific details on the steps it will take to update this estimate to  ensure that it would include likely program costs and risks, such as the  cost data it plans to include or the assumptions it may reassess. Further,  since the Navy will likely submit its budget request to Congress as early  as February 2020, Congress may be asked to authorize and fund lead  submarine construction without the benefit of any changes to the estimate  that may occur as a result of recommendations stemming from an  independent review of the update.

Further, although the Navy reports Columbia program cost information to  Congress through annual matrices submissions, updates to the program  cost estimate will not be reflected in these reports. For example, the Navy  plans to report program manager and contractor cost estimates for  individual submarines in the matrices once the submarines are under  construction. Since these estimates are based on shipbuilder contract  performance, they are initially calculated only after construction of each  submarine is 15 percent complete, when sufficient data are available to  show performance trends. While the Navy plans to award the contract for  the lead submarine in October 2020, limited contractor performance data  will be available in time for the February 2021 matrix submission. As a  result, the earliest opportunity to report on the cost of the lead submarine  would be the Navy’s next submission in February 2022, at which point the  Navy will have already requested funding for the second and third  Columbia submarine.

Navy Is Using National Sea-Based Deterrence Fund and Associated Authorities, but Anticipated Savings May Be Overestimated

In 2014, Congress created a National Sea-Based Deterrence Fund (the  Fund) that provides DOD with greater discretion to fund the design,  construction, purchase, alteration, and conversion of the Columbia  class. Since then, Congress has provided the Navy with enhanced  acquisition authorities to buy and construct submarines and certain key  components early, in bulk, and continuously, when using these funds.  The Navy anticipates saving over $1.9 billion through use of these  authorities, but these savings, which were not included in the Columbia  class program’s cost estimate, may be overestimated.

Navy Executes Columbia Program through the Fund and Its Associated Authorities

Since its inception in 2014, Congress has expanded the special  acquisition authorities under the Fund, in part, to allow the Columbia class  program to gain economic efficiencies and realize cost savings. The  timeline of the establishment of the Fund and legislative changes are  shown in figure 14.

The following authorities have been included as part of the use of the  Fund:

Economic order quantity: Permits awarding of contracts that provide  a quantity of supplies that will result in a total cost and unit cost most  advantageous to the government by achieving economic efficiencies  based on production economies.

Advance construction: Allows for manufacturing and fabrication  efforts prior to ship authorization.

Multiyear procurement authority: Permits a single contract for more  than one year of critical components.

Incremental funding authority: Facilitates the purchase of long lead  items through partial funding of a contract with the expectation that full  funding will be provided later.

Using the Fund’s associated authorities, the Navy is able to purchase  significant components and start advance construction prior to receiving  Congress’s authorization of and funding to purchase each submarine. In  total, the Navy will have requested and received $8.6 billion in funding,  including 33 percent of funding for the lead submarine, before it receives  authorization and funding to begin construction of the lead submarine in  October 2020. At that point, the Navy will also have already requested  funding for the propulsor and advance construction for the second  submarine.

Under law, the Navy is required to deposit all appropriations for the  Columbia class construction and design into the Fund. To date, the Navy  has made three deposits from the Shipbuilding and Conversion, Navy  account into the Fund, totaling over $1.6 billion. The Navy is using initial  deposits of $773 million in fiscal year 2017 and $862 million in fiscal year  2018 for detail design and continuous production of missile tube  components.

The Navy Comptroller initiates all deposits into the Fund, which are  approved by the DOD Comptroller as internal reprogramming actions, as  shown in figure 15.

Anticipated Savings from Use of Fund’s Associated Authorities May Be Overestimated

The Navy anticipates achieving over $1.9 billion in savings through the  use of the Fund’s associated authorities, but the Navy did not evaluate  these savings when it developed the program office cost estimate. Table  4 provides a description of each authority and the Navy’s plans and  estimated potential savings resulting from use of the authorities.

Overall, while we were unable to fully assess the methodology and  assumptions the Navy used to estimate anticipated savings, the  information we reviewed indicated that the Navy may have overestimated  some of the savings the program can realistically achieve through use of  the Fund’s associated authorities. While the Navy provided some  documentation of the cost estimate methodologies, we could not fully  validate that the estimated savings were realistic because, in general, the  documentation provided by the Navy did not include a detailed description  of how the estimates were calculated or how historical data were used to  develop the estimate—a best practice identified in GAO’s cost estimating  guide. In some cases, such as for individual critical components, the  total value of the component costs was not documented. For other  savings, such as advance construction, the Navy could not provide  documentation of the calculations or a rationale for the estimated savings.

In addition, the Navy assumes a higher rate for Columbia multiyear  procurement savings than what has been typically achieved for other  programs. The Navy has generally used multiyear procurement contracts  after production has begun and some units have already been purchased.  For example, according to the Navy, it did not receive multiyear  procurement authority for the DDG 51 Arleigh Burke-class destroyer  program—until 1998—more than 10 years after the contract for the lead  ship was awarded and 38 ships had been purchased. We have reported  that DOD typically overestimates savings from multiyear procurement  authority. Further, in a 2017 presentation to Congress, the Navy stated  that multiyear procurement savings are historically 10 to 12 percent.  When the Navy requested multiyear procurement authority for the DDG  51 program in fiscal year 2013, it estimated achieving a savings of 8.7  percent. Similarly, when planning material purchases for the Virginia class  submarine, the shipbuilder estimated that it would achieve 10 to 15  percent savings through the use of multiyear procurement authority.  However, the Navy estimates that the Columbia class program will realize  savings of 15 to 20 percent using multiyear procurement authority. A  realistic estimate of savings is essential because program management is  essentially relying on these savings as the only cost reserve to address  any issues that arise during design and construction of the submarines.  Updating the cost estimate to reflect these savings will provide program  management with a more realistic assessment of the margin available  and resources needed to achieve their costs.

The Columbia class program is driven by the continued and pressing  need to meet the Navy’s nuclear deterrent requirements as the legacy  submarine fleet cannot extend its life any longer. From the outset this has  translated into an aggressive and concurrent schedule for lead submarine  construction. To counterbalance this schedule risk, the program plans to  complete a substantial amount of the design before starting construction,  which may prove challenging as the shipbuilder must complete an  increasingly higher volume and complexity of disclosures. This, coupled  with failures in missile tubes already delivered to the shipyard, highlight  the potential for management challenges ahead. This is not to suggest  that in a program of this size and complexity that some issues are not to  be expected. Rather, the challenge for the Columbia class program is that  the Navy has a limited ability to slow the pace of the program given the  mission imperatives.

At present, the need for additional resources appears likely because the  Navy’s margin to mitigate any cost growth from issues that develop during  design and construction relies on overestimated savings from use of the  Fund’s associated authorities. The steps that the Navy takes between  now and the fiscal year 2021 budget request to understand and plan for  likely program costs will determine whether sufficient funding is in place to  cover potential cost growth. The Navy plans to update the lead submarine  cost estimate to reflect its current acquisition strategy and, in doing so,  the Navy has the opportunity to incorporate more realistic information into  the risk analysis and lead submarine cost estimate. In addition, a realistic  and well-documented estimate of savings from use of the Fund’s  associated authorities would help ensure that the Navy has allocated the  necessary resources to address any issues that emerge during design or  construction of the lead submarine. Such steps will likely improve the  reliability of the lead submarine cost estimate and would position the  Navy to better align its fiscal year 2021 budget request with funding it will  likely need to construct the lead submarine—the next key decision point  in the Columbia class program. Without an updated cost estimate with  more realistic assumptions, Congress will be asked to commit billions of  dollars for the lead submarine without knowing the full potential cost of  construction and the possible effect on other shipbuilding programs.

We are making three recommendations to the Secretary of the Navy:

The Secretary of the Navy should direct NAVSEA to incorporate  current cost and program data and an updated cost risk analysis in its  planned update of the Columbia class lead submarine cost estimate.  (Recommendation 1)

The Secretary of the Navy should direct NAVSEA to develop a  realistic and well-documented estimate of savings from use of the  authorities associated with the Fund and incorporate the savings  associated with the lead submarine into the Columbia lead submarine  cost estimate. (Recommendation 2)

The Secretary of the Navy should direct the Columbia class program  office to update the lead submarine cost estimate and cost risk  analysis prior to requesting funds for lead submarine construction.  (Recommendation 3)

We provided a draft of the sensitive report to DOD for comment. DOD’s  written comments on the sensitive report are reprinted in appendix IV and  summarized below. DOD concurred and described the actions they have  taken or plan to take in response to all three of our recommendations.

Regarding our recommendations to update its cost estimate update prior  to requesting funds for lead submarine construction, the Navy has stated  that it incorporated current cost and program data and an updated risk  analysis into its cost estimate for the lead submarine in 2018, as part of  an annual review. The Navy also stated that it will continue to update the  lead submarine cost estimate with current data prior to requesting funding  for lead submarine construction in fiscal year 2021. Until the updated  estimate is independently validated—an essential cost estimating step— we cannot determine that the updated estimate is credible.

Further, in response to our recommendation regarding the development  of a realistic and well-documented estimate of savings from use of the  Fund’s associated authorities, the Navy stated that it incorporated savings  in its updated cost estimate. However, it has not provided any additional  evidence to demonstrate that estimated savings from use of the Fund’s  associated authorities are realistic and well-documented. Based on  documentation that the Navy provided to us, it did not include a detailed  description of how the estimates were calculated or how historical data  were used to develop the estimate. Until these estimates are  independently validated, the Navy cannot be confident that the program  will achieve the planned amount of savings.

The Navy also provided technical comments, which we incorporated as  appropriate. DOD also raised a number of issues related to our  assessment of the cost estimate, advance construction, and technology  development, which we address in appendix IV.

We are sending copies of this report to the appropriate congressional  committees, the Acting Secretary of Defense, the Secretary of the Navy,  and other interested parties. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.

Should you or your staff have questions, please contact me at (202) 512- 4841 or oakleys@gao.gov. Contact points for our Offices of   Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

This report evaluates the Navy’s Columbia class submarine program.  Specifically, we assessed (1) the Navy’s progress and challenges, if any,  associated with meeting design goals and preparing for lead submarine  construction; (2) the reliability of the Navy’s cost estimate for the  Columbia class submarine program; and (3) how the Navy is  implementing the National Sea-Based Deterrence Fund (the Fund) and  associated authorities to construct Columbia class submarines.

This report is a public version of a sensitive report that we issued in  March 2019. The Department of Defense (DOD) deemed some of the  information in our March report to be sensitive, which must be protected  from public disclosure. Therefore, this report omits sensitive information  about the Navy’s development of critical technologies for the Columbia  class program, including specific details about the technologies. Although  the information provided in this report is more limited, the report  addresses the same objectives as the sensitive report and uses the same  methodology.

To assess the Navy’s progress and what challenges, if any, are  associated with meeting design goals and preparing for lead submarine  construction, we reviewed Navy and shipbuilder documents, including  program briefings, schedules, and contract status reports to assess the  schedule and performance risks of the Columbia class program. To  evaluate the shipbuilder’s progress in maturing the Columbia class  design, we reviewed the Navy’s plans for design management and  completion, evaluated the shipbuilder’s design schedule, and compared  them against design progress reports to identify any delays. To evaluate  the Navy’s plans for advance construction, we analyzed metrics reported  in Navy and shipbuilder documents, briefing slides, and other  documentation including key dates and estimated construction plans. We  compared design knowledge on the Columbia class program to our prior  work on shipbuilding best practices. We reviewed ongoing development  efforts and schedules for the Columbia class program’s critical  technologies to determine remaining risks to their development and  integration. We also reviewed the matrices submitted by the Navy to  Congress in February and October 2018, to determine the status of the  program and identify any changes to the Navy’s design and construction  goals for the program since our last report in December 2017. We also  analyzed available documentation related to the status of the nuclear  reactor and integrated power system. We reviewed the shipbuilder’s  construction plans for its new facilities and its workforce hiring plans. We  also reviewed the shipbuilder’s and Navy’s process for evaluating its  suppliers.

To corroborate documentary evidence and gather additional information  in support of our review, we met with officials from the Navy’s Columbia  class submarine program office; Naval Nuclear Propulsion Directorate;  Naval Surface Warfare Center Philadelphia; Office of the Chief of Naval  Operations; Supervisor of Shipbuilding, Groton; the Office of the Deputy  Assistant Secretary of Defense for Systems Engineering; and the Office  of Undersecretary of Defense for Acquisition and Sustainment.  Additionally, we met with shipbuilding representatives from General  Dynamics Electric Boat—the prime contractor—as well as their main  subcontractor, Huntington Ingalls Industries Newport News Shipbuilding  to understand their role in Columbia class design and construction.

To assess the reliability of the Navy’s cost estimate for the Columbia  class submarine program, we determined the extent to which the estimate  met best practices as identified in GAO’s Cost Estimating and  Assessment Guide. We examined cost estimate documentation, such as  the Columbia class program life-cycle cost estimate, briefs, memoranda,  and other documents that contain cost, schedule, and risk information.  We also examined the independent cost estimate conducted by the Office  of the Secretary of Defense’s Office of Cost Assessment and Program  Evaluation (CAPE), the independent cost assessment conducted by the  Naval Center for Cost Analysis (NCCA), and the cost estimate conducted  by the Congressional Budget Office, to determine what methodologies  and assumptions differed from the program cost estimate. We met with  Navy officials who were responsible for developing the cost estimate to  understand the processes used by the cost estimators, to clarify  information, and to allow the Navy to provide additional documentation on  the data and methodologies used in the estimate. We also observed  portions of the Columbia class program’s cost model during a  presentation and discussion with Navy cost estimators. We also reviewed  the matrices submitted by the Navy to Congress to identify any changes  to the Navy’s cost goals and reported information. To further corroborate  documentary evidence and gather additional information in support of our  review, we conducted interviews with relevant DOD and Navy officials  responsible for developing, updating, and assessing the Columbia class  program cost estimate, including CAPE; NCCA; the Naval Sea Systems  Command’s (NAVSEA) Cost Engineering and Industrial Analysis Group;  and the Columbia class program office.

To evaluate how the Navy is implementing the Fund and associated  authorities to construct Columbia class submarines, we reviewed the  legislation establishing and modifying the Fund, program budget request  documents, and DOD reprogramming approvals. We also reviewed the  Navy’s basis of estimate for the savings it plans to achieve from these  authorities. To further corroborate documentary evidence and gather  additional information in support of our review, we met with officials from  the Office of the Assistant Secretary of the Navy for Financial  Management and Comptroller; Office of the Under Secretary of Defense  (Comptroller); and the Columbia class program office to discuss the  Navy’s plans to use and execute the Fund and DOD’s role in approving  transfers into the Fund.

Appendix II: Technology Readiness Levels

Appendix II: Technology Readiness Levels Description   Lowest level of technology readiness. Scientific research begins to be translated into applied  research and development. Examples might include paper studies of a technology’s basic  properties.

Invention begins. Once basic principles are observed, practical applications can be invented.  The application is speculative and there is no proof or detailed analysis to support the  assumption. Examples are still limited to paper studies.

Active research and development is initiated. This includes analytical studies and laboratory  studies to physically validate analytical predictions of separate elements of the technology.  Examples include components that are not yet integrated or representative.

Basic technological components are integrated to establish that the pieces will work together.  This is relatively “low fidelity” compared to the eventual system. Examples include integration  of “ad hoc” hardware in a laboratory.

Fidelity of breadboard technology increases significantly. The basic technological components  are integrated with reasonably realistic supporting elements so that the technology can be  tested in a simulated environment. Examples include “high-fidelity” laboratory integration of  components.

Representative model or prototype system, which is well beyond the breadboard tested for  TRL 5, is tested in a relevant environment. Represents a major step up in a technology’s  demonstrated readiness. Examples include testing a prototype in a high-fidelity laboratory  environment or in a simulated realistic environment.

Prototype near or at planned operational system. Represents a major step up from TRL 6,  requiring the demonstration of an actual system prototype in a realistic environment, such as  an aircraft, vehicle, or space. Examples include testing the prototype in a test bed aircraft.

Technology has been proven to work in its final form and under expected conditions. In almost  all cases, this TRL represents the end of the true system development. Examples include  developmental test and evaluation of the system in its intended weapon system to determine if  it meets design specifications.

Actual application of the technology in its final form and under mission conditions, such as  those encountered in operational test and evaluations. In almost all cases, this is the end of  the last “bug fixing” aspects of true system development. Examples include using the system  under operational mission conditions.

Appendix III: GAO’s Assessment of the Reliability of the Navy’s Cost Estimate for the Columbia Class Submarine Program

To assess the reliability of the Navy’s cost estimate, we determined the  extent to which the estimate was consistent with cost estimating best  practices as identified in GAO’s Cost Estimating and Assessment Guide.  This guide groups the best practices into four general characteristics: well  documented, comprehensive, accurate, and credible.

We reviewed documentation the Navy submitted for its cost estimate  including limited portions of the Navy’s cost model, conducted numerous  interviews, and reviewed relevant sources. We determined that the  Columbia class cost estimate substantially met one, and partially met  three of the four characteristics of a reliable cost estimate, shown in figure  16. We determined the overall assessment rating by assigning each  individual rating a number: Not Met = 1, Minimally Met = 2, Partially Met =  3, Substantially Met = 4, and Met = 5. Then, we calculated the average of  the individual assessment ratings to determine the overall rating for each  of the four characteristics as follows: Not Met = 1.0 to 1.4, Minimally Met  = 1.5 to 2.4, Partially Met = 2.5 to 3.4, Substantially Met = 3.5 to 4.4, and  Met = 4.5 to 5.0. We consider a cost estimate to be reliable if the overall  assessment ratings for each of the four characteristics are substantially or  fully met. If any of the characteristics are not met, minimally met, or  partially met, then the cost estimate does not fully reflect the  characteristics of a high-quality estimate and is not considered reliable.

Appendix IV: Comments from the Department of Defense

In addition to responding to our recommendations, DOD also provided  observations on a number of issues related to our assessment of the cost  estimate, advance construction, and technology development. Our  response to DOD’s observations is as follows.

Assessment of Columbia Class Program’s Cost Estimate

In paragraph 4, page 1 of the letter above, the Navy did not agree with  our assessment of the accuracy of the cost estimate and stated that  the life cycle cost estimate includes accurate calculations, proper  inflation tables, and updates to requirements. DOD also stated that  GAO or other stakeholders did not identify any errors. This is  incorrect. While the Navy allowed us to observe the model, we did not  independently check the accuracy of the calculations because Navy  officials stated that the cost model, which contains the cost  calculations, could not be released. We informed the Navy that this  would affect parts of our assessment. After we provided a draft of the  report, the Navy provided a briefing summarizing the results of a  program office cost checkpoint conducted in September 2018. At the  briefing, we received information on updates that the Navy made to  the program cost estimate. As a result, we updated our assessment to  reflect that the Navy substantially met the best practice to regularly  update the cost estimate to reflect significant changes. However, the  additional information provided by the Navy did not change our  assessment of the accuracy and, therefore, our overall assessment of  the Columbia cost estimate remains valid.

In paragraph 1, page 2, the Navy did not agree with our assessment  of the credibility of the cost estimate and stated that the life cycle cost  estimate includes analyses that address sensitivity, risks, and  uncertainty within the estimate. As we point out in the report, the  estimate is based, in part, on optimistic assumptions regarding the  number of labor hours needed to construct Columbia class  submarines. The Navy has made updates to the program cost  estimate based on a 2018 checkpoint review and stated that the cost  risk analysis has been updated and program costs are less than  originally estimated. The Navy provided us with a high-level brief of  these updates. However, due to the timing of this report, we were not  able to fully assess the update to the cost model. Given the size and  complexity of the Columbia class program, we continue to believe that  the program’s cost estimate does not adequately account for program  risks.

In paragraph 3, page 1, DOD stated that our findings were largely  informed by an assessment conducted by the Naval Center for Cost  Analysis (NCCA). However, our process for assessing program cost  estimates is based on the extent to which the estimate met best  practices outlined in GAO’s Cost Estimating and Assessment Guide.  In conducting our assessment, we examined multiple sources of  information, including the Columbia class program life cycle cost  estimate, NCCA’s independent cost assessment, DOD’s Office for  Cost Assessment and Program Evaluation’s (CAPE) independent cost  estimate, and the cost estimate conducted by the Congressional  Budget Office (CBO), to determine what methodologies and  assumptions differed from the program cost estimate. We also relied  on prior experience examining and reporting on the cost performance  of Navy shipbuilding programs, issuing 26 reports over the past 10  years. We found, for example, that the cost estimate is based on  optimistic labor assumptions which, while in agreement with NCCA’s  assessment and CBO’s estimate, results from our independent  assessment of the evidence we reviewed and on our prior work.

In paragraph 2, page 2, the Navy stated that it identified super  modules and selected components where acceleration would reduce  construction schedule risk. We acknowledge in the report that the  design for these components will be complete prior to starting  construction. However, we continue to believe that starting  construction for components of the lead submarine before the  arrangements for the submarine are complete increases design and  construction risk. Even if the components included in advance  construction are fully designed, risk remains for the adjoining and  interfacing components within the module that may have ongoing  design work, potentially requiring costly and time-intensive rework.

In paragraph 4, page 2, the Navy notes that fully maturing all of the  key technologies identified in our 2017 report—such as the advanced  propulsor bearing—would require substantial investments in money  and time. However, we continue to reinforce that a tenet of achieving  design maturity is based on demonstrating a prototype in its final form,  fit, and function in a realistic environment—which requires a design  resembling the final configuration.

Integrated Power System Motor Manufacturing Delays

In paragraph 6, page 2, the Navy stated that it does not agree with our  characterization that the Navy is continuing to experience  manufacturing problems with the electric drive of the integrated power  system. DOD stated that while the vendor experienced delays in  manufacturing the prototype motor, it has taken proactive measures to  deliver the motor to the shipyard, as scheduled. However, the Navy’s  plan to concurrently test and finalize the design increases risk that any  issues identified in testing could delay the delivery of the system to  the shipyard. As a result, we continue to identify this as a key risk to  the program. Additional details on this system are classified.

Appendix V: GAO Contact and Staff Acknowledgments

Shelby S. Oakley, (202) 512-4841 or oakleys@gao.gov.

In addition to the contact above, the following staff members made key  contributions to this report: Diana Moldafsky, Assistant Director; Laura  Jezewski; Jessica Karnis; and Nathaniel Vaught. Other contributions were  made by Brian Bothwell; Daniel Glickstein; Kurt Gurka; Stephanie  Gustafson; and Robin Wilson.

The Navy will start taking certain submarines out of service in 2027 and is relying on its new Columbia class submarines to replace them. The schedule for construction and delivery of the first submarine is aggressive and leaves little room for error.

However, the Navy's cost estimate to construct 12 Columbia class submarines—$115 billion—is not realistic because it is based on several overly optimistic assumptions, such as the amount of labor needed for construction.

We made 3 recommendations to help the Navy calculate a more realistic cost estimate.