NASA Human Space Exploration: Persistent Delays and Cost Growth Reinforce Concerns over Management of Programs

NASA is undertaking a trio of closely related programs to continue human space exploration beyond low-Earth orbit. All three programs (SLS, Orion, and supporting ground systems) are working toward a launch readiness date of June 2020 for the first mission.

The House Committee on Appropriations included a provision in its 2017 report for GAO to continue to review NASA's human space exploration programs. This is the latest in a series of reports addressing the mandate. This report assesses (1) how NASA's human space exploration programs are performing relative to cost and schedule commitments, and (2) the extent to which NASA's use of contract award fees is achieving desired program outcomes. To do this work, GAO examined program cost and schedule reports and contractor data, and interviewed officials. This report does not assess the effect, if any, of the government shutdown that ended in January 2019.

Due to continued production and testing challenges, the National Aeronautics and Space Administration's (NASA) three related human spaceflight programs have encountered additional launch delays and cost growth. In November 2018, within one year of announcing an up to 19-month delay for the three programs—the Space Launch System (SLS) vehicle, the Orion spacecraft, and supporting ground systems—NASA senior leaders acknowledged the revised date of June 2020 is unlikely. Any issues uncovered during planned integration and testing may push the launch date as late as June 2021. Moreover, while NASA acknowledges about $1 billion in cost growth for the SLS program, it is understated. This is because NASA shifted some planned SLS scope to future missions but did not reduce the program's cost baseline accordingly. When GAO reduced the baseline to account for the reduced scope, the cost growth is about $1.8 billion.

In addition, NASA's updated cost estimate for the Orion program reflects 5.6 percent cost growth. The estimate is not complete, however, as it assumes a launch date that is 7 months earlier than Orion's baseline launch date. If the program does not meet the earlier launch date, costs will increase further. Updating baselines to reflect current mission scope and providing complete cost estimates would provide NASA management and Congress with a more transparent assessment of where NASA is having difficulty controlling costs.

NASA paid over $200 million in award fees from 2014-2018 related to contractor performance on the SLS stages and Orion spacecraft contracts. But the programs continue to fall behind schedule and overrun costs. Ongoing contract renegotiations with Boeing for the SLS and Lockheed Martin for the Orion program provide NASA an opportunity to reevaluate its strategy to incentivize contractors to obtain better outcomes.

GAO is making four recommendations to NASA, including that the SLS program should calculate cost growth based on costs that are currently included in the first mission and the Orion program should update its cost estimate to reflect the schedule agreed to in its baseline. In addition, the SLS and Orion programs should reevaluate their strategy for incentivizing contractors. NASA concurred with three recommendations, and partially concurred with the recommendation related to the Orion program's cost estimate. GAO believes the recommendation remains valid, as discussed in the report.

The National Aeronautics and Space Administration (NASA) is developing  systems planned to transport humans beyond low-Earth orbit, including  the Moon and eventually Mars. Deep space exploration requires the  capability to transport crew and large masses of cargo beyond low-Earth  orbit to distant destinations. The systems NASA is developing to achieve  this include:  the Space Launch System (SLS) program, which is developing a  vehicle to launch a crew capsule and cargo beyond low-Earth orbit;  the Orion Multi-Purpose Crew Vehicle (Orion) program, which is  developing a crew capsule to transport humans beyond low-Earth  orbit; and  the Exploration Ground Systems (EGS) program, which is developing  systems and infrastructure to support assembly, test, and launch of  the SLS and Orion crew capsule, and recovery of the Orion crew  capsule.

Each of these programs represents a large, complex technical and  programmatic endeavor and is in the integration and test phase of  development. Our prior work has shown this phase of the acquisition  process often reveals unforeseen challenges leading to cost growth and  schedule delays.

GAO has designated NASA’s management of acquisitions as a high-risk  area for almost three decades. In our March 2019 high-risk report, we  reported there was a lack of transparency in NASA’s major project cost  and schedules, especially for its human spaceflight programs. We  reported that the agency has not taken action on several  recommendations related to understanding the long-term costs of its  human exploration programs. For example, EGS and SLS do not have a  cost and schedule baseline that covers activities beyond the first planned  flight, and Orion does not have a baseline beyond the second planned  flight. We have previously reported that without transparency into these  estimates, NASA does not have the data to assess long-term affordability  and Congress cannot make informed budgetary decisions.

The House Committee on Appropriations included a provision in its 2017  report for GAO to continue to review NASA’s human space exploration  programs, specifically the SLS, EGS, and Orion programs. This report is  the latest in a series of reports addressing the mandate. This report  assesses (1) how NASA’s human space exploration programs are  performing, including any technical challenges, relative to their cost and  schedule commitments, and (2) the extent to which NASA’s use of  contract award fees is achieving desired program outcomes.

To assess the performance of the human space exploration programs,  including any technical challenges, relative to their cost and schedule  commitments, we obtained and analyzed program cost and schedule  estimates and compared them against baselines to determine cost and  schedule growth. We also interviewed program officials to determine how  the programs phase costs for future flights outside the current baseline.  We also obtained and reviewed program risk registers and discussed the  potential impacts of cost and schedule risks, including risk mitigation  efforts to-date, with program officials. In addition, we assessed program  schedules over time and compared performance against program plans  to identify potential and realized schedule delays, including the impact of  delays across the programs. We based our assessment on data collected  prior to the federal government shutdown that occurred in December  2018 and January 2019. We determined the data we used were  sufficiently reliable for the purposes of this engagement. This assessment  does not reflect the effect, if any, of the shutdown on the programs’ costs  and schedules or NASA’s March 2019 announcement that it is studying  how to accelerate the SLS schedule.

To determine the extent to which NASA’s use of contract award fees is  achieving desired program outcomes, we analyzed award fee plans and  fee determination records for the Orion crew spacecraft and SLS core  stage—or stages—contracts. We selected these contracts because they  represent the largest development efforts for each program. We analyzed  contract documentation to determine the amount of award fee available  on these contracts compared to other incentives, such as milestone  incentives, and calculated fees paid to date. Moreover, we compared fee  determination results to overall program outcomes since program  confirmation. For more information on our scope and methodology, see  appendix I.

We conducted this performance audit from March 2018 to June 2019 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The NASA Authorization Act of 2010 directed NASA to develop a SLS, to  continue development of a crew vehicle, and to prepare infrastructure at  Kennedy Space Center to enable processing and launch of the launch  system. To fulfill this direction, NASA formally established the SLS  launch vehicle program in 2011. Then, in 2012, NASA aligned the  requirements for the Orion program with those of the newly created SLS  vehicle and the associated ground systems programs. The Exploration  Systems Development (ESD) organization reports to NASA’s Associate  Administrator for Human Exploration and Operations Mission Directorate  and is responsible for managing and integrating the human space  exploration programs. Figure 1 provides details about each SLS hardware  element and its source as well as identifies the major portions of the  Orion spacecraft.

NASA established the EGS program to modernize the Kennedy Space  Center to prepare for integrating hardware, as well as processing and  launching SLS and Orion, and recovery of the Orion crew capsule. The  EGS program consists of a number of components and processing  centers including the Vehicle Assembly Building, Mobile Launcher, and  Crawler-Transporter.

The Mobile Launcher consists of (1) a two-story base that is the platform  for the rocket and (2) a tower equipped with a number of connection lines,  called umbilicals, and launch accessories that will provide SLS and Orion  with power, communications, coolant, fuel, and stabilization prior to  launch. During preparations for launch, the Crawler-Transporter will pick  up and move the Mobile Launcher into the Vehicle Assembly Building.  Inside the Vehicle Assembly Building, NASA will stack the SLS and Orion  vehicle on the Mobile Launcher and complete integration for launch.  Before launch, the Crawler-Transporter will carry the Mobile Launcher  with SLS and Orion to the launch pad where engineers will lower the  Mobile Launcher on to the pad and remove the Crawler-Transporter.  During launch, each umbilical and launch accessory will release from its  connection point, allowing the rocket and spacecraft to lift off from the  launch pad. Figure 2 is a picture of the Mobile Launcher positioned on top  of the Crawler-Transporter outside of the Vehicle Assembly Building.

During Exploration Mission 1 (EM-1), the SLS vehicle is to launch an  uncrewed Orion to a distant orbit some 70,000 kilometers beyond the  Moon. All three programs—SLS, Orion, and EGS—must be ready on or  before the EM-1 launch readiness date to support this integrated test  flight. Exploration Mission 2 (EM-2) will be a 10- to 14-day crewed flight  with up to four astronauts that will orbit the moon and return to Earth to  demonstrate the baseline Orion vehicle capability.

History of Program Cost and Schedule Changes

NASA establishes an agency baseline commitment—the cost and  schedule baselines against which the program may be measured—for all  projects that have a total life cycle cost of $250 million or more. A  rebaseline is a process initiated if the NASA Administrator determines the  development cost growth is more than 30 percent of the estimate  provided in the baseline of the report, or if other events make a rebaseline  appropriate. A replan is a process generally driven by changes in  program or project cost parameters, such as if development cost growth  is 15 percent or more of the estimate in the baseline report or a major  milestone is delayed by 6 months or more from the baseline date. A  replan does not require a new project baseline to be established.

When the NASA Administrator determines that development cost growth  is likely to exceed the development cost estimate by 15 percent or more,  or a program milestone is likely to be delayed from the baseline’s date by  6 months or more, NASA must submit a report to the Committee on  Science, Space, and Technology of the House of Representatives and  the Committee on Commerce, Science, and Transportation of the  Senate. Should a program exceed its development cost baseline by  more than 30 percent, the program must be reauthorized by the Congress  and rebaselined in order for the contractor to continue work beyond a  specified time frame. NASA tied the SLS and EGS program cost and  schedule baselines to the uncrewed EM-1 mission and the Orion  program’s cost and schedule baselines to EM-2.

Over the past 5 years, we have issued several reports assessing the  progress of NASA’s human space exploration programs relative to their  agency baseline commitments. In April 2017, we found that given the combined effects of ongoing technical challenges in conjunction with  limited cost and schedule reserves, it was unlikely that these programs  would achieve the committed November 2018 launch readiness date.  We recommended that NASA confirm whether this launch readiness date  was achievable and, if warranted, propose a new, more realistic EM-1  date and report to Congress on the results of its schedule analysis. NASA  agreed with both recommendations and stated that it was no longer in its  best interest to pursue the November 2018 launch readiness date.  Subsequently, NASA approved a new EM-1 schedule of December 2019,  with 6 months of schedule reserve available to extend the date to June  2020, and revised costs (see table 1).

Because NASA delayed the EM-1 schedule by up to 19 months, the SLS  and EGS programs—that are both baselined to EM-1—reported a replan  to the Congress. The EGS program also reported its development costs  increased by about 23 percent over the baseline. At the same time, NASA  reported that the SLS program development costs would only increase by  about 2 percent.

Under the Federal Acquisition Regulation (FAR), a variety of contract  types are available including those that incentivize a contractor in areas  that may include performance, cost, or delivery. The type of contract used  for any given acquisition inherently determines how risk is allocated  between the government and the contractor. According to the FAR, since  the contract type and the contract price are interrelated, the government  must consider them together. The government can choose a contract  type and negotiate price (or estimated cost and fee) that will result in  reasonable contractor risk and provide the contractor with the greatest  incentive for efficient and economical performance. For example, under  firm-fixed-price contracts, the contractor assumes full responsibility for  performance costs. Under cost-reimbursement contracts, the government  provides for the payment of allowable incurred costs, to the extent  prescribed in the contract. The government uses cost-reimbursement  contracts when, for example, there are uncertainties involved in contract  performance.

Incentive contracts can be either fixed-price or cost-reimbursement type  contracts. The contractor’s responsibility for the performance costs and  the profit or fee incentives in incentive contracts are tailored to the  uncertainties involved in contract performance. Incentive contracts— including award fee and predetermined, formula-type incentive fee  contracts—are designed to attain specific acquisition objectives by, in  part, including appropriate incentive arrangements that (1) motivate  contractor efforts that might not otherwise be emphasized, and (2)  discourage contractor inefficiency and waste. Award fees generally  emphasize multiple aspects of contractor performance in areas that the  government assesses more subjectively. In contrast, predetermined  formula-type incentives are generally associated with a cost incentive, but  can also emphasize performance in areas that the government assesses  more objectively.

The FAR indicates that award fee contracts are suitable when  it is neither feasible nor effective to devise predetermined objective  incentive targets,  the likelihood of meeting acquisition objectives will be enhanced by  using a contract that provides the government with the flexibility to  evaluate both actual performance and the conditions under which it  was achieved, and  the administrative effort and cost are justified.

Table 2 provides an overview of cost-plus-incentive-fee and cost-plus- award-fee contracts because these are the type used in the Orion and  SLS programs.

Multiple-incentive contracts contain more than one incentive. For  example, these contracts may include both subjective award fee criteria  and predetermined, formula-type incentives. Agencies can use incentive  contracts to promote certain acquisition outcomes, such as keeping costs  low, delivering a product on time, and achieving technical performance of  the product.

NASA awarded incentive contracts to both Boeing and Lockheed  Martin—a cost-plus-incentive-fee/award-fee contract to Boeing for the  SLS stages effort and a cost-plus-award-fee contract to Lockheed Martin  for the Orion crew spacecraft effort. For the SLS stages incentive  contract with Boeing, the contract includes both incentive and award fees,  broken into these three components:

Milestone-incentive fees. These fees are paid for successful  completion of each program milestone event.

Cost-incentive fees. These fees are initially negotiated and later  adjusted by a formula and are paid based on the relationship of total  allowable costs to total target costs.

Award fees. These fees are determined through subjective  evaluations relative to factors in the contract’s award fee plan.

For the Orion crew spacecraft incentive contract with Lockheed Martin,  the contract includes fee broken into three components. The government  typically uses award fees when it is not feasible or effective to use  predetermined objective criteria. Therefore, as noted above, award fees  are typically determined against subjective criteria. However, this contract  includes award fee with both subjective and objective criteria:

Milestone award fees. These fees are paid for completing critical  criteria and dates associated with each milestone.

Performance incentive fee. These fees are paid for completing criteria  and dates associated with each performance incentive.

Period of performance award fee. These fees are determined through  subjective evaluations relative to factors in the contract’s award fee  plan.

For purposes of discussion within this report, we group each of the fees  for each contract into one of four categories—milestone fee, performance  incentive fee, cost incentive fee, and award fee. When award fees are  used that require a subjective assessment by the government, NASA  generally defines award fee periods of at least 6 months for the duration  of the contract and establishes performance evaluation boards to assess  the contractor’s performance relative to the performance evaluation plan.  For the contracts we reviewed, NASA evaluates contractor performance  based on weighted evaluation factors to determine the award fee. Table 3  includes a description of the evaluation factors and the weighted  percentages for each factor assigned to the SLS stages and Orion crew  vehicle contracts.

When developing a contractor’s evaluation for a period of performance,  the members of the performance evaluation boards for each contract use  descriptive ratings in their evaluations. Performance monitors for different  areas within the programs compile a list of the contractor’s strengths and  weaknesses relative to specific criteria and defined activities for each of  the evaluation factors. The performance monitors then consider other  factors, such as government-directed changes and obstacles that arose  that may have affected the contractor’s performance, and prepare  performance reports. Members of the performance evaluation boards  consider the performance monitor’s reports and assign the scores and  descriptive ratings for the specific evaluation period. Table 4 below  outlines award fee adjectival ratings, award fee pool available to be  earned, and descriptions of the award fee adjectival ratings from the  Federal Acquisition Regulation.

Continued Underperformance Has Led to Additional Schedule Delays and Cost Growth

In November 2018—within 1 year of announcing a delay for the first  mission—senior NASA officials acknowledged that the revised EM-1  launch date of December 2019 is unachievable and the June 2020 launch  date (which takes into account schedule reserves) is unlikely. These  officials estimate that there are 6 to 12 months of schedule risk  associated with this later date, which means the first launch may occur as  late as June 2021 if all risks are realized. This would be a 31-month delay  from the schedule originally established in the programs’ baselines.  Officials attribute the additional schedule delay to continued production  challenges with the SLS core stage and the Orion crew and service  modules. NASA officials also stated that the 6 to 12 months of risk to the  launch date accounts for the possibilities that SLS and Orion testing and  final cross-program integration and testing at Kennedy Space Center may  result in further delays. These 6 to 12 months of schedule risk do not  include the effects, if any, of the federal government shutdown that  occurred in December 2018 and January 2019.

In addition, NASA’s reporting of cost data for the SLS and Orion  programs is not fully transparent. NASA’s estimates for the SLS program  indicate 14.7 percent cost growth as of fourth quarter fiscal year 2018, but  our analysis shows that number increases to 29.0 percent when  accounting for costs that NASA shifted to future missions. Further, in  summer 2018, NASA reported a 5.6 percent cost growth for the Orion  program. However, this reported cost growth is associated with a program  target launch date that is 7 months earlier than its agency baseline  commitment launch date. If the Orion program executes to the launch  date established in its agency baseline commitment, costs will increase  further.

SLS: First Mission Will Incur Additional Delay as Challenges with Core Stage Production Continue, and Cost Growth Underreported

SLS Will Not Meet June 2020 Replan Schedule

The SLS program will not meet the June 2020 launch date for the first  mission due, in part, to ongoing development issues with the core stage.  For this mission, the SLS launch vehicle includes solid rocket boosters,  an upper stage, and a core stage—which includes four main engines and  the software necessary to command and control the vehicle. As of fall  2018, the program reported that the boosters, engines, and upper stage  all had schedule reserves—time allocated to specific activities to address  delays or unforeseen risks— to support a June 2020 launch. The core  stage, however, did not have schedule reserves remaining as the  program continues to work through development issues.

According to the SLS program schedule, core stage development  culminates with “green run” testing. For this test, NASA will fuel the  completed core stage with liquid hydrogen and liquid oxygen and fire the  integrated four main engines for about 500 seconds. The green run test  carries risks because it is the first time that several things are being done  beyond just this initial fueling. For example, it is also the first time NASA  will fire the four main engines together, test the integrated engine and  core stage auxiliary power units in flight-like conditions, and use the SLS  software in an integrated flight vehicle. In addition, NASA will conduct the  test on the EM-1 flight vehicle hardware, which means the program would  have to repair any damage from the test before flight.

The program has no schedule margin between the end of core stage  production and the start of the green run test, and is tracking risks that  may delay the test schedule. For example, as the NASA Office of  Inspector General (OIG) found in its October 2018 report, the Stage  Controller—the core stage’s command and control hardware and  software needed to conduct the green run test—is 18 months behind  schedule and may slip further. Any additional delays with the  development of the core stage and stage controller will further delay the  start of the green run test. In addition, the SLS program has no schedule  margin between the green run test and delivery of the core stage to  Kennedy Space Center for integration to address any issues that may  arise during testing.

In November 2018, senior NASA officials stated that they have accounted  for the potential of continued core stage development delays—along with  risks to the Orion and EGS programs—and stated that there are an  additional 6 to 12 months of risk to the EM-1 launch date. We found that a  delay of this length would push the launch date for EM-1 out as far as  June 2021 should all of the risks be realized. This would represent a 31- month delay from the original schedule baseline. Further, these 6 to 12  months of schedule risk do not include the effects, if any, of the federal  government shutdown that occurred in December 2018 and January  2019. Figure 3 below compares schedules of key events for the core  stage shortly after NASA established the program baseline in August  2014, the December 2017 replan, and the program’s schedule as of  November 2018.

Officials from the SLS program and Boeing, the contractor responsible for  building the core stage, indicated that an issue driving core stage delays  was underestimation of  the complexity of manufacturing and assembling  the core stage engine section—where the four RS-25 engines are mated  to the core stage—and those activities have taken far longer than  expected. For example, around the time of the December 2017 replan,  the SLS program schedule indicated that it would take 4 months to  complete the remaining work. By late 2018, the estimate for the same  work had increased to 11 months. Part of that delay included time  required to resolve residue and debris discovered in the fuel lines, which  was present because Boeing had not verified the processes that its  vendors were using to clean the fuel lines. Further, installation of the fuel  lines overlapped with other work in the engine section, making work in the  limited space more difficult and complex than it otherwise would have  been.

NASA officials indicated that there have been additional issues behind  core stage delays, including the following:

Boeing underestimated the staffing levels required to build the core  stage in the time available. According to a NASA official, as core  stage production began, Boeing was focused on minimizing the  number of technicians, in part to keep costs low, and hired about 100  technicians. The official stated that Boeing now has about 250  technicians on staff in order to address ongoing delays, however,  because a number of the additional staff came from non-spaceflight  projects, some time was lost getting those staff up to speed on SLS.  In addition, the official noted that technicians were spending time  performing work away from the vehicle, such as collecting tools and  parts for the work they were completing. According to the official,  Boeing has since hired additional support staff to perform off-vehicle  tasks such as pre-packaging tools in order to allow technicians to  spend their time working on the vehicle.

The build plans for the core stage were not adequately mature when  the contractor began work on the hardware itself, which led to  additional delays. For example, according to NASA officials, they  expected the work instructions—detailed directions on how the vehicle  should be built—to be largely complete by the program’s critical  design review, which precedes the production decision. In this case,  however, the build plans were not complete by the start of production.  Officials stated that the lack of build plans slowed progress, as  technicians can only perform work that they have instructions to carry  out.

In addition, the time to perform some work activities needed to build  the designed vehicle was not included in the schedule. For example,  more than 900 engine section brackets that were in the design were  not on the schedule and, according to NASA officials, Boeing had to  install the brackets later, adding complexity to the work schedule.

Boeing officials provided three additional perspectives regarding the  delays.

Boeing officials explained that they did not anticipate any changes  from NASA for the loads—impacts and stresses of mass, pressure,  temperature, and vibration that the vehicle will experience—following  the program’s critical design review, but instead NASA provided three  significant updates to those loads. In some cases, the changes were  significant enough that they invalidated legacy systems Boeing had  planned to use, which required rework. However, SLS program  officials stated that they continued to update loads data as the  environments anticipated during launch became clearer.

Boeing officials also stated that they alerted NASA in September 2014  that a decision to decrease funding in fiscal year 2015 would require  the contractor to delay the core stage delivery date. In October 2018,  however, the NASA OIG reported that while Boeing anticipated  receiving $150 million less than planned in fiscal year 2015, the  company received only $53 million less; that a funding increase was  received in fiscal year 2016; and that the value of Boeing’s contract  increased by nearly $1 billion in May 2016.

Finally, Boeing officials stated that it has been challenging to execute  NASA’s development approach that called for the first set of hardware  built to be used for the initial launch. Boeing officials stated that they  are more used to an approach in which they use the first hardware  built to qualify the design and that hardware is never flown. The  challenge with the current approach, according to Boeing officials, is  that all the learning associated with a first build is occurring on the  flight unit, which requires extra scrutiny and slows down the process.  SLS program officials stated that this approach has been part of the  development plan since the initial contract with Boeing was signed.

One area in which the program has benefited from the core stage delay is  that development of SLS test and flight software, which has been a  schedule concern for the program, now has additional time to complete  development. Delays to date have been due to late hardware model  deliveries and requirements changes according to program officials. The  SLS program completed the qualification test—a verification that the  software meets documented requirements—for the green run software in  March 2018. Program officials stated that the verified test software  release will be complete by April 2019, and the EM-1 flight software  release will be complete by October 2019. The earlier they are able to  complete the software before launch, the more time they will have to  complete testing, fix any defects they find, and work with EGS to integrate  with the ground software. Measuring to a June 2020 launch date, flight  software development has about 6 months of additional time to address  issues should they arise. However, the program has a number of test  cycles remaining and the program continues to assess a risk regarding  the potential impact that late requirements changes could have on  software completion.

SLS Program Has Shifted Some Costs to Future Missions, Resulting in an Underreporting of Cost Growth for EM-1

The SLS program has been underreporting its development cost growth  since the December 2017 replan because of a decision to shift some  costs to future missions while not adjusting the baseline downward to  reflect this shift. The SLS development cost baseline established in  August 2014 for EM-1 includes cost estimates for the main vehicle  elements—stages, liquid engines, boosters—and other areas. According  to program officials, because of the December 2017 replan process,  NASA decided that costs included as part of the SLS EM-1 baseline cost  estimate would be more appropriately accounted for as costs for future  flights. Thus, NASA decided not to include those costs, approximately  $782 million, as part of the revised SLS EM-1 cost estimate. However,  NASA did not lower the $7 billion SLS development cost baseline to  account for this significant change in assumptions and shifting of costs to  future flights, and NASA officials told us that they were not sure what the  benefit to NASA would be in adjusting the baseline.

This decision presents challenges in accurately reporting SLS cost growth  over time. NASA’s decision not to adjust the cost baseline downward to  reflect the reduced mission scope obscures cost growth for EM-1.  NASA’s cost estimate as of fourth quarter fiscal year 2018 for the SLS  program indicated development cost growth had increased by $1 billion,  or 14.7 percent. However, our analysis shows that development cost  growth actually increased by $1.8 billion or 29.0 percent, when the  development baseline is lowered to account for the reduced mission  scope. Essentially, NASA is holding the baseline costs steady, while  reducing the scope of work included in current cost estimates (see figure  4).

NASA’s current approach for reporting cost growth misrepresents the cost  performance of the program and thus undermines the usefulness of a  baseline as an oversight tool. NASA’s space flight program and project  management requirements state that the agency baseline commitment for  a program is the basis for the agency’s commitment to the Office of  Management and Budget (OMB) and the Congress based on program  requirements, cost, schedule, technical content, and an agreed-to joint  cost and schedule confidence level. Removing effort that amounts to  more than a tenth of a program’s development cost baseline is a change  in the commitment to OMB and the Congress and results in a baseline  that does not reflect actual effort.

Further, the baseline is a key tool against which to measure the cost and  schedule performance of a program. A program must be rebaselined and  reauthorized by the Congress if the Administrator determines that  development costs will increase by more than 30 percent. Accounting  for shifted costs, our analysis indicates that NASA has reached 29.0  percent development cost growth for the SLS program.

In addition, as we previously reported in May 2014, NASA does not have  a cost and schedule baseline for SLS beyond the first flight. As a result,  NASA cannot monitor or track costs shifted beyond EM-1 against a  baseline. We recommended that NASA establish cost and schedule  baselines that address the life cycle of each SLS increment, as well as for  any evolved Orion or ground systems capability. NASA partially  concurred with the recommendation, but has not taken any action to date.

By not adjusting the SLS baseline to account for the reduced scope,  NASA will continue to report costs against an inflated baseline, hence  underreporting the extent of cost growth. NASA’s Associate Administrator  and Chief Financial Officer stated that they understood our rationale for  removing these costs from the EM-1 baseline and agreed that not doing  so could result in underreporting of cost growth. Further, the Associate  Administrator told us that the agency will be relooking at the SLS  program’s schedule, baseline, and calculation of cost growth.

Orion: Challenges Contribute to Additional Delay for First Mission and Program Cost Estimate Not Complete

Orion Is Not on Schedule to Meet June 2020 Replan Schedule for First Mission

The Orion program is not on schedule to meet the June 2020 launch date  for the first mission due to delays with the European Service Module and  ongoing component issues with the avionics systems for the crew  module, including issues discovered during testing.

European Service Module (ESM). Through a barter agreement, the  European Space Agency developed and produced the ESM, which  provides propulsion, air, water, and power to the crew module while in  space. The European Space Agency delivered the ESM to NASA in  November 2018, following several delays with its development. According  to program officials, the most recent set of delays prior to delivery were  due to issues and failures during ESM propulsion system testing as well  as the need to redesign power system components.

Orion and EGS officials explained that a total of 20 months is required  from receipt of the ESM to prepare it for launch. This time frame includes  14 months for the Orion program to finalize testing of each module and  complete program-level integration and testing and 6 months for the EGS  program to complete integrated test and checkout with SLS and EGS.  As a result, the earliest the Orion program could be ready to support a  first mission based on the service module schedule alone is July 2020, 20  months after NASA accepted delivery in November 2018. ESD officials  told us that the 6 to 12 months of risk that could push EM-1 to June 2021  includes ESM-related delays. These 6 to 12 months of schedule risk do  not include the effects, if any, of the federal government shutdown that  occurred in December 2018 and January 2019.

Figure 5 compares schedules of key events for the Orion program,  including delays with the ESM, from shortly after NASA established the  program’s baseline in September 2015, the December 2017 replan, and  as of November 2018.

Crew Module. While the ESM remains the critical path—the path of  longest duration through the sequence of activities that determines the  earliest completion date—for the Orion program, the crew module is  nearly the critical path due in part to component failures within the  avionics systems during testing. Figure 6 is a picture of a crew module  test article.

In May 2018, we reported that the Orion program was addressing  component issues in its avionics systems after they failed during vibration  testing. For example, components throughout the crew and service  module relied on computer cards used to regulate power. When those  cards cracked during testing, the program needed to redesign the cards,  retest them, and reinstall them for system tests. Since then, additional  avionics failures have surfaced. In one instance, one of the vehicle’s  global positioning system receivers failed to power up. In another, a part  failed on one of the inertial measurement units, which provide navigation  information like vehicle rotation and acceleration. In March 2019, program  officials told us that they have addressed these issues in the avionics  systems and all flight hardware is installed.

Testing. The ability for Orion, SLS, and EGS to complete testing in the  integrated test laboratory facility—where software and hardware or  hardware simulators are tested together—remains an ongoing risk for  both the first mission and then the timing of the second mission. The lab  has limited time and test resources to complete the testing necessary for  EM-1, and NASA officials indicated that at times it has more demand than  it can support. In addition, some testing is taking longer than planned,  delaying later tests. The risk associated with these delays is that the later  the program discovers an issue, the less time there is to address the  issue prior to launch.

At the same time that the Orion program is completing EM-1 work in the  integrated test lab, the program will also need to modify the lab’s  configuration in order to support EM-2 efforts because of hardware and  software differences between missions. The schedule currently includes  periods of time during EM-1 testing where EM-1 efforts will be shut down  in order to work on lab modifications for EM-2. Although program officials  indicated that test lab delays for EM-1 will not adversely affect lab efforts  for EM-2, resources directed to EM-2 will mean less resources will be  available during those times to support EM-1.

The Orion program has reported development cost growth but is not  measuring that growth using a complete cost estimate. In summer 2018,  the Orion program reported development cost growth of $379 million, or  5.6 percent above its $6.768 billion development cost estimate. The  program explained that the major drivers of this cost growth were the slip  of the EM-1 launch date, which reflected delays in the delivery of the  service module; Orion contractor underperformance; and NASA-directed  scope increase.

However, during our review, Orion program officials stated that this cost  estimate assumes an EM-2 launch date of September 2022, which is 7  months earlier than the program’s agency baseline commitment date of  April 2023 that forms the basis for commitments between NASA, the  Congress, and OMB. As a result, NASA’s current cost estimate for the  Orion program is not complete because it does not account for costs that  NASA would incur between September 2022 and April 2023.  Subsequently, program officials told us that its cost projections fund one  of those seven months. See figure 7.

NASA officials originally told us that they do not have an Orion cost  estimate through the EM-2 agency baseline commitment launch date of  April 2023 because they plan to launch by September 2022, if not earlier.  According to scheduling best practices, performance is measured against  the program’s baseline even if a program is working to an earlier date.

By not estimating costs through its baseline launch date, the Orion  program is limiting the NASA Associate Administrator’s insight into how  the program is performing against the baseline. According to federal law,  the Administrator must be immediately notified any time that a designated  official has reasonable cause to believe that either the program’s  development cost is likely to exceed the estimate in the agency baseline  commitment by 15 percent or more or a program milestone will slip 6  months or more beyond its schedule agency baseline commitment date.  If the Administrator confirms the cost growth or schedule delay exceeds  the given threshold, the Administrator must submit a report to the  Committee on Science and Technology of the House of Representatives  and the Committee on Commerce, Science, and Transportation of the  Senate. Given that the program is already reporting cost growth to a date  earlier than its baseline schedule, updating the cost estimate relative to  the EM-2 baseline schedule would provide NASA management and  Congress with more complete cost data and increased awareness of  whether additional oversight is merited.

EGS: Delays and Development Challenges Have Eroded the Schedule, but Program Remains within Replanned Schedule and Costs

Since the December 2017 replan, the EGS program has had to address  several technical challenges that consumed schedule reserves.  Nevertheless, officials expect to have EGS facilities and software ready  by June 2020, the planned launch date. The program has completed  many of its projects, including the renovation of the Vehicle Assembly  Building and the launch pad. Since the replan, however, the project has  had to address technical challenges with the Mobile Launcher. Figure 8  below compares the EGS schedule—including timeframes for the Mobile  Launcher and software completion—shortly after NASA established the  program’s baseline in September 2014, the December 2017 replan, and  as of November 2018. It also shows the potential launch window  reflecting the 6-12 months of risk NASA is tracking that could push EM-1  to June 2021.

Mobile Launcher. The Mobile Launcher schedule deteriorated since the  December 2017 replan due to problems with finalizing construction work  prior to moving it to the Vehicle Assembly Building. Moving the Mobile  Launcher into the Vehicle Assembly Building was intended to allow the  program to begin multi-element verification and validation, a process that  checks that the various launch and processing systems at Kennedy  Space Center meet requirements and specifications and can operate  together to fulfill their intended purpose.

Challenges the program experienced with the Mobile Launcher included  having to add structural supports after determining that the design was  not adequate to carry the load of the SLS vehicle and fuel. In addition,  program officials stated that construction work overall did not progress to  the point desired to move the Mobile Launcher to the Vehicle Assembly  Building. As a result, the program did not move the Mobile Launcher into  the Vehicle Assembly Building until September 2018, 5 months later than  in the schedule established after the December 2017 replan. Moving  forward, the program has to complete the multi-element verification and  validation process for the Mobile Launcher and Vehicle Assembly  Building.

We have reported on a number of issues related to the EGS program’s  management of the Mobile Launcher, as well as the now-completed  Vehicle Assembly Building project. For example, in 2016, we found that  the program did not mature requirements and designs for the Mobile  Launcher before beginning construction. In addition, the EGS program  completed all major structural changes to the Mobile Launcher prior to  completing the design and installation of the ground support equipment  and the nine umbilicals that connect the Mobile Launcher directly to the  SLS and Orion. There have also been ground support equipment and  umbilical design changes both during and after the Mobile Launcher’s  design phase because of vehicle requirement changes from SLS and  Orion. Officials indicated this approach was problematic because the  concurrency increased program risk. Further, according to officials, the  decision to have separate contracts for design and construction  exacerbated these challenges. Officials indicated that this contracting  strategy meant that design changes required multiple levels of review and  approval from NASA and each of the program’s contractors, which in turn  led to numerous contract modifications.

According to EGS officials, the program plans to incorporate lessons  learned from developing the first Mobile Launcher into the acquisition  approach for a second Mobile Launcher that NASA is building to allow for  future configurations of the SLS vehicle. Specific lessons officials plan to  carry forward to the second Mobile Launcher include:  implementing an integrated design process, including establishing a  process to better handle requirement changes during design and  construction; developing and maintaining a three-dimensional (3D) model to  facilitate integrated design; and enabling builder involvement during the design process to avoid  pitfalls during construction.

However, these lessons learned do not address metrics to assess design  stability before starting construction. Our work on acquisition best  practices show that good processes that mature designs early in  development and ensure that the design meets requirements can position  a program for future success and lead to more predictable cost and  schedule outcomes. Traditionally, we have used the number of  releasable engineering drawings as a metric to assess design stability.  Specifically, our work has found that achieving design stability at the  product critical design review, usually held midway through product  development, is a best practice. Completion of at least 90 percent of  engineering drawings at this point provides tangible evidence that the  product’s design is stable.

We have also found that the U.S. Navy and the commercial shipbuilding  industry use 3D product models as tools to document design stability.  We found that there are aspects of shipbuilding that are analogous to  building a Mobile Launcher in that both involve designing and building a  large metal structure and installing multiple complex integrated systems  to support complex functions such as launching spacecraft, or in the case  of the Navy, launching aircraft and/or missile systems. NASA officials  agreed that developing a Mobile Launcher is analogous to shipbuilding.  Best practices for commercial shipbuilding indicate that 3D product  models documenting 100 percent of the system’s basic and functional  designs should be complete before construction begins.

Basic design includes fixing the ship steel structure; routing all major  distributive systems, including electricity, water, and other utilities; and  ensuring the ship will meet the performance specifications.

Functional design includes providing further iteration of the basic  design, providing information on the exact position of piping and other  outfitting in each block, and completing a 3D product model.

The combined basic and functional designs in conjunction with the 3D  product model provide the shipbuilder a clear understanding of the ship  structure as well as how every system is set up and routed throughout the  ship. This detailed knowledge allows commercial shipbuilders to design,  build, and deliver complex ships such as floating production storage and  offloading vessels, which are able to collect, process, and store oil from  undersea oil fields, within schedule estimates.

The improved design processes the EGS program is pursuing in the  development of the second Mobile Launcher, including the development  of a 3D model to facilitate integrated design, have the potential to improve  program outcomes. Further, achieving design stability before beginning  construction would also improve this potential.

Software. The program’s two software development efforts represent the  EGS critical path, and program officials stated that recent changes have  begun to address previous challenges with the software development. For  example, officials explained that the program has implemented iterative  integration testing and has identified lead engineers for each software  development area. The iterative integration testing involves conducting  tests on smaller segments of software throughout the development  process instead of waiting to conduct testing when a software release is  fully complete. According to officials, these efforts allow the program to  identify and correct errors prior to completing a full software drop. These  changes have also resulted in lower numbers of issues found in some  software releases. Further, the 6-month delay to the SLS and Orion  programs has provided additional flexibility to EGS’s software  development schedule.

Finally, with respect to EGS’s performance against its cost baseline, EGS  updated its cost estimate as part of the December 2017 replan. The EGS  program continues to operate within costs established for the June 2020  launch date, $3.2 billion, but any delays beyond June 2020 will result in  additional cost growth.

Contractors Received Majority of Award Fees but NASA Experienced Poor Program Outcomes

NASA’s award fee plans for the SLS stages and Orion crew spacecraft  contracts provide for hundreds of millions of dollars to incentivize  contractor performance, but the programs continue to fall behind  schedule and incur cost overruns. Our past work shows that when  incentive contracts are properly structured, the contractor has profit  motive to keep costs low, deliver a product on time, and make decisions  that help ensure the quality of the product. Our prior work also shows,  however, that incentives are not always effective tools for achieving  desired acquisition outcomes. We have found that, in some cases, there  are significant disconnects between contractor performance for which the  contractor was awarded the majority of award fees possible without  achieving desired program results. Additionally, we have found that some  agencies did not have methods to evaluate the effectiveness of award  fees.

The incentive strategies for both the SLS stages and the Orion crew  spacecraft contracts include multiple incentives—milestone fees,  performance incentive fees, cost incentive fees, and award fees—aimed  at incentivizing different aspects of contractor performance. These  contracts’ milestone fees, performance incentive fees, and cost incentive  fees are generally determined against objective criteria, such as meeting  a date and application of predetermined formulas. For example, NASA  will pay a milestone fee to Boeing under the SLS contract when it meets a  specific program milestone such as transferring the core stage to the  government for the green run test. Under this contract, Boeing receives  additional milestone fee when it beats a milestone date and reduced fee  when it misses a milestone date. Likewise, pre-determined formula-type  incentives—such as these contracts’ performance incentive fees and cost  incentive fees—are typically determined based on objective criteria, such  as meeting technical metrics or predetermined cost targets.

Award fees on these types of contracts are generally determined at 6 to  12-month periodic evaluations of the contractor’s performance against  criteria outlined in the award fee plan. For example, according to officials,  NASA may evaluate the contractor against technical performance and  criteria, such as the ability to avoid and predict cost overruns, manage  risk, or accomplish small business goals. Upon the completion of a formal  review, performance evaluation board officials make recommendations to  the fee determination official on the amount of fee to be paid. Figures 9  and 10 provide overviews of the total incentive fee available on the  current contracts for the SLS stages contract and the Orion crew  spacecraft contract, by type and percentage.

Under the terms of the current contracts, Boeing has earned about $271  million in award fee and Lockheed Martin has earned about $294 million  in award fee. Since each program held its confirmation review, the point  in time when a program established its cost and schedule baselines,  NASA has paid the majority of available award fee to both contractors.  Specifically, NASA has paid Boeing about 81 percent of available award  fee—or about $146 million—and Lockheed Martin about 93 percent—or  about $88 million—since their respective program confirmation reviews.

During the annual award fee periods, the descriptive ratings both  contractors received ranged from good to excellent. In the subjective  appraisals supporting these ratings, NASA identified both strengths that  indicate areas of good contractor performance and weaknesses that  indicate areas of poor contractor performance. Table 5 includes the  results of award fee determinations since the respective program  confirmations. The numerical score for each evaluation period represents  the percentage of fee paid to the contractor from the available fee pool.

Examples of strengths and weaknesses NASA identified in the award fee  letters include the following:

For the Boeing award fee period ending February 2015, NASA  identified several strengths, including effective and timely  communication, but stated that its subcontractor management for the  vertical assembly center was inadequate. In particular, the program  discovered during this time that the as-built design of the vertical  assembly center tool was not capable of serving its purpose, which is  to build core stage hardware. The design issue resulted in several  months of schedule delays. NASA also raised concerns about  Boeing’s ability to manage to the baseline schedule in a subsequent  award period.

For the Lockheed Martin award fee period ending April 2017, NASA  identified several strengths, including addressing top program  development risks such as establishing a robust mitigation plan to  address risks related to the heatshield block architecture. At the same  time, NASA noted that Lockheed Martin was not able to maintain its  schedule for the crew service module and that the contractor’s  schedule performance had decreased significantly over the previous  year.

While both the SLS and Orion contractors have received the majority of  available award fee in each award fee period, the programs have not  always achieved overall desired outcomes. For example, in its December  2018 award fee letter to Boeing—representing the good assessment for  the September 2017 through October 2018 period of performance—the  fee determination official noted that the significant schedule delays on this  contract have caused NASA to restructure the flight manifest for SLS. As  previously discussed, within 1 year of announcing a delay for the first  mission, senior NASA officials acknowledged that the SLS and Orion  programs will not meet the new EM-1 schedule of December 2019, and  the 6 months of schedule reserve available to extend the date to at least  June 2020 has been consumed. In addition, the officials identified 6 to12  months of risk to that date, which could increase the delay up to 31  months. These 6 to 12 months of schedule risk do not include the effects,  if any, of the federal government shutdown that occurred in December  2018 and January 2019 due to a lapse in appropriations for fiscal year  2019.

Both the contractors and government bear responsibilities for these  delays. We have previously found that NASA has made programmatic  decisions—including establishing low cost and schedule reserves,  managing to aggressive schedules, and not following best practices for  earned value management—that have compounded technical challenges  that are expected for inherently complex and difficult large-scale  acquisitions. Further, we previously reported that NASA did not follow  best practices for establishing cost and schedule baselines for these  programs nor update cost and schedule analyses to reflect new risks.  As a result, NASA overpromised what it could deliver from a cost and  schedule perspective.

At the same time, both contractors have had challenges that contributed  to past delays. For example, in 2015, Boeing was unable to manufacture  an intertank panel—which resides between the liquid oxygen and liquid  hydrogen tanks—without significant cracking. At the time, NASA  estimated that resolving this issue could result in a 6-month slip to the  production schedule. Further, as previously discussed, NASA discovered  during installation that fuel lines used in the engine section were  contaminated with residue and other debris. According to a program  official, Boeing had not verified the processes that its vendors were using  to clean the fuel lines, resulting in about 2 months’ delay to resolve  residue and debris issues. SLS officials indicated that the engine section  has a very complex design with many parts in a relatively small, cramped  area, so any time problems are found with parts that have already been  installed, removing, repairing or replacing them often requires that other  parts be removed. Furthermore, as some of the tubing sections had  already been installed, resolving this issue, including inspecting, shipping,  and cleaning the tubing, affected the overall program schedule.

In addition, NASA determined in 2017 that Lockheed Martin would not  meet the delivery date for the crew module—even if the European  Service Module were on schedule—when numerous problems including  design issues, damage during testing, and manufacturing process  changes resulted in major schedule impacts to the program. Lockheed  Martin also had a number of issues with subcontractor-supplied avionics  system components failing during testing that have required time to  address. NASA has highlighted concerns over Lockheed Martin’s ability  to manage subcontractors in award fee evaluation periods from 2016 to  2018, and the resulting significant cost, schedule, and technical risk  impacts to the program. In an attempt to resolve these issues and to  improve subcontractor oversight moving forward, Lockheed Martin  officials told us that they have placed staff in the subcontractor facilities.

Because of these cost increases and delays, the agency plans to  renegotiate the Boeing contract for SLS. NASA officials stated that  Boeing expects its costs to exceed the cost-reimbursement contract’s not- to-exceed estimated total cost, which will lead to contract renegotiation.  Consequently, the contractor has been executing work under an  undefinitized contract action since September 2018. Contract actions  such as these authorize contractors to begin work before reaching a final  agreement with the government on contract terms and conditions.

Orion program officials stated that NASA is modifying the cost and period  of performance aspects of its contract with Lockheed Martin for Orion  development and negotiating a new contract with Lockheed Martin for  Orion operations and production. Officials told us the following:

NASA is modifying the Orion development contract with Lockheed  Martin because the contractor will exceed the cost reimbursement  contract’s not-to-exceed estimated total cost. Orion program officials  indicated that poor performance on the part of the contractor resulted  in the contractor exceeding the costs allowed under the contract  without completing the full scope of work. Consequently, NASA is  modifying the contract to allow increased costs. Orion officials  indicated that since the cost growth is contractor caused, the  contractor will not have the ability to earn any fees on this increased  cost.

NASA is also modifying the Orion development contract to extend the  contract period of performance. The current contract’s period of  performance ends in December 2020, which is earlier than NASA’s  planned EM-2 launch date of June 2022. Orion program officials  stated that this extension is largely driven by delays in receipt of the  European Service Module.

According to officials, NASA is negotiating the terms of the Orion  production and operations contract with Lockheed Martin. This  contract is expected to support future production of the Orion  spacecraft from Exploration Mission-3 potentially through 2029. In  addition to production, this effort will include sustaining engineering  and flight operations support, with limited development to allow  mission kits to be built to specifications as mission objective are  defined. Orion program officials indicated that NASA plans to  eventually transition the contract to a fixed-price type contract for  production, but that the development of mission kits will remain under  a cost-reimbursement type contract with some type of incentive fee.

In November 2018, senior leaders within the ESD organization told us  that it was not clear whether NASA would renegotiate how incentive fees  are distributed among milestone incentive fee, or cost incentive fee, and  award fee as part of the upcoming Boeing contract renegotiations. NASA,  however, has made these types of changes in the past. For instance, the  Orion program redistributed fees in 2014 to include an incentive fee  component when the contract transitioned from the Constellation program  to the Orion program.

The Federal Acquisition Regulation and NASA contracting guidance  indicate that award fee is appropriate when the work to be performed is  such that it is neither feasible nor effective to devise predetermined  objective incentive targets applicable to cost, schedule, and technical  performance. However, now that the SLS and Orion programs are further  into the acquisition life cycle, the programs are at the point in  development wherein it may be possible to determine more objective  targets for cost, schedule, and technical performance, especially for the  first mission. Further, a principle of federal internal controls is that  management should design control activities to achieve objectives and  respond to risks. This includes management conducting reviews to  compare actual performance to planned or expected results, and taking  corrective actions to achieve objectives. Without reevaluating its strategy  for incentivizing contractors, NASA will miss an opportunity to consider  whether changes to the incentive structure could better achieve expected  results, such as motivating the contractor to meet upcoming milestone  events within cost and schedule targets.

NASA’s SLS, Orion, and EGS programs are a multi-billion dollar effort to  transport humans beyond low-Earth orbit, but the agency has been  unable to achieve agreed-to cost and schedule performance. NASA  acknowledges that future delays to the June 2020 launch date are likely,  but the agency’s approach in estimating cost growth for the SLS and  Orion programs is misleading. And it does not provide decision makers,  including the Administrator, complete cost data with which to assess  whether Congress needs to be notified of a cost increase, pursuant to  law. By not using a similar set of assumptions regarding what costs are  included in the SLS baseline and updated SLS cost estimates, NASA is  underreporting the magnitude of the program’s cost growth. Similarly,  NASA is underreporting the Orion program’s cost performance by  measuring cost growth to an earlier-than-agreed-to schedule date. As a  result, Congress and the public continue to accept further delays to the  launch of the first mission without a clear understanding of the costs  associated with those delays.

Further, NASA is now turning its attention to new projects to support  future missions, including building a second Mobile Launcher. Ensuring  design stability before construction start would better position NASA to  improve its acquisition outcomes for this next Mobile Launcher.

Finally, contractor performance to date has not produced desirable  program cost and schedule outcomes. Ongoing and planned contract  negotiations present an opportunity to restructure the government’s  approach to incentives. Such steps may better position the agency to  obtain better outcomes going forward.

We are making the following 4 recommendations to NASA:  We recommend the NASA Administrator ensure that the NASA Associate  Administrator for Human Exploration and Operations direct the SLS  program to calculate its development cost growth using a baseline that is  appropriately adjusted for scope and costs NASA has determined are not  associated with the first flight, and determine if the development cost  growth has increased by 30 percent or more. (Recommendation 1)

We recommend the NASA Administrator ensure that the NASA Associate  Administrator for Human Exploration and Operations direct the Orion  program to update its cost estimate to reflect its committed EM-2 baseline  date of April 2023. (Recommendation 2)

We recommend the NASA Administrator ensure that the NASA Associate  Administrator for Human Exploration and Operations direct the EGS  program to demonstrate design maturity by completing 3D product  modeling of the basic and functional design of the second Mobile  Launcher prior to construction start. (Recommendation 3)

We recommend the NASA Administrator ensure that the NASA Associate  Administrator for Human Exploration and Operations direct the SLS and  Orion programs to reevaluate their strategies for incentivizing contractors  and determine whether they could more effectively incentivize contractors  to achieve the outcomes intended as part of ongoing and planned  contract negotiations. (Recommendation 4)

NASA provided written comments on a draft of this report. These  comments, and our assessment of them, are included in appendix II.  NASA also provided technical comments, which were incorporated as  appropriate.

In responding to a draft of this report, NASA concurred with three  recommendations and partially concurred with a fourth recommendation,  and identified actions that they plan to take.

NASA partially concurred with our recommendation to direct the Orion  program to update its cost estimate to reflect its committed EM-2 baseline  date of April 2023. In its response, NASA stated providing the estimate to  the forecasted launch date—September 2022—rather than to the  committed baseline date of April 2023 is the most appropriate approach.  Further, NASA stated that any additional slips to the program involve  considerable uncertainty associated with “unknown-unknowns” which are,  by their very definition, impossible to predict or forecast and that  attempting to forecast these at this point is neither practical nor useful to  help manage the program. If the schedule projections go beyond  September 2022, NASA stated that the Orion program will follow standard  Agency processes and update its cost estimate to reflect the updated  schedule projections.

NASA established Orion’s EM-2 launch date of April 2023 as part of the  agency’s program confirmation process in 2015. According to federal law,  NASA is required to track and report progress relative to the cost and  schedule baselines established at the program’s confirmation review.  While programs often pursue goals trying to beat these dates and/or cost  estimates, the primary purpose of a cost and schedule baseline is to  provide a consistent basis for measuring program progress over time. By  developing cost estimates only to the program’s goals and not relative to  the established baseline, the Orion program is not providing the Agency  or the Congress the means of measuring progress relative to the  baseline. We agree that it is difficult to forecast the potential impacts of  unexpected problems. NASA guidance, however, provides instructions to  programs on the percentage/relative level of cost reserves that should be  maintained to deal with potential unknown-unknowns that are likely to  come up late in development. We continue to believe that NASA should  fully implement this recommendation.

We are sending copies of this report to the NASA Administrator and  interested congressional committees. In addition, the report is available at  no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix III.

To assess the performance of the human space exploration programs,  including any technical challenges, relative to their cost and schedule  commitments, we obtained and analyzed cost and schedule estimates for  the Space Launch System (SLS), Orion Multi-Purpose Crew Vehicle  (Orion), and Exploration Ground Systems (EGS) programs through  November 2018. We then compared these estimates against program  baselines to determine cost growth and schedule delays. We also  interviewed SLS program officials and reviewed cost data to determine  how the program phases costs for future flights outside the current  baseline. We then analyzed the SLS program’s current cost estimate to  determine how the scope of the current estimate had changed relative to  the scope of the SLS baseline cost estimate. Moreover, we obtained and  reviewed quarterly reports and the programs’ risk registers, which list the  top program risks and their potential cost and schedule impacts, including  mitigation efforts to-date. We then discussed risks with program officials.  We also compared program schedules across three points in time— schedules from when NASA first established baselines for each program,  schedules established for each program following the replan in December  2017, and schedules as of November 2018—to assess whether program  components and software were progressing as expected  Furthermore, for the EGS program, we reviewed program-level lessons  learned regarding the acquisition of the Mobile Launcher against  acquisition best practices to determine the extent to which the program  plans to incorporate these best practices as part of its acquisition  planning for the second Mobile Launcher.

To determine the extent to which NASA’s use of contract award fees are  achieving desired outcomes, we analyzed contract modifications, award  fee plans, and fee determination records for the Orion crew spacecraft  and SLS stages—or stages—contracts. We selected these contracts  because they represent the largest development efforts for each program.  We analyzed contract documentation to determine the amount of award  fee available on these contracts compared to other incentives, such as  milestone incentives, and calculated fees paid to date. Specifically, for  award fee on both contracts, we reviewed fee determination records for  evaluation periods after the SLS program’s confirmation review in 2014  and the Orion program’s confirmation review in 2015 to determine fees  paid, numeric and descriptive ratings awarded for each period and  contractor strengths and weaknesses identified by the program.  Moreover, we reviewed award fee documentation to identify broader  program challenges and compared fee determination results to overall  program outcomes since program confirmation. For the Orion contract,  the scope of our incentive fee analysis included the full scope of incentive  fees available for developing and manufacturing the Orion spacecraft  from the beginning of the contract. For the SLS contract the scope of our  incentive fee analysis included the incentive fees available for 1) contract  line item number 9 of the contract which includes the full scope of stages  work supporting SLS’s EM-1 effort, and 2) contract line item number 12  indefinite-delivery, indefinite-quantity support task activities for contract  line item number 9.

We performed our work at Johnson Space Center in Houston, Texas; the  Boeing Company in Huntsville, Alabama; Marshall Space Flight Center in  Huntsville, Alabama; Kennedy Space Center in Kennedy Space Center,  Florida; Lockheed Martin Space Systems Company in Houston, Texas;  and NASA headquarters in Washington, DC.

We based our assessment on data collected prior to the federal  government shutdown that occurred in December 2018 and January 2019  due to a lapse in appropriations for fiscal year 2019. This assessment  does not reflect the effect, if any, of the shutdown on the programs’ costs  and schedules or a March 2019 announcement that NASA is studying  how to accelerate the SLS schedule. We assessed the reliability of  program data we used to support this engagement using GAO reliability  standards as appropriate, including reviewing related documentation,  interviewing knowledgeable agency officials, and performing selected  testing of data. We determined the data was sufficiently reliable for the  purposes of this engagement.

We conducted this performance audit from March 2018 to June 2019 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Comments from the National Aeronautics and Space Administration

1.  This report acknowledges the complexity of NASA’s deep space  exploration systems. The introduction section of this report  acknowledges that NASA is developing systems planned to  transport humans beyond low-Earth orbit, including the Moon and  eventually Mars, and that each of these programs represents a  large, complex technical and programmatic endeavor. The  introduction also notes that these programs are in the integration  and test phase of development, which our prior work has shown  often reveals unforeseen challenges leading to cost growth and  schedule delays.   2.  Senior NASA officials told us that the revised EM-1 launch date of  December 2019 is unachievable and the June 2020 launch date  (which takes into account schedule reserves) is unlikely. These  officials then estimated that there are 6 to 12 months of schedule  risk associated with the June 2020 date. It would be misleading for  us to continue to report the June 2020 launch date when we were  told there was substantive risk to that date. Without a new  approved schedule, Figure 3, Figure 5, and Figure 8 all present a  notional launch window including the acknowledged schedule  risks. We then used the information NASA provided us to report  that the first launch may occur as late as June 2021, if all risks are  realized.

Further, this substantial delay to the first mission was  acknowledged by senior officials less than one year after NASA  announced up to a 19 month delay. We maintain that continued  underperformance contributed to these additional schedule delays  and associated cost increases. For example, for SLS, NASA  discovered during installation that fuel lines used in the engine  section were contaminated with residue and other debris.  According to a program official, Boeing had not verified the  processes that its vendors were using to clean the fuel lines,  resulting in about 2 months’ delay to resolve residue and debris  issues. For the Orion program, NASA determined in 2017 that  Lockheed Martin would not meet the delivery date for the crew  module—even if the European Service Module were on  schedule—when numerous problems including design issues,  damage during testing, and manufacturing process changes  resulted in major schedule impacts to the program.

As a result, we also maintain that these delays and cost growth  reinforce concerns over the management of the programs. In  addition to the underperformance, NASA’s management decisions  on how to report cost growth is not fully transparent and, in  particular, obscures the difficulties the SLS program has faced  controlling costs.  3.  We agree that that these are long-term, “multi-decadal” programs  and that content is subject to change. As a result, we maintain that  arbitrarily focusing on a single mission and not looking at long- term costs may have negative impacts to this human spaceflight  system. We previously reported in May 2014, that NASA does not  have a cost and schedule baseline for SLS beyond the first flight.  As a result, NASA cannot monitor or track costs shifted beyond  EM-1 against a baseline. We recommended that NASA establish  cost and schedule baselines that address the life cycle of each  SLS increment, as well as for any evolved Orion or ground  systems capability. NASA partially concurred with the  recommendation, but has not taken any action to date. Until action  is taken to do so, as noted above, NASA’s decision to shift some  SLS costs to future missions while not adjusting the baseline  downward not only underestimates cost growth for the first  mission, but also results in there being no mechanism to track  these costs that NASA shifted to future missions.  4.  Through the course of this review, NASA was transparent in its  discussions with us of how it calculated costs for each of the  programs. The findings of this report are not meant to convey that  NASA is withholding information, but rather, that decisions NASA  has made about how to calculate costs do not provide sufficient  transparency into cost growth or cost estimates. Further, we have  previously reported that without transparency into costs for future  flights, NASA does not have the data to assess long-term  affordability and Congress cannot make informed budgetary  decisions.

Cristina T. Chaplain, (202) 512-4841 or chaplainc@gao.gov.

In addition to the contact named above, Molly Traci, Assistant Director;  Andrea Bivens; Sylvia Schatz; Ryan Stott; Tanya Waller; John Warren;  Alyssa Weir; and Robin Wilson made significant contributions to this  report.

NASA is developing 3 systems to put astronauts into space—the Orion crew vehicle, Space Launch System, and Exploration Ground Systems.

In the past we’ve reported on concerns over the way NASA is managing these large and complex efforts—such as working to overly optimistic schedules.

NASA is unlikely to meet its recently revised launch date for the first test flight. In addition, its reporting of cost growth for SLS and Orion is distorted. We recommended that NASA adopt more transparent cost reporting practices.

NASA's acquisition management has been on our High Risk List since 1990.