F-35 Joint Strike Fighter: Actions Needed to Address Manufacturing and Modernization Risks

The acquisition cost for the F-35 program increased substantially in 2019, partially due to the program's addition of estimated costs for modernization of hardware and software systems, referred to as its Block 4 efforts.

This is the fifth report under the provision that Congress included in statute for GAO to review the F-35 program annually until the program reaches full-rate production. This is also the first report under another provision in statute to review the program's production and Block 4 progress annually through 2024. Among other objectives, this report assesses (1) the program's production performance and (2) the program's modernization cost estimate and development progress. GAO reviewed Department of Defense (DOD) and contractor documentation and interviewed DOD officials and contractor representatives.

The F-35 program is at risk of missing its test schedule and not meeting manufacturing leading practices. In 2019, the F-35 program conducted much of its planned operational testing but extended the schedule by 9 months, which delays the program's full-rate production decision to between September 2020 and March 2021. Over that time, the program will continue to deliver aircraft.

In addition, while the F-35 program has increased the production rate and negotiated lower aircraft prices, it is not meeting manufacturing leading practices identified by GAO. Specifically, only about 3,000 of the over 10,000 airframe contractor's manufacturing key processes meet predefined design standards for ensuring product quality. Also, the fielded aircraft, over 500 so far, do not meet the program's reliability and maintainability goals. Although the contractor is changing manufacturing processes to address problems and improve efficiency, more remains to be done. Unless the program office evaluates the risks of not meeting these leading practices, the military services and international partners are at risk of not receiving the quality aircraft they purchased.

In addition, the July 2019 suspension of Turkey from the F-35 program—due to security concerns after its acquisition of Russian defense equipment—is likely to compound production risks. The program has identified new sources for 1,005 parts produced by Turkish suppliers, but the program is assessing the effect of 15 key parts not currently being produced at the needed production rate.

In 2019, estimated development costs to modernize the F-35's hardware and software systems—known as Block 4—increased by over $1.5 billion. The cost increase puts estimated Block 4 development costs at $12.1 billion. However, the cost estimate did not fully adhere to leading practices, such as including all life cycle costs. In addition, while development will continue through 2026, reports on Block 4 that the program submits to Congress are slated to end in 2023. Without continued Block 4 reporting through the development phase, Congress will lack important oversight information.

Congress should consider extending DOD's reporting requirement for Block 4 modernization beyond 2023. GAO is also making five recommendations to DOD. While DOD did not concur with two of these recommendations—including to evaluate production risks and update its Block 4 cost estimate with a program-level plan, it identified actions that, if implemented, will meet the intent of these recommendations. DOD concurred with GAO's three other recommendations.

Also known as the Joint Strike Fighter, the F-35 Lightning II is the  Department of Defense’s (DOD) most expensive weapon acquisition  program in U.S. military history. DOD is now in its 19th year of developing  this family of fifth-generation strike fighter aircraft for the United States Air  Force, Marine Corps, and Navy, as well as seven international partners.  The F-35’s key capabilities include low-observable, or stealth, technology  combined with advanced sensors and computer networking capabilities.  The F-35’s acquisition cost increased by over $22 billion in 2019, in part  due to the addition of estimated costs for its modernization efforts, or  Block 4. The total acquisition costs for the F-35 exceed $428 billion and  include the procurement of 2,470 U.S. aircraft through fiscal year 2044.

To date, the program has delivered almost 500 aircraft to the warfighter  even though operational testing—which will determine if the aircraft is  operationally effective and suitable—is ongoing. In October 2019, the  program delayed its full-rate production decision, a review that authorizes  entry into the production and deployment phase, to sometime between  September 2020 and March 2021 so it could complete this testing.  However, the program still faces risks ahead of that decision. We have  reported on these and other program risks in the past and made  recommendations for improvement. DOD has taken action to address  some, but not all, of our recommendations. For a list of our  recommendations to the F-35 acquisition program and a summary of  DOD’s actions in response, see appendix I. In addition, a list of related  GAO products is included at the end of the report.

This report fulfills two mandates. First, the National Defense Authorization  Act (NDAA) for Fiscal Year 2015 included a provision for us to review the  F-35 program annually until the program reaches full-rate production.  This is our fifth report under that provision. Second, the NDAA for Fiscal  Year 2020 included a provision for us to review the program’s production  and Block 4 progress annually through 2024. In this report, we (1)  provide information on the program’s progress toward completing  operational testing and resolving deficiencies found in testing; (2) assess  the program’s production performance and manufacturing efficiency  initiatives; and (3) assess the program’s modernization cost estimate and  progress with Block 4 development efforts.

For all objectives, we interviewed DOD officials and contractor  representatives regarding the program’s activities, progress, and plans.

To provide information on what progress the program has made in  operational testing and resolving deficiencies, we reviewed test event  progress and schedules, program briefings, and internal DOD  briefings. We analyzed program documentation and updates on  resolved and newly identified deficiencies. We also interviewed DOD  officials and contractor representatives regarding the deficiencies and  resolution time frames.

To assess the program’s production performance and manufacturing  efficiency initiatives, we collected and analyzed production  performance data from the program office, the prime airframe  contractor, and the prime engine contractor. We analyzed the extent  to which the program has met GAO’s manufacturing leading practices,  which programs should follow prior to making a full-rate production  decision. We also reviewed the Defense Contract Management  Agency’s reports on F-35 production.

To assess the program’s Block 4 cost estimate and development  progress, we reviewed program office planning and implementation  documents as well as documentation of the cost estimate, such as  cost models and analyses. We assessed the cost estimating  methodologies, assumptions, and results against leading practices for  developing a comprehensive, accurate, well-documented, and  credible cost estimate, identified in GAO’s Cost Estimating and  Assessment Guide. Appendix II contains more information about how  we applied our cost estimating leading practices.

We determined that all the data we used were sufficiently reliable for the  purposes of our reporting objectives. For example, we collected and  analyzed the program’s production data for all production lots and  corroborated these metrics by interviewing contractor representatives and  DOD officials in oversight offices such as the Defense Contract  Management Agency. In addition, we reviewed official program  documentation on the Block 4 efforts and corroborated it with officials  across DOD involved in the effort, such as the F-35 Joint Program Office  cost estimating team and DOD’s Cost Assessment and Program  Evaluation office, regarding Block 4. Appendix III contains a detailed  description of our scope and methodology.

We conducted this performance audit from June 2019 to May 2020 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

DOD started the F-35 program in 2001 to develop a fifth-generation  fighter aircraft intended to replace a range of aging aircraft in the U.S.  military services’ inventories and to provide enhanced capabilities to  warfighters that capitalized on technological innovations. Among other  capabilities, the program designed the F-35 aircraft to be difficult to  observe using radar and include sensors that can provide insights into  potential targets and other warfighting information. The program is  producing and delivering three variants of the F-35 aircraft:  the F-35A conventional takeoff and landing variant for the Air Force,  the F-35B short takeoff and vertical landing variant for the Marine  Corps, and  the F-35C carrier-suitable variant for the Marine Corps and the Navy.

The characteristics of the services’ variants are similar, but each variant  also has unique operating requirements. For example, the Marine Corps  requires that the F-35B be capable of operating from aircraft carriers,  amphibious ships, and main and austere operating bases alike, requiring  the capability to conduct short takeoffs and vertical landings. Figure 1  shows an F-35B exercising this capability.

While DOD plans to purchase 2,470 aircraft for the U.S. services, the F- 35 program is acquiring more than just aircraft. The complete F-35 air  system has eight elements, including training and maintenance systems.  Figure 2 shows the eight elements that make up the entire F-35 Air  System and how they each support the aircraft.

For example, the program intends for the Automated Logistics Information  System (ALIS) to provide the necessary logistics tools to F-35 program  participants as they operate and sustain the F-35 aircraft. To do this, ALIS  consists of multiple software applications designed to support different  squadron activities, such as supply chain management, maintenance,  training management, and mission planning. For the F-35 aircraft to have  full capability, each element of the air system has to be developed and  fielded in sync with the aircraft. However, we found in March 2020 that  problems with ALIS still pose significant challenges to day-to-day F-35  operations. According to DOD, it plans to replace ALIS with a new  system named the Operational Data Integrated Network (ODIN).  Furthermore, DOD reports that it is currently developing a strategy for  ODIN, which will include key tasks, milestones and schedule, risks and  opportunities, governance structure, and cost estimates. We concluded  that, as DOD proceeds with replacing ALIS with ODIN, it will be important  for the department to carefully consider and assess the key technical and  programmatic uncertainties that we reported in March 2020. These  include how much of ALIS will be incorporated in ODIN and the extent to  which DOD has access to the data it needs to play a more active role in  the management of the system. These issues are complex, and will  require significant direction and leadership to resolve.

Further, we reported in March 2020 that the F-35 program office was not  able to provide us with historic costs showing how much the department  had spent on ALIS over the years. Also, because DOD had not answered  key questions about the future of the system, such as the extent to which  the re-design will incorporate current ALIS software, DOD has not been  able to develop accurate cost estimates for the ALIS re-design. We  recommended that DOD develop and implement a strategy for the re- design of ALIS. The strategy should be detailed enough to clearly  identify and assess the goals, key risks or uncertainties, and costs of re- designing the system. DOD concurred with the recommendation.

Status of F-35 Program Development and Costs as of April 2019

DOD began development of the F-35 aircraft in 2001 without adequate  knowledge of its critical technologies or a solid design, as we reported in  March 2005. DOD’s acquisition strategy also called for high levels of  concurrency between development and production—building aircraft  while continuing to refine the designs of key components—which runs  counter to GAO’s leading practices for major defense acquisition  programs. In our prior work, we identified the F-35 program’s lack of  adequate knowledge and high levels of concurrency as the major drivers  of the program’s eventual significant cost and schedule growth, among  other performance shortfalls.

Since 2001, the program has been rebaselined with new cost and  schedule estimates three times. DOD initiated the most recent  restructuring in 2010 when the program’s cost estimates for each aircraft  exceeded critical thresholds established by statute—a condition known as  a Nunn-McCurdy breach. DOD then established a new acquisition  program baseline that increased the program’s cost estimates by $162.7  billion and extended delivery schedules 5-6 years into the future. This last  revision is the current program baseline, reflecting the cost and schedule  estimates to deliver the aircraft and systems and to meet the original  program requirements.

From 2018 to 2019, the total cost estimate of the F-35 acquisition  program increased by $22 billion, from $406 billion to over $428 billion.  This increase was partially due to the addition of the estimated Block 4  modernization costs. Block 4 includes efforts to enhance and add  capabilities—beyond the F-35 baseline program—through hardware and  software upgrades. In April 2019, the F-35 program estimated that Block  4 development and procurement costs would add $13.9 billion to the  program’s total baseline cost. Beyond this Block 4 increase, the F-35  program baseline costs also increased by $8 billion over the program’s  2018 estimate.

Table 1 outlines the program’s baseline costs, the Block 4 modernization  costs, and the sum total of the baseline and Block 4 cost estimates since  2001.

In addition to the acquisition costs above, the program estimates that the  sustainment costs to operate and maintain the F-35 fleet for its planned  66-year life cycle are $1.2 trillion, bringing the total cost of the F-35  program to over $1.6 trillion.

Status of Testing, Production, and Reliability and Maintainability as of December 2019

The F-35 program office, in coordination with the Director of Operational  Test and Evaluation (DOT&E), received approval to conduct some  preliminary operational testing in January 2018. This included weapons,  cybersecurity, and cold weather testing, among other things. The  program’s formal operational testing (conducted by DOT&E) started in  December 2018 and was ongoing in 2019. The purpose of operational  testing is to assess the effectiveness, suitability, survivability, lethality,  and mission capability of the F-35, including the information systems and  the air vehicle, in an operationally representative environment.

Operational testing includes cybersecurity assessments, some of which  the program has conducted. The program plans for the remaining testing  to take place through at least September 2020, while the program  continues to produce and deliver aircraft.

Through 2019, F-35 program test officials had identified over 3,200  deficiencies. Deficiencies represent specific instances where the weapon  system either does not meet requirements or where the safety, suitability,  or effectiveness of the weapon system could be affected. The test  officials categorize deficiencies according to their potential impact on the  aircraft’s performance.

Category 1 deficiencies are critical and could jeopardize safety,  security, or another requirement.

Category 2 deficiencies are those that could impede or constrain  successful mission accomplishment.

In June 2018, we recommended that the program resolve all critical  deficiencies before making a full-rate production decision, in part, to  reduce the potential for additional concurrency costs stemming from  continuing to produce aircraft while testing was ongoing. DOD  concurred with our recommendation and stated that it would resolve  critical deficiencies before full-rate production, currently planned to occur  between September 2020 and March 2021.

Production of the aircraft began one year after testing started in 2007,  while development was in its early stages. Due to the concurrency of  testing and production, according to an F-35 program official, as many as  550 aircraft delivered through 2020 will need retrofits to fix deficiencies  and design issues found during testing. The program refers to the cost  of these fixes as its concurrency cost, which the program estimates at  $1.4 billion; this estimate did not change with the program’s last update in  2019. Until operational testing is complete, there is a risk that the  program may identify additional deficiencies. As a result, as we have  previously reported, the concurrency costs of retrofitting delivered aircraft  could increase.

In our June 2018 report, we found that the program was not on track to  meet its reliability and maintainability (R&M) performance targets. R&M  targets indicate how much time the aircraft will be in maintenance rather  than operations. We concluded that the program was missing a prime  opportunity to infuse affordability into the aircraft’s future with better R&M  performance. As a result, we recommended that the F-35 program office  identify what steps it needed to take to ensure the F-35 meets R&M  requirements and update the R&M Improvement Program with these  steps. DOD concurred with the recommendations, noting that the F-35  program office would update the R&M Improvement Program with the  steps needed to ensure continued progress towards its goals. In April  2019, we found that F-35 R&M performance had shown some small  improvements but that the program could take more actions to meet the  R&M targets. We made additional recommendations to the Secretary of  Defense, with which DOD concurred and has taken some actions to  implement. Currently, the Office of the Under Secretary of Defense for  Acquisition and Sustainment (OUSD (A&S)) is the acquisition decision  authority for the F-35 program, and would direct the F-35 program office  to take any further actions. In 2019, the program’s R&M performance  generally remained unchanged. However, measurable improvements in  R&M can take time to manifest. For example, fielded aircraft must be  modified and flown for many hours before the program can measure  improvements. For details about the R&M performance, see appendix IV.

Block 4 Modernization’s Development Approach

As we have previously reported, even though operational testing of the  baseline program remains ongoing, the F-35 program office has turned its  attention to Block 4 modernization activities using a different development  approach. DOD refers to this approach as Continuous Capability  Development and Delivery (C2D2). This method is loosely based on the  Agile software development process. With this approach, the program  plans to deliver capabilities to the warfighter faster than it did during the  baseline development program. For example, rather than take years to  develop and deliver all the required capabilities to the warfighter, the  program intends to incrementally develop, test, and deliver small groups  of capabilities every 6 months. In January 2018, to transition from the  baseline development program to its Block 4 activities, the F-35 program  started using the C2D2 approach to develop and test software updates to  address deficiencies identified during testing.

The planned $13.9 billion Block 4 effort exceeds the statutory and  regulatory thresholds for what constitutes a major defense acquisition  program, and Block 4 is more expensive than many of the other major  weapon acquisitions already in DOD’s portfolio. To provide better  oversight into Block 4 activities, in 2016, we recommended that the  Secretary of Defense hold a milestone B review—a critical point in an  acquisition program leading to the engineering and manufacturing  development phase—and manage it as a separate major defense  acquisition program. DOD did not concur with our recommendation, and  it continues to manage Block 4 within the larger F-35 program. We  maintain that DOD should manage the Block 4 activities as a separate  program.

Operational Testing Delays Provide More Time to Address Deficiencies before Full-Rate Production Decision Completion of Operational Testing Delayed by 9 Months

In 2019, the F-35 program conducted a majority of its planned operational  testing but added 9 months to the schedule to complete the remaining  tests. Specifically, as of February 2020, according to test officials, the  program completed 156 flight tests. The program must still conduct four  open-air flight tests, the remaining cybersecurity tests of the air vehicle and mission systems, and 64 simulated flight tests. The 9-month delay  needed to complete testing, however, also provides additional time for the  program to address our June 2018 recommendation that it resolve critical  deficiencies before making its full-rate production decision, currently  planned to occur between September 2020 and March 2021.

Figure 3 shows the test schedule as of 2019, the delay to the schedule  into 2020, and the remaining tests events planned.

The completion of operational testing hinges on three main tasks: (1) the  final four open-air flight tests; (2) cybersecurity testing; and (3) the final  development, integration, verification and validation of its simulator and  64 simulated flight tests.

First, the program expects to complete the four remaining open-air tests  between March and April 2020. To conduct these tests, the program must  finish moving the Radar Signal Emulators—test assets that simulate long- range threat radars—from the Nevada Test and Training Range to the  Point Mugu Sea Range in California. According to test officials, there is  some risk with this move, such as damage to the sensitive test equipment. The test facilities will have to integrate the equipment into the  testing infrastructure at Point Mugu.

Second, while the program has conducted cybersecurity testing on  several aspects of the F-35 aircraft and support systems, three air vehicle  subsystems tests and two enterprise-level ALIS tests remain. The  program expects to complete these by August 2020.The tests completed  to date have identified multiple cybersecurity vulnerabilities. The F-35  program office has taken steps to address some identified vulnerabilities  and is working to address the remainder. Test officials stated that some of  the delays to cybersecurity testing of the aircraft are due to safety  concerns and the risk of losing the use of a test aircraft before testing is  complete.

According to DOD policy, cybersecurity testing should be conducted as  early in the operational test cycle as possible. Leaving this critical  testing to the end of operational testing adds risk to the program because  the program will not know the extent to which the aircraft may have  cybersecurity vulnerabilities until near the expected decision to proceed to  full-rate production. If the program cannot finish these tests by  September 2020, officials stated that DOT&E could require that the  cybersecurity testing be completed in follow-on testing and not hold up  the full-rate production decision. Any additional cybersecurity  vulnerabilities may require more time to develop and implement plans to  address vulnerabilities in aircraft that have already been produced and  those slated for production.

Lastly, the program has not been able to complete the F-35 Joint  Simulation Environment, which we refer to as the aircraft simulator, on  time. The simulator runs the F-35’s mission systems software along with other software models (such as other weapons and modern threat  systems) to provide complex test scenarios that the program cannot  replicate in a real-world environment. We reported in April 2019 that the  simulator’s development was behind schedule and was a risk to  operational testing. Since then, the program has struggled to develop  the complex software and functionality needed to complete the simulator.

The difficulties stem, in part, from the program office’s original plan to  have the contractor, Lockheed Martin, develop the simulator. However, in  August 2017, program office officials decided that the contractor’s  proposal was considered to be too expensive. To mitigate concerns over  the cost of the proposal, the program decided to have the Navy complete  the work. The program originally expected the Navy’s simulator to be  ready for testing in 2017, but it is now 3 years behind schedule. According  to program office officials, the simulator’s development effort has taken  longer than expected to integrate F-35 aircraft and sensor data, in part  because the contractor claimed the data as its own intellectual property.  These issues were resolved by 2019 when the contractor provided the  necessary data. Because of these delays, the program now expects that  the simulator will be ready by August 2020, with the planned simulator  testing expected to take about 3 weeks. According to test officials, there  is increased risk that the completion of the simulator may face additional  delays to correct deficiencies and add needed capabilities, but also stated  that they can complete the tests by August 2020.

Due to these delays to completing operational testing, the program has  delayed its full-rate production decision by at least an additional 9  months. Though the program is working toward September 2020, the  program has acknowledged this decision could be made as late as March  2021. Any additional delays due to challenges with moving the  emulators, completing the simulator, or cybersecurity testing could further  delay the end of operational testing and the program’s decision to enter  into full-rate production. This delay, however, gives the program more  time to complete two key steps consistent with statute and DOD policy.

Complete operational testing, which is intended to demonstrate that  the aircraft are operationally suitable.

Resolve all deficiencies, which should be done prior to full-rate  production, and is discussed below.

Even with these delays, the program plans to have produced and  delivered over 550 aircraft before operational testing is complete, adding  to the risk of finding more deficiencies that will require retrofits—at  additional cost—for the delivered aircraft. Statute and DOD policy states  that the preliminary low-rate production quantities will be set at the  development request for proposal decision point. If, at that time, low-rate  initial production quantities are determined to be above 10 percent of the  total quantity planned, the Secretary of Defense must explain the reasons  for the increase in a report to Congress. When a program reaches the  planned low-rate initial production quantity, and requires to exceed the  quantity, the program may seek approval to produce quantities above that  amount. The F-35 program will have delivered more than 10 percent of  the total planned production quantities—due to the necessity to prevent a  break in production—before operational testing and the full-rate  production decision are complete. As noted above, this approach has  contributed to the $1.4 billion in concurrency costs already incurred by the  program.

The program reports that none of the category 1 deficiencies is a  safety of flight concern and all of them have operational workarounds. In 2019, the  program split the category 1 deficiencies into two groups. Group A are deficiencies that  may cause death, severe injury, severe occupational illness, or major loss or damage to  equipment and has no workaround. The program has none of these deficiencies currently.  Group B are deficiencies that may critically restrict the combat readiness capabilities or  may result in adequate performance but not be able to accomplish the primary or alternate  missions. All of the 9 category 1 deficiencies are in group B.

F-35 Program Made Progress Resolving Deficiencies during 2019, but Many Remain Unresolved

In 2019, the F-35 program resolved nearly 300 of the deficiencies it had  identified in developmental and operational testing, but discovered even  more over the same period. Specifically, 331 new deficiencies were  identified in operational testing during 2019. As of December 2019, the F- 35 program had 870 open deficiencies.

Of the 870 open deficiencies, the program characterizes nine as category  1 and 861 as category 2.

Of the 9 open category 1 deficiencies, the program reports all have  operational workarounds—procedures that avoid encountering the  deficiency. This represents four fewer open category 1 deficiencies than  we reported in April 2019, reflecting the resolution of previously identified  deficiencies and the addition of new ones, some of which were  resolved. For example, the program fielded a software fix to a category  1 deficiency, which showed that the F-35’s cockpit display could falsely  indicate its AIM-9X weapon—an air-to-air missile—selection status as  “selected” though the weapon’s status is not selected. Figure 5 shows the  F-35 firing an AIM-9X missile.

The program office plans to continue to address the open deficiencies,  but officials report that some will not be fully resolved for several years.  Further, some deficiencies may not be resolved ever and some may be  resolved well after the program has completed testing, and after it  expects to have made a full-rate production decision. According to  DOT&E, there are many significant deficiencies the program should  address to ensure the F-35 baseline aircraft configuration is stable prior to  adding all of the new capabilities planned in Block 4. As of December  2019, the program office and the contractor have resolved over 2,300  deficiencies and program office officials stated that they have a process in  place to address the high priority ones.

F-35 Aircraft Prices Decreased but Manufacturing and Supply Chain Risks Remain

In 2019, the program reported continuing to negotiate lower unit prices  across all F-35 aircraft variants and delivered more aircraft on time.  However, officials also reported that the airframe and engine contractors  demonstrated some declines in production performance, such as the  number of labor hours to produce each aircraft, as production rates  increased. We also identified other risk indicators that could affect the  contractors’ future production performance. Specifically, the airframe  contractor’s manufacturing processes do not meet all manufacturing  leading practices that programs should meet before full-rate production.  Additionally, parts shortages increased significantly in 2019 and Turkey’s  suspension from the program will likely further complicate existing supply  chain challenges.

Aircraft Prices Decreased and More Aircraft Were Delivered on Time, but Other Measures of Production Performance Declined

According to the program office, the negotiated prices for all F-35 variants  have generally been decreasing with each production lot and as more  aircraft are being procured in each lot. In April 2019, we reported that the  program set a goal of reducing the negotiated unit price of an F-35A to  less than $80 million by lot 13. According to a program official, in  October 2019, the program finalized the contract action for lots 12-14 that  met this goal. Specifically, with the most recent contract, the program  agreed to purchase 351 F-35As, with unit costs declining to $73 million in  lot 14. Figure 6 shows how the negotiated price for an F-35A has  decreased since production began, as reported by the program office.

According to the program office, it negotiated lower unit prices by working  with the airframe contractor to leverage economic order quantity  purchases and invest in cost reduction initiatives. Economic order  quantities involve the contractor making large purchases of components  that it will use across multiple procurement lots of aircraft to reduce  production costs by achieving economies of scale. The program office  estimates that the economic order quantity purchases for lots 12-14 will  save the program about $225.5 million. In addition, the program office  and prime contractors have continued to invest in various initiatives to  lower production costs. Specifically, the program office spent $320 million  in efforts to improve manufacturing processes that it estimates could  result in up to $10.5 billion in savings over the life of the program. The  airframe contractor told us that it has invested $170 million as of January  2019 to further lower its production costs. The engine contractor also told  us that it spent $33 million to potentially realize over $4 billion of cost  savings.

The airframe contractor—Lockheed Martin—delivered 43 more aircraft in  2019 than in 2018, and as of October 2019, there were 229 aircraft in  various stages of assembly worldwide. The contractor also delivered  more aircraft on time in 2019. According to contractor officials, the  improved rates of on-time delivery are partially a result of the contractor’s  efforts to obtain a performance incentive fee that was added to the lot 11  production contract. The program intended the incentive fee to focus the  contractor on improving its performance in the final assembly phase of  production, which was expected to improve its on-time deliveries. To earn  the incentive fee, contractor representatives told us they took several  steps to improve production rates. For example, because the F-35Cs  were taking longer to produce and all variants had to move through the  same final assembly area, the contractor made a separate final assembly  line for the F-35Cs so work could proceed without delaying the other  variants. This step, according to program office officials, allowed the  contractor to improve on-time deliveries for F-35As and F-35Bs. Figure 7  highlights progress in the contractor’s aircraft deliveries since 2016.

Other production metrics associated with the airframe, however,  demonstrated varied performance over the last two years as production  increased. For example, the average number of hours needed to build an  aircraft decreased slightly for the F-35A but increased for the B and C  variants. Defense Contract Management Agency officials told us the  increase was partly attributable to new personnel. In particular, since  January 2017, the contractor has hired and trained nearly 1,700 new  personnel to accommodate increased production rates—nearly doubling  its workforce. New personnel take time to train and gain experience on  the production line. According to contractor representatives, as these new  employees become more experienced and produce more aircraft, they  expect the metric to improve.

The contractor’s amount of rework needed was also mixed. During the  course of production, the contractor may identify issues with a part or a  process, which, in turn, may lead to scrap, rework, and repair to replace  or fix the issue. Between 2016 and 2017, most F-35 variants realized  improvements in the amount of scrap, rework, and repair needed. In 2018  and 2019, however, only the F-35A continued to show improvements.  Figure 8 shows the average total hours for scrap, rework, and repair for  each variant since 2016.

According to the program office, the increased production rate posed a  challenge, and because the contractor has not built as many F-35Cs, this  has added to the increase in scrap, rework and repair. To improve  performance in this regard, the contractor put teams in place to focus on  addressing the main drivers of scrap, rework, and repair.

Similarly, the engine contractor—Pratt & Whitney—increased its  production rate by roughly 51 percent in 2019. However, engine on-time  delivery performance has continued to decline which officials attribute to  production quality issues and parts delays. Specifically, in 2019, 91  percent of engines delivered were late. In 2019, the airframe contractor  was able to work around the late engine deliveries to deliver the entire  aircraft on time. Figure 9 shows the engine contractor’s on-time and late  deliveries since 2016.

In addition, the average number of quality notifications per engine— production defects indicating a quality issue—has increased by 16  percent in 2019. Figure 10 highlights the engine contractor’s quality  notifications per engine over the last 4 years.

According to the Defense Contract Management Agency’s performance  reports, engine test failures, among other quality issues, have affected  engine deliveries. According to an official from this agency, there have  been 18 engine test failures in 2019, which is eight more than in 2018,  each requiring disassembly and rework. The engine contractor stopped  deliveries due to the test failures, which has slowed engine acceptance  and reduced on-time deliveries. These issues are affecting engines built  at the engine contractor’s production facility in West Palm Beach, Florida,  which opened in 2014. To address this issue, the engine contractor has  developed new tooling for the assembly line and has established a team  to identify characteristics leading to the test failures. Plans are also in  place for additional training for employees.

F-35 Program Has Not Met All Manufacturing Leading Practices, Indicating Risk to Future Production

While F-35 aircraft have been in production since 2007 and have reached  a high level of manufacturing readiness per DOD guidance, the program  is not meeting two of eight manufacturing leading practices GAO has  identified as indicators of a program’s readiness for full-rate production, or  milestone C review. To date, the program is meeting or plans to meet  six leading practices for this milestone:

Demonstrating processes on a pilot production line.

Building and testing production-representative prototypes to  demonstrate product in intended environment.

Collecting statistical process control data.

Conducting an independent cost estimate.

Conducting an independent program assessment.

Conducting major milestone decision review to begin production.

However, we also found that the production processes are not in control  according to the Process Capability Index. This index is a tool to  measure how closely the production steps result in a part or subsystem  that meets predefined standards. According to the leading practices,  meeting these standards provides greater confidence that the contractor  can produce a high quality product consistently, to minimize variation  which results in fewer defects or the need for rework. Additionally, the F- 35 aircraft have not achieved their reliability goals through testing of  production representative prototypes. These two leading practices focus  on gathering sufficient knowledge to determine the relative ease of  manufacturing and whether the product is of high quality and sufficiently  mature to move forward into full-rate production.

Our analysis of contractor data shows that the airframe contractor’s  production processes are in flux. The contractor continues to change  some of its production processes, and in other cases, is not following its  own established processes well, which has led to several quality issues  over the years. For example, in 2018, we reported that the contractor had  halted deliveries of aircraft after the Air Force identified corrosion between  the aircraft’s surface panels and the airframe because the contractor did  not apply a primer when it attached the panels. We reported in 2019  that the program office, the contractor, and the F-35 Program Executive  Officer reached a mutual agreement on the cost to resolve this issue, the  details of which they did not disclose publicly. In November 2019, a  mechanic identified titanium fasteners installed in an area of the aircraft  where the design calls for a fastener stronger than titanium. According to  the program office, the incorrect fasteners were installed on most already- fielded F-35 aircraft. That same month, the contractor started  implementing its corrective action plan. As of March 2020, the F-35  program office had reviewed and approved the contractors’ analysis as  well as its durability and damage reports on the use of these fasteners.  We describe other key F-35 technical risks in appendix V.

Over the years, the airframe contractor has continued to change and  refine production processes, aiming to improve efficiency amidst  concurrent development and production. For example, the airframe  contractor identified a particular process that installs wiring harnesses into  the aircraft wings as a driver of one of its production quality issues. To  address this issue, the prime contractor developed a new tool that helps  the installer route the wires more consistently.

While process changes like these can improve the quality of the product,  they also indicate that the overall production process are not in control  less than a year before the program’s planned full-rate production  decision, or milestone C review. In 2019, according to our analysis, the  total number of key F-35 manufacturing processes identified in the final  assembly phase increased 70 percent, to a total of over 10,000 critical  processes. Furthermore, of these critical processes, only 30 percent are  currently able to produce a product within predefined design standards.  According to manufacturing leading practices, critical processes should  be repeatable, sustainable, and consistent in producing parts within  quality standards. Meeting these practices provides confidence that the  contractor can produce the product within cost, schedule, and quality  targets. Without processes in control, the program could face continued  quality issues that will add to the overall cost of the program. Figure 11  shows the F-35 aircraft in the final assembly phase of production where  some of these processes take place.

Another leading practice that should be met before making a full-rate  production, or milestone C decision, is to demonstrate that a production  representative prototype can meet the program’s R&M goals. The R&M  goals lay out specific quantitative goals aimed at ensuring that an aircraft  will be available for operations as opposed to out of service for  maintenance. We reported in April 2019 that the F-35 aircraft in service  around the world were still not meeting all of their R&M goals and  recommended the program take actions to ensure that the aircraft would  meet those goals. Despite some improvement in 2019, the program is  not meeting half of its R&M goals. Until the program does so, the  warfighter will continue to accept aircraft for delivery that are less reliable  and more costly to maintain than originally planned. For details on the F- 35’s R&M performance, see appendix IV.

The program has not met these two leading manufacturing practices, in  part, due to the changes the airframe contractor made and continues to  make to the production line and the program’s concurrent approach to  acquisition. We have repeatedly found that DOD programs that moved  into full-rate production carrying manufacturing risks experienced billions  of dollars in cost growth in production, and nearly two-thirds reported  increases in average procurement unit costs. With the risks the F-35  program still faces, it may realize additional cost and schedule growth if  these production risks are not evaluated. Despite these risks, the program  has continued to push forward with increased production rates and has  not taken actions to determine the potential impact of not meeting these  leading practices may have on future production and overall life-cycle  costs.

Furthermore, according to a program official, the F-35 program has not  completed a comprehensive assessment of production risks and does not  plan to ahead of its full-rate production decision. However, according to  DOD officials, the F-35 program office and prime contractor convene a  monthly Joint Risk Management Board, which identifies and manages  overall program risk, and has completed an independent technical risk  assessment to support the full-rate production decision, which identified  production risks. Title 10 section 2366c of the U.S. Code requires the  milestone decision authority for a major defense acquisition program to  provide Congress with a report that includes, among other things, a  summary of any manufacturing risks associated with the program;  however, this summary is not required until 15 days after the authority  grants approval for the program to enter the production and deployment  phase. The program currently plans to obtain this approval between  September 2020 and March 2021. In this case, however, the F-35  program has not met all of the manufacturing leading practices that  should be met before the full-rate production decision. Furthermore, the  underlying risks, such as not meeting R&M goals, have persisted for  years and the program has yet to take steps to fully address these risks. If  an evaluation of these risks is not provided ahead of the full-rate  production decision, Congress will not be fully aware of the risks the  program is taking by committing to increased production rates.

F-35 Supply Chain Challenges Continue and May Be Exacerbated by Turkey’s Suspension from the Program

According to program officials, some suppliers for the F-35 struggled to  meet increased production demands in 2019 and, as a result, the  program witnessed increased rates of late deliveries or parts shortages.  In particular, the number of parts delivered late to the airframe contractor,  as well as parts shortages, have grown steadily over the past 2 years.  According to the Defense Contract Management Agency:

Between August 2017 and July 2019, the number of parts delivered  late increased from under 2,000 to more than 10,000.

Between July 2018 and July 2019, the parts shortages per month  increased from 875 to over 8,000. According to contractor  representatives, roughly 60 percent of parts shortages are attributable  to 20 suppliers.

To mitigate late deliveries and parts shortages—and deliver more aircraft  on time—the airframe contractor has utilized methods such as  reconfiguring the assembly line and moving planned work between  different stations along the assembly line. According to the program  office, such steps can cause production to be less efficient, which, in turn,  can increase the number of labor hours necessary to build each aircraft.

Airframe contractor representatives and a program office official cited  measures they are taking to improve supplier performance in light of the  upcoming full-rate production decision. For example, the contractor  instituted action plans to help problematic suppliers, sent task teams to  struggling suppliers to help resolve issues, and, in some cases, is seeking  alternative sources. Additionally, the program office has established joint  meetings with the prime contractor to monitor progress on a weekly basis  and holds a semiannual review to achieve executive-level coordination.  While prime contractor representatives told us that they have been  actively managing underperforming suppliers for several years, some of  their efforts are new and will need time before results materialize.

These supply chain risks may compound as the program continues to  produce, deliver, operate, and maintain more aircraft each year. For  example, in April 2019, we found that fielded, operational F-35 aircraft  were not meeting warfighter requirements, largely due to spare parts  shortages and difficulty in managing and moving parts around the world.

We recommended that the program assess what actions it should take to  meet warfighter requirements, which could include adjusting the amount  of spare parts acquired. DOD concurred and is working toward  addressing the recommendation to identify warfighter gaps with regard to  the supply chain. However, with the aircraft in production also facing  significant shortages, this problem could get worse as the program  prepares to further increase the production rate from 141 aircraft in 2019  to 169 in 2022.

We found that Turkey’s recent suspension from the F-35 program is likely  to compound these existing supply chain issues. In July 2019, Turkey  was suspended from the F-35 program. In particular, the Under Secretary  of Defense for Acquisition and Sustainment directed that the F-35  program establish alternative sources and to stop placing orders from  Turkish suppliers after March 2020. According to an official with that  office, Turkish suppliers will provide parts through the end of lot 14  deliveries (scheduled to take place through 2022), in part, to avoid  disruptions to aircraft deliveries and additional cost growth from standing  up new suppliers. The F-35 program office identified that Turkish  companies supplied 1,005 parts for the F-35 airframe and engine and  some of these parts have been provided by only one supplier. As of  December 2019, the program has identified new suppliers for all of these  parts, but it still needs to bring roughly 15 parts currently produced in  Turkey up to the current production rate.

During our review, the program reported that production through lot 14  should not be adversely affected if it continues to accept parts from  Turkey until lot 14 aircraft are delivered, but risk remains with the  transition to alternate sources. However, lots 12-14 still face some risk  receiving parts from Turkey. According to program officials, some of these  new parts suppliers will not be producing at the rate required until next  year, as roughly 10 percent are new to the F-35 program. Airframe  contractor representatives stated it would take over a year to stand up  these new suppliers, with lead times dependent on several factors, such  as part complexity, quantity, and the supplier’s production maturity. In addition, these new suppliers are required to go through qualification and  testing to ensure the design integrity for their parts. According to an  official with the Under Secretary of Defense for Acquisition and  Sustainment, by accepting parts from Turkish suppliers through lot 14, the  program will have additional time to ensure new suppliers can meet  demands for parts. Additionally, the program reported that it intends to  utilize alternative sources for parts currently made in Turkey for aircraft  delivered under lots 13 and 14 contracts. Furthermore, according to a  program office official, it is also not clear how the prices for parts that will  be obtained from new suppliers after Lot 14 will compare with the prices  under the contracts with the suppliers from Turkey, but the official noted  that alternative sources could be more costly.

Block 4 Reporting Requirement Expires Before Completion Date, and Cost Estimate Does Not Fully Reflect Leading Practices

In its May 2019 report to Congress, DOD outlined its plans for Block 4  with a development cost estimate of $10.6 billion for activities through  fiscal year 2024. Since the 2019 report, we found the program office has  increased its estimate by about 14 percent, to $12.1 billion, primarily due  to schedule delays. The program now expects to extend the delivery of  Block 4 capabilities by 2 additional years, through 2026. In the meantime,  DOD’s Block 4 annual reporting requirement to Congress is scheduled to  end in 2023, 3 years before development is complete. Additionally, most  of the capabilities the F-35 program planned to deliver in 2019 were  delayed. Furthermore, we found that the program’s cost estimate used to  support its report to Congress does not fully meet cost estimating leading  practices.

Block 4 Modernization Will Not Be Completed before Reporting Requirements Expire, in Light of Schedule Delays

The Block 4 development cost and schedule have grown considerably  since DOD’s last report to Congress. In 2016, GAO recommended that  DOD manage Block 4 as a major defense acquisition program with its  own reporting requirements, separate from the original F-35 development  program. DOD did not concur with our recommendation, citing the F-35  as DOD’s most closely managed system and its existing F-35 program  oversight. The NDAA for Fiscal Year 2017 required DOD to report  annually on elements of a Block 4 baseline, such as development and  retrofit cost estimates, beginning no later than one year after the award of  the development contract for follow on modernization, until March 31,  2023. At that time, we reported that DOD had requested funding for the  development and delivery of Block 4 through the end of 2022. However,  over the last year, the program has revised its Block 4 schedule and now  expects to field Block 4 capabilities into fiscal year 2026. As a result,  there is no requirement for DOD to report on Block 4 progress for at least  3 years even though those efforts will be ongoing.

In its May 2019 Block 4 report to Congress, DOD reported that the total  cost to develop 66 Block 4 capabilities—both hardware and software— would be $10.6 billion for activities planned from fiscal years 2018 to  2024. The report also included the F-35 program office estimate of an  additional $6.4 billion in fiscal year 2018 through 2024 funding to retrofit  aircraft from the baseline F-35 configuration to a full Block 4 configuration.  The F-35 program based the costs in this report on its Block 4  development cost estimate from July 2018. However, we found that  reported Block 4 costs did not include all Block 4 costs. In particular, the  report did not include Block 4 costs the program incurred prior to 2018 or  costs that the effort will incur after 2024. Because the F-35 program office  is not managing the Block 4 effort as a separate program, it has chosen  to exclude the past and future costs in the Block 4 cost estimate it  reported to Congress. Instead, the program reported on Block 4 costs for  the future years defense program—which is DOD’s projected spending  for the current budget year and the next four years. By excluding any  costs prior to 2018 and those that would be incurred after 2024, the  program did not report on the total costs of Block 4.

In May 2019, the program also updated its Block 4 development cost  estimate, increasing both the time and cost to complete the work, but this  updated estimate was not included in its May 2019 report to Congress.  The updated cost estimate reflects that the program office will be fielding  Block 4 capabilities into fiscal year 2026. This new schedule adds 2 years  to the costs DOD reported to Congress in May 2019. Additionally, our  analysis of DOD’s updated cost estimate indicates the total cost of Block  4 development grew by $1.5 billion to a total of $12.1 billion for activities  in fiscal years 2018 through 2026. Furthermore, in addition to the Block  4 development costs, the program also estimates it will need another $2.9  billion to develop other capabilities, such as upgrades to ALIS. Program  officials attributed this schedule and cost growth to having better insight  into the scope of work to develop and test Block 4 capabilities and noted  that they would continue to refine and update these costs annually as  modernization efforts progress further into development.

Once the existing statutory reporting requirement expires in 2023, DOD  will no longer be required to provide Congress key information that would  be useful in making informed decisions regarding the Block 4 effort— which now extends until 2026. Furthermore, without a complete cost  estimate for Block 4, inclusive of costs already incurred and those not yet  incurred but estimated through completion, Congress is left without a  complete picture of what DOD intends to spend on the total Block 4 effort.  Without a complete picture of these costs, the Congress’s ability to  assess the program’s cost and schedule performance in the future will be  hindered.

Program Office Delayed Delivery of Most 2019 Block 4 Capabilities to the 2020s

The airframe contractor did not deliver the Block 4 capabilities it planned  to deliver in 2019. Specifically, according to the plan outlined in its May  2019 report to Congress, the F-35 program was going to deliver eight  Block 4 capabilities in 2019. However, the program delivered only one—a  software capability called the auto ground-collision avoidance system.  This capability enables the aircraft to perform an automatic recovery  when it predicts that the aircraft will strike the ground. This was ahead of  schedule as the program had originally planned to deliver this capability  after 2019. According to program officials, the development of the other  capabilities is taking longer than planned and, as a result, the program  pushed their delivery schedule into 2020.

Development and delivery of the capabilities within the Block 4 effort are  complex, and the program does not consider development complete until  the products for all elements of the F-35 air system are ready. In  particular, full capability delivery occurs when the contractor delivers all of  the software and hardware needed for all of the F-35 air system elements  to support the planned capability. Program officials stated they are still  working to put the processes in place to synchronize the delivery of the  late capabilities for all of the F-35 air system elements. For example, the  airframe contractor had planned to deliver a capability called the interim  full motion video for the Marine Corps in 2019. The contractor developed  the software needed, but it is late in developing the hardware needed for  the software to operate and, as a result, the contractor did not deliver the  capability in 2019 as planned.

DOD test officials we met with at Edwards Air Force Base stated that in  2019, using the C2D2 approach, the contractor delivered other, partial  Block 4 capabilities to be tested. However, test officials told us those  capabilities were delivered later than expected. Since the program could  not fully test those capabilities on the aircraft, the program office deferred  them to the next incremental update scheduled for 2020. Changes such  as these have contributed to the Block 4 cost and schedule growth.

The program is also discovering issues during Block 4 testing, causing  the testing to take longer than anticipated. According to a DOT&E official,  Block 4 software changes caused issues with functionality of F-35  baseline aircraft capabilities that worked before the program installed new  Block 4 software onto the aircraft. The program discovered issues with  each new software version during flight testing and has been working to  fix these issues in subsequent software updates. Testing and DOD  officials stated that the contractor had not performed adequate testing of  the software before delivering it to the test fleet as the reason for these  issues. Contractor representatives acknowledged these issues and stated  that they will conduct additional lab testing for future software releases to  avoid such problems going forward.

Block 4 Development Cost Estimate Does Not Fully Meet Leading Practices, Which Limits Congress’ Understanding of Costs

We found that the F-35 program office’s Block 4 cost estimate did not fully  meet the four key characteristics of GAO’s cost estimating leading  practices when projecting Block 4 development costs. Table 2 presents  key points from our assessment, and appendix II provides additional  detail on our rationale.

As reflected in table 2, our assessment of the F-35 Block 4 development  cost estimate identified a number of missing elements. Specifically, the  estimate does not rely on a product-oriented work breakdown structure  (WBS), it does not address cost risk and uncertainty, it does not take into  account risk related to technology maturity, and it does not have an  independent cost estimate, as leading practices reflect. While the  program office updates its cost estimate regularly, officials told us that  they do not intend to address some of these missing elements in future  updates.

Work breakdown structure. According to cost estimating leading  practices, the program should base its cost estimate on a program- level, product-oriented WBS that allows a program to track cost and  schedule by defined deliverables, such as hardware or software  components. The WBS ensures that the program does not leave out  any portions of the work and makes it easier to compare it to similar  systems and programs. According to program officials, the Block 4  cost estimate does not rely on a single WBS; rather, multiple,  contractor-derived WBSs exist for the program. Without its own,  program-office-level WBS, the program lacks a framework to develop  a schedule and cost plan that it can use to track progress and  accomplishments.

Risk and uncertainty analyses. The program did not perform cost  risk and uncertainty analyses. Program officials said they do not plan  to conduct a formal risk analysis. The program office works jointly with  the contractor to identify and manage risks for the F-35 program. For  example, there are monthly Joint Risk Management Boards attended  by both program office and contractor leadership. However, overall  program risk management is different from quantitative cost risk and  uncertainty analyses in that program risk management is not specific  to costs and it is not used to assess the cost variance of the cost  estimate itself. When planning for funding decisions for a program of  this scale, analyzing program-level risks alone is inadequate. Without  a risk analysis, the cost estimate will not be fully accurate or credible  because it will not account for the effects of potential schedule slips or  other risks that the program could realize.

Technology maturity. A program office official stated that in  developing the cost estimate they did not consider that technologies  would not be mature, but rather assumed that most technologies  needed to deliver each Block 4 capability would be mature before the  program begins development for that capability. The official stated  that the complexity of design, development, and testing based on the  baseline program experience was reflected in the estimate, but the  cost estimate did not identify if there were specific costs associated  with maturing these technologies. The official further noted that Block  4 costs would increase if a capability takes longer than planned to  design, integrate, and test due to its immaturity. In 2019, we  recommended that the Secretary of Defense ensure that the F-35  program office completes an independent technology readiness  assessment, as part of its business case for the initial Block 4  capabilities, before initiating additional development work. DOD did  not concur with our recommendation. According to a program official,  as of December 2019, the program office had not completed any  technology readiness assessments even though the contractor has  started development of over half of the capabilities within Block 4.  Going forward, the program is considering holding incremental  technology readiness assessments as it plans for and develops a new  set of capabilities, in accordance with the C2D2 schedule. Program  officials told us that, going forward, as they update the Block 4 cost  estimate, they will consider the results of future technology readiness  assessments. Until the program office does so, management cannot  determine a reasonable level of additional resources that might be  necessary to cover increased costs resulting from unexpected design  complexity, incomplete requirements, technology uncertainty, and  other uncertainties.

Independent cost estimate. In 2019, we also recommended the F- 35 program office include an independent cost estimate as part of its  business case for Block 4. As noted in table 2, the Block 4 effort still  lacks an independent cost estimate. The program is planning for the  Office of the Secretary of Defense, Cost Assessment and Program  Evaluation to have a draft independent cost estimate for an interim  program review scheduled for March 2020 and to have a complete  independent cost estimate in June 2020. This estimate will evaluate  the entire F-35 program, including Block 4.

With these pieces currently missing, the Block 4 development cost  estimate does not present a full picture of Block 4’s cost. Ultimately,  without a complete understanding of Block 4 costs, the program could  face additional cost growth, which will be hard to track without a complete  cost baseline. The lack of a complete cost baseline hinders insight and  oversight into the program’s costs, plans, and progress to date and going  forward. Moreover, if a cost estimate does not fully or substantially meet  all four characteristics of cost estimating leading practices, it cannot be  considered reliable.

DOD plans for the F-35 to be central to the warfighter prevailing in future  conflicts. However, the program has been behind schedule and over cost  almost since its inception.

DOD is slated to move into full-rate production despite several key  challenges in the production of aircraft. We acknowledge that the current  F-35 program’s production rates are more commonly associated with  programs already in full-rate production. However, the F-35 aircraft in the  field have not met standards for reliability and maintainability, indicating  that the program is not delivering aircraft at the level of quality expected.  Additionally, the program’s concurrent approach and the contractor’s  continual changes to the production line indicate that the production line  processes are not in control. Leading practices indicate that mature  production lines—production lines ready for full-rate production—should  meet metrics for consistency. Furthermore, to minimize production risk  and potential cost growth, suppliers should routinely meet quality and  delivery schedules, although this is not yet true of the F-35 program. Not  meeting these leading practices poses risks that DOD and the  international partners will not routinely receive the F-35’s they specified  and need. The long-standing challenges with receiving parts on time and  efforts underway to replace Turkish suppliers of parts for the F-35  compound these production challenges and may raise additional risks.  Unless the program office assesses and reports on these manufacturing  risks ahead of the milestone C review, Congress may not have key  insights into the risks that remain with the program and to the overall  effort to deliver F-35s to the warfighter.

Since the F-35 program is not managing the Block 4 effort as a separate  program with traditional oversight tools, we are particularly concerned as  Block 4 efforts proceed through development and testing. Specifically,  because of the delays to the program, after 2023, DOD will not be  required to provide Congress information on Block 4’s development  efforts as the current reporting requirements will end. Furthermore, the  program’s cost estimate, as presented in its report to Congress, does not  fully present all incurred and future costs for Block 4. Without this  information, Congress may not have the insight it needs to assess Block  4 cost and schedule progress as well as to make informed oversight and  budgeting decisions.

In addition, the Block 4 development cost estimate does not fully meet  leading practices, lacking a full reflection of all costs. Specifically, the cost  estimate does not have a program office level work breakdown structure,  a risk and uncertainty analysis, and consideration of technology  readiness. Without a comprehensive and credible cost estimate, DOD  and Congress lack a sound basis for informed investment decision  making, realistic budget formulation, meaningful progress measurement,  proactive course correction when warranted, and program and contractor  accountability for results.

Congress should consider revising Section 224(d) of the National  Defense Authorization Act for Fiscal Year 2017, Pub. L. No. 114-328, to  extend DOD’s Block 4 reporting requirement until all Block 4 capabilities  are fielded to ensure that Congress is aware of cost and schedule growth  beyond 2023. (Matter for Consideration 1)

We are making the following five recommendations to the Secretary of  Defense to direct the Undersecretary of Defense for Acquisition and  Sustainment (OUSD (A&S)).

The OUSD (A&S) should direct the F-35 program office to provide  information that is similar to that which is statutorily required after the  milestone C review to Congress ahead of the milestone C review (full-rate  production decision). This submission should include an evaluation of the  production risks associated with critical production processes that are not  in control, reliability and maintainability (R&M) targets that are not met,  and supplier readiness—particularly for those replacing Turkish suppliers,  along with the steps it is taking to address those risks. (Recommendation  1)

The OUSD (A&S) should direct the F-35 program office to establish a  Block 4 cost estimate baseline that includes all Block 4 costs, including  incurred costs and future costs in its reports to Congress as required by  the NDAA for Fiscal Year 2017, so that Congress has a complete  understanding of all Block 4 costs and can compare this baseline to future  cost estimates and performance. (Recommendation 2)

The OUSD (A&S) should direct the F-35 program office to complete a  program office level, product-oriented work breakdown structure for the  next update to its Block 4 cost estimate to ensure that the estimate meets  the comprehensive leading practices. (Recommendation 3)

The OUSD (A&S) should direct the F-35 program office to conduct risk  and uncertainty analyses for the next update to its Block 4 cost estimate  to ensure that the estimate meets the credible leading practices.  (Recommendation 4)

The OUSD (A&S) should direct the F-35 program office to consider the  results of its future technology readiness assessment of all Block 4  technologies and incorporate the cost and schedule risks of developing  those technologies in the next update to its Block 4 cost estimate to  ensure that the estimate meets the comprehensive leading practices.  (Recommendation 5)

We provided a draft of this report to DOD for review and comment. DOD  provided written comments, which we have reproduced in appendix VI.  DOD concurred with three of the recommendations related to the Block 4  modernization effort (recommendations 2, 4, and 5 above). While DOD  did not concur with the other two recommendations, it outlined planned  actions that we believe, if implemented, would meet the intent of our  recommendations. DOD also provided technical comments, which we  incorporated as appropriate. We will continue to monitor the program and  evaluate implementation of these recommendations.

DOD officials did not concur with the first recommendation, which, in the  draft report, was to evaluate production risks and provide a statutorily  required report to Congress ahead of the program’s full-rate production  decision. While DOD did not concur with the draft recommendation, it  agreed to keep the Congress apprised of these matters in its quarterly  briefings to the defense committees. To clarify the actions we intended  DOD to take to address our findings, we revised the recommendation to  indicate that DOD should provide information to Congress on the  production risks we identified in our report, ahead of the milestone C  review. If the DOD provides a substantive assessment highlighting these  production risks, as well as the steps it will take to mitigate them, during  its quarterly briefing to Congress ahead of the milestone C review, it  would address the intent of our recommendation.

DOD also did not concur with our third recommendation for the F-35  program office to complete a program-level, product-oriented work  breakdown structure (WBS) for the next update to its Block 4 cost  estimate. DOD noted that its next scheduled update was due in April  2020, after we provided our report for comment. While DOD noted it  would be unable to complete a program-level WBS by the April 2020  update, it agreed to evaluate moving to a program-level, product-oriented  WBS in 2021. If the F-35 program office utilizes a program-level, product- oriented WBS for this cost estimate update, it would meet the intent of our  recommendation.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; and the Under Secretary of  Defense for Acquisition and Sustainment, the Secretary of the Air Force,  the Acting Secretary of the Navy, and the Commandant of the Marine  Corps. In addition, the report is available at no charge on the GAO  website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or ludwigsonj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix VII.

Appendix II: F-35 Block 4 Development Cost Estimate Analysis

To assess the reliability of the F-35 Block 4 development cost estimate,  we obtained and reviewed cost estimate documentation such as the Joint  Program Office briefing on its May 2019 estimate, the Air System  Procurement Playbook—a planning document for Block 4—and its cost  estimate models. Additionally, we met with relevant staff in the F-35  program office and the Department of Defense’s Office of Cost  Assessment and Program Evaluation. We analyzed this information and  determined the extent to which the program office’s practices for  developing the F-35 Block 4 development cost estimate were consistent  with the leading practices identified in the GAO Cost Estimating and  Assessment Guide. These practices have been found to be the basis for  reliable cost estimates. We assessed each practice as being one of the  following:

Met—the agency provided data and documentation that satisfies the  entire leading practice criterion.

Substantially met—the agency provided data and documentation that  satisfies a large portion of the leading practice criterion.

Partially met—the agency provided data and documentation that  satisfies about half of the leading practice criterion.

Minimally met—the agency provided data and documentation that  satisfies a small portion of the leading practice criterion.

Not met—the agency provided data and documentation that does not  satisfy any portion of the leading practice criterion.

For our reporting needs, we collapsed GAO’s 18 leading practices into  four general characteristics: comprehensive, well-documented, accurate,  and credible. The assessment of each characteristic was based on an  average of the F-35 program office’s scores for the leading practices  included in that category. A second analyst verified the assessment and  management reviewed the results. We determined the overall  assessment rating by assigning each individual rating a number:

Then, we took the average of the individual assessment ratings to  determine the overall rating for each of the four characteristics. The  resulting average becomes the Overall Assessment as follows:

A cost estimate is considered reliable if the overall assessment ratings for  each of the four characteristics are substantially or fully met. If any of the  characteristics are not met, minimally met, or partially met, then the cost  estimate does not fully reflect the characteristics of a high-quality estimate  and cannot be considered reliable. See table 4 for a high level summary  of each leading practice and the reasons for the overall scoring.

Appendix III: Objectives, Scope, and Methodology

This report fulfills two mandates. First, the National Defense Authorization  Act for fiscal year 2015 included a provision for GAO to review the F-35  acquisition program annually until the program reaches full-rate  production. This is the fifth report under that provision. Second, the  National Defense Authorization Act for Fiscal Year 2020 includes a  provision for GAO to review the program’s production and Block 4  progress annually through 2025. In this report, we (1) provide information  on the program’s progress toward completing operational testing and  resolving deficiencies found in testing; (2) assess the program’s  production performance and manufacturing efficiency initiatives; and (3)  assess the program’s modernization cost estimate and progress with  Block 4 development efforts.

To provide information on the program’s progress in operational testing  and the resolution of deficiencies, we first reviewed the baseline  program’s costs, schedule, and performance plans and compared the  actual progress in each area with the goals established in its 2012  baseline to identify any significant trends. We reviewed progress on test  events completed versus those that remain, test schedules, program  briefings, and DOD briefings. We traveled to Edwards Air Force base to  interview DOD test authorities and met with officials from the program  office, DOD test authorities, and the contractor Lockheed Martin (the  prime aircraft contractor), to discuss key aspects of F-35 development  progress, including flight testing, future test plans, and recent findings  from test events. Specifically, we obtained updates on key events that are  required to complete testing according to the program office’s current  schedule. We also interviewed the Director, Operational Test and  Evaluation office and F-35 program developmental and operational test  pilots. To provide information on the program’s progress resolving  deficiencies, we interviewed the same officials mentioned above and  discussed how the number of open and closed deficiencies changed in  2019. We reviewed program and contractor information on deficiency  reports, mitigations, resolutions, and the deficiency resolution process.

To assess the program’s production performance and manufacturing  efficiency initiatives, we obtained and analyzed the production metrics  from Lockheed Martin and Pratt & Whitney (the prime engine contractor)  and their aircraft and engine delivery rates from 2012 through 2019. We  reviewed metrics and briefings provided by the program office, Lockheed  Martin, Pratt & Whitney, and the Defense Contract Management Agency  to identify progress in improving manufacturing processes. We analyzed  delivery dates for lot 11 aircraft delivered in 2019. We traveled to the  production facility in Fort Worth, Texas to discuss reasons for any delivery  delays and plans for improvements with officials from Lockheed Martin.  We obtained cost investment and savings estimates and discussed cost  and manufacturing efficiency initiatives, such as the economic order  quantity purchases, with the contractors and program office officials to  understand potential cost savings and plans. We collected and analyzed  the extent to which the program has met leading practices identified by  GAO for full-rate production. We also obtained and analyzed metrics on  parts and aircraft quality through December 2019 and discussed steps  taken to improve quality and deliveries with Lockheed Martin and Pratt &  Whitney officials. We interviewed officials from Lockheed Martin, Pratt &  Whitney, and Northrop Grumman (a key subcontractor) regarding the  administration’s decision to suspend Turkey from the program and the  implications of the suspension for the contractors. We determined that the  contractors’ production metrics and delivery dates were sufficiently  reliable for our purposes of determining production efficiency and  deliveries. We collected and analyzed production and supply chain  performance data from the program office, Lockheed Martin, and Pratt &  Whitney.

To assess the reliability of the May 2019 Block 4 development cost  estimate, we evaluated documentation supporting the estimate, such as  the cost estimating models, the F-35’s Air System Procurement Playbook,  its updated acquisition strategy, the Decision Memorandum requirements  document, and briefings provided to the DOD decision authority. We  assessed the cost estimating methodologies, assumptions, and results  against leading practices for developing a comprehensive, accurate, well- documented, and credible cost estimate, identified in GAO’s Cost  Estimating and Assessment Guide. We also interviewed program  officials responsible for developing and reviewing the cost estimate to  understand their methodology, data, and approach for developing the  estimate. We found that the cost estimate was not reliable. To assess  progress with Block 4 development efforts, we interviewed DOD and  program office officials, and contractor representatives regarding the  program’s Block 4 planning, development, testing, and production  activities to date. We reviewed other program documentation, such as the  F-35’s fiscal year 2020 budget request, to identify costs associated with  the Block 4 effort. We compared the program’s accomplishments in 2019  to its plans and identified what capabilities the program office delivered to  the fleet. We reviewed the program office’s plans to develop and deliver  additional Block 4 capabilities from 2020 through 2025.

We conducted this performance audit from June 2019 to May 2020 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix IV: The F-35’s Reliability and Maintainability Metrics

The Joint Strike Fighter Operational Requirements Document, which  outlines the requirements Department of Defense and the military  services agreed the F-35 should meet, defines all eight reliability and  maintainability (R&M) metrics. Table 5 shows each F-35 variants’  performance against these metrics’ targets, as of August 2019.

Appendix V: Status of Selected F-35 Technical Risks

F-35B Thrust Cutback: An F-35B aircraft can experience an  unanticipated cutback in thrust during vertical landings (hover). The  contractor put hover weight restrictions in place to mitigate the effect and  has identified the root cause. The contractor is developing software and  hardware fixes.

F-35C Nose Landing Gear: During shipboard landings, the F-35C can  experience bending stress, which causes cracking of the coating on a  part in the nose landing gear. In the short term, this part will be inspected  for damage every 400 flying hours. The contractor is also redesigning the  part that is cracking and expects to test it between early 2020 to June  2021.

F-35B Three Bearing Swivel Module: The module is mounted at the  back of the aircraft and allows the thrust from the engine to be vectored  from straight aft for conventional flight to straight down for short takeoff  and vertical landing operations. In June 2019, an F-35B experienced a  warning indicator in its short takeoff mode due to a hardware component.  However, according to the contractor, this component should not cause a  warning indicator or loss of functionality for the aircraft. The contractor  has identified the root cause of the hardware issue and a gap in the  software’s logic that led to the warning. As a result, the contractor is  making manufacturing changes to the hardware and implementing  software changes to address the issue.

Technical Risks Identified in Our Previous Reports

Canopy Coating Delaminations: The F-35 fleet has experienced over  50 incidents of the canopy transparencies delaminating after less than  100 flight hours since August 2017. This is over 30 more than we  reported in 2019. The contractor tested solutions for the delaminations in  2019 and implemented a solution of adding a vent to the canopy’s frame.

Since October 2019, the contractor has added a vent to 146 canopies  with one subsequent delamination.

Helmet Mounted Display: During low-light flights, the Helmet Mounted  Display’s technology cannot display pure black images, instead  presenting a green glow on the screen, which makes it difficult to see the  full resolution of the night vision video feed. The contractor developed a  new display to avoid this effect. According to F-35 program officials, they  placed an initial order of 62 displays with 35 delivered by December 2019  to support U.S. Marine Corps and Navy F-35C fleet operations. Three F- 35C pilots completed initial day and night testing using the new display in  July 2019 on a carrier. The contractor expects to have a fully qualified  redesign by August 2021 and will incorporate it into the production of lot  12 aircraft.

Appendix VII: GAO Contact and Staff Acknowledgments

In addition to the contact named above, the following staff members  made key contributions to this report: Justin Jaynes and Alissa Czyz  (Assistant Directors); Diana Maurer, Desirée E. Cunningham (Analyst-in- Charge), Jillena Roberts, Tim Moss, Rose Brister, Juaná Collymore,  Emile Ettedgui, Jennifer Leotta, and Jeff Hubbard, Other staff who  contributed include Leslie Ashton, Priyanka Sethi Bansal, Vinayak  Balasubramanian, Julia DiPonio, Christine Pecora, Ralph Roffo, Roxanna  Sun, Jessica Waselkow, and Alyssa Weir.

Weapon System Sustainment: DOD Needs a Strategy for Re-Designing  the F-35’s Central Logistics System. GAO-20-316. Washington, D.C.:  March 6, 2020.

F-35 Aircraft Sustainment: DOD Faces Challenges in Sustaining a  Growing Fleet. GAO-20-234T. Washington, D.C.: November 13, 2019.

Defense Acquisitions: Observations on the F-35 and Air Force’s  Advanced Battle Management System. GAO-19-456T: Washington, D.C.:  May 2, 2019.

F-35 Joint Strike Fighter: Action Needed to Improve Reliability and  Prepare for Modernization Efforts. GAO-19-341. Washington, D.C.: April  29, 2019.

F-35 Aircraft Sustainment: DOD Needs to Address Substantial Supply  Chain Challenges. GAO-19-321. Washington, D.C.: April 25, 2019.

F-35 Joint Strike Fighter: Development is Nearly Complete, but  Deficiencies Found in Testing Need to Be Resolved. GAO-18-321.  Washington, D.C.: June 5, 2018.

Warfighter Support: DOD Needs to Share F-35 Operational Lessons  Across the Military Services. GAO-18-464R. Washington, D.C.: April 25,  2018.

F-35 Aircraft Sustainment: DOD Needs to Address Challenges Affecting  Readiness and Cost Transparency. GAO-18-75. Washington, D.C.:  October 26, 2017.

F-35 Joint Strike Fighter: DOD’s Proposed Follow-on Modernization  Acquisition Strategy Reflects an Incremental Approach Although Plans  Are Not Yet Finalized. GAO-17-690R. Washington, D.C.: August 8, 2017.

F-35 Joint Strike Fighter: DOD Needs to Complete Developmental  Testing Before Making Significant New Investments. GAO-17-351.  Washington, D.C.: April 24, 2017.

F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans  to Begin Development of New Capabilities. GAO-16-390. Washington,  D.C.: April 14, 2016.

F-35 Sustainment: DOD Needs a Plan to Address Risks Related to Its  Central Logistics System. GAO-16-439. Washington, D.C.: April 14, 2016.

F-35 Joint Strike Fighter: Preliminary Observations on Program Progress.  GAO-16-489T. Washington, D.C.: March 23, 2016.

F-35 Joint Strike Fighter: Assessment Needed to Address Affordability  Challenges. GAO-15-364. Washington, D.C.: April 14, 2015.

F-35 Sustainment: Need for Affordable Strategy, Greater Attention to  Risks, and Improved Cost Estimates. GAO-14-778. Washington, D.C.:  September 23, 2014.

F-35 Joint Strike Fighter: Slower Than Expected Progress in Software  Testing May Limit Initial Warfighting Capabilities. GAO-14-468T.  Washington, D.C.: March 26, 2014.

F-35 Joint Strike Fighter: Problems Completing Software Testing May  Hinder Delivery of Expected Warfighting Capabilities. GAO-14-322.  Washington, D.C.: March 24, 2014.

F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but  Affordability Challenges and Other Risks Remain. GAO-13-690T.  Washington, D.C.: June 19, 2013.

F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term  Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March  11, 2013.

Fighter Aircraft: Better Cost Estimates Needed for Extending the Service  Life of Selected F-16s and F/A-18s. GAO-13-51. Washington, D.C.:  November 15, 2012.

The F-35 program produced more aircraft and negotiated lower prices in 2019. However, the program is not meeting standards aimed at ensuring consistent, high-quality products, and fielded aircraft do not meet reliability goals.

Also, the cost to modernize aircraft systems went up about $1.5 billion (14%) since the program’s May 2019 annual report to Congress. Due to development delays, this reporting requirement will expire before the effort is complete. We suggest Congress consider extending it.

We made 5 recommendations, including some to help the Defense Department make its modernization cost estimate more comprehensive and credible.