Generic Drug Development: Stakeholders' Views of Risk Evaluation and Mitigation Strategies Differ

To manage the risks posed by some drugs, FDA requires drug companies to establish risk evaluation and mitigation strategies. Companies developing generic drugs generally need samples of the reference standard drug to conduct bioequivalence testing. Generic companies may also have to negotiate a shared system with the reference drug company, when that company's drug is subject to certain REMS requirements.

FDA and FTC officials acknowledge that some drug companies have used certain practices that prevent or delay the development of generic drugs. The practices include limiting access to samples of reference standard drugs with and without REMS and delaying negotiations for creating required shared systems. GAO was asked to review drugs subject to REMS and drug companies' experience with these practices. This report describes (1) the drugs subject to REMS, and (2) FDA and FTC's efforts to address these practices, and stakeholders' views on agencies' efforts.

GAO analyzed FDA data on the conditions these drugs treat and the REMS requirements that apply to the drugs. GAO also interviewed FDA and FTC officials and representatives from five reference drug companies and four generic drug companies, which GAO selected based on a variety of factors, including the companies' experiences with drugs subject to REMS . GAO also reviewed public comments and related documents from FDA and FTC.

HHS and FTC provided technical comments on a draft of this report, which GAO incorporated as appropriate.

The Food and Drug Administration (FDA) can require drug companies to establish risk evaluation and mitgation strategies (REMS) for drugs with serious safety concerns to ensure that a drug's benefits outweigh its risks. As of March 18, 2019, FDA approved 74 active REMS that cover 523 drugs that treat various conditions. One hundred forty-three of the drugs are reference standard drugs, which are drugs generic drug companies must use to conduct bioequivalence testing. Of these 143, 64 have at least one approved generic that is also subject to REMS. Ten of the REMS are shared systems that allow health care providers to obtain information from multiple companies on a drug's risks and satisfy other administrative requirements through one REMS system.

According to FDA and the Federal Trade Commission (FTC), drugs with and without REMS have been the subject of practices that can delay or prevent generic drug development and marketing. FDA and FTC have taken actions designed to address some of these practices. According to FDA officials, they are more limited in what actions they can take when drugs without REMS are involved. Drug company officials that GAO interviewed had different views on these actions. To address practices that may limit access to samples of reference standard drugs and keep generic drugs from the market:

FDA issued draft guidance in 2014 on how generic companies could obtain a letter stating that the agency would not consider it a REMS violation to provide reference standard drug samples to the generic company requesting the letter. Three of the four generic companies GAO interviewed said these letters were not useful because they do not require drug companies to share samples. In contrast, officials from three of five reference drug companies said the letters addressed their safety concerns about providing samples to generic companies. FDA does not issue such letters for drugs without REMS.

In February 2019, FDA published a list of drug companies whose reference standard drugs were the subject of access inquires made to FDA by generic drug companies. One of the four generic companies GAO spoke with said FDA's list was helpful, and one reference drug company said it was uncertain why it was included on the list.

FTC has reviewed inquiries it received from FDA and generic companies, and has filed amicus briefs in two cases involving drugs with REMS. According to FTC, to date, the agency has not brought a case charging a drug company with violating federal antitrust law for refusing to provide samples to a generic drug company.

To address practices that may delay negotiations between reference drug and generic drug companies for creating required shared systems, FDA issued waivers and related guidance that allowed generic companies to develop a separate, but comparable, REMS shared system. One generic drug company said the guidance on waivers was helpful; however, one drug company said the waivers put added burden on health care providers who have to use multiple REMS systems.

All drugs pose some level of safety risk to patients, but certain drugs have  more serious risks associated with them than others, such as a risk for  severe adverse events. To ensure the benefits of certain drugs outweigh  their risks, the Food and Drug Administration (FDA) can require drug  companies to establish risk evaluation and mitigation strategies (REMS)  for these drugs. For example, FDA may require drug companies to place  certain conditions on a drug subject to REMS, such as who can distribute  the drug and where it can be done, such as in hospitals.

In recent years, generic companies have raised concerns that some drug  companies have used certain practices involving REMS requirements to  hinder competition by delaying or preventing generic drugs from being  developed or coming to market. Drug companies have also used these  practices for drugs that are not subject to REMS. Reference listed drugs,  typically brand name drugs, are drugs already approved by FDA. Generic  drug companies generally use samples of reference standard drugs to  test whether their generic drugs are bioequivalent to the corresponding  reference listed drug. Once available, generic drugs can provide  substantial cost savings for patients and third-party payers, including  government health programs.

FDA and the Federal Trade Commission (FTC) have identified two  practices that could hinder generic drug companies’ ability to develop  generic drugs: (1) limiting generic drug companies’ access to samples of  reference standard drugs, which are necessary to test whether the  generic drug is bioequivalent to the reference listed drug, and (2) delaying  negotiations between reference drug and generic drug companies that  must be completed before certain generic drugs can be marketed.

You asked us to review drugs subject to REMS and the practices  identified by FDA and FTC that may prevent generic drugs from coming  to market. This report describes  1.  the drugs subject to REMS and the requirements established by these  REMS; and  2.  FDA and FTC’s efforts to address practices that may hinder the  development and marketing of generic drugs and what drug  companies and stakeholders have said about these efforts.

To describe drugs subject to REMS and the requirements established by  the REMS, we analyzed FDA data, as of March 18, 2019, on drugs for  which FDA has approved REMS. To further describe drugs subject to  REMS, we analyzed FDA data to identify additional characteristics of  these drugs, including orphan drugs. To determine the medical  conditions that drugs subject to REMS are intended to treat, we identified  the therapeutic classes of these drugs using RED BOOK data from March  2018, and characterized the therapeutic classes by medical conditions.  We also estimated Medicare Part D and Medicaid drug spending for a  select number of reference standard drugs for which cost data were  available. We also compared spending for a select number of reference  standard drugs to spending for a corresponding generic drug. To do this,  we used publicly available data from the Medicare Part D Drug Spending  Dashboard and the Medicaid Drug Spending Dashboard, maintained by  the Centers for Medicare & Medicaid Services (CMS). These data  covered drug spending and utilization for both of these programs for  calendar year 2017, the most current data available at the time of our  review. To assess the reliability of the FDA data, we performed data  checks and interviewed agency officials. To assess the reliability of the  Medicare and Medicaid drug spending data, we interviewed agency  officials. We determined that these data were sufficiently reliable for the  purposes of our reporting objective.

To identify FDA’s and FTC’s efforts to address practices that could hinder  the development and marketing of generic drugs and what drug  companies and other stakeholders have said about these agencies’  efforts, we interviewed FDA and FTC officials and reviewed agency  documents, such as guidance documents, about their efforts. We  interviewed officials from two stakeholders representing drug companies.  We also interviewed officials from nine drug companies who have  experience with drugs subject to REMS about these agencies’ efforts.  We used the following three criteria to identify a range of generic drug  companies with whom to speak: (1) whether they made six or more  inquiries to FDA regarding their inability to access samples of reference  standard drugs; (2) whether they had experience negotiating a required  shared system (a system implemented jointly by two or more drug  companies to coordinate certain REMS activities, which is required for  some generic drugs); and (3) the different medical conditions their drugs  were intended to treat in order to obtain a range of drugs. After applying  our criteria, we selected four generic drug companies. To obtain the  reference drug companies’ perspectives, we selected five drug  companies whose reference standard drugs were the subject of sample  access inquiries from the four generic companies that we selected. We  asked these drug companies about their opinions on FDA and FTC’s  efforts to address these specific practices. To supplement our interviews  with stakeholders and drug companies, we reviewed 16 comments to the  Federal Register related to FDA’s efforts. We selected these 16  comments out of a total of 42 comments because they were submitted by  stakeholders representing professional associations, patient advocacy  groups, and drug companies. These comments included two additional  drug companies beyond those we interviewed as well as organizations  representing professional associations, such as pharmacists. We also  excluded comments submitted by individuals.

We conducted this performance audit from March 2018 through October  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

FDA approves reference listed drugs and generic drugs that meet safety  and efficacy standards for marketing in the United States. Generic drug  companies must show that their drug is (1) the same as the reference  listed drug with respect to the active ingredient(s), conditions of use, route  of administration, dosage form, strength, and labeling (with certain  permissible differences, as approved by FDA); and (2) bioequivalent to  the reference listed drug, meaning it generally delivers the same amount  of active ingredient(s) in the same amount of time as the reference listed  drug.

When the reference listed drug is available, it is also designated as the  reference standard drug, which is the product generic drug companies  must use to conduct bioequivalence testing. When the reference listed  drug is not available, FDA will select an approved generic of the reference  listed drug to serve as the reference standard drug.

All drugs pose some level of safety risk to patients. According to FDA, for  most drugs, routine, risk-minimization measures, such as FDA-approved  professional labeling, are sufficient to protect the public from the drug’s  risks. However, in some cases, FDA may require a drug company to  take additional actions to ensure that the benefits of the drug outweigh its  risks and to help mitigate or prevent serious risks of adverse side effects.  Specifically, FDA may require the drug company to establish a REMS that  includes one or more risk-mitigation strategies beyond the drug’s  professional labeling.

According to FDA, most REMS are designed to reinforce patients’ and  health care providers’ behaviors and actions that support the safe use of  the particular drug they cover. For example, FDA may require drug  companies to give patients and health care providers additional  information to reinforce certain safe use conditions or specific risks  described in the approved labeling of a certain drug. FDA may require a  REMS either before a drug is approved or after approval if FDA becomes  aware of new safety information. When determining whether a REMS is  necessary, FDA considers several factors, including, for example, the  estimated size of the population likely to use the drug and the  seriousness of the disease or condition being treated.

FDA may require a REMS to include one or more components. For  example, FDA may require drug companies to provide patients with  certain information in the form of medication guides. Generally,  medication guides include information on serious side effects, including  those that might require emergency medical care or involve life- threatening conditions. Similarly, FDA may require drug companies to  develop communication plans for how the drug company will disseminate  information to health care providers. Communication plans can include,  for example, information on any serious risks of the drug and any safety  protocols to ensure its safe use. Thus, for one REMS, FDA could require  a drug company to provide a medication guide. For a second REMS, FDA  could require a drug company to provide both a medication guide and a  communication plan. Table 1 below includes a list of selected REMS  components. Additionally, if a reference listed drug is subject to a  REMS, an approved generic drug is also subject to some of the same  REMS requirements.

FDA can also require drug companies to implement another REMS  component, called “elements to assure safe use” (ETASU), if a drug has  been shown to be effective, but is associated with a specific serious risk.  Depending on the risk, FDA may require any or all of the following  ETASU measures:

Prescribers have specific training or special certifications;

Pharmacies or health care settings where the drug is dispensed have

Drugs are dispensed only in certain health care settings, such as  hospitals;

Drugs are dispensed with evidence of safe-use by the patient, such as  requiring a patient’s acknowledgement that she has been counseled  on a drug’s risks and understands and accepts these risks;

Patients are monitored, for example, while taking the drug for specific  adverse events or outcomes; or

Patients are enrolled in a registry for collection of certain information,  such as patient outcomes and adverse reactions associated with the  drug.

According to FDA, these measures are for drugs that can be marketed  only if there are requirements in place to mitigate a specific serious risk  listed in the drug’s labeling. If FDA requires certain ETASU measures, it  may also require a drug company to develop an implementation system  to enable the drug company to monitor and evaluate implementation of  the ETASU measures by health care providers, pharmacists, and other  responsible parties.

Also, if a reference listed drug is subject to a REMS with ETASU and a  generic version is being developed, the reference drug company and the  generic drug company are required to develop a shared system—a  system that is used by participating companies to coordinate their REMS  activities and information about a drug’s risks. Under a required shared  system, the generic drug company and the reference drug company use  the same REMS documentation and other materials on the drug’s risks  and generally share in the implementation and maintenance of any  database and infrastructure (e.g., call center). According to FDA, shared  systems can be beneficial in reducing the burden for patients and health  care providers, such as prescribers and pharmacies, when accessing  REMS informational materials or completing administrative requirements,  including any required training or certifications for providers.

Generic drug companies must submit REMS documentation and  materials as part of their generic drug application. Generally, before FDA  can approve generic drugs that are subject to REMS with ETASU,  reference drug companies and generic companies must reach agreement  on a required shared system. According to FDA, generic drug companies  that are developing a required shared system should submit their  proposed REMS materials to FDA by the midpoint of the application  review process or another time as specified by the agency. Any delays  in the development of a required shared system can affect FDA’s ability to  approve a generic drug application. A generic company may request a  waiver from FDA, which if granted, would allow the generic company to  develop a separate system that includes the same ETASU measures  required for the reference listed drug. For example, if the reference  listed drug’s ETASU measures require prescriber certification and the  dispensing of the drug in certain health care settings, then the generic  drug company’s separate system must also include the same ETASU  measures.

Practices Identified by FDA and FTC that May Hinder Generic Drug Development and Marketing

In recent years, FDA and FTC have identified two practices that can  hinder competition by preventing or delaying the development and  marketing of generic drugs. The first practice the agencies identified  involves limiting access to samples of reference standard drugs, which  generic companies generally need to conduct bioequivalence testing.  This practice can apply both to reference standard drugs subject to  REMS, specifically those subject to certain ETASU measures, and those  not subject to REMS. For example, some drug companies might limit  access to samples of reference standard drugs subject to REMS, citing  ETASU measures that limit distribution, such as the measure that limits  distribution of drugs subject to REMS to only certain health care  settings. Additionally, drug companies may limit access to samples of  reference standard drugs that are not subject to REMS. Typically, generic  companies obtain samples through normal distribution channels such as  wholesale distributors. However, drug companies could, for example, limit  the sale of their reference standard drugs to certain pharmacies, such as  specialty pharmacies. FDA and FTC have testified before Congress that  these distribution limits—for reference standard drugs with and without  ETASU-related distribution measures—can hinder generic companies’  ability to develop generic drugs and to submit a generic drug application  to FDA for review.

The second practice involves circumstances when a reference drug  company delays its negotiations with generic drug companies on a  required shared system. The negotiations to develop a required shared  system can be complex because all parties must agree on the  implementation of the REMS as well as issues related to cost-sharing,  confidentiality, and product liability concerns. As part of their generic drug  application, generic companies must include an adequate REMS program  in order to be approved. Therefore, delays in the development of a  required shared system can affect FDA’s ability to approve a generic drug  application.

Drugs Subject to REMS Vary in the Risks They Pose, Treat a Variety of Conditions, and About Half of Approved REMS Place Limits on Distribution

Drugs Subject to REMS Vary in Risks, Treat a Variety of Conditions, and Accounted for at Least $11 Billion in Federal Spending

Our analysis of FDA data shows that as of March 18, 2019, there were 74  approved active REMS that apply to 523 drugs. These drugs pose a  variety of risks to users, treat a variety of conditions, and some are  generics. A REMS can apply to one drug, more than one drug, or to a  large number of drugs. Specifically, the approved REMS apply to:

136 drugs because they pose a high risk of serious medical side  effects,

384 drugs because they pose a high risk of serious medical side  effects from misuse and abuse, and

Three drugs because they have the risk of medical side effects from  both the use of the drug and from misuse and abuse.

These drugs also treat at least 15 different types of medical conditions  such as cancer, cardiovascular, and respiratory conditions. Twenty-two  are orphan drugs, which are drugs intended to treat rare diseases. One  hundred forty-three of these drugs are reference standard drugs, and 64  of these reference standard drugs have one or more approved generics  that are also subject to REMS. (See Table 2) For example, FDA  approved a generic of the drug Clozaril, which is used to treat mental and  mood disorders. Both Clozaril and its generic, Clozapine, are subject to a  REMS to prevent adverse medical side effects.

Medicare and Medicaid paid at least $11.8 billion in 2017 for reference  standard drugs subject to REMS. Specifically, in 2017 Medicare paid at  least $8.5 billion for 83 of 139 reference standard drugs subject to  REMS. This amount accounted for at least 8 percent of all Medicare drug spending in 2017. In the same year, Medicaid paid at least $3.3  billion—or at least 15 percent of all Medicaid drug spending—for 83 of the  139 reference standard drugs subject to REMS. Appendix I provides  information from available data on Medicare and Medicaid spending on  reference standard drugs subject to REMS.

Almost Half of FDA’s 74 Active REMS Include Limits on How Drugs Are Distributed, and 10 Established a Shared System

Of the 74 active REMS in our analysis, 51 have at least one required  ETASU measure, and 35 specifically limit how drugs are distributed.  Thirty-one of the 74 active REMS also require medication guides  explaining the risks of the drug to be given to patients, and 12 require a  communication plan for how the company will disseminate information to  health care providers. Similar to how the 74 active REMS can have more  than one REMS component, the 51 active REMS with ETASU measures  can have more than one required measure. For example, 19 active  REMS have an ETASU measure requiring patients to be enrolled in a  registry and an ETASU measure requiring drug companies to provide  training to prescribers of the drugs.

Over half of the 51 active REMS with ETASU include measures that may  limit how drugs are distributed. Specifically, 35 active REMS with ETASU  measures include a requirement for drug companies to ensure drug  dispensing settings are specially certified before they distribute the  drugs. The certification process may require dispensing pharmacies to  enroll in education programs provided by the drug companies. For  example, to mitigate the risk of accidental overdoses from the misuse and  abuse of fentanyl products, dispensing pharmacies are required to  complete an education program that addresses—among other things— the risks of fentanyl products, patient selection, drug dosage, and patient  counseling.

In addition, for 10 of the 74 active REMS, companies have entered into a  shared system. In three of the 10 shared systems, generic companies  received a waiver from the shared system requirement after they were not  successful in negotiating a shared system with the reference drug  companies. In these three cases, the generic drug companies entered  into shared systems that are separate from the reference drug company  systems. For example, after developing generics of Lotronex, which is  subject to REMS with ETASU and intended to treat gastrointestinal  conditions, the generic companies were required to enter into a shared  system with the reference drug company. When these companies were  not successful in negotiating a required shared system, FDA determined  the burden of developing a required shared system with the reference  drug company outweighed the benefits of having one and waived the  requirement. Once FDA granted the waiver, multiple generic companies  were allowed to share REMS materials and administrative requirements  with heath care providers via one shared system that is separate from  Lotronex’s REMS system.

FDA and FTC Have Taken Actions to Address Practices They Identified; Drug Companies and Stakeholders Disagreed on the Usefulness of the Actions

FDA and FTC Have Taken Actions to Facilitate Access to Samples of Reference Standard Drugs and Required Shared Systems Negotiations

FDA and FTC have taken four actions to address circumstances when  generic drug companies cannot access samples of reference standard  drugs or experience delays in negotiating required shared systems.  According to FDA and FTC, both circumstances can hinder generic drug  companies’ ability to develop and market generic drugs. Three of the  actions focus on making samples of reference standard drugs accessible  and the fourth focuses on facilitating the development of a required  shared system. While all four of the actions pertain to drugs subject to  REMS, only two of the actions pertain to drugs both subject to REMS and  not subject to REMS. According to FDA officials, the agency is even more  limited in what actions it can take when drugs not subject to REMS are  involved.

Drug Companies’ Perspectives on Limited  Access to Samples of Reference Standard  Drugs with Elements to Assure Safe Use  (ETASU) that Limit Distribution   Officials from all four of the generic drug  companies we interviewed told us that their  inability to access samples of reference  standard drugs with ETASU measures that  limit distribution either delayed or discouraged  them from developing generic drugs.  Officials from two of the five reference drug  companies we interviewed told us they were  unaware of specific instances when generic  companies had difficulty obtaining samples or  that generic companies had requested  samples of reference standard drugs with  ETASU measures that limit distribution. Also,  officials from two reference drug companies  cited safety concerns as the reason for  limiting the distribution of their drug.

FDA issued draft guidance on how to obtain a safety  determination letter. One of FDA’s actions focused on facilitating  generic drug companies’ access to reference standard drugs with  ETASU-related limited distribution measures. (See sidebar for drug  companies’ perspectives on this practice.) In 2014, FDA issued draft  guidance describing how a generic drug company could ask the  agency to send what is known as a safety determination letter to the  reference drug company on the generic drug company’s behalf. The  draft guidance explains how FDA could send a letter stating that the  agency had reviewed the generic company’s plans for its  bioequivalence testing and determined that these plans included  safety measures that were comparable to those in the ETASU  measures for the reference standard drug. For example, if the  reference standard drug’s ETASU required protections to prevent fetal  exposure to the drug, the generic company’s plans should include the  same protections. The safety letter would also note that FDA would  not consider it a REMS violation to provide reference standard drug  samples to the generic company requesting the safety determination  letter. According to FDA, some reference drug companies were  concerned that providing samples to the generic drug company would  violate REMS requirements.

From 2016 to 2018, FDA issued 12 safety determination letters to  reference drug companies on behalf of generic companies, according  to agency data. However, FDA did not issue a safety determination  letter for all of the requests it received. According to FDA officials,  there are various reasons why they might not issue a safety  determination letter to the reference drug company. For example, a  generic company must sign a disclosure form in order for FDA to send  the letter to the reference drug company, but the generic company  does not always choose to do this. Additionally, the generic  company might have withdrawn its request for a safety determination  letter, or FDA might be waiting for additional information from the  generic company in order to complete its review. According to FDA  officials, there is no need for a safety determination letter (which  assures the reference drug company that providing samples to the  generic drug company will not be considered a violation of their  REMS) when there is no REMS for the product in the first place.

Drug Companies’ Perspectives on Limited  Access to Samples of Reference Standard  Drugs Not Subject to Risk Evaluation and  Mitigation Strategies (REMS)  Officials from three of the four generic  companies in our review told us they had  experience with drug companies’ imposed  distribution limits on reference standard drugs  not subject to REMS. Of the four generic  companies in our review, officials from one  company said they were not able to obtain the  samples they needed and chose not to  pursue developing a particular drug.   Officials from one of the five reference drug  companies said they have limited distribution  for reference standard drugs not subject to  REMS. They said their companies do so to  ensure that their products are efficiently  distributed, in part by using certain  pharmacies.

FDA published a web page with information about inquiries that  included drugs both subject to and not subject REMS. In  February 2019, FDA published a web page with information on  inquiries made to FDA by generic companies seeking to obtain  samples of reference standard drugs in order to develop generic  drugs. (See sidebar for drug companies’ perspectives on this  practice.) FDA officials said they published this list to increase  transparency about continuing issues related to accessing samples  and to raise awareness about the potential effect these issues might  have on reducing competition in the drug market. This list included  drugs subject to and not subject to REMS, the names of reference  drug companies, and the number of inquiries made. According to the  web page as of February 2019, inquiries were made for 54 reference  standard drugs, including 25 drugs with ETASU-related limited  distribution measures and 29 drugs without such measures.  According to FDA data, the number of inquiries had been generally  decreasing in the years prior to when the list was published.

FTC officials reviewed inquiries the agency received from FDA  and generic companies and filed two briefs. FTC told us it  reviewed inquiries the agency had received from generic companies  and FDA, including those related to information on FDA’s published  web page. However, FTC officials said, to date, they have not brought  a case charging a reference drug company with violating federal  antitrust law for refusing to provide samples to a generic drug  company. In order to take enforcement action, FTC needs to find  sufficient evidence of activity that violates the Federal Trade  Commission Act or the Sherman Act. For example, FTC would need  to find that a reference drug company’s practice constituted  monopolization in violation of the Sherman Act. According to FTC  officials, they have not brought any antitrust cases to the courts, but  have filed two amicus briefs related to cases involving drugs subject  to REMS. In both of these briefs, FTC noted that the generic  companies’ respective allegations, if true, established an antitrust  violation and that the generic companies’ lawsuits should be allowed  to continue.

Drug Companies’ Perspectives on  Negotiating Required Shared Systems  Officials from one generic company said that  the respective reference drug companies  would not meet with them to negotiate the  development of a required shared system  REMS. Officials from another generic  company said the negotiation process with the  reference drug company lasted almost 2  years.    Officials from four of the five reference drug  companies we interviewed had experience  negotiating required shared systems. Officials  from three of these four companies told us  that developing a shared system is a difficult,  challenging, and complex process. Officials  from one reference drug company said that  the level of complexity can increase based on  the number of companies and the different  people involved.

FDA issued waivers that allowed generic drug companies to  develop a separate system from the REMS of the reference listed  drug and issued draft guidance on how to obtain such a waiver.  According to FDA, since 2007, the agency has received 13 requests  for a waiver from the shared system requirement, and at the time of  our data collection and analysis, FDA had approved three, the first in  2013. (See sidebar for drug companies’ perspective on required  shared systems.) These waivers allowed the generic drug company to  develop a separate system that includes the same ETASU measures  required for the reference listed drug. According to officials, FDA was  unable to grant the remaining waivers for different reasons. For  example, the agency may still be reviewing the generic drug  application submitted by the company that requested a waiver.  Officials explained that the waiver request is part of the overall generic  drug application and the agency cannot approve a waiver without  approving the application as well. To further facilitate the process, in  2018, FDA issued draft guidance describing what factors the agency  considers when granting waivers. The statute authorizes FDA to  grant a waiver (1) if the burden of creating a required shared system  outweighs the benefit of having it, taking into account the impact on  the health care providers, patients, and drug companies involved or  (2) if an aspect of the ETASU is covered by an unexpired patent or  entitled to trade secret protection, and the generic company was  unsuccessful in obtaining a license for use. FDA’s guidance  describes examples of the potential benefits of having a shared  system and the burdens of forming a shared system on health care  providers, patients and drug companies that FDA will consider. For  example, having a shared system could benefit drug companies by  making a REMS for multiple products more efficient. In contrast, the  drug companies negotiating a required shared system could be  market competitors and involved in patent litigation related to the drug  product.

Selected Drug Companies Had Differing Views on the Usefulness of FDA’s and FTC’s Efforts

In general, the four generic drug companies and five reference drug  companies we interviewed disagreed on the usefulness of FDA’s and  FTC’s efforts to address the practices that may affect the development of  generic drugs.

FDA’s safety determination letters. Officials from three of the  generic companies in our review said that the safety determination  letters were not useful because they were not enforceable and did not  require a reference drug company to provide a generic company with  samples of a reference standard drug. In its comments on FDA’s draft  guidance on obtaining a safety determination letter, one stakeholder  representing generic companies expressed concern that reference  drug companies now use safety determination letters as another  requirement to obtain samples.

In contrast, officials from three reference drug companies we  interviewed told us that FDA’s safety determination letters addressed  their safety concerns regarding sharing samples of reference standard  drugs with generic companies. Further, officials from two of these  three reference drug companies said they request these letters from  generic companies that request samples of reference standard drugs.  Officials from the remaining two reference drug companies we  interviewed said they were not aware of FDA’s safety determination  letters or did not have concerns or a position on the issue.

FDA’s publication of its web page. Officials from one of the four  generic companies we interviewed told us they thought the inquiries  web page published by FDA was helpful. However, this same  company said it had not noticed a significant effect in being able to  access samples of reference standard drugs because of the web  page. Officials from another generic company said it was too early to  tell about the usefulness of FDA’s web page. Of the remaining two  generic companies, officials from one company were unaware of the  web page and officials from the second company noted that they were  uncertain why a generic company would be included in the list of  companies on FDA’s web page.

Officials from two of the five reference drug companies we  interviewed, and whose companies appeared on the web page, said  they were unaware of any inquiries made to their companies  requesting samples of reference standard drugs. Additionally, officials  from one company told us they did not know why they were on FDA’s  published web page because the company had sold the reference  standard drug to another company and had informed FDA that this  had occurred.

According to FDA, the web page reflects the owner of the reference  standard drug at the time the agency received an inquiry, regardless  of whether the drug was later sold. Additionally, some generic  companies might contact FDA directly without contacting the  reference drug company because they anticipate having difficulties  accessing samples of the reference standard drug. FDA notes on its  web page that the agency did not independently investigate or confirm  the access limitations described in the inquiries it received.

FTC’s filing of amicus briefs. Officials from two of the generic  companies in our review said FTC’s filing of amicus briefs was  generally a positive step. Officials from two companies said the  amicus briefs helped negotiations with reference drug companies. A  third generic company said the amicus briefs helped raise awareness  about issues generic companies are having. Officials from a fourth  generic company said FTC’s actions could impact the company’s  efforts to develop generic versions of reference listed drugs in the  future. Officials from the five reference drug companies we  interviewed did not have any comments on FTC’s specific amicus  briefs.

Waivers for a required shared system. Officials from three of the  four generic companies we spoke with had experience with waivers.  Officials from one of these three companies said the waiver guidance  was helpful. However, officials from this generic company and a  second company said it took FDA almost a year to grant their waivers.  According to officials from a third company, they obtained their waiver  within a month, in part, because negotiations had been ongoing for  more than a year. According to FDA officials, the review of a waiver  request is part of the generic drug company’s drug application. FDA  will not grant a waiver unless the generic drug company meets the  waiver requirements and its generic drug application is approved.

Reference drug companies and other stakeholders expressed  concerns about these waivers. Officials from three of the five  reference drug companies we spoke with said the burden on health  care providers or patients should be considered when granting  waivers. Officials from one company specifically expressed concerns  that as FDA grants additional waivers, it could place an additional  burden on the health care system. For example, health care providers  could be required to use multiple systems to access REMS  information on the drug’s risks or to complete administrative  requirements, such as required certification. The remaining reference  drug companies did not have comments on the topic.

In comments we reviewed on FDA’s draft guidance on these waivers,  stakeholders noted concerns similar to those raised by the reference   drug companies. For example, two groups representing pharmacists  and pharmacies said that if FDA grants additional waivers, it could  place a burden on the health care system. Historically, FDA has  attempted to limit the number of required shared systems created  under waivers. If a generic drug company is granted a waiver, it is  allowed to create a separate system that includes the same ETASU  measures required of the reference listed drug. However, to date,  FDA has only granted waivers to generic drug companies that agree  to share their systems with other drug companies that concurrently or  subsequently develop generic or brand versions of the same  reference listed drugs.

We provided a draft of this product to FDA and FTC for their review and  comment. Both agencies provided technical comments, which we  incorporated as appropriate.

As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of HHS, and other interested  parties. In addition, the report is available at no charge on the GAO  website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or dickenj@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

Appendix I: 2017 Medicare and Medicaid Spending on Reference Standard Drugs Subject to REMS

In order to estimate Medicare and Medicaid spending on reference  standard drugs subject to risk evaluation and mitigation strategies  (REMS), we compared data from the Food and Drug Administration  (FDA) on drugs subject to REMS as of March 18, 2019, to publicly  available 2017 data on the Medicare Part D Drug Spending Dashboard  and the Medicaid Drug Spending Dashboard, maintained by the Centers  for Medicare & Medicaid Services (CMS). These data covered drug  spending and utilization for both of these programs for calendar year  2017, the most current data available. However, not all spending and  utilization data for reference standard drugs subject to REMS were  available. Since we analyzed data as of March 18, 2019, we were able to  identify 139 reference standard drugs subject to REMS with  corresponding cost data. To assess the reliability of these data, we  interviewed knowledgeable agency officials. We determined that the data  were sufficiently reliable for the purposes of our report.

Medicare and Medicaid paid at least $11.8 billion in 2017 for reference  standard drugs subject to REMS, according to cost data available from  the CMS’s drug pricing dashboard. Specifically, Medicare Part D paid at  least $8.5 billion for reference standard drugs subject to REMS. This  amount—which includes Medicare Part D plan sponsors and beneficiaries  Part D payments such as copays, but not price concessions, such as  manufacturers’ rebates—accounted for at least 8 percent of all Medicare  drug spending in 2017. Similarly, Medicaid paid at least $3.3 billion for  reference standard drugs subject to REMS, or at least 15 percent of all  Medicaid drug spending in 2017.

Of the 139 reference standard drugs in our analysis, the greatest share of  these programs’ spending, across medical conditions, was on reference  standard drugs subject to REMS for cancer, based on our analysis of  available data. Specifically, Medicare and Medicaid spent at least $4.6  billion on 8 reference standard drugs that treat cancer. See table 3 below  for Medicare and Medicaid spending for reference standard drugs subject  to REMS by medical condition treated.

Further, our analysis of available data showed that Medicare and  Medicaid spent the most on Revlimid, a drug used to treat cancer, totaling  $3.6 billion with Medicare accounting for $3.3 billion of this total. More  than 37,000 Medicare beneficiaries used this drug, at an average cost per  dosage unit of $626.94. (See table 4.)

In contrast, Medicaid spent the most on Suboxone, a drug used to treat  opioid dependence, totaling $0.7 billion, based on available data. More  than 3 million Medicaid claims were filed for this drug in 2017, at an  average cost per dosage unit of $7.89. Vivitrol was the third most utilized  drug under Medicaid. (See table 5.)

Based on our analysis of available data, the selected examples of  reference standard drugs subject to REMS had higher average cost per  dosage unit compared to the generic. For example, Medicare spent an  average cost per unit of $12.20 for Clozaril, a drug used to treat mental  health conditions, compared to $0.99 for clozapine, a generic version of  Clozaril. Table 6 below shows selected examples comparing Medicare  spending for reference standard drugs to Medicare spending for a generic  version, based on our analysis of available data.

Our analysis of available Medicaid data showed similar results to our  analysis of Medicare data. For example, Medicaid spent an average cost  of $11.71 for Clozaril, a drug used to treat mental health conditions,  compared to $0.97 for clozapine, a generic version Clozaril. Table 7  below shows selected examples comparing Medicaid spending for  reference standard drugs to Medicaid spending for a generic version,  based on our analysis of available data.

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Geri Redican-Bigott and Tom  Conahan, Assistant Directors; Carolyn Garvey, Analyst-in-Charge; Zhi  Boon, Gay Hee Lee, and McKenna Storey made key contributions to this  report. Also contributing were Sam Amrhein, Kaitlin Farquharson, Cathy  Hamann, and Diona Martyn.

The Food and Drug Administration can require drug companies to establish risk-reduction strategies for drugs with serious safety concerns to help ensure their benefits outweigh their risks. For example, a company’s strategy may include restricting a drug’s distribution to prescribers with special training.

Some strategies like these have been used to keep drugs out of the hands of companies looking to develop generic versions of them, according to FDA and the Federal Trade Commission.

In response, FDA and FTC have tried to make drug samples easier to obtain. Generic drug company representatives said some of these efforts have had mixed results.


Defense Acquisitions: Observations on the F-35 and Air Force's Advanced Battle Management System

In 2018, the F-35 program began operational testing. Also in 2018, the Air Force continued planning for the acquisition of ABMS, intended to modernize how DOD maintains command and control over and manages the future battlefield. Both the F-35 and ABMS are expected to play key roles in DOD's modernization efforts.

This testimony statement discusses (1) the F-35 program's development and modernization efforts, and progress in improving the aircraft's R&M and (2) DOD's current planning efforts for ABMS. This statement is based on two GAO reports on the F-35 published in April 2019 and on GAO's ongoing work examining ABMS. To conduct this work, GAO analyzed DOD management reports; discussed the efforts with program and contractor officials; and compared both efforts to DOD policy and GAO acquisition best practices.

The Department of Defense (DOD) wrapped up the F-35 development program in April 2018 and expects to complete operational testing in December 2019. DOD has turned its attention to modernization efforts—referred to as Block 4—to add new capabilities to address evolving threats. The program office estimates Block 4 to cost at least $10.5 billion through 2024. DOD plans to start Block 4 development without a complete business case identifying baseline cost and schedule estimates. Key documents for establishing a business case, such as an independent cost estimate, will not be ready before the program plans to award Block 4 development contracts in May 2019 (see figure).

Without a business case—consistent with acquisition best practices—program officials cannot be confident that the risk of committing to development has been reduced adequately prior to planned contract awards.

The program made slow, sustained progress in improving the F-35's reliability and maintainability (R&M). F-35 aircraft are assessed against eight R&M metrics, which inform how much time the aircraft will be in maintenance rather than operations. Half of these metrics are not meeting targets. While the program office has a plan for improving R&M, its guidance is not in line with GAO's acquisition best practices or internal control standards as it does not include specific, measurable objectives, align improvement projects to meet those objectives, and prioritize funding to match resources to R&M requirements. If the R&M requirements are not met, the warfighter will have to settle for a less reliable and more costly aircraft than originally planned. This contributes to the F-35's $1.12 trillion estimated sustainment costs and challenges with maintaining an expanding fleet that also has supply chain and logistics system problems.

GAO's ongoing work indicates that the Air Force's Advanced Battle Management System (ABMS)—intended to provide battle management command and control and surveillance across air, land, and sea—is in the early stages of planning. The capabilities and the strategy to deliver those capabilities are still to be determined. The Air Force plans to manage ABMS as a family of systems, integrating sensors from existing and future weapons programs, and overseen by a Chief Architect—whose role is still to be determined. The Air Force expects to further define ABMS after analyzing different options for delivering the capability. That analysis is expected to be complete in summer 2019.

In April 2019, GAO recommended that the F-35 program office complete its Block 4 business case before making more contract awards. DOD did not concur, citing that it has adequate knowledge to begin Block 4 development. GAO maintains that completing its business case before awarding its Block 4 development contracts would put DOD and the program in a better position to successfully develop Block 4 capabilities. GAO also recommended that DOD take action to improve its R&M performance. DOD concurred and noted the actions it would take.

Thank you for the opportunity to discuss our work on the F-35 Lightning II  program and the Air Force’s Advanced Battle Management System  (ABMS). The Department of Defense’s (DOD) 2018 National Defense  Strategy outlines its strategic approach to build a more lethal force, which  includes modernizing key capabilities to address future threats. Both the  F-35 and the ABMS are expected to play a key role in DOD’s  modernization efforts.

DOD is now in its 18th year of developing the F-35—a family of fifth- generation strike fighter aircraft for the United States Air Force, Marine  Corps, and Navy, as well as eight international partners. The F-35  program has developed and is delivering three variants; the F-35A  conventional takeoff and landing variant for the Air Force, the F-35B short  takeoff and vertical landing variant for the Marine Corps, and the F-35C  carrier-suitable variant for the Navy. The F-35 is DOD’s largest acquisition  program in U.S. military history, with total acquisition costs expected to  exceed $406 billion to develop and procure more than 2,400 aircraft  through fiscal year 2044. In addition, the Office of the Secretary of  Defense’s (OSD) Cost Assessment and Program Evaluation office  estimates the F-35 program’s sustainment costs to operate and maintain  the fleet over the next 52 years to be $1.12 trillion.

ABMS is in the early planning stages and is intended to change the way  the Air Force provides battle management command and control  capabilities across air, land, sea, and space. ABMS would shift the  emphasis from the current capabilities delivered by manned battle  management platforms, such as the Airborne Early Warning and Control  System (AWACS) and Joint Surveillance and Target Attack Radar  System (JSTARS), to an integrated network of sensors providing new and  upgraded capabilities.

This testimony is based on two reports we issued on the F-35 program in  April 2019, our past F-35 work, and an oral briefing we provided to  Congress on ABMS in February 2019. This statement (1) assesses the  F-35 program’s development progress, plans for modernization, and  sustainment and supply chain efforts; and (2) describes DOD’s current  planning efforts for ABMS.

To assess the F-35 program’s development and modernization plans, we  reviewed program development documents as well as modernization  planning documents that should be completed prior to awarding a  development contract, according to GAO best practices. We interviewed  DOD officials and contractor representatives regarding the program’s  development and modernization efforts. We analyzed monthly contractor  reliability and maintainability (R&M) reports and compared these to  program requirements. We reviewed the F-35 R&M Improvement  Program—the program’s plan to improve R&M metrics’ performance. We  also interviewed DOD officials and contractor representatives regarding  the program’s R&M performance. We also summarized our past findings  on the F-35 program’s supply chain and sustainment efforts.

To describe the Air Force’s ABMS planning efforts, we reviewed and  analyzed DOD program and planning documents. We also interviewed  officials from DOD, including the Office of Cost Assessment and Program  Evaluation, the Office of the Assistant Secretary of the Air Force for  Acquisition, Technology and Logistics, Air Combat Command, and  multiple Air Force program offices.

We assessed the reliability of the information we gathered regarding the  F-35 program and ABMS by reviewing supporting documentation and  interviewing knowledgeable officials. Based on these steps, we  determined that all the data we used were sufficiently reliable for the  purposes of this written statement. We discussed the information in this  written statement with DOD officials and incorporated their comments as  appropriate. The work on which this statement is based, has been  conducted in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

F-35 Modernization, Reliability, and Sustainment and Supply Chain Efforts Face Risks and Challenges

The F-35 plays a key role in DOD’s modernization efforts. However, it  faces concerns in several areas that will inform the program’s cost and  performance in the future. These include the risk in its modernization  efforts, its aircraft not meeting all reliability targets, and sustainment and  supply chain challenges. Specifically, the F-35 program plans to award  Block 4 development contracts before it has key business case  documents that would normally inform this decision. Also, the program is  not meeting all of its Reliability and Maintainability (R&M) targets. Finally,  the F-35 program’s sustainment costs are rising as it also faces  significant supply chain challenges.

The F-35 Program Will Start Block 4 Development without a Full Business Case

The F-35 baseline aircraft program completed development in April 2018.  It started formal operational testing of the baseline aircraft in December  2018 after a 3-month delay. This testing was delayed for two main  reasons: (1) to resolve critical deficiencies identified in developmental  testing, and (2) to accommodate an unexpected grounding following the  crash of an F-35B in September 2018. According to a test official, the  program expects to complete testing in December 2019, about 3 months  later than planned due to delays with the simulator that is used for more  complex testing. Until that testing is complete, there is still a risk that  additional deficiencies may be identified. With the program wrapping up  development of the baseline program, it is transitioning to early  development and testing activities for the Block 4 modernization efforts,  which the F-35 Joint Program Office estimates will cost about $10.5  billion. With Block 4, DOD plans to add new capabilities and modernize  the F-35 aircraft to address evolving threats.

In April 2019, we found that DOD will not have a complete business case  for Block 4 before it plans to award development contracts in 2019.  Section 224 of the National Defense Authorization Act for Fiscal Year  2017 required DOD to submit a report containing certain elements of an  acquisition program baseline—in essence, a business case—to include  cost, schedule, and performance information and independent  estimates—for Block 4. In 2018, we found that DOD’s report to Congress  was incomplete. In its report, DOD stated that the acquisition program  baseline would continue to be refined over the next year. DOD officials  stated that the updated F-35 program baseline, with the Block 4 efforts  included, will be released in April 2019.

Over the past year, the program has already invested over $1.4 billion, in  part to gain the knowledge it needs to develop that business case, such  as a preliminary design review, as well as to establish Block 4 testing  facilities and support early capabilities’ development. The program  incorporated some Block 4 activities into its acquisition strategy, which  was approved in October 2018. However, we found that three key Block 4  business case documents will not be ready before the program’s planned  development contract awards in May 2019:  Independent Technology Readiness Assessment: A Technology  Readiness Assessment is a systematic, evidence-based process that  evaluates the maturity of hardware and software technologies critical  to the performance of a larger system or the fulfillment of the key  objectives of an acquisition program. The program office plans to  conduct a partial independent assessment of initial capabilities  sometime between October and December 2019 with additional  assessments to follow. A program official stated that technologies will  not be integrated into the aircraft until they are adequately mature.  However, without a complete independent Technology Readiness  Assessment, the program will not have identified potential critical  technology elements and, as a result, may be at risk of delaying the  delivery of new capabilities.

Test and Evaluation Master Plan: Although the F-35 program has  begun testing Block 4 capabilities, it does not have an approved Test  and Evaluation Master Plan. The Test and Evaluation Master Plan  documents the overall structure, strategy, and objectives, as well as  the associated resources needed for execution. Developmental and  operational test officials have expressed concerns about the lack of  an approved test plan, uncertain funding, the number of test aircraft  available, and the draft test schedule, among other things. An  approved, properly resourced test plan is essential for planning and  preparing for adequate testing of the Block 4 capabilities. According to  these officials, without an approved plan, the F-35 program is  providing the test authorities with capabilities to be tested without  giving them the necessary direction on how to adequately prepare to  conduct the tests, making it difficult to execute testing. While this is  still a concern, F-35 program officials explained that over the past 3  months they have been providing the test authorities with the direction  needed to conduct testing.

Independent Cost Estimate: The Block 4 Independent Cost  Estimate, which details the program’s total estimated life cycle cost, is  not complete. In August 2017, we reported that DOD estimated the  development funding needed for the first phase of Block 4 was  projected to be over $3.9 billion through 2022. Since then, the  program incorporated more fidelity and specific Block 4 efforts that  were not in the original estimate into its Block 4 cost estimate. Based  on the program office’s latest estimate, the cost of Block 4 capabilities  is expected to be $10.5 billion through 2024. According to OSD’s Cost  Assessment and Program Evaluation office, it will provide the  Independent Cost Estimate between October and December 2019 to  support the F-35 program’s pending full-rate production decision, but  this would occur several months after the program plans to award the  Block 4 development contracts. According to the GAO Cost Guide, an  Independent Cost Estimate is considered one of the best and most  reliable estimate validation methods as it provides an independent  view of expected program costs that tests the program office’s  estimate for reasonableness. Without an Independent Cost Estimate,  Congress does not have insight into the full potential cost of Block 4.

The expected completion dates for these documents are between  October and December 2019, at the earliest. Figure 1 shows key Block 4  dates such as the Block 4 re-plan, which included revising the cost  estimate for Block 4 that DOD established in 2017, the planned  development contract awards, and planned completion dates for the three  remaining critical business case documents.

As seen in figure 1, the program office plans to award Block 4  development contracts in May 2019, at least five months before any of  the critical business case documents will be available. Based on best  practices identified by GAO, without an independent Technology  Readiness Assessment, Test and Evaluation Master Plan, or an  Independent Cost Estimate, program officials cannot have a high level of  confidence that the requirements are firm and that risk has been  adequately reduced before beginning efforts estimated to cost $10.5  billion in funding to develop Block 4. If program officials move ahead with  Block 4 contracts without gaining the knowledge that a full business case  would provide, Block 4 modernization efforts will be at risk of experiencing  the same kind of cost and schedule growth the baseline development  program experienced.

To address this risk, in April 2019, we recommended to the DOD that it  should ensure the F-35 program office complete its business case, to  include the three documents discussed above, at least for the initial Block  4 capabilities under development before initiating additional development  work. DOD did not concur with this recommendation. In its comments,  DOD stated that the F-35 program office has adequate knowledge to  begin Block 4 development. We maintain, however, that completing its  business case before awarding its Block 4 development contracts would  put DOD and the program in a better position to effectively and  successfully develop Block 4 capabilities.

The F-35 Program Is Still Not Meeting All Reliability and Maintainability Targets

As we reported in April 2019, the program has made slow, consistent  progress in improving the F-35’s R&M metrics’ performance but half of  the metrics are not achieving targets. All F-35 variants are generally  performing near or above targets for four of the eight R&M metrics, while  still falling short for the other four. Each F-35 aircraft variant is measured  against eight R&M metrics, four of which are in part of the contract. All  eight R&M metrics are described in the program’s Operational  Requirements Document (ORD)—the document that outlines the targeted  performance levels for these metrics that DOD and the military services  agreed the F-35 should meet in 2000. Based on our analysis, while the  program is on track to meet half of the targets, the program office has not  taken adequate steps to ensure the others will be met. Additionally, in  December 2018, the Director, Operational Test & Evaluation reported  that, although performance for the four under-performing metrics has  shown slow growth over the years, none of these metrics are meeting  interim goals needed to reach requirements at each variant’s maturity.  Each F-35 variants’ R&M performance against these metrics is shown in  table 1.

Since the program began tracking R&M performance in 2009, it has seen  small, annual improvements. Over the past year, all variants showed a  slight improvement in targeted performance levels for one metric, the  mean flight hours between failure—design controlled, but saw little or no  discernable improvement for the four metrics not meeting targets.  However, based on current performance, the program does not expect to  meet those targets by full aircraft maturity. According to F-35 program  officials, the ORD R&M metrics should be re-evaluated to determine more  realistic R&M performance metrics, but the program has not yet taken  actions to do so. Until the program office does so, it remains accountable  for ensuring those ORD R&M metrics are achieved.

In June 2018, we recommended that the F-35 program identify steps it  needs to take to ensure the F-35 aircraft meet R&M requirements before  each variant reaches maturity and update its R&M Improvement Program  (RMIP)—DOD’s action plan for improving R&M—with these steps. DOD  concurred with our recommendation but has yet to take substantive  actions to address it. DOD did, however, complete 16 improvement  projects since we last reported on this. Despite completing these projects,  there were not significant gains in the R&M metrics not meeting targets.  Program officials advised, however, that measurable improvements in  R&M can take time to manifest. To speed this process, the program is  accelerating planned upgrades to older aircraft where appropriate, which  officials stated should translate to an overall improvement in the  program’s R&M performance.

The F-35 Program Office’s Improvement Plan Does Not Address Under- Performing Targets

The F-35 program office has estimated that implementing all of the  identified improvement projects currently contained in its RMIP could  result in potential life cycle cost savings of over $9.2 billion by improving  the F-35’s R&M. However, we found that, as of December 2018, the  guidance the F-35 program office has used to implement the RMIP does  not define specific, measurable objectives for what the desired goals for  the F-35’s R&M performance should be or align improvement projects  with R&M goals. Furthermore, the RMIP has not been a funding priority.

Federal internal control standards state that programs should define  objectives when implementing programs such as the RMIP. Although  the F-35 program RMIP’s guidance has a general goal of improving R&M,  it does not identify achieving the eight R&M targets listed in the ORD as  an objective. Program officials acknowledged that the RMIP’s guidance  does not include such an objective. Instead, officials stated they are using  the RMIP to prioritize and fund projects that will improve aircraft  availability and mission capability—neither of which are included in the  eight R&M metrics, but are necessary and important initiatives. The  program is focusing on these two areas in part because a September  2018 memorandum from the Secretary of Defense to the Secretaries of  the military departments included a goal for the F-35 fleet to attain a  mission capable rate of 80 percent by the end of fiscal year 2019.  According to program officials, improving these two areas will translate  into improvements in the F-35 overall R&M. However, we found that the  RMIP’s guidance does not discuss these priorities or align how any  improvement projects would ensure targets under all eight R&M targets  will be met.

In our prior work on weapon system acquisitions, we have identified a  number of best practices for improving program outcomes if implemented,  such as clearly establishing well-defined requirements and securing  stable funding that matches resources to requirements. We found that  the program office has not prioritized or dedicated funding in its budget to  improve R&M, in part because program officials explained that they were  focused on initiatives intended to lower the cost of the aircraft. In  addition, any current funding for R&M improvement projects comes from  the program’s operation and maintenance funds, which are only available  for one fiscal year. Officials explained that, if the funding runs out or is  used by the program for other efforts, then R&M projects will go unfunded  or be suspended until new funding is available. In fiscal year 2018, for  example, while some projects were completed, several other projects  were suspended when that year’s funding ran out. As of December 2018,  according to a contractor representative, all of the identified improvement  projects currently unfunded in the program’s RMIP would cost about $30  million to implement, but were not funded.

Program officials also stated that they are in the process of revising the  RMIP and have considered including more specific objectives in addition  to improving aircraft availability and mission capability, such as more  focus on improving R&M performance where ORD R&M targets are not  currently being met. According to the program, any revisions to the RMIP  and changes to how it will be funded, however, will not be complete until  April 2019.

By not defining objectives in its RMIP guidance for meeting all eight R&M  metrics, aligning which improvement projects will ensure those metrics  are met, and prioritizing funding for those projects, the program is at risk  of not fully meeting its R&M targets. As a result, the warfighter may  accept aircraft that are less reliable than originally planned, and whose  operation and sustainment costs may raise affordability questions. In  addition, the military services recently identified the need to cut  sustainment costs—by 43 percent in the case of the Air Force—to  improve the F-35’s affordability in sustainment. Increasing costs from less  reliable aircraft will add strain to an already unaffordable program.

To address these issues, in April 2019, we recommended to DOD that it  should ensure that the F-35 program office  1.  assess whether the ORD R&M targets are still feasible and revise the  2.  as it revises its RMIP, identify specific and measurable R&M  objectives in its RMIP guidance;  3.  as it revises its RMIP, identify and document which RMIP projects will  achieve the identified objectives of the RMIP guidance; and  4.  prioritize funding for the RMIP.

DOD concurred with these recommendations and stated that it will take  actions to address them.

Continued Concerns with F-35 Sustainment Costs and Supply Chain, and Logistics System

We have previously reported on the F-35 program’s rising estimated  sustainment costs and challenges maintaining an expanding fleet. In  October 2017, we reported that estimated F-35 life-cycle sustainment  costs increased by 24 percent from fiscal years 2012 through 2016 due to  an increase in projected flying hours and other factors. We also reported  that sustainment costs were not fully transparent to the military services.  For example, the Marine Corps received an initial funding requirement for  fiscal year 2017 sustainment of $293 million, which then increased to  $364 million in the execution without a full explanation from the program  office. We recommended that DOD take steps to improve communication  with the services and provide more information about how F-35  sustainment costs they are being charged relate to the capabilities  received. DOD concurred with the recommendation and has begun taking  actions to address it.

In addition, DOD faces substantial supply chain challenges that are  lowering F-35 aircraft performance. In April 2019, we reported that F-35  aircraft performance was falling short of warfighter requirements—that is,  aircraft could not perform as many missions or fly as often as required.  Specifically, F-35A aircraft were mission capable only 52 percent of the  time from May through November 2018—far short of the 80 percent target  set by the former Secretary of Defense. This lower-than-desired aircraft  performance is due largely to F-35 spare parts shortages and limited part  repair capabilities. For example, during this time period, F-35 aircraft were  unable to fly about 30 percent of the time due to spare parts shortages.

Additionally, DOD’s capabilities to repair F-35 spare parts at its depots  are years behind schedule, which has resulted in an average of 188 days  to repair an F-35 part and a backlog of about 4,300 spare parts awaiting  repair at military depots or manufacturers. We also reported that DOD  faces challenges managing, moving and maintaining accountability of F- 35 parts within the supply chain. We made eight recommendations to  DOD, including that DOD determine what actions are needed to close the  gap between warfighter requirements for aircraft performance and F-35  supply chain capabilities. DOD concurred with the recommendations and  identified actions that it was taking or planned in response.

Finally, the F-35’s Autonomic Logistics Information System (ALIS) has the  potential to lead to increased costs for the program if key issues are not  addressed. ALIS is the F-35’s central logistics system intended to support  operations, mission planning, supply-chain management, maintenance,  and other processes. In April 2016, we identified several risks, including  that ALIS (1) was not initially designed to be deployable, (2) lacked  redundant infrastructure, (3) did not communicate well with legacy aircraft  systems, (4) had data accuracy and accessibility issues, and (5) had  security risks. In addition, DOD had not included certain analyses and  information, such as historical cost data, to increase the credibility and  accuracy of ALIS’s estimated costs. Further, a 2013 DOD-commissioned  study found that schedule slippage and functionality problems with ALIS  could lead to between $20 billion and $100 billion in additional costs.

We have made several recommendations to DOD to improve ALIS  planning and cost estimates, and to develop a performance measurement  process for ALIS to better address problems based on actual system  performance and user requirements. DOD generally concurred with our  recommendations and has taken some actions, including developing a  plan that identifies and prioritizes key ALIS risks. However, more work  remains. We are currently conducting a review examining DOD’s  progress in implementing our ALIS-related recommendations, addressing  concerns from ALIS users, identifying emergent financial and operational  risks associated with ALIS, taking near-term actions to improve ALIS  functionality, and assessing DOD’s actions regarding the long-term  viability of ALIS to ensure capable sustainment of the F-35 fleet. We plan  to issue a report based on our current work later in 2019.

Air Force’s Advanced Battle Management System Acquisition Strategy Is in the Early Planning Stages

Based on our ongoing work, ABMS is early in the acquisition process, as  the specific capabilities and overarching acquisition strategy are still to be  determined by the Air Force. As a result, the Air Force has not yet  established a cost and technical baseline for ABMS. When ABMS  planning began in 2017, program officials stated that the intent of the  program was to replace and modernize the capabilities of the AWACS  system—which provides the warfighter with the capability to detect,  identify, and track airborne and maritime threats. But changes in Air Force  expectations for how it would fight during future conflicts led the  department to assess options for developing a more robust and  survivable air, land, and sea battle management system that can operate  in contested environments. In July 2018, the ABMS Initial Capabilities  Document—which describes capability needs and associated gaps—was  approved by the DOD Joint Requirements Oversight Council.

Our ongoing work also found that, in December 2018, the Air Force  determined it would not continue its planned JSTARS Recapitalization  program—which was intended to provide surveillance and information on  moving ground targets—well into the future, as initially expected. As a  result of a recent study, the Air Force has extended the estimated service  life of the JSTARS fleet, and will incorporate its capabilities into the ABMS  in the short term, and retire JSTARS in the 2030s.

Our preliminary observations indicate that the details about ABMS are still  to be determined. The Air Force expects to fully define ABMS through an  Analysis of Alternatives (AOA) that it plans to complete by the summer of  2019, as shown in figure 2.

The ABMS AOA, led by the Air Force’s Air Combat Command, will assess  how ABMS will deliver air-centric capabilities, such as those currently  provided by AWACS. Air Force officials explained that they plan to utilize  an existing AOA completed for the JSTARS Recapitalization program,  approved in May 2012, to identify and assess ABMS’s potential ground  target tracking capabilities. Originally planned as a 9-month study, Air  Force officials stated that the ABMS AOA was shortened to a 6-month  effort. As a result, the Air Force received conditional approval to reduce  the number of alternatives studied from five to three.

Our ongoing work indicates that the Air Force plans to develop ABMS  over three phases. The first phase began in fiscal year 2018 and goes  through 2023. In this phase, the Air Force plans to integrate existing  sensors, improve battle management systems, and upgrade  communication networks across 10 existing acquisition programs. Table 2  includes information on three existing programs the Air Force plans to  enhance during the first phase of ABMS.

According to an Air Force acquisition official, the technologies associated  with the first phase are considered to be mature but there may be risks as  the Air Force integrates technologies.

Air Force officials explained that their approaches to the second and third  phases of ABMS are not fully developed, but noted that the phases would  be informed by the AOA results. That said, the Air Force expects to start  phase 2 in 2024 by integrating advanced sensors and software into its  existing battle management command and control platforms while at the  same time retiring JSTARS. Air Force officials have reported that the third  phase, planned for the mid-2030s, is expected to provide multi-sensor,  resilient battle management command and control capability using  multiple types of communications methods, with an initial operational  capability planned for 2035. The Air Force estimates that ABMS’s  acquisition cost through fiscal year 2024 will be $3.8 billion.

Because ABMS is composed of many different defense acquisition  programs, the Air Force intends to manage it as a family of systems  directed by a Chief Architect and not a traditional acquisition program  manager. According to the Air Force, the ABMS Chief Architect is the first  of its kind, and the Air Force believes the position will be instrumental in  integrating the various programs and technologies into an overall system.  Based on our preliminary analysis, the roles and responsibilities of the  Chief Architect have not been fully defined. However, according to the Air  Force, the Chief Architect is expected to be responsible for (1) leading a  high-level analysis and determining the overall design of ABMS, (2)  coordinating with the service-level commands and the acquisition  programs involved to make sure they are aligned with the ABMS  development, and (3) identifying the enabling technologies for integration  into ABMS.

Chairman Norcross, Ranking Member Hartzler, and members of the  Subcommittee, this completes my prepared statement. I would be  pleased to respond to any questions you may have. We look forward to  continuing to work with the Congress as we to continue to monitor and  report on the progress of the F-35 program and the ABMS.

If you or your staff have any questions about this testimony, please  contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov.  Contact points for our Office of Congressional Relations and Public  Affairs may be found on the last page of this statement. Individuals  making key contributions to this statement are Justin Jaynes (Assistant  Director), Diana Maurer, Jennifer Baker, Desirée E. Cunningham, Alissa  Czyz, Stephanie Gustafson, Kasea Hamar, Jeff Hubbard, Jessica Karnis,  Matt Metz, Robin Wilson, and Lauren Wright.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

This testimony provides updates on 2 acquisitions: the F-35 fighter aircraft and the Advanced Battle Management System (ABMS), which is intended to replace the systems that currently track air, land, and sea targets.

The F-35 program is embarking on a $10.5 billion modernization effort through FY 2024 to meet evolving threats. We found that the program needs to complete its business case—the program’s estimated cost and proposed schedule—before beginning. The aircraft also has yet to meet half of its reliability and maintainability goals.

The Air Force is still determining the exact capabilities it would like ABMS to provide.