Military Depots: DOD Can Benefit from Further Sharing of Best Practices and Lessons Learned

DOD operates depots nationwide to maintain complex weapon systems and equipment through overhauls, upgrades, and rebuilding. These depots are crucial to sustaining military readiness by ensuring that the military services can regularly maintain critical weapon systems and return them to the warfighter for use in training and operations. For fiscal year 2018, DOD reported $19 billion in total maintenance expenditures and about 84,000 personnel performing depot-level maintenance. In June 2018, the Senate Armed Services Committee, in a report accompanying a bill for the National Defense Authorization Act for Fiscal Year 2019, included a provision for GAO to review DOD's sharing and implementation of best practices and lessons learned among the depots. GAO evaluated the extent to which DOD experiences benefits and has challenges with (1) sharing and (2) implementing best practices and lessons learned among the depots. GAO reviewed agency guidance; surveyed 17 depots; conducted site visits at five depots; and interviewed DOD, military service, and depot officials.

The Department of Defense (DOD) experiences benefits from sharing best practices and lessons learned among its depots, but communication and organization challenges exist. Best practices and lessons learned are shared among the depots through a variety of venues, including networking, working groups, and benchmarking trips to other depots. However, DOD has communication challenges, such as the lack of awareness of venues for sharing information. While Office of the Secretary of Defense officials reported posting a list of working groups, the list only contains three of the more than 60 working groups GAO identified. Without a centralized list of sharing venues and points of contact, it is unclear what groups exist and who to contact to participate, which may impede sharing of best practices and lessons learned. Further, while the Army stated it established lessons learned organizations for sharing maintenance best practices and lessons learned, it did not maintain them due to organizational restructuring and resource constraints. Establishing and maintaining effective organizations dedicated to sharing materiel best practices and lessons learned would encourage knowledge sharing among the Army depots.

DOD is experiencing benefits and taking steps to mitigate challenges with implementing best practices and lessons learned among the depots. Depots reported that implementing some best practices and lessons learned has led to benefits, including time and cost savings. For example, Navy Fleet Readiness Center Southwest, California, implemented an intermittent fault detection system from Ogden Air Logistics Complex, Utah, on its F/A-18 aircraft generators. According to officials, the depot reduced repair time from 90 days to 30 days and quadrupled the generators' time between failures. Depots reported a variety of challenges to implementing lessons learned and best practices, including a lack of resources, lengthy approval processes, and acquisition and technology restrictions. DOD is taking steps to mitigate challenges to implementation, such as creating a new technology tool for viewing metrics on weapon systems' cost and availability which will allow senior leaders to steer resources to needed programs.

GAO is making two recommendations to improve the depots' ability to share best practices and lessons learned by creating a comprehensive list of sharing venues, including points of contact, and re-establishing and maintaining materiel lessons learned organizations. DOD concurred with the recommendations.

The Department of Defense (DOD) operates depots nationwide to  maintain complex weapon systems and equipment through overhauls,  upgrades, and rebuilding. These depots are crucial to sustaining military  readiness by ensuring that the military services can regularly maintain  critical weapon systems and return them to the warfighter for use in  training and operations. For fiscal year 2018, DOD reported $19 billion in  total maintenance expenditures and about 84,000 personnel performing  depot-level maintenance. However, our prior work shows that DOD is  continually experiencing challenges at its depots, including deteriorating  equipment and facility condition, filling critical personnel skills, and  meeting service repair needs. These challenges can lead to delays in the  maintenance of weapon systems, which ultimately affects readiness by  impeding the military services' ability to conduct training and to provide  forces with sufficient equipment to perform operations around the world.  According to DOD officials, these challenges could be better addressed  within a culture of collaboration that shares best practices as well as  leaders and processes that foster a culture of assessment and feedback.

To address these challenges and learn more about DOD’s efforts to  share best practices and lessons learned, the National Defense  Authorization Act for Fiscal Year 2018 directed the Secretary of Defense  to submit to the congressional defense committees a ‘‘comprehensive  plan for the sharing of best practices for depot-level maintenance among  the military services.’’ In March 2018, DOD submitted a report to  Congress describing a number of groups, committees, and activities  related to a governance framework of joint collaboration. In June 2018,  the Senate Armed Services Committee, in a report accompanying a bill  for the National Defense Authorization Act for Fiscal Year 2019, stated  that it is not clear if DOD is effectively sharing and implementing best  practices and lessons learned identified by its individual depots. As such,  the Senate Armed Services Committee report included a provision for us  to review DOD’s sharing and implementation of best practices and  lessons learned among the depots. In this report, we examine the extent  to which DOD experiences benefits and has challenges with (1) sharing  and (2) implementing best practices and lessons learned among the  depots. This report is the first in a series of reports examining depot  maintenance requirements and timeliness for aviation, ground vehicles,  and naval shipyards.

To address these objectives, we reviewed relevant laws and DOD and  military service guidance that govern depot maintenance. We conducted  a survey of 17 DOD depots performing depot-level maintenance to gain  an understanding of how each depot shares with each other and  implements best practices and lessons learned. The response rate for  the survey was 100 percent. To gather detailed examples of DOD’s  efforts to share and implement best practices and lessons learned, we  visited a non-generalizable sample of five depots. To select our sample,  we considered variation in geographic location, military service  representation, and types of weapon systems maintained. At these sites,  we conducted group discussions with depot officials and maintainers to  gain insight into their roles in sharing and implementing best practices  and lessons learned.

Additionally, we interviewed officials from the Office of the Secretary of  Defense, military headquarters, military logistics or materiel components,  and military lessons learned centers. We reviewed our prior reports  related to challenges experienced at DOD depots and DOD’s report to Congress on the sharing of best practices for depot-level maintenance  among the military services. We obtained and analyzed documentation of  sharing, such as working group charters and trip reports documenting  results from visiting another depot, as well as benefits experienced from  implementing a best practice or lessons learned, including time and cost  savings. We assessed the documentary and testimonial evidence we  collected against DOD and military service guidance, as well as the  Standards for Internal Control in the Federal Government related to  information and communication. A detailed discussion of our scope and  methodology is in appendix I.

We conducted this performance audit from January 2019 to January 2020  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Depots are government-owned, government-operated industrial  installations that maintain, overhaul, and repair a multitude of complex  military weapon systems and equipment for the Department of Defense.  Depots are essential to maintaining readiness for DOD and play a key  role in sustaining weapon systems and equipment in meeting operational,  contingency, and training requirements. There are 17 depots operated by  the military services that perform depot-level maintenance on a wide  range of vehicles and other military assets, including aircraft, engines,  helicopters, combat vehicles, ships, and software. Five are Army depots,  four are Naval shipyards, three are Navy fleet readiness centers, two are  Marine Corps production plants, and three are Air Force air logistics  complexes. Figure 1 below shows the location of these 17 depots across  the United States.

The depots are part of a larger, DOD-wide logistics enterprise that  involves a number of different organizations (See fig. 2.).

Office of the Under Secretary of Defense for Acquisition and  Sustainment. This office is responsible for, among other things, ensuring  the defense industrial base, including depots, is robust, secure, resilient  and innovative.

Office of the Assistant Secretary of Defense for Sustainment. This  office serves as the principal assistant and advisor to the Under Secretary  of Defense for Acquisition and Sustainment on material readiness.  Among other responsibilities, the Assistant Secretary of Defense for  Sustainment prescribes policies and procedures on maintenance,  materiel readiness, and sustainment support.

Office of the Deputy Assistant Secretary of Defense for Materiel  Readiness. This office establishes and maintains maintenance policies  and programs to maintain the desired levels of weapon systems and  military equipment readiness to accomplish the Department's missions.  Further, according to DOD officials as well as DOD’s March 2018 report  to Congress on sharing best practices, the Office of the Deputy Assistant  Secretary of Defense for Materiel Readiness has established a  governance framework for materiel maintenance at DOD depots. There  are a number of stakeholders involved in this framework, including the  Maintenance Executive Steering Committee (Committee) and the Joint  Group-Depot Maintenance.

Maintenance Executive Steering Committee. This Committee consists  of senior maintenance and logistics representatives from the Office of the  Secretary of Defense, the Joint Staff, the Defense Logistics Agency, and  the military services. According to DOD, this Committee advises the  Deputy Assistant Secretary of Defense for Materiel Readiness on  initiatives affecting efficiency, effectiveness, and affordability of  maintenance management and operations. The Committee also serves  as a forum for a coordinated review of maintenance policies, systems,  programs and activities and helps optimize and steer DOD enterprise  maintenance practices and strategy.

Joint Group–Depot Maintenance. As a standing committee of the  Maintenance Executive Steering Committee, the mission of the Joint  Group–Depot Maintenance is to promote and review depot maintenance  functions at the enterprise level to achieve effective and affordable depot  maintenance support for weapon systems and to execute responsibilities  assigned in DOD maintenance of military materiel policy.

Military service organizations. Each military service has its own  logistics or materiel command component, which provides day-to-day  management and oversight of the military services’ depots.

DOD Guidance for Sharing Best Practices and Lessons Learned

The Chairman of the Joint Chiefs of Staff is responsible for formulating  policies for gathering, developing, and disseminating joint lessons learned  for the armed forces. Chairman of the Joint Chiefs of Staff (CJCS)  Instruction 3150.25G, Joint Lessons Learned Program, defines: best practice as “a validated method or procedure which has  consistently shown results superior to those achieved with other  means, and appears to be worthy of replication,” and   lesson learned as “a resolved issue or best practice that improves  operations or activities and results in an internalized change to  capability, process, or procedure.”

The Joint Staff’s Joint Lessons Learned Program collects, validates, and  disseminates lessons learned to support sustainment and improvement of  joint force readiness and effectiveness via refinements in doctrine,  organization, training, materiel, leadership and education, personnel,  facilities, and policy. Specific military service guidance on their  respective lessons learned programs share the same purpose. Best  practices and lessons learned are captured in the Joint Lessons Learned  Information System—DOD’s system of record for lessons learned—and  are generally focused on sharing operational information from after-action  reports and joint training exercises, rather than maintenance-related  lessons learned. The DOD maintenance community, including the military  service logistics or materiel command component and depots, do not  typically coordinate with the military services’ lessons learned centers or  enter lessons learned into the Joint Lessons Learned Information System.

Our prior work has identified multiple challenges that can affect depot  performance, including having the right facilities and having personnel  with the right skills, among other challenges (See fig. 3.).

Specifically, in April 2019 we reported on the condition of facilities at DOD  depots, such as the condition of these depots are poor and the age of  equipment is generally past its useful life, and the military services do not  consistently track the effect that these conditions have on depot  performance. To address these challenges, we recommended that DOD  improve its data collection on the effect of facilities and equipment  condition on depot performance, among other things. DOD concurred,  and stated, in general, that the Service Chiefs for the Army, Navy, Air  Force, and Marine Corps will ensure that their respective material  commands take actions to implement the recommendations for their  respective service. Also, in December 2018 we reported on depot  workforce challenges, such as hiring personnel in a timely manner and  providing inexperienced personnel with the training necessary to become  proficient in skilled operations. According to DOD officials, these  workforce challenges contributed to delays in the maintenance of some  weapon systems. To address these workforce challenges, we  recommended that the military services assess the effectiveness of the  actions they have taken to maintain critical skills in the depot workforce.  DOD concurred, and stated that each of the four services will take action  to assess the effectiveness of the hiring, training, and retention programs  at their respective depots, shipyards, fleet readiness centers, and air  logistics complexes. The Related GAO Products page at the end of this  report provides a list of our depot-related reports and testimonies.

DOD Experiences Benefits from Sharing Best Practices and Lessons Learned among the Depots, but Communication and Organization Challenges Exist

DOD Experiences Benefits from Sharing Best Practices and Lessons Learned among the Depots through a Variety of Venues

DOD shares best practices and lessons learned among the depots  through a variety of venues, including networking, working groups, and  benchmarking.

Networking. DOD shares best practices and lessons learned through  informal networking, such as personal contacts and conferences. All 17  depots reported engaging in networking to share best practices and  lessons learned and coordinating with their materiel commands, program  managers and/or program offices, and academia. The majority of the  depots also coordinated with industry, other depots, and/or a point of  contact or group within the Office of the Secretary of Defense (see table 1  below).

All 17 depots reported that the DOD Maintenance Symposium  (Symposium), an annual department-wide conference addressing the  maintenance of weapon systems and equipment, is the most regularly  attended and most beneficial venue for networking. All 17 depots  reported attending the Symposium regularly or occasionally, with depot  officials stating in the survey and interviews that the Symposium provides  opportunities to build relationships and network with peers in DOD and  external contacts in industry. Depots reported in our survey that the  Symposium was valuable because it offered opportunities to make  contacts with equipment vendors and other services, as well as break-out  sessions and informal discussions to exchange ideas. During the  Symposium, a number of maintenance awards, including the Robert T.  Mason Award for Depot Maintenance Excellence, are awarded to  recognize maintenance excellence (see sidebar). Three depots  reported that the recognition of the award-winning depots gives other  depots the opportunity to reach out to the award-winning depots for  relevant information.

This success has been shared with Fleet  Readiness Centers East and Southeast,  which are both implementing similar systems.  Successfully training new artisans is  particularly important for depot performance,  as our prior work has shown that this  workforce is aging and the Department of  Defense faces challenges in hiring and  retaining workers with key skills. Officials cited  examples of maintenance taking months or  years longer than expected, in part due to  shortages in skilled personnel.

Working Groups. DOD depots’ leadership and staff use working groups  and communities of practice as venues for the DOD maintenance  community to collaborate and to share expertise on specific topics.  When surveyed, 13 of 17 depots reported they share best practices and  lessons learned in working groups, and they identified more than 60 such  working groups. Our analysis of survey responses shows that depots  value working groups because they improve depot support to the  warfighter by allowing the depot to evaluate best practices, review new  technology, exchange data, initiate relationships, and gain stakeholder  support. In our interviews, depot officials affirmed the value of working  groups to promote collaboration and open discussions among peers  focused on specific topics of common interest.

We found that the working groups fall into three topic areas: new  technologies, specific weapon systems, and depot management. For  example:

New technologies. The Joint Technology Exchange Group was  chartered to improve coordination in the introduction of new or  improved technology, new processes, or new equipment into DOD  depot maintenance activities. To do this, the Joint Technology  Exchange Group facilitates a number of forums and working groups  centered on specific technologies, which allow representatives from  the depots to learn from other services, academia, and industry (See  fig. 4.). One example of this is cold spray, a new technology that  sprays high velocity metal particles to repair worn surfaces and  damaged parts that are unrepairable by traditional processes.  Working groups facilitated by the Joint Technology Exchange Group  have shared the usefulness of cold spray technology, and 12 depots  from all service branches reported that they have begun adopting the  technology. One depot estimates that its annual savings from using  cold spray will be $202,000 annually, as well as additional time  savings.

Weapon systems. According to Navy officials, depot officials and  maintainers for the CH-53E/MH-53E heavy lift helicopter participate in  the H-53 Fleet Support Team working group. Fleet Readiness Center  East reported that its production team was able to implement lessons  learned from this group for repairing misalignment in a piece of the  helicopter’s tail. As a result, the safety of the helicopter was  increased. See figure 5 for details on this heavy lift helicopter.

Depot management. Depot commanders participate in the Industrial  Base Commanders’ monthly teleconference to share best practices  and lessons learned related, in part, to management of depot  operations. Twelve of the 17 depots indicated that the Industrial Base  Commanders’ monthly teleconference is beneficial. The depots  reported that the Industrial Base Commanders’ monthly  teleconference allows base commanders time to share and to work on  specific depot maintenance problems and is particularly productive in  the areas of personnel and policy.

Benchmarking. To benchmark, depot officials visit another depot to  compare performance and find improvement ideas, particularly best  practices and lessons learned related to weapon systems and depot  management. Our analysis of site visit and survey data shows 10 of the  17 depots reported benchmarking trips. For example, in 2018 the Marine  Corps Albany Production Plant sent a team of managers and technicians  from their electronics and fabrications branches on a benchmarking trip to  learn best practices from the team at Tobyhanna Army Depot. They  visited six areas, where they observed processes and ideas that they  could take back to their plant. In its trip report, the Marine Corps Albany  Production Plant team highlighted a number of processes that increased  efficiency in the electronics shop at Tobyhanna Army Depot, such as  steps to eliminate unnecessary travel in sheet metal processes and  updated electronics workstations.

According to our prior work, benchmarking is useful for reducing internal  resistance to change—a barrier to sharing best practices and lessons  learned cited by the depots—because knowing what others actually are  accomplishing changes perceptions of what can be done and what should  be attempted. One depot told us that it intentionally brings maintainers  and depot officials together on benchmarking trips so that the maintainers  can benefit firsthand from seeing the best practices and lessons learned.

DOD Has Communication Challenges That May Hinder Ability of the Depots to Share Best Practices and Lessons Learned

DOD has communication challenges, such as the lack of awareness of  venues, that may hinder the ability of the 17 depots to share best  practices and lessons learned. While many sharing venues exist, such as  working groups, the depots’ knowledge of them has gaps. According to  our survey, 12 of the 17 depots reported being unaware of the existence  of some venues where best practices and lessons learned can be shared.  Additionally, 7 of the 17 depots reported not knowing who to contact to  participate in some venues for sharing best practices and lessons  learned. Moreover, in our interviews officials explained that staff turnover  is also a challenge. Specifically, officials from one depot said that when  the depot representative to a venue leaves, the institutional knowledge of  the venue and its point of contact can be lost. They recounted having to  resort to cold-calling other depots for information. Depots also reported  that their staff did not attend best practices and lessons learned venues  because they believed that those venues were for higher command  levels. For example, one depot expressed confusion about the Industrial  Base Commanders’ meeting and reported that while the depot officials  were aware of the meeting, they believed that it was for officials at a  higher level, such as their Materiel Command.

Department of Defense Instruction 4151.18 states that DOD materiel  maintenance programs should adopt business practices and quality  management processes to continuously improve maintenance operations  and maintenance production, achieve cost savings and avoidance, and  realize process cycle time reduction. Further, GAO’s Standards for  Internal Control in the Federal Government states that management  should communicate quality information down and across reporting lines  to enable personnel to perform key roles in achieving objectives.  However, the Office of the Secretary of Defense has not created, shared,  or maintained a comprehensive and updated list of all depot-specific DOD  sharing venues (i.e., working groups) that includes points of contact.  Officials from the Office of the Secretary of Defense stated that the Joint  Technology Exchange Group maintains a list on its website. However, the  list is incomplete, only containing three of the over 60 working groups we  identified in our analysis of our interview and survey data. Moreover, we  found that not all depot officials were aware of the Joint Technology Exchange Group and so would not be familiar with the Joint Technology  Exchange Group’s website. Without a centralized list of venues and  points of contact, it is unclear what groups exist and who to contact to  participate, which may impede sharing of best practices and lessons  learned.

The Army Has Not Maintained Lessons Learned Organizations, Potentially Hindering the Ability of the Depots to Share Best Practices and Lessons Learned

Each military service has initiatives or organizations to encourage the  sharing of best practice and lessons learned; however, the Army has not  maintained its lessons learned organizations. The depots from the Navy,  Marine Corps, and Air Force reported, in our survey and interviews, that  their military services have initiatives and organizations that encourage  knowledge sharing regarding best practices and lessons learned among  the depots. For example:

Navy’s Fleet Readiness Center’s Naval Sustainment System. The  Naval Sustainment System is an initiative to increase maintenance  capacity and readiness among the Navy’s fleet readiness centers by  process reviews and benchmarking. The depots reported in our  survey that it improves production by encouraging them to identify  constraints and to share lessons learned. The Naval Sustainment  System is also in the process of being adopted by the shipyards.

Navy’s “One Shipyard” Concept. The “One Shipyard” concept is a  Navy workforce initiative in which maintainers are exchanged among  the shipyards to ensure that the shipyards will have the required  number of workers and skill sets to meet current and planned  maintenance requirements. A Navy depot stated that as a result of the  communication required by this concept, they are better able to share  best practices.

Marine Corps’ Marine Depot Maintenance Command. Based on  responses to our survey, Marine Corps officials stated that the Marine  Corps depots have a single command structure. With this structure, all  process improvement meetings are held with both depots in  attendance, resulting in the sharing of best practices and lessons  learned between the two depots.

Air Force’s Art of the Possible. The Air Force Sustainment Center  created this management program to focus attention on restrictions in  workflow in the depots. Depots report that it creates a culture of  collaboration and sharing of best practices and lessons learned  because it focuses on process improvement and creates a culture in  which it is acceptable to discuss problems with other depots.

Competition for Workload   To determine which depot will receive new  workload, the Department of Defense (DOD)  Instruction 4151.24, Depot Source of Repair  Determination Process (Oct.13, 2017) outlines  a process under which workloads necessary  to sustain core logistics capabilities are  assigned to DOD depots that have the  requisite competencies. Two Army depots  reported that this process created competition  for workload that hinders sharing for them.  Depot officials stated that they fear that other  depots will take workload from them if they  share weapons system maintenance best  practices. In one such instance, Marine Corps  depot officials stated they visited an Army  depot and observed a best practice for  repairing 50-caliber machine gun receivers.  However, when the Marine Corps depot  reached out for technical details, the Army  depot was not inclined to share, for a variety  of reasons including competition for the same  workload. Then, the Marine Corps depot  asked Marine Corps Logistics Command to  facilitate, and they resolved the issue by  finding a Navy depot that had similar  technology and was willing to share.

In contrast, the Army does not have similar initiatives or organizations.  Army regulations direct the establishment and maintenance of two  organizations for sharing depot best practices and lessons learned. First,  Army Regulation 750-1 directs the Army Materiel Command to establish  and maintain the Army Materiel Lessons Learned Analysis Program to  identify potential systemic materiel sustainment issues and examine root  and contributing causes. Second, Army Regulation 11-33 directs Army  Materiel Command to establish and maintain the Center for Army  Acquisition and Materiel Lessons Learned to provide support in the  collection, analysis, dissemination, and archiving capability of materiel  lessons learned, with the objective of creating a knowledge sharing  culture within the Army. Moreover, the Standards for Internal Control in  the Federal Government states that management should establish an  organizational structure, assign responsibility, and delegate authority to  achieve the entity’s objectives.

Senior Army officials concurred that  competition between depots for jobs can be a  barrier for sharing, particularly when it  involves the preservation of specific depot  workloads. However, depots in other services  did not report competition for workload to be a  barrier to sharing.

The Army stated it established these organizations for sharing materiel  best practices and lessons learned; however, Army Headquarters, Army  Materiel Command, and Army depot officials stated that they were not  aware of analysis or knowledge sharing of depot best practices and  lessons learned that were performed by these organizations. Further, the  Army did not maintain these organizations for sharing materiel best  practices and lessons learned. First, officials from Army Futures  Command confirmed that the Army Materiel Lessons Learned Analysis  Program was transferred from Army Materiel Command to Army Futures  Command in July 2018 and no longer focuses specifically on materiel  lessons learned. Second, the officials confirmed that the Army ceased to  maintain the Center for Army Acquisition and Materiel Lessons Learned  in early 2017 due to direct funding limitations. In addition, some Army  depots reported being unable to identify peers in other depots to share  with, and they reported that competition hinders sharing (see sidebar).  Senior Army officials concurred that there are cultural challenges, which  result in the depots being less open to sharing and implementing best  practices and lessons learned. Establishing and maintaining effective  organizations dedicated to sharing materiel best practices and lessons  learned would encourage knowledge sharing among the Army depots.

DOD Is Experiencing Benefits and Taking Steps to Mitigate Challenges with Implementing Best Practices and Lessons Learned among the 17 Depots

DOD Is Implementing Some Best Practices and Lessons Learned That Has Led to Benefits

DOD is implementing some best practices and lessons learned among  the 17 depots that have led to benefits, including cost and time savings.  In response to our survey, 16 of the 17 depots reported benefits from  successfully implementing best practices and lessons learned, such as  sharing technology to reduce costs and improving maintenance  processes to repair parts and systems. These implemented best practices  and lessons learned can be defined as intra-service (within a military  service), inter-service (between two or more military services), or DOD  and external entities (between a military service and private industry).

Intra-service collaboration. Depots within each military service are  collaborating to implement best practices and lessons learned to improve  depot management processes and repairs related to weapon systems.  For example, Red River Army Depot implemented a best practice learned  from Anniston Army Depot to improve its depot management process in  meeting its production schedule. The production schedule is a plan that  identifies, among other things, working hours for maintainers, available  storage, and parts supply. To facilitate the implementation of this best  practice, Army Tank-Automotive and Armaments Command, which  oversees these two depots, hosted a joint event for the purpose of  Anniston’s sharing how a small group of individuals at its depot is  responsible for maintaining visibility of all end-item (i.e., components and  parts ready for their intended use) production schedules. According to  Army officials, Red River did not have an organization that performed a  similar function, and during the joint event, depot officials from Red River  saw this as a lesson learned that they could take back to their depot and  implement. Additionally, Anniston shared how it conducts its risk  assessments, or program reviews, and weekly execution meetings,  among other processes, in meeting its production schedules. As a result,  Army officials told us that Red River implemented the best practices they  thought would be beneficial in helping them make progress in meeting  their production schedules.

In another example, two Air Force depots that maintain the Navy’s C-130  aircraft are working together to implement a best practice, which,  according to program documentation, has led to cost and time savings  (See fig. 6.).

Specifically, the Navy’s C-130 aircraft, which, according to Ogden  officials, is maintained at Ogden Air Logistics Complex and Warner  Robins Air Logistics Complex, contains a shelf bracket, which holds the  pieces of the aircraft together. The aircraft becomes structurally  vulnerable and unfit for operations and training if the shelf bracket is  removed. The process of blasting, inspecting, plating, and reinstalling the  shelf bracket takes an average of 63 days. During this time, some  maintenance activities cannot occur until the shelf bracket is reinstalled.  To address this issue, engineers at Ogden told us they created a series of  specially-sized pins to lock the Navy’s C-130 aircraft in place to help  maintain the structural integrity of the airframe while other areas of the  aircraft are being repaired. As a result of this best practice, maintainers  have eliminated 16 days in the maintenance process for the C-130. Also,  depot officials told us for a one-time cost of $13,000 for one set of  specially-sized pins, eliminating 16 days in the maintenance process in  turn generates a cost avoidance of $32,000 per day (the cost to dock the  aircraft) or more than $500,000 per aircraft. In implementing this best  practice, the total annual benefit to the C-130 fleet at Ogden amounts to  288 days of aircraft availability and about $9 million in cost avoidance.  Officials at Ogden told us they have implemented this new process and  are discussing this best practice with maintainers at Warner Robins for  implementation at their depot as well.

Further, Air Force depots are partnering to further implement another best  practice, cold spray technology, which allows depots to repair damaged  parts instead of replacing them. Replacing these damaged parts can be  expensive or difficult if they are low in supply. Also, limited parts and long  lead times can cause delays in the supply system, and existing repair  processes have a long turnaround time. Cold spray technology has not  been fully implemented; however, even with its limited implementation,  cold spray technology has yielded cost and time benefits (See fig. 7.).

According to Air Force officials, Ogden has been collaborating with the  Oklahoma City Air Logistics Complex to cold spray its F-16 gearboxes  until Ogden can obtain adequate workload to sustain the cold spray  technology. According to Ogden officials and program documentation,  cold spraying each gearbox costs about $1,300 whereas replacing each  gearbox costs about $38,000; at 13 units per year, this amounts to almost  $500,000 in annual cost avoidances. Additionally, it would take 95 weeks  to build and receive a new gearbox unit; however, with the cold spray  repair the unit is back in service in 4 weeks. Ogden officials are currently  working to include cold spraying gearboxes for the F-15, C-5 and E-3  weapon systems to its workload.

Inter-service collaboration. Depots from two or more military services  are collaborating to implement best practices and lessons learned which  has led to benefits. For example, the Navy’s Fleet Readiness Center  Southwest implemented a best practice learned from Ogden Air Logistic  Complex to improve testing of electrical circuits. Specifically, according to  depot officials, a maintainer at Ogden created a method—Intermittent  Fault Detection and Isolation System—which tests systems and software  to detect, isolate, and repair intermittent problems due to open circuits,  short circuits, and poor wiring by replicating the environment of the  aircraft in flight (See fig. 8.). According to Ogden officials and program  documentation, by implementing this best practice, they have recovered  out-of-service assets and generated about $62 million in cost savings. For  example, after testing its F-16 chassis, Ogden officials recovered 138 out- of-service assets—amounting to $42 million of flight hardware returning to  service.

Moreover, officials at Fleet Readiness Center Southwest visited Ogden  during a benchmarking trip to discuss the process of implementing the  Intermittent Fault Detection and Isolation System to test their systems.  According to officials from the Office of the Secretary of Defense, the  intermittent faults due to aircraft electrical systems amounted to more  than $300 million in operating and support costs in fiscal year 2014. The  Fleet Readiness Center Southwest used the Intermittent Fault Detection  and Isolation System to test its F/A-18 aircraft generators, which provide  electrical power to the aircraft. As a result of testing these generators  using the Intermittent Fault Detection and Isolation System, the mean  time between failures for the generators has increased, according to  officials, from 104 flight hours to over 400 flight hours, and the Navy  anticipates a reduction of about 30 to 90 days of repair time.

DOD and external entities. Depots are also partnering with private  industry to implement best practices and lessons learned, which has led  to time-savings benefits (See fig. 9.).

For example, according to program officials, the Air Force, Navy, original  equipment manufacturer, and contractor collaborated to implement a best  practice for the U-2 aircraft. Specifically, in 2018, generators for the Air  Force’s U-2 aircraft had decreased their mean time between failures from  1,000 hours to 400 hours. To sustain the fleet, the Air Force was  cannibalizing—removing parts from one aircraft to another—generators  from aircraft in depot maintenance to those preparing for deployment. The  U-2 program office identified the Navy’s F/A-18 A/B generator as similar  to the U-2 generator and learned valuable information on the repair and  overhaul process, root cause analysis of failure of critical parts, and the  Navy’s recommendation for procuring and building overhaul generator  kits. In order to implement the Navy’s processes, the Air Force program  office, working with the original equipment manufacturer and contactor,  incorporated the Navy’s best practices in overhauling its generator kit  concept. As a result, the Air Force is no longer cannibalizing these  generators and the mean time between failures has returned to about  1,000 hours of flight time.

DOD Has Not Been Able to Implement Some Best Practices and Lessons Learned among the 17 Depots, but Is Taking Steps to Mitigate Challenges

DOD has not been able to implement some best practices and lessons  learned among the 17 depots, but DOD is taking steps to mitigate  challenges to implementation. In its March 2018 Report to Congress on  Sharing of Best Practices for Depot-Level Maintenance Among the  Military Services, DOD noted some of the challenges in implementing  best practices such as differing military service priorities, strategies, and  resourcing of technologies and infrastructure. In responding to our  survey, 15 of the 17 depots reported challenges in implementing best  practices and lessons learned, including insufficient resources,  restrictions related to information technology, approval process, and  acquisition and contracting policies, among others (See table 2.).

Insufficient resources. Ten of the 17 depots reported insufficient  resources as a challenge to implementation for various reasons. First,  depots reported not having adequate time, staff, or funding to attend  knowledge sharing activities or to analyze data from best practices and  lessons learned. According to depot officials, not being able to attend  knowledge sharing activities has made networking more difficult because  these activities allowed them to discuss best practices and lessons  learned with colleagues from other depots and industry. Second, in  addition to not having adequate funding, depots also reported identifying  sources of funding as a challenge to implementing best practices and  lessons learned for specific weapon systems. For example, according to  officials from one depot, they have been unable to identify a funding  source to implement the laser de-painting system for the F-16, which  would allow the aircraft to stay in service longer and would produce less  hazardous materials than the current blasting process to remove paint  from the aircraft. Third, depots reported insufficient equipment to  implement a best practice. For example, one depot reported not having  enough hand-held tablets, which contain electronic technical data and  best practices from private industry to assist maintainers working on a  weapon system. Another depot reported that it has not implemented the  tablets and are relying on paper documentation to maintain its weapon  systems. According to depot officials, the lack of tablets has had direct  effects at the depot, such as delays in standing-up new capability and  maintainers waiting on available tablets to perform their work.

To mitigate challenges with insufficient resources, DOD, military service,  and depot officials have taken a variety of steps. For example, officials  from the Office of the Secretary of Defense held an event through the  Joint Technology Exchange Group to discuss available funding sources  for new and emerging technologies, such as the funding sources for the  cold spray technology. According to officials at a Navy depot, depots can  petition the Office of Naval Research for federal laboratory designation.  With this designation, depots can partner with private industry to evaluate  technology in any area that is consistent with the federal laboratory’s  mission and may receive funds from private industry for technology  research and development. Specific to the tablets, depot officials told us  that the materiel command has taken responsibility for managing the  funding of these assets and the depots will receive a technical upgrade  every 4 years. Moreover, in February 2019 the Office of the Secretary of  Defense launched the Enterprise Sustainment Dashboard (Dashboard), a  web-based tool that will provide access to an online central repository of  sustainment data for the military services and will allow senior leaders to  steer resources to needed programs. The Dashboard will allow users to  analyze metrics such as materiel availability (condition of a weapon  system to perform an assigned mission), operational availability  (availability of active inventory to conduct military service operations), and  cost per day availability (maintenance cost per day for a population of  weapon systems by type, model, and series). The Dashboard will also  consolidate inventory, availability, and cost data systems from each of the  military services. This Dashboard is in its early phase and the  implementation plan includes milestones extending into fiscal year 2020.

Restrictions related to information technology. Ten of the 17 depots  reported restrictions related to information technology as a challenge to  implementation of best practices. Specifically, depots reported having  outdated and incompatible software systems and a lack of a consolidated  database for departments and product lines, which may hinder their ability  to connect computer systems to automate a repair process. Additionally,  depots stated that it may take years to obtain authority and approval to  operate information technology systems, making data collection, sharing,  and implementation of best practices difficult. For example, one depot  reported a technology tool was not user friendly and had a rigid  infrastructure, making it difficult for maintainers to use to analyze metrics  to improve depot maintenance. Specifically, depot officials told us that this  technology tool performs its functions as designed but is limited in its  scope of meeting depot requirements, such as identifying bottlenecks in  the maintenance process. In another example, one depot reported  cybersecurity concerns with commercial off-the-shelf products, which may  not be compatible with the depot’s information technology system.

To mitigate challenges related to information technology, depots reported  using information systems, such as SharePoint, as a primary source for  collecting, storing, organizing, sharing, and accessing information via a  web browser. For example, Navy officials told us that there are  SharePoint sites for different departments within their organization,  including portals dedicated to training, aircraft, and business processes  and procedures, which capture best practices and lessons learned from  subject matter experts. In another example, an Air Force depot reported  that its SharePoint portal includes a section focused on practical problem  solving methods for some of its continuous process improvement  projects, such as balancing weight on an aircraft and issues related to the  wings of the C-130T. Further, depot officials told us they conducted an  analysis to mitigate concerns about a technology tool, mentioned above,  that was not user friendly and had a rigid infrastructure. Based on this  analysis, depot officials found a modeling and simulation tool that would  help resolve challenges in several key areas, including projecting  workload and personnel required to perform depot maintenance and  determining the depot’s capability for the volume of work that can be  inducted into the depot, among other areas. The modeling and simulation  tool has not been implemented yet because it was recently funded in  September 2019.

Moreover, in 2018, we reported on steps that DOD is taking to improve its  information technology systems. Specifically, the Secretary of Defense  asked the Defense Business Board to provide actionable  recommendations that DOD could adopt to transform its six core business  processes, including acquisition and procurement, logistics and supply,  and real property management, and their supporting information  technology systems. We recommended, in part, that DOD identify  timeframes and deliverables for identifying and adopting optimal  information technology solutions. DOD concurred with this  recommendation and is taking steps to improve its information technology  systems, such as issuing its initial plan for business operations reform in  April 2019, collecting federal and private industry benchmarks, and  reviewing information technology costs.

Approval process. Eight of the 17 depots reported that the approval  process and guidance for implementing best practices is challenging.  Specifically, depots reported that the layers of leadership approval  prevent timely implementation of best practices and, at times, can cause  enthusiasm for a project’s implementation to wane. Depot officials also  told us that implementing new ideas for maintaining or repairing weapon  systems is challenging because they have to get multiple approvals from  their chain of command as well as the program manager for a specific  weapon system, thus making implementation more difficult and less  timely. For example, depot officials told us that implementing best  practices at the depot from one weapon system to another requires  retesting of the practice and approval from each program manager.  Additionally, in response to the survey, a depot reported that many of the  essential, time-sensitive engineering decisions for one of its new weapon  system reside at another location, which has caused delays in making  timely decisions. In another example, depot officials told us that they had  to get approval from individual program managers to implement the cold  spray technology and the Intermittent Fault Detection Isolation System.

To mitigate challenges in the approval process, such as these, depot  officials told us it is beneficial when technological development that affect  the DOD-wide logistics enterprise or an entire military service occurred at  a higher organizational level, making it easier for new ideas to be  implemented at the lower levels. For example, one depot reported on the  Navy’s approach of implementing a best practice across its platforms to  eliminate corrosive plating on its weapon systems. Navy officials told us  that these decisions are made at the headquarters level and implemented  across the depots. Moreover, one depot reported allowing decision  authority for specific weapon systems to reside within the depot, rather  than at another location, to help the depot make timely decisions on  implementing new ideas. Finally, the Office of the Assistant Secretary of  Defense is providing specific guidance in implementing best practices and  lessons learned, such as the memorandum issued in April 2019 on the  Intermittent Fault Detection and Isolation System directing the military  services to adopt this best practice.

Acquisition and contracting policies. Five of the 17 depots reported  acquisition and contracting policies as a challenge to implementation.  Specifically, depots reported that current acquisition and contracting  policies are complex and time consuming, which causes government to  lag behind industry in implementing best practices. For example, officials  from one depot told us that even when two depots need the same item to  repair a weapon system, each depot was encouraged to pursue a  separate contract. Depot officials described this as an inefficient and  burdensome process, which sometimes resulted in an inferior item.  Similarly, officials from another depot told us that they started an initiative  to make equipment and software more similar across their service’s  depots; however, they were unable to implement this initiative for similar  reasons.  Further, officials from one depot told us that the procurement of  a weapon system does not always include access to all data necessary to  maintain the system. According to depot officials, this limits their ability  to implement a best practice or lesson learned from a similar weapon  system because the contractor retains ownership of the intellectual  property needed to repair or optimize the system.

To mitigate challenges related to acquisition and contracting policies,  depot officials told us that military services are purchasing enough new  technology for all their depots rather than have each depot purchase  technology individually. For example, according to Navy officials, they  purchased the equipment to implement cold spray technology across all  four shipyards, which makes implementing the best practice or lesson  learned more timely. Additionally, officials from one depot told us that they  use public-private partnerships to bridge gaps for systems that lack  access to the necessary data rights to conduct maintenance on the  systems. Our February 2019 report identified additional steps DOD is  taking to mitigate challenges related to intellectual property, especially  software sustainment. First, our prior work found that DOD is in the  early stages of addressing a statutory provision for DOD to (1) develop  policy on the acquisition or licensing of intellectual property; and (2)  establish a cadre of intellectual property experts to help support the  acquisition workforce on intellectual property matters. Second, in our  prior work, we reported that DOD officials we spoke with emphasized that  there are situations in which the data rights needed may not be known  until years into sustainment and that it would be useful if data rights could  have a pre-negotiated price and be an option as part of the initial contract.  Such an option would give the government the right, but not the  obligation, to purchase the data rights at the pre-negotiated price if  needed in the future.

The sharing and implementation of best practices and lessons among the  17 depots is crucial to sustaining military readiness by ensuring that the  military services can regularly maintain critical weapon systems and  return them to the warfighter for use in training and operations.  Successful collaboration of maintenance best practices and lessons  learned across military services, private industry, and academia is  increasingly essential as DOD operates, and thus needs to maintain,  weapon systems. DOD shares best practices and lessons learned among  the depots through a variety of venues, including networking, working  groups, and benchmarking. However, DOD has communication  challenges, including a lack of awareness of many sharing venues, which  may hinder the ability of the depots to share best practices and lessons  learned. The Office of the Secretary of Defense has not created, shared,  or maintained a comprehensive and updated list of all depot-specific DOD  sharing venues (i.e., working groups) that includes points of contact.  Without a centralized list and points of contact, it is unclear what groups  exist and who to contact to participate, which may impede sharing of best  practices and lessons learned. Further, while the Army stated it  established lessons learned organizations for sharing materiel best  practices and lessons learned, it did not maintain them due to  organizational restructuring and resource constraints. Establishing and  maintaining effective organizations dedicated to sharing materiel best  practices and lessons learned would encourage knowledge sharing  among the Army depots.

We are making two recommendations, including one to the Under  Secretary of Defense for Acquisition and Sustainment and one to the  Secretary of the Army. Specifically, the Secretary of Defense should  direct that:  The Under Secretary of Defense for Acquisition and Sustainment should  ensure that the Deputy Assistant Secretary of Defense for Materiel  Readiness create, share, and maintain a comprehensive and up-to-date  list of all DOD sharing venues (i.e., working groups), including points of  contact, related to depot maintenance. (Recommendation 1)

The Secretary of the Army should ensure that Army Materiel Command  reestablish and maintain organizations dedicated to sharing materiel best  practices and lessons learned, as required by Army regulations.  (Recommendation 2)

We provided a draft of this report to DOD for review and comment. In  written comments on a draft of this report, DOD concurred with the  recommendations. DOD’s comments are reprinted in their entirety in  appendix III. DOD also provided technical comments, which we  incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, the Secretaries of the Army, Navy,  and Air Force, and the Commandant of the Marine Corps. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff has any questions about this report, please contact  Diana Maurer at (202) 512-9627 or maurerd@gao.gov. Contact points for  our Office of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff that made key contributions to this  report are listed in appendix IV.

To conduct the work for our reporting objectives, we reviewed relevant  laws and the Department of Defense (DOD) and military service guidance  that govern depot maintenance and the sharing of best practices and  lessons learned. We included in our scope DOD depots performing major  depot-level maintenance. We conducted a survey of DOD’s 17 depots  performing depot-level maintenance to gain an understanding of how  each depot shares with each other and implements best practices and  lessons learned. The response rate for the survey was 100 percent.  These depots included:

Corpus Christi Army Depot, Corpus Christi, Texas

Letterkenny Army Depot, Letterkenny, Pennsylvania

Tobyhanna Army Depot, Tobyhanna, Pennsylvania

Norfolk Naval Shipyard, Portsmouth, Virginia

Pearl Harbor Naval Shipyard, Honolulu, Hawaii

Portsmouth Naval Shipyard, Kittery, Maine

Puget Sound Naval Shipyard, Bremerton, Washington

Fleet Readiness Center East, Cherry Point, North Carolina

Fleet Readiness Center Southeast, Jacksonville, Florida

Fleet Readiness Center Southwest, San Diego, California

Albany Production Plant, Albany, Georgia

Barstow Production Plant, Barstow, California

Ogden Air Logistics Complex, Ogden, Utah

Oklahoma City Air Logistics Complex, Oklahoma City, Oklahoma

Warner Robins Air Logistics Complex, Warner Robins, Georgia  We analyzed survey responses to gain an understanding, for example, of  which depot officials are coordinating with others to share best practices  and lessons learned, which sharing venues are attended, and the extent  to which this information sharing is beneficial. To ensure that the survey  questions were clear, comprehensible, and technically correct, we  conducted expert reviews of our draft survey with four subject matter  experts with knowledge and experience in auditing DOD depots. We also  conducted two pre-tests of our draft survey with the depot commanders of  Anniston Army Depot and Warner Robins Air Logistics Complex,  respectively.During each pre-test, conducted by teleconference, we read  the instructions and each survey question aloud and asked the depot  commanders to tell us how they interpreted the question. We then  discussed the instructions and questions with each depot commander to  identify any problems and potential solutions by determining whether (1)  the instructions and questions were clear and unambiguous, (2) the terms  we used were accurate, (3) the survey was unbiased, and (4) the survey  did not place an undue burden on the depot officials completing it. We  noted any potential problems and modified the survey based on feedback  from the subject matter experts and depot commanders, as appropriate.  We sent a fillable survey and a cover email to 17 depots on May 29,  2019, and asked them to complete the survey and email it back to us by  June 14, 2019. We closed the survey on July 3, 2019. Data were auto- extracted from the Adobe PDF form into an Excel spreadsheet. Our  examination of the survey results included both a quantitative data  analyses on closed-ended questions and a review of open-ended  responses to identify common themes.

Additionally, to gather detailed examples of DOD’s efforts to share best  practices and lessons learned, we visited a non-generalizable sample of 5  depots (Anniston Army Depot, Anniston, Alabama; Norfolk Naval  Shipyard, Portsmouth, Virginia; Fleet Readiness Center Southwest, San  Diego, California; Marine Corps Albany Production Plant, Albany,  Georgia; and Ogden Air Logistics Complex, Ogden, Utah). To select our  sample, we considered variation in geographic location, military service  representation, and types of weapon systems maintained. At these sites,  we conducted group discussions with individuals across the depot to gain  insight into their roles in sharing best practices and lessons learned.  Qualitative data analyses were conducted by our staff who have subject  matter expertise to identify themes and select examples of best practices  or lessons learned shared through collaboration with another depot. We  then obtained and analyzed documentation of sharing, such as working  group charters and trip reports documenting results from visiting another  depot; as well as benefits experienced from implementing a best practice  or lessons learned, including time and cost savings.

We interviewed officials from the Office of the Under Secretary of  Defense (Acquisition and Sustainment) (Deputy Assistant Secretary of  Defense for Materiel Readiness), Joint Chiefs of Staff (Joint Lessons  Learned Division), and the military service headquarters (Headquarters,  Department of Army G4; Deputy Assistant Secretary of the Navy for  Expeditionary Programs and Logistics Management; Headquarters  Marine Corps, Installations & Logistics; and Air Force Acquisition,  Logistics & Product Support. We also interviewed officials from the  military service logistics or materiel components (Army Materiel  Command; Naval Sea Systems Command; Naval Air Systems Command  (Commander, Fleet Readiness Center); Marine Corps Logistics  Command; and the Air Force Materiel Command) as well as the military  lessons learned centers (Center for Army Lessons Learned, Naval  Warfare Development Command, Marine Corps Center for Lessons  Learned, and the Air Force LeMay Center for Lessons Learned).

Finally, we reviewed our prior reports related to challenges experienced  at DOD depots and DOD’s report to Congress on the sharing of best  practices for depot-level maintenance among the military services. We  assessed the documentary and testimonial evidence we collected against  DOD and military service guidance on lessons learned and materiel  maintenance and GAO’s Standards for Internal Control in the Federal  Government. Specifically, the information and communication  component of internal control—the actions management uses to internally  communicate the necessary quality information to achieve the entity’s  objectives—was significant to this audit.

We conducted this performance audit from January 2019 through January  2020 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Appendix II: Depot Working Groups and Communities of Practice

During the course of our work examining the extent to which the  Department of Defense (DOD) experiences benefits and has challenges  with (1) sharing and (2) implementing best practices and lessons learned  among the depots, we collected information from the depots on the  working groups and communities of practice in which they participate.  The list below is compiled from analysis of our survey data, in which we  surveyed all 17 of DOD’s depots, as well as the interviews we conducted  during our site visits to a non-generalizable sample of five depots. Note  that this is not a list of all the possible working groups and communities of  practice which exist among the depots, simply those which the depots  shared with us.   1.  448th Supply Chain Management Wing   2.  Air Force Metrology and Calibration Working Group  3.  Air Force Sustainment Center Logistics Directorate’s Strategic  4.  Aircraft Cyber Threat Working Group  5.  Aircraft Maintenance Group Summit  6.  Aircraft Storage Strikeboard   8.  Army Safety and Occupational Health Information Management  9.  Army Safety and Occupational Health Management System Working  10. Carrier Team One  11. Cold Spray Action Team     12. Commander, Fleet Readiness Centers Advanced Technology &  13. Commercial Technologies for Maintenance Activities Working Group –  14. Commodities, Electronics, Missiles, & Propulsion Maintenance  15. Coordinate Measuring Machine Community of Practice  16. Corporate Electrical Community of Practice  17. Corrosion Control Working Groups  18. Cyber Resiliency Office for Weapon Systems Working Groups    19. Depot Maintenance Activation Working Group  20. Depot Maintenance Enterprise Action Group   21. Diminishing Manufacturing Sources and Material Shortages  22. DOD Digital Manufacturing Users Group  23. DOD Unmanned Systems & Robotics Summit  24. DOD Voluntary Protection Programs  26. Enterprise IT Systems Strikeboard  27. F-35 Joint Risk Working Group  28. H-53 Fleet Support Team  29. Heavy Metal Working Group  30. Industrial Base Commander’s Meetings  31. Integrated Quality Teams   32. Investment Working Group  33. Joint Additive Manufacturing Steering Group  34. Joint Additive Manufacturing Working Group and Community of  35. Joint Intermittence Team   36. Joint Requirements Working Group  37. Joint Robotics Working Group  38. Joint Technology Exchange Group  39. Metrics Community of Practice   40. Modernization Working Group  41. National Center for Defense Manufacturing and Machining  42. Naval Surface Warfare Center, Carderock Division Human  43. Naval Undersea Warfare Center Division, Keyport Human  44. Navy Forum for Small Business Innovation Research/Small Business  Technology Transfer Transition  45. Non-Destructive Inspection Forum  46. Non-Destructive Testing Working Group  47. Norfolk Naval Shipyard Technology and Innovation Community of  48. Organic Industrial Base Commander’s Summit  49. Project Management Executive Steering Committee  50. Public-Private Partnership Community of Practice  51. Quality Performance System Community of Practice   52. Quality Work Environment Working Group  53. Residential Economic Development Inc.  54. RepTech Working Group  55. Shipyard departmental level Communities of Practice: C200, C1200,  C1200N, C600, C400, etc.  56. Shipyard-only Community of Practice  57. Software Engineering Institute Agile Collaboration Group  58. Software Maintenance Group Summit  59. Sub Team One  60. Tri-Air Logistics Complex Summits   61. Weapon-system Specific Enterprise Cross-talks: C-130 Enterprise  Crosstalk, A-10 Enterprise Crosstalk, etc.

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

In addition to the contact listed above, Jodie Sandel (Assistant Director),  Laura Czohara (Analyst-in-Charge), Clarine Allen, Felicia Lopez, Amie  Lesser, Christina Murphy, Clarice Ransom, Andrew Stavisky, and  Courtney Tepera made key contributions to this report.

Navy Maintenance: Persistent and Substantial Ship and Submarine  Maintenance Delays Hinder Efforts to Rebuild Readiness. GAO-20-257T.  Washington, D.C.: December 4, 2019.

Naval Shipyards: Key Actions Remain to Improve Infrastructure to Better  Support Navy Operations. GAO-20-64. Washington, D.C.: November 25,  2019.

F-35 Aircraft Sustainment: DOD Faces Challenges in Sustaining a  Growing Fleet. GAO-20-234T. Washington, D.C.: November 13, 2019.

Depot Maintenance: DOD Should Adopt a Metric That Provides Quality  Information on Funded Unfinished Work. GAO-19-452. Washington, D.C.:  July 26, 2019.

Military Depots: Actions Needed to Improve Poor Conditions of Facilities  and Equipment That Affect Maintenance Timeliness and Efficiency.  GAO-19-242. Washington, D.C.: April 29, 2019.

Weapon System Sustainment: DOD Needs to Better Capture and Report  Software Sustainment Costs. GAO-19-173. Washington, D.C.: February  25, 2019.

Army Modernization: Steps Needed to Ensure Army Futures Command  Fully Applies Leading Practices. GAO-19-132. Washington, D.C.: January  23, 2019.

DOD Depot Workforce: Services Need to Assess the Effectiveness of  Their Initiatives to Maintain Critical Skills. GAO-19-51. Washington, D.C.:  December 14, 2018.

Navy and Marine Corps: Rebuilding Ship, Submarine, and Aviation  Readiness Will Require Time and Sustained Management Attention.  GAO-19-225T. Washington, D.C.: December 12, 2018.

Navy Readiness: Actions Needed to Address Costly Maintenance Delays  Facing the Attack Submarine Fleet. GAO-19-229. Washington, D.C.:  November 19, 2018.

Depot Maintenance: DOD Has Improved the Completeness of Its Biennial  Core Report. GAO-19-89. Washington, D.C.: November 14, 2018.

Air Force Readiness: Actions Needed to Rebuild Readiness and Prepare  for the Future. GAO-19-120T. Washington, D.C.: October 10, 2018.

Weapon System Sustainment: Selected Air Force and Navy Aircraft  Generally Have Not Met Availability Goals, and DOD and Navy Guidance  Need to Be Clarified. GAO-18-678. Washington, D.C.: September 10,  2018.

Military Readiness: Analysis of Maintenance Delays Needed to Improve  Availability of Patriot Equipment for Training. GAO-18-447. Washington,  D.C.: June 20, 2018.

F-35 Aircraft Sustainment: DOD Needs to Address Challenges Affecting  Readiness and Cost Transparency. GAO-18-75. Washington, D.C.:  October 26, 2017.

Depot Maintenance: Executed Workload and Maintenance Operations at  DOD Depots. GAO-17-82R. Washington, D.C.: February 3, 2017.

Depot Maintenance: Improvements to DOD’s Biennial Core Report Could  Better Inform Oversight and Funding Decisions. GAO-17-81. Washington,  D.C.: November 28, 2016.

Naval Shipyards: Actions Needed to Improve Poor Conditions that Affect  Operations. GAO-17-548. Washington, D.C.: September 12, 2017.

Army Working Capital Fund: Army Industrial Operations Could Improve  Budgeting and Management of Carryover. GAO-16-543. Washington,  D.C.: June 23, 2016.

Military Readiness: Progress and Challenges in Implementing the Navy’s  Optimized Fleet Response Plan. GAO-16-466R. Washington, D.C.: May  2, 2016.

Defense Inventory, Further Analysis and Enhanced Metrics Could  Improve Service Supply and Depot Operations. GAO-16-450.  Washington, D.C.: June 9, 2016.

Navy Working Capital Fund: Budgeting for Carryover at Fleet Readiness  Centers Could Be Improved. GAO-15-462. Washington, D.C.: June 30,  2015.

Sequestration: Documenting and Assessing Lessons Learned Would  Assist DOD in Planning for Future Budget Uncertainty. GAO-15-470.  Washington, D.C.: May 27, 2015.

Operational Contract Support: Actions Needed to Enhance the Collection,  Integration, and Sharing of Lessons Learned. GAO-15-243. Washington,  D.C.: March 16, 2015.

DOD operates depots nationwide to overhaul, upgrade, and rebuild complex weapons and equipment. These depots are crucial to keeping the military ready to conduct operations and training.

While depots have sometimes struggled to meet repair needs in recent years, they reported benefits from sharing best practices and lessons learned with each other. For example, one shared an improvement that reduced service time for a type of aircraft gearbox—from a 95-week replacement to a 4-week repair.

We found DOD could do more to help depots share information. We recommended, among other things, creating a comprehensive list of depot working groups.


Tribal Broadband: FCC Should Take Efforts to Promote Tribal Access to Spectrum

Broadband service on tribal lands continues to lag behind the rest of the country, especially on rural tribal lands. Broadband service can be delivered through wireless technologies using radio frequency spectrum. According to FCC, increasing tribal access to spectrum would help expand broadband service on tribal lands.

This statement is based on GAO's November 2018 report ( GAO-19-75 ) related to spectrum use for broadband services by tribal entities and selected updates. Specifically, it discusses (1) tribal entities' ability to obtain and access spectrum to provide broadband services and the reported barriers that may exist, and (2) the extent to which FCC promotes and supports tribal efforts to obtain and access spectrum. For that report, GAO interviewed 16 tribal entities that were using wireless technologies. Selected entities varied geographically, among other characteristics. GAO analyzed FCC's license and auction data as of September 6, 2018, reviewed FCC's rulemakings on spectrum for broadband services, and interviewed other tribal and industry stakeholders and FCC officials. The information obtained was not generalizable to all tribes or industry participants. As an update, GAO reviewed FCC's June 2019 draft order related to spectrum in the 2.5 GHz band.

The tribal entities—tribal governments and tribally owned telecommunications providers—GAO contacted for its November 2018 report cited various barriers to obtaining spectrum licenses in bands that can be used to provide broadband services. Based on data from the Federal Communications Commission (FCC) as of September 2018, GAO identified 18 tribal entities that held active spectrum licenses in such bands. For example, of these 18 tribal entities, 4 obtained licenses through secondary market transactions—that is, they bought or leased the license from another provider, and 2 obtained a license through an FCC spectrum auction. The barriers tribal officials identified to obtaining licensed spectrum include high costs at auctions and, in the case of secondary market transactions, a lack of information on who holds licenses over tribal lands. Because most spectrum allocated for commercial use has already been assigned, the secondary market is one of the few avenues available to tribal entities that would like to access licensed spectrum.

At the time of GAO's November 2018 report, FCC had taken some actions to increase tribal access to spectrum. For example, FCC issued proposed rulemakings in 2011 and 2018 that sought comment on tribal-specific proposals, such as establishing tribal-licensing priorities and initiating processes to transfer unused spectrum licenses to tribal entities. FCC had not finalized these rules at the time of GAO's report, but FCC published a draft order in June 2019 that would create a tribal-licensing priorty window, whereby tribal entities would have an opportunity to obtain spectrum in the 2.5 gigahertz (GHz) band prior to the spectrum being auctioned. FCC adopted the order on July 10, 2019. FCC stated that it will implement spectrum initiatives and that it recognizes the importance of promoting a robust secondary market to improve communications throughout the United States, including tribal lands. However, GAO found that FCC had not consistently collected data related to tribal access to spectrum. For example:

FCC did not collect data on whether spectrum auction applicants are tribal entities and therefore did not have a comprehensive understanding of the extent that tribal entities are attempting to obtain licensed spectrum.

FCC did not analyze the extent that unused licensed spectrum exists over tribal lands. Although FCC officials said evaluating the effectiveness of FCC's secondary market policies is a way to increase the use of unused spectrum, FCC's approach did not include an analysis of unused spectrum licenses on tribal lands. As a result, FCC's evaluations of the secondary market may not accurately reflect how its policies affect tribal entities.

By collecting data on the extent that tribal entities are obtaining and accessing spectrum, FCC could better understand tribal spectrum issues and use this information as it implements ongoing spectrum initiatives. Further, given that the secondary market is one of few ways for tribal entities to access licensed spectrum to provide Internet service, FCC could promote a more robust secondary market by analyzing unused licensed spectrum over tribal lands and using that information to inform FCC's oversight responsibilities.

In the November 2018 report, GAO made three recommendations to FCC, including that FCC should collect data on tribal access to spectrum and analyze unused licensed spectrum over tribal lands. FCC agreed with the recommendations and described actions to address them.

Thank you for the opportunity to discuss our November 2018 report on  tribal access to and use of spectrum for broadband services. Broadband  service on tribal lands continues to lag behind the rest of the country,  especially on rural tribal lands. According to the Federal  Communications Commission (FCC), this gap could hinder tribal efforts to  promote self-governance, economic opportunity, education, public safety,  and cultural preservation. In prior work, we found that one barrier to  increasing broadband access on tribal lands is the cost of deploying  infrastructure in areas with challenging terrain. However, broadband  service can be delivered through wireless technologies using radio  frequency spectrum, and according to FCC, such wireless technologies  are more cost-effective for some remote and sparsely populated areas,  such as tribal lands. In 2010, FCC reported that increasing tribal access  to and use of spectrum would create additional opportunities to expand  broadband service on tribal lands.

My remarks today are based on the November 2018 report and selected  updates and address (1) the ability of tribal governments and  telecommunications providers, which we refer to as “tribal entities,” to  obtain and access spectrum to provide broadband services on tribal lands  and the reported barriers that may exist, and (2) the extent to which FCC  promotes and supports tribal efforts to obtain and access spectrum for  broadband services. For our report, we reviewed relevant statutes and  regulations, FCC documents, and academic and government  publications on spectrum-related issues on tribal lands. We analyzed  FCC data on spectrum auction applicants and license holders as of  September 6, 2018 and reviewed the list of federally recognized tribes  published in the Federal Register. To obtain stakeholder views, we  interviewed stakeholders, such as tribal associations, regional  consortiums, private providers that deliver Internet services to tribal lands,  industry associations, and companies that work with tribal entities. In  addition, we interviewed 24 tribal entities—16 of which were using  wireless technologies to provide Internet services—that we selected to  include variation in terms of geographic location, level of broadband  deployment, population size and density, and urban or rural distinction.  Stakeholders were selected to represent a range of views and experience  working with tribes and broadband service; their views are not  generalizable. To assess FCC’s efforts, we interviewed FCC officials and  reviewed relevant FCC-rulemaking proceedings and related public  comments, and compared FCC’s actions against recommendations made  in FCC’s National Broadband Plan, FCC’s current strategic plan, and  federal internal control standards related to using quality information. For  this testimony, we reviewed the status of related FCC rulemakings and  contacted FCC about action FCC has taken in response to our  recommendations. Further details on our scope and methodology are  included in our report.

We conducted the work on which this statement is based in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.

Wireless broadband connects users to the Internet using spectrum to  transmit data between the customer’s location and the service provider’s  facility, and can be transmitted using fixed wireless and mobile  technologies, as shown in figure 1.

Fixed wireless broadband technologies establish an Internet connection  between fixed points—such as from a radio or antenna that may be  mounted on a tower, to a stationary wireless device located at a home— and generally requires a direct line of sight. Mobile wireless broadband  technologies also establish an Internet connection that requires the  installation of antennas, but this technology provides connectivity to  customers wherever they are covered by service, including while on the  move, such as with a cell phone. Spectrum is the resource that makes  wireless broadband connections possible. Spectrum frequency bands  each have different characteristics that result in different levels of ability to  cover distances, penetrate physical objects, and carry large amounts of  information. Examples of some of the frequency bands that can be used  by commercial and nonfederal entities for broadband services are shown  in figure 2.

The frequency bands that can be used for broadband services are either  licensed or unlicensed. For licensed spectrum, FCC can assign licenses  through auctions, in which prospective users bid for the exclusive rights to  transmit on a specific frequency band within geographic areas. Having  exclusive rights ensures there will be no interference from other spectrum  users in that band. License holders may sell or lease their license, in  whole or in part, to another provider, a process that is known as a  secondary market transaction, with FCC’s approval. FCC has assigned  licenses administratively in two frequency bands that can be used for  broadband services.

FCC also authorizes the use of unlicensed spectrum, where an unlimited  number of users can share frequencies without a license, such as  wireless microphones, baby monitors, and garage door openers. In  contrast to users of licensed spectrum, unlicensed users have no  regulatory protection from interference by other licensed or unlicensed  users in the bands.

In March 2010, FCC issued the National Broadband Plan that included a  centralized vision for achieving affordability and maximizing use of high- speed Internet. The plan made recommendations to FCC, including that  FCC should take into account the unique spectrum needs of tribal  communities when implementing spectrum policies and evaluate its  policies and rules to address obstacles to spectrum access by tribal  communities. With regard to tribal lands, the plan recommended that FCC  increase its commitment to government-to-government consultation with  tribal leaders and consider increasing tribal representation in  telecommunications planning. FCC established the Office of Native  Affairs and Policy in July 2010 to promote the deployment and adoption of  communication services and technologies to all native communities, by,  among other things, ensuring consultation with tribal governments  pursuant to FCC policy.

Few Tribal Entities Had Obtained Licensed Spectrum and Face Barriers Doing So

For our November 2018 report, we identified 18 tribal entities from FCC’s  license data that held active spectrum licenses in bands that can be used  to provide broadband services as of September 2018. Of those 18, 4  obtained the spectrum through a secondary market transaction and 2  from an FCC spectrum auction.

We interviewed 16 tribal entities that were using wireless technologies at  the time to provide service, and 14 told us that they were accessing  unlicensed spectrum to do so. While representatives from most of the 16  tribal entities reported some advantages of unlicensed spectrum, such as  the spectrum is available at no cost, they also discussed their  experiences with the limitations of unlicensed spectrum, including issues  with interference and speed or capacity. Some of the stakeholders we  contacted and FCC have highlighted the importance of exclusive-use  licensed spectrum for tribal entities. For example, FCC’s Office of Native  Affairs and Policy reported in 2012 that unlicensed spectrum is not an  option across all tribal lands and that tribal access to robust licensed  spectrum is a critical need. In addition, representatives from the  stakeholders we interviewed told us that there are non-technological  benefits for tribal entities to obtain greater access to licensed spectrum,  including: enhanced ability to deliver additional Internet services, enhanced ability to sell or lease spectrum for profit, and additional opportunities to obtain federal funding that requires entities  to hold or have access to licensed spectrum.

Furthermore, two tribal stakeholders and representatives from several  tribal entities told us that having access to licensed spectrum would  enable tribes to exercise their rights to sovereignty and self- determination. For example, representatives from four of the tribal entities  told us that having access to licensed spectrum would ensure that  spectrum is being used in a way that aligns with tribal goals and  community needs, further supporting their rights to self-determination.

In our November 2018 report, we described barriers tribal entities  reported facing in accessing licensed spectrum. First, representatives  from tribal entities we contacted said that obtaining a spectrum license  through an auction was too expensive for many tribal entities. Indeed,  over 60 percent (983 of 1,611) of the winning bids from a 2015 spectrum  auction were more than $1 million. Representatives from some tribal  entities told us they were unable to obtain financing to participate in  auctions because tribal governments cannot use tribal lands as collateral  to obtain loans and that participating in spectrum auctions requires  auction-specific expertise that tribal entities may not have.

Second, tribal entities reported facing barriers obtaining spectrum through  secondary market transactions. Most of the spectrum allocated for  commercial use has already been assigned through spectrum auctions  and other mechanisms to private providers that may not be providing  service on tribal lands. As such, there may be tribal areas where  providers hold licenses for bands but are not using the spectrum to  provide Internet service. All three of the tribal associations we contacted  confirmed that there were unused spectrum licenses over tribal lands,  and representatives from a nationwide provider indicated that they only  deploy services if there is a business case to support doing so.  Accordingly, the secondary market is one of few avenues available to  tribal entities that would like to access licensed spectrum. However,  representatives from tribal entities we contacted told us it could be  challenging to participate in the secondary market because there is a lack  of willing sellers, license holders are not easily identified, and tribal  entities may not be aware of how to pursue secondary market  transactions. For example, representatives from a tribal entity that had  been successful in obtaining a license through the secondary market told  us that an Indian-owned telecommunications consulting company was  pivotal in identifying the license holder and facilitating the transaction, and  without such assistance, the transaction would not have occurred.

FCC Had Taken Some Actions to Increase Access, but Does Not Collect or Communicate Key Spectrum-Related Information to Tribal Entities

At the time of our November 2018 report, we found that FCC had taken  some actions to increase tribal access to spectrum. In particular:

FCC issued a proposed rulemaking in March 2011 that sought  comments on three proposals to create new spectrum access  opportunities for tribal entities (see fig. 3). As of July 12, 2019, FCC  had not adopted new rules or taken further action on the 2011  rulemaking.

FCC issued a proposed rulemaking in May 2018 that sought  comment on establishing a priority window for tribal nations located in  rural areas to obtain a license in the Educational Broadband Service  spectrum band (also known as the 2.5 GHz band). In the proposed  rulemaking, FCC had found that significant portions of this band were  not being used, primarily in rural areas. FCC had not finalized this rule  at the time of our November 2018 report, but published a draft order in  June 2019 that would establish a priority filing window so that tribal  entities could get access to unassigned spectrum in the 2.5 GHz band  on rural tribal lands prior to an FCC auction. FCC adopted this order  on July 10, 2019.

FCC’s Office of Native Affairs and Policy conducts training,  consultation, and outreach to tribal entities on spectrum-related  issues, such as communicating with tribal entities prior to FCC  auctions or when FCC regulatory actions or policies would affect tribal  governments and spectrum over their lands.

FCC’s 2010 National Broadband Plan stated that ongoing measurement  of spectrum utilization should be developed to better understand how  spectrum resources are being used because some studies indicated that  spectrum goes unused in many places much of the time. The plan stated  that any spectrum utilization studies that FCC conducts should identify  tribal lands as distinct entities. The plan also stated that FCC should  make data available that would promote a robust secondary market for  spectrum licenses, such as information on how and to whom spectrum is  allocated on tribal lands. In FCC’s 2018 strategic plan, FCC stated that  it will implement ongoing initiatives that will assist in spectrum policy  planning and decision making, promote a robust secondary market in  spectrum, and improve communications services in all areas of the United  States, including tribal areas. Additionally, federal internal control  standards state that agencies should use quality information, including  information that is complete, to inform the decision-making processes and  communicate with external entities. Tribal governments are an example  of such external entities.

However, in our 2018 report, we found that FCC had not consistently  collected data related to tribal access to spectrum or communicated  important information to tribes. In particular:

FCC did not collect data on whether spectrum license-holders or  auction applicants are tribal entities. Without this information, FCC did  not have a comprehensive understanding of the extent that tribal  entities are attempting to obtain or access licensed spectrum or have  been successful at obtaining and accessing it.

FCC did not analyze the extent that unused licensed spectrum exists  over tribal lands, even though FCC had the information—broadband  availability data from providers and information on geographic areas  covered by spectrum licenses—needed for such an analysis.  Although FCC officials told us evaluating the effectiveness of FCC’s  secondary market policies is a way to increase the use of unused  spectrum, FCC’s approach did not include an analysis of unused  spectrum licenses on tribal lands. As a result, FCC’s evaluations of  the secondary market may not have accurately reflected how its  policies affect tribal entities. Because the secondary market is one of  few ways for tribal entities to access licensed spectrum, such an  assessment would enable FCC to better promote a robust secondary  market that provides opportunities for tribes to access spectrum.

FCC did not communicate information to tribes that could benefit them  in their efforts to obtain spectrum in the secondary market. As  described earlier, the secondary market is a significant mechanism for  tribal entities to obtain spectrum licenses, but representatives from the  tribal entities we interviewed reported challenges related to  participating in the secondary market, such as not knowing whom to  contact should they wish to engage in a secondary market transaction  to obtain a spectrum license.

We concluded that FCC’s efforts to promote and support tribal entities’  access to spectrum had done little to increase tribal use of spectrum. In  particular, FCC lacked information that could help inform its decision- making processes related to spectrum policy planning, which is intended  to improve communications services in all areas of the United States,  including tribal lands. By collecting data on the extent that tribal entities  are obtaining and accessing spectrum, FCC could better understand tribal  spectrum issues and use this information as it implements ongoing  spectrum initiatives. Furthermore, the ability of tribal governments to  make informed spectrum planning decisions and to participate in  secondary market transactions is diminished without information from  FCC on the spectrum transactions that occur over tribal lands. Providing  this information directly to tribal entities could enable them to enter into  leasing, partnership, or other arrangements to obtain spectrum. In our  November 2018 report, we recommended that FCC (1) collect data on the  extent that tribal entities are obtaining and accessing spectrum and use  this information as FCC implements ongoing spectrum initiatives; (2)  analyze data to better understand the extent that unused spectrum  licenses exist over tribal lands, such as by analyzing the data for a  sample of tribal lands, and as appropriate use this information to inform  its oversight of the secondary market; and (3) make information on  spectrum-license holders more accessible and easy to understand for  interested parties, including tribal entities, to promote their ability to  purchase or lease spectrum licenses from other providers. FCC agreed  with these recommendations and described the actions it plans to take to  implement them. For example, according to FCC, it will consider ways to  collect data on the extent to which tribal entities are obtaining and  accessing spectrum; analyze data from a sample of spectrum licenses on  tribal lands to inform FCC’s spectrum policies; and transition to a more  user-friendly system for its licensing data.

Chairman Hoeven, Vice Chairman Udall, and Members of the Committee,  this concludes my prepared statement. I would be pleased to respond to  any questions that you may have.

If you or your staff have any questions about this testimony, please  contact Andrew Von Ah, Director, Physical Infrastructure Issues at (202)  512-2834 or vonaha@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. GAO staff who made key contributions to this testimony  are Sally Moino and Anne Doré. Other staff who made contributions to  the report cited in this testimony are identified in the source product.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

Broadband service on tribal lands continues to lag behind the rest of the country, limiting tribes’ access to the Internet—especially in rural areas. In such areas, wireless broadband can help keep people connected.

We testified that the Federal Communications Commission could do more to promote and support tribes' access to radio frequency spectrum that can be used for such wireless service.

In the report on which this testimony is based, we made 3 recommendations to FCC to help improve tribes' access.