Navy Ship Maintenance: Evaluating Pilot Program Outcomes Could Inform Decisions to Address Persistent Schedule Challenges

The Navy relies on its fleet of over 150 surface ships to be ready to operate when needed for the defense of the United States. The Navy spends billions annually in maintaining this fleet. In 2015, the Navy changed how it contracts for such maintenance work, aiming to better control costs and improve quality. The new approach, called MAC-MO, generally uses firm-fixed-price contract delivery orders for individual ship availabilities competed among pre-qualified contractors at Navy regional maintenance centers.

House Report 115-676 included a provision for GAO to review the Navy's implementation of the MAC-MO strategy. This report (1) examines outcomes under the strategy; (2) evaluates actions the Navy has taken related to recent lessons learned; and (3) describes contractors' considerations when planning for hiring and facilities. GAO analyzed data on ship repair under MAC-MO; reviewed six case studies involving different availability types, classes of ships, maintenance centers, and contractors; and interviewed Navy officials and contractors.

Since shifting to the Multiple Award Contract-Multi Order (MAC-MO) contracting approach for ship maintenance work in 2015, the Navy has increased competition opportunities, gained flexibility to ensure quality of work, and limited cost growth, but schedule delays persist. During this period, 21 of 41 ship maintenance periods, called availabilities, for major repair work cost less than initially estimated, and average cost growth across the 41 availabilities was 5 percent. Schedule outcomes were less positive and Navy regional maintenance centers varied in their performance (see figure).

To mitigate these delays, the Navy has identified and taken actions to implement lessons learned, including negotiating and funding undefined but expected increases in work at the time of contract award. However, these actions have not resolved the delays that result from the approval process the Navy often must use to obtain funds to complete this maintenance work. Namely, if an availability extends into a new fiscal year and needs more than $4 million in additional prior-year funding, both Navy and Defense Department approvals are required. GAO found this approval process took between 26 and 189 days based on Defense Department data. In December 2019, Congress established a pilot program that would potentially allow the Navy to avoid this process. Leading practices GAO identified for pilot programs call for development of an analysis plan to track implementation and performance and for evaluating final results. As the Navy moves into implementation of its pilot program, developing an analysis plan would provide it with a means to identify opportunities to evaluate schedule outcomes of pilot program availabilities, as compared to non-pilot program availabilities, and document a process for evaluating lessons learned from the pilot program. Such evaluations would provide information to determine if the pilot approach should expand to help address persistent schedule challenges.

Ship repair contractors now operating in the MAC-MO environment told GAO that two key considerations drive their decisions on workforce and facilities investments: visibility regarding planned workloads within a given port and their assessment of the share of that work they are most likely to win. In recognition of these considerations, Navy officials have begun taking steps to increase predictability of workloads at each port. These officials anticipate that these steps, coupled with increasing workloads at the ports, will help increase contractors' confidence in their ability to forecast their share of future work.

GAO recommends that the Navy establish an analysis plan for the evaluation of the pilot program. The Navy concurred with GAO's recommendation.

In support of its various missions in defense of the United States, the  Navy plans to spend over $20.7 billion through fiscal year 2024 to  maintain its fleet of over 150 surface combatant and amphibious warfare  ships. The Navy relies on these non-nuclear surface ships to be ready to  operate when needed and to meet their expected service lives. To  accomplish this, the Navy sustains these ships using scheduled periods  of repair work called availabilities. Our prior work has shown that the  Navy has faced persistent challenges in maintaining its fleet, resulting in  thousands of days that ships were unavailable for training or operations.

The Navy contracts with private shipyards and other firms, which conduct  or support ship repair and maintenance availabilities. In 2015, the Navy  transitioned to a new contracting strategy with the goals to control costs,  increase competition, and improve the quality of work while supporting  operational readiness of the fleet. Under this strategy, which we refer to  as Multiple Award Contract, Multi Order (MAC-MO), the Navy generally  competes orders for individual ship repair availabilities among the  shipyards under firm-fixed-price terms, after a third-party contractor plans  the repair work. The Navy uses the MAC-MO strategy for maintenance  and repair of six classes of ships, including amphibious ships, cruisers,  and destroyers, and plans to include the Littoral Combat Ship in the near  future.

In a November 2016 report, we assessed the early implementation of the  MAC-MO contracting strategy in response to a provision in House of  Representatives Report 114-102, which accompanied the Fiscal Year  2016 National Defense Authorization Act. We found that the Navy  identified several lessons learned from pilot availabilities it had  undertaken to demonstrate MAC-MO. We concluded that the Navy was  still learning, and we subsequently recommended that that the Navy  perform systematic assessments of MAC-MO to harness new lessons  learned and effectively implement the strategy. The Navy completed its  first such assessment in December 2018 in response to our  recommendation. House of Representatives Report 115-676  accompanying the fiscal year 2019 National Defense Authorization Act  included a provision for us to review the Navy’s more recent progress in  implementing the MAC-MO contracting strategy. Specifically, this report  (1) examines competition, cost, schedule, and quality outcomes under the  strategy; (2) evaluates actions the Navy has taken related to recent  lessons learned; and (3) describes considerations informing contractors’  plans for future hiring and facilities investments under the MAC-MO  contracting strategy.

To examine competition, cost, schedule, and quality outcomes the Navy  has achieved under the MAC-MO strategy, we obtained available data on  ship repair contract competition, number of offers received, costs, and  schedules. To examine the competition outcomes of the MAC-MO  strategy, we obtained Federal Procurement Data System-Next  Generation (FPDS-NG) entries for all of the delivery orders associated  with MAC-MO contract numbers from the start of the strategy through  March 31, 2019. To assess the reliability of the FPDS-NG data, we  reviewed documentation, interviewed Navy officials, performed logic  checks, and compared the FPDS-NG data to contract documents. We  found the FPDS-NG data sufficiently reliable to examine competition  outcomes. We reviewed data for the planned cost and schedule of Chief  of Naval Operations (CNO) availabilities, as well as the actual cost and  schedule for the availabilities that the Navy closed out between February  2, 2011 and January 15, 2019.

To assess the reliability of the cost and schedule data, we (1) gathered  information from the Navy’s users of the data related to its reliability, (2)  compared different snapshots of the data over time to check the  consistency of completed entries including the version that the Navy used  to publish its first assessment of the MAC-MO strategy, and (3) compared  availability documentation from our completed case study CNO  availabilities. We found these data sufficiently reliable to examine cost  and schedule outcomes. We reviewed contracting documentation for six  ship repair availabilities under the MAC-MO contracting strategy, four of  which were completed at the time of our review, which we used as case  studies. This documentation included availability completion reports that  identified work progress along with any quality deficiencies at the  conclusion of each availability. We selected these case study availabilities  because they provided examples of performance across a mix of different  types of availabilities, repair complexity, ship classes, and locations. The  case studies are meant to be illustrative and are not generalizable.

To evaluate the actions the Navy has taken related to recent MAC-MO  strategy lessons learned, we reviewed documentation of initiatives to  improve the MAC-MO contracting approach, the Navy’s December 2018  assessment of MAC-MO implementation, and prior GAO work. We  reviewed Navy documentation and identification of key lessons learned  by Navy officials knowledgeable of MAC-MO implementation. We also  reviewed contract files for our six case study availabilities to identify  lessons learned.

To describe considerations informing contractors’ plans for future hiring  and facilities investments under the MAC-MO strategy, we conducted  semi-structured interviews with and reviewed questionnaire responses  from 11 non-nuclear surface ship repair contractors, including all eight  contractors responsible for executing major MAC-MO ship repair work.  We also randomly selected a non-generalizable sample of three small  business contractors performing noncomplex ship repair work at the three  home ports implementing MAC-MO, to obtain the views of small  businesses executing MAC-MO contracts. In addition, we obtained  information from the Navy on planned future availabilities in home ports  where MAC-MO has been implemented.

For each objective, we also interviewed Navy officials responsible for  planning, administering, and funding the ship repair contracts, including  the offices of the Deputy Commander, Surface Ship Maintenance and  Modernization, (SEA 21); Commander, Naval Surface Force, Atlantic;  Commander, Naval Surface Force, Pacific; Commander, Navy Regional  Maintenance Center; the Mid-Atlantic Regional Maintenance Center  (MARMC) in Norfolk, Virginia; the Southwest Regional Maintenance  Center (SWRMC) in San Diego, California; and the Southeast Regional  Maintenance Center (SERMC) in Mayport, Florida. These are the three  maintenance centers where the Navy is implementing the MAC-MO  strategy. See appendix I for more information about our scope and  methodology.

We conducted this performance audit from November 2018 to May 2020  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The Navy oversees the planning and execution of non-nuclear surface  ship repair and maintenance through several organizations (see fig. 1),  including the following:

The Chief of Naval Operations is the senior military officer of the  Department of the Navy and is responsible to the Secretary of the  Navy for the command, utilization of resources, and operating  efficiency of the operating forces of the Navy and of the Navy shore  activities assigned by the Secretary.

The Assistant Secretary of the Navy for Research, Development  and Acquisition, as the Navy Acquisition Executive, has overall  authority, responsibility, and accountability for all acquisition and  sustainment functions and programs, including surface ship repair and  maintenance.

Naval Sea Systems Command (NAVSEA) and its subordinate  organizations maintain ships to meet fleet requirements within cost  and schedule parameters, among other duties for combat systems  design and operation.

Types of Availabilities for Surface Ship Repair

The Navy contracts with private shipyards and other firms—collectively  known as the ship repair industrial base—for the repair and maintenance  of non-nuclear surface ships. This work may be performed in either  government-owned or contractor-owned facilities, potentially including  shipyards with piers, cranes or facilities for pipefitting and valve repair.  Certain types of work, such as inspecting, repairing or otherwise  maintaining a ship’s hull, might require placing a ship in the ship repair  contractor’s dry dock.

Ship repair availabilities can range from a few weeks to years depending  on the extent of work required and degree of complexity. The types of  availabilities include the following:

Chief of Naval Operations (CNO) availabilities accomplish major  repair work. This includes industrial maintenance requiring complex  processes to complete restorative work, such as structural,  mechanical, and electrical repairs. These may include modernization  work to upgrade a ship’s capabilities along with repair work, and can  last for over a year. Larger contractors typically execute these types of  availabilities rather than small businesses.

Continuous Maintenance availabilities (CMAV) accomplish non- major repair work, which includes routine maintenance work requiring  relatively little time compared to CNO availabilities—typically only  weeks to a few months in duration. Small business contractors  commonly execute CMAVs, and, at some ports, larger companies that  have contracts for CNO availabilities also take on this type of work.

NAVSEA’s Oversight of MAC-MO Strategy Implementation

Within NAVSEA, several organizations oversee MAC-MO strategy  implementation (see fig. 2), including key functions such as contract  administration, program management, and planning for future  availabilities.

Availability Planning under the MAC-MO Strategy

Prior to awarding a contract for ship repair work under MAC-MO, the  Navy plans and defines requirements for upcoming availabilities as  depicted in figure 3 below.

Availability Execution under the MAC-MO Strategy

Contracting for availability execution under the MAC-MO strategy differs  from that under the Navy’s previous strategy, known as Multi-Ship, Multi- Option (MSMO), in several key ways, including by calling for: establishment of fixed contractual prices and completion time frames  for an upcoming availability, rather than payment of contractors’  incurred costs; use of a third-party planning contractor under a cost-reimbursement  contract to define contract specifications, rather than relying on  planners employed by ship repair contractors;  and award of indefinite delivery contracts to multiple contractors that can  then compete for future availabilities, rather than all availabilities for a  particular class of ships going to one contractor.

Under the MAC-MO strategy, the Navy normally places fixed-price orders  for availabilities with expected durations of 10 months or less using  indefinite delivery/indefinite quantity (IDIQ) contracts. IDIQ contracts do  not specify exact times for delivery of supplies or services at contract  award; the Navy establishes those via orders placed during contract  performance. With MAC-MO, the Navy generally solicits and awards  contracts for five-year periods to a set of qualified contractors at specific  home ports. These periods include an initial execution year and four  additional option years. As a result, several qualified contractors are  available to subsequently compete for availabilities in a specific home  port under firm-fixed-price availability delivery orders until contract  expiration.

Availabilities that the Navy expects to last more than 10 months are not  restricted to the ships’ home port. This allows for contractors outside the  home port to compete for this work. The Navy then awards contracts for  these coast-wide availabilities as stand-alone contracts to a single prime  contractor, potentially at a port different from the home port of the ship.  Shorter availabilities may be limited to the home port area provided there  is adequate competition, which the Navy defines as the presence of two  or more qualified bidders. If adequate competition is not available in the  home port area, the geographic area for solicitation is expanded equally  in all directions until adequate competition exists. Figure 4 below depicts  contracting processes used for soliciting and awarding work under the  MAC-MO strategy.

In November 2016, we reviewed the Navy’s implementation of the MAC- MO strategy through pilot maintenance periods, including its potential  benefits and effects on the industrial base. We found MAC-MO had  some potential benefits compared to the previous MSMO contracting  strategy, including increased opportunities for competition and control of  costs through fixed-price contracts. We additionally found that some  contractors saw uncertainty associated with  the need to continually compete for work, which could result in  decisions to reduce their workforce and facilities, and  the stability of ship repair workloads in their ports, irrespective of  contract type.

The Navy Has Met Most of Its MAC-MO Goals, but Schedule Delays Persist

The Navy has achieved some, but not all, of the goals it set under the  MAC-MO strategy. Among the achievements, the Navy provides more  opportunities for competition—and received more offers—under MAC-MO  than under the prior strategy. Further, MAC-MO’s fixed-price contracts  help enable the Navy to ensure quality of work, and we found no  evidence of deficient work at availability completion in our review of four  completed case studies. At the same time, the Navy also desired  improved availability cost and schedule outcomes under the MAC-MO  strategy. The Navy’s results in these two areas have been mixed.  Through April 2019, the Navy had completed 41 CNO availabilities under  its MAC-MO strategy with, on average, 5 percent cost growth and 30  percent schedule growth. Unplanned work, which can often be  unavoidable in ship repair, has detracted from both cost and schedule  performance.

MAC-MO Strategy Has Increased Opportunities for Competition and Helps the Navy Ensure Quality Standards Are Met

The MAC-MO strategy has provided more opportunities than MSMO for  competition by awarding a delivery order for each ship repair availability.  The Navy has competed over 500 delivery orders under the MAC-MO  strategy from April 2015 to March 2019. This represents a departure from  the MSMO strategy under which a single contract was awarded to one  contractor to execute multiple availabilities for a class of ship. The MAC- MO strategy also allows small businesses in Norfolk and San Diego to  compete for noncomplex maintenance. Previously, under the MSMO  strategy, small businesses said that they were more likely to work as  subcontractors for the businesses that held one of the MSMO contracts.  Navy officials have since stated that small businesses are now acting as  prime contractors.

The Navy has also achieved competition for soliciting its delivery orders  under the MAC-MO strategy. According to our analysis of data from the  Federal Procurement Data System-Next Generation (FPDS-NG), from the  start of the MAC-MO strategy (April 2015) through March 2019, at least  78 percent (435 of 554) of MAC-MO awards solicited within home ports  received two or more offers. Further, in the 18 percent of instances when  the Navy awarded a delivery order after receiving only a single offer, it may have attained the benefits of having solicited that delivery order in a  competitive environment. Table 1 shows the number of offers for both  complex and noncomplex MAC-MO awards through March 2019.

The MAC-MO strategy also gives the Navy flexibility to ensure that a  contract’s quality requirements are met under a fixed price by the time of  availability completion. The Navy identified improving the quality of  workmanship as a goal when it switched from MSMO to MAC-MO. The  previous MSMO contracting strategy relied on use of cost-reimbursement  contracts, which only require the government to reimburse the contractor  its allowable incurred costs, regardless whether the contractor completed  the work. The MAC-MO strategy uses firm-fixed-price contracts, which  provide for a price that is not subject to any adjustment on the basis of the  contractor’s cost experience in performing the contract. This contract type  places upon the contractor maximum risk and full responsibility for all  costs and performance, including meeting the quality requirements of the  contract. NAVSEA officials stated that in the event that the contractor  doesn’t meet the quality terms of the contract, the Navy has two options:  (1) require the contractor to complete the deficient work, at the  contractor’s cost, to meet the specifications or (2) reduce the contract  price to reflect the reduced value of the services performed and descope  the related work requirements from the existing contract for performance  on a future availability.

Our review of availability completion reports from the four case study  availabilities that were complete at the time of our assessment, out of six  total, showed no instances where the Navy accepted quality deficiencies  at availability completion. Navy contracting officials stated that in a firm- fixed-price contracting environment, they would not agree to accept  deficient work without first obtaining concessions from the performing  contractor, which would require modifying the delivery order. In one of  these availabilities, we found evidence that the Navy elected to descope a  non-option work item and defer it to a future availability. The maintenance  team stated that this decision followed poor planning of the work item,  which would have caused delays in completing the availability if not  deferred. We also discussed these four availabilities with the responsible  Navy maintenance teams, and none of those teams reported to us any  deficient work at the time each availability completed.

CNO Availabilities Have Experienced Limited Cost Growth under the MAC- MO Strategy, but Schedule Delays and Growth Work Persist

Between April 2015, when the Navy implemented the MAC-MO strategy,  and April 2019, the Navy completed 41 CNO availabilities with an  average cost growth per availability of 5 percent, or $1.7 million in fiscal  year 2020 dollars. However, more than half of these availabilities (21 of  41) were completed at a lower cost than the Navy initially estimated. The  cost growth of the remaining CNO availabilities (20 of 41) ranged  between 1 percent and 78 percent and drove the aggregate average  increase. Figure 5 shows the variation in cost performance, or the actual  cost compared to the Navy’s estimate, for the 41 CNO availabilities.

Figure 6 shows the cost performance, or actual cost compared to the  Navy’s estimate, for the 41 CNO availabilities grouped by their location.

Figure 7 shows cost performance, or actual cost compared to the Navy’s  estimate, for the 41 CNO availabilities grouped by ship class.

Between the start of the MAC-MO strategy in April 2015 and April 2019,  the Navy completed 41 CNO MAC-MO ship repair availabilities with an  average schedule growth, or actual number of days from availability start  to completion, compared to the Navy’s estimate, of 30 percent, or 64  days. Twelve of 41 availabilities finished on time, and none finished  ahead of schedule. In addition, two availabilities more than doubled in  length, with one finishing with 123 percent schedule growth. We discuss  some factors that can contribute to schedule growth below. Figure 8  shows the schedule growth for individual CNO availabilities.

Figure 9 shows the schedule growth for the 41 CNO availabilities grouped  by location.

Figure 10 shows the schedule growth for the 41 CNO availabilities  grouped by ship class.

Navy officials stated that one potential source of delays is unplanned  work, which consists of both growth work and new work. The Navy  defines growth work as additional work that is identified or authorized  after contract award that is related to a work item included in the original  contract. We previously found that growth work contributed to cost and  schedule increases, and it remains a contributing factor. Navy officials  stated they expect some growth work in availabilities, as officials stated  that certain tasks are difficult to fully scope within the original contract.

As an example, one official stated that they cannot fully inspect ballast  tanks and accurately write work specifications for their repair until the ship  is at the repair yard and the availability has begun. Alternatively, the Navy  defines new work as any additional work that is identified or authorized  after contract award that is not related to a work item included in the  original contract. Maintenance team officials stated that new work can  originate when an item that needs repair breaks or the maintenance team  first discovers it after the Navy awards the contract. The Navy can also  add new work to an availability whenever it sees fit.

In our six case study availabilities, we found that five added growth work,  including examples of growth items that the Navy considered  unavoidable. Our analysis of RMC data showed that the USS Stout (DDG  55) CNO Availability had 60 instances of growth work that the Navy  considered unidentifiable prior to the start of the availability, including  welding for the fuel tanks and repair to the bulkheads. The maintenance  team did not consider these growth items to be unusual. Some non-CNO  availabilities, like Continuous Maintenance availabilities, are smaller in  scope and less susceptible to growth work. Maintenance team officials at  SERMC consequently stated that they can often complete CMAVs on  schedule. We found that the Navy completed one of our case study  availabilities, the USS Iwo Jima CMAV, on schedule, and maintenance  team officials stated they had time to add three new work items to the  availability.

Figure 11 describes the USS Stout (DDG 55) case study.

Figure 12 describes the USS Iwo Jima (LHD 7) case study.

According to Navy officials, managing growth work under firm-fixed-price  contracts has contributed to schedule delays. In our November 2016  report on the Navy’s transition to the MAC-MO strategy, we described the  importance of contractors and RMC staff negotiating contract changes  and agreeing on costs in a timely manner in order to minimize schedule  impact. In our current review, Navy officials stated that negotiating  change orders for unplanned work under MAC-MO is more difficult and  time consuming than under the prior MSMO strategy because the Navy  can no longer direct the contractor to continue to work without agreeing  on the cost. In one of our case study availabilities, the USS Whidbey  Island (LSD 41) CNO Availability, the maintenance team officials stated  that they had difficulties negotiating contract changes. As a result, the  officials stated that the Navy used unilateral modifications to direct the  contractor to execute growth work items and avoid further schedule  disruptions. See Figure 13 below for more detail on the USS Whidbey  Island (LSD 41) case study.

The Navy Has Taken Action to Respond to MAC-MO Lessons Learned, but Funding Process Continues to Contribute to Delays

The Navy recognizes the negative schedule outcomes it currently faces  with MAC-MO strategy implementation and has worked to mitigate them.  It has implemented new contracting provisions and is moving key  availability milestones to earlier in the process in an effort to better plan  availabilities and facilitate their on-time completion. The Navy has also  tried to better coordinate with the third-party planner to plan for  availabilities and improve schedule performance. Statutory requirements  and their implementation, however, have hindered the Navy’s ability to  further mitigate schedule delays. Specifically, the Navy must obtain  approval from the Under Secretary of Defense (Comptroller) before  funding growth work that occurs in subsequent fiscal years and exceeds  $4 million—an amount established under a 1990 law. Late last year,  Congress established a pilot program in fiscal year 2020 that affords the  Navy the ability to use procurement funds for availabilities, and these  funds remain available for obligation for three years. A congressional  statement accompanying the appropriations law that established the pilot  program states that the Navy is to submit quarterly reports on the  execution of ship availabilities funded through the pilot program.

The Navy Has Taken Action to Address Key Lessons Learned with MAC-MO Implementation

In our November 2016 report, we identified several key lessons learned  stemming from MAC-MO pilot maintenance availabilities. When we  revisited these lessons learned during interviews with Navy officials, they  discussed two persistent MAC-MO strategy attributes that remain points  of emphasis for lessons learned from 2016. These strategy attributes,  namely the use of firm-fixed-price contracts and the use of a third-party  planner, led to two new key lessons learned and another ongoing lesson  learned from our 2016 report. Most of these center on the importance of  the Navy accurately planning for and anticipating needs during  availabilities in order to avoid schedule delays—a theme that was also  evident in our November 2016 report.

According to NAVSEA leadership officials, the Navy primarily relies on  two activities to determine lessons learned and identify actions that  NAVSEA needs to take to improve ship repair maintenance, including  under the MAC-MO strategy.

Surface Team One compiles lessons learned that the individual  RMCs recommend and reviews the implementation and status of  actions to address those lessons learned.

Performance to Plan (P2P) is a data-centric, analytical approach the  Navy uses for a variety of improvement initiatives, including ship  maintenance, to clearly characterize availability performance goals  and develop solutions to improve availability duration outcomes.

As shown in Table 2 below, the Navy has developed new contracting  provisions and milestones to respond to lessons learned the Navy has  identified. Additional information on each action follows the table.

The Navy Has Recently Implemented New Contract Provisions and Revised Milestones Based on Lessons Learned to Reduce Disruptive Effects from Growth Work

In 2018, the Navy began implementing two new contract provisions  originating from lessons learned regarding the MAC-MO strategy—Small  Dollar Value Growth and Level of Effort to Completion—in an effort to  mitigate schedule delays typically associated with growth work.

The November 2018 SDVG provision specifically addressed schedule  delays due to growth items that cost $25,000 or less. Under SDVG,  during availability planning the Navy and contractor agree on a set price  to be used anytime a growth work item equal to or under the $25,000  threshold is added to the work specification. This provision eliminates the  need for the Navy and the contractor to engage in time-consuming  negotiations on small dollar items during the availability. According to the  Navy’s 2018 biennial assessment, small dollar growth work negotiations  accounted for around 70 percent of all contract changes. According to  Navy documentation, contract negotiations for small dollar growth work  caused delays of up to a week. In our discussion with officials from the  USS Whidbey Island maintenance team, they reported that the availability  required 972 contract changes, which they suggested SDVG would have  helped expedite. The Navy’s SDVG policy memo states that in using  SDVG, the contractor can now typically begin work on the growth item 24  hours after discovery. Figure 14 describes the SDVG process.

While it can expedite work on smaller dollar value items, the use of SDVG  carries cost risk for the Navy and the executing contractors, which RMC  leadership officials and contractor representatives acknowledged.  According to these officials, under SDVG the Navy, at times, will likely  pay more for growth items than it would if it devoted increased time to  negotiate prices, with the same being true for the contractors. For  example, the Navy awarded a contract delivery order for the USS  Bulkeley (DDG 84) availability in February 2019 that included SDVG. The  SDVG line item provided for up to 291 changes for growth during that  availability at a firm-fixed-price of $7,144 per change based on historical  needs of similar availabilities. This meant that the Navy could use SDVG  up to 291 times during the availability, and each of those growth items  would cost the Navy $7,144 regardless of whether the actual cost to the  contractor underran or exceeded that amount. After the contractor  identifies the in-scope growth item, the Navy only must determine that the  cost is equal to or less than the $25,000 threshold in the contract.  Nonetheless, Navy officials expressed that the benefit of significantly  decreased negotiation time outweighs the potential cost risk.

As reflected in table 2, the Navy implemented a second new contract  change process, known as LOE to Completion, in November 2018. This  process is used for growth work items when the price exceeds the SDVG  threshold of $25,000. LOE to Completion allows the Navy, within the  already awarded contract for the availability, to fund growth work that  contractors regularly discover during availability execution without having  to separately negotiate each item. Through LOE to Completion, RMC  leadership officials stated they have decreased negotiations and schedule  delays during availability execution.

LOE to Completion allows the Navy to obligate funding for labor-hours  and material costs for estimated growth work at the time of award, rather  than having to obtain appropriate funds after repair work begins. The  Navy can then use those labor-hours and materials for individual growth  work items over the course of the availability. According to RMC  leadership officials, this provision allows them to avoid incurring additional  delays. To establish the amounts of funding, the Navy reviews historical  cost for growth work by class type and whether the availability is a  docking or non-docking availability. For example, the Navy provided up to  134,002 work hours and $1.4 million for materials under the LOE to  Completion contract process for the USS Bulkeley (DDG 84) availability.  Figure 15 describes the LOE to Completion process.

Because the Navy just recently implemented this process in November  2018, it has collected only limited data to date on its effectiveness.  However, as described in figure 16, an availability involving complex ship  repair work for the USS Princeton (CG 59) included contract terms that  Navy officials described as a precursor to LOE to Completion.

In August 2019, the Navy began targeting award of delivery orders for  individual availabilities 120 days prior to the scheduled work start date.  Previously, the Navy awarded these delivery orders 60 days prior to the  scheduled work start date. According to Navy supply officials, awarding  the delivery orders 120 days prior to the start of scheduled work allows  the officials involved in the planning process to procure long lead-time  materials early enough so that material delays do not impact schedule—a  challenge they cited under the 60-day schedule. Figure 17 shows how the  change awarding delivery order 120 days before work is scheduled to  begin will affect availability milestones.

As reflected in figure 17, another change is that long lead time materials  are now ordered 365 days ahead of the start of work, as opposed to the  prior schedule of 170 days ahead. Navy supply officials said that some  materials require lead times from 1 year to 18 months. Consequently,  ordering these materials 170 days before an availability begins increased  the likelihood that they would arrive too late to fulfill the Navy’s stated goal  of procuring all materials 30 days prior to the start of repair work. Unless  repair work requiring these materials is nonessential and can be deferred  to a future availability, these material delays can delay completion of  availabilities by several months. Several ship repair contractor  representatives we interviewed with pointed to long lead-time materials as  drivers for schedule growth.

While noting the potentially positive effects of shifting award date to 120  days before the availability begins, Navy officials also raised some  challenges. They said that locking ship repair requirements almost a full  year before an availability actually begins means that the Navy could  finalize a ship’s upcoming availability work specifications before a ship  even begins its next deployment. During this deployment, equipment  breakages or other deficiencies not anticipated and subsequently not  included in the work package could arise on the ship, all of which would  likely become growth work during the availability.

This new time frame for delivery orders has only recently been  implemented. The first MAC-MO delivery order awarded 120 days prior to  the start of work occurred in January 2020, with another awarded since  then. The Navy was scheduled to award availabilities 120 days prior to  the start of work in November 2019, but, according to Navy officials,  lacked necessary funds to award several availabilities due to the  continuing resolution in place at the time. The Navy is not yet certain  whether awarding delivery orders earlier will improve the Navy’s ability to  provide long lead-time materials on time.

The Navy Has Taken Action to Address Availability Planning Lessons Learned, but Views Are Mixed on the Results

Both the Navy and the third-party planner recognize the need for the two  parties to work closely together to produce the best specifications and  work packages possible under MAC-MO. As within the Navy, third party  planner staff also seek to identify lessons learned, in order to improve the  quality of ship repair specifications they produce.

According to third-party planning contractor representatives, they monitor  contract changes involving growth work, assess whether that growth is  due to planning deficiencies or other causes, and then identify lessons  learned, which they use to improve their specification writing process. For  example, contractor representatives stated that they used lessons  learned during the USS Bainbridge (DDG 96) availability to create a  template for a section of the forecastle deck plate. This template could be  used on future availabilities for ships of the same destroyer class,  providing potential cost savings to future availabilities.

However, RMC officials across the three ports implementing the MAC-MO  strategy expressed concerns over the quality of third-party planning  contractor specifications used in ship repair availability solicitations and  contracts. They stated that the specifications developed by the third-party  planning contractors have frequently included errors and discrepancies.  As a result, the maintenance teams have had to work with the third-party  planning contractor to resolve the issues prior to award.

According to RMC officials, maintenance teams within a given port have  their own preferences with regard to how the third-party planning  contractor writes specifications. Consequently, a specification written and  approved in one RMC is sometimes deemed inadequate within another  RMC. Figure 18 describes how specification deficiencies and other  events affected a USS Roosevelt (DDG 80) availability.

Even with the issues that Navy maintenance teams have encountered  with third-party planner-developed specifications, RMC officials stated  that they continue to find ways to enhance their coordination with the  third-party planning contractor. For instance, according to MARMC  officials, they found that when availability maintenance teams physically  worked alongside third-party planning contractor staff, the planning  process went much more smoothly. After SWRMC officials learned of this  practice, SWRMC’s maintenance teams were co-located with the third- party planning contractor staff in an effort to improve its process as well.  According to RMC staff, they found that having all parties coordinating  closely in the planning process to be an effective way to mitigate some of  the specification writing issues.

In contrast to RMC officials, from NAVSEA leadership officials’  perspective, the third-party planning contractor is currently accomplishing  the goals the Navy has set forth and has provided accurate enough  specifications to earn the incentive fees outlined in its contract. The  NAVSEA officials noted that the contractor has also received annual  incentive fees for providing recommendations to the Master Specification  Catalog utilized by the Navy to incorporate lessons learned and improve  specifications written at all RMCs.

Congressional Action Offers Relief to the Navy’s Lengthy Funding Approval Processes, but Navy Does Not Have Plans to Assess Results

Historically the Navy has used its operation and maintenance account to  pay for ship repair. By law, those funds have generally only been  available for new obligations for one fiscal year—which corresponds with  the fiscal year in which the availability contract is awarded– after which  the funds expire. In order for the Navy to use any remaining expired funds  in the subsequent fiscal year for an in-scope contract change, the  executing RMC must request what is called an upward obligation. The  Navy can request an upward obligation at the fleet level as long as the  request for a specific availability is less than $4 million. RMC officials  stated this type of request involves a short process. However, if the  upward obligations request exceeds $4 million for an availability, the  executing RMC must receive approval from the Office of the Under  Secretary of Defense (OUSD) Comptroller. According to RMC leadership  officials, this process can take several months. We found that the Navy  has requested upward obligations from the OUSD Comptroller 25 times  across 14 ship repair availabilities since implementing the MAC-MO  strategy in April 2015.

In November 2016, we reported that the Navy identified the need for  training for staff on how to obtain upward obligations funding. In our  interviews with RMC leadership officials and Navy financial officials, some  said they now had experience with upward obligations because of their  regular need to obtain funding for ship availabilities that crossed fiscal  years.

Nonetheless, in our discussions with the RMC commanding officers, they  described the upward obligations process to obtain OUSD Comptroller  approval for upward obligations as cumbersome and unnecessarily  complicated. Other Navy officials and contractors echoed these views  and highlighted the upward obligations request process as a significant  impediment to schedule performance. According to the RMC  commanders, it requires several months to successfully execute and  complete the upward obligations process for many availabilities because  of reviews required within the Navy and the Office of the Secretary of  Defense before approval is granted. Navy officials said the process also  results in significant delays to the availabilities, as work cannot proceed  without funding. For example, of six availabilities for which the Navy  provided data, the shortest upward obligations request took 26 days, with  the longest request spanning 189 days. Figure 19 describes how for one  of our case studies, the USS Chosin (CG 65), the Navy experienced  several months of schedule delay due in part to the upward obligations  process.

Navy officials stated they have attempted to identify legislative solutions  to reduce the frequency under which they must obtain upward obligations,  given the negative schedule effects this process precipitates. In 2018, the  Office of the Assistant Secretary of the Navy (Financial Management and  Comptroller), in conjunction with the Office of the Under Secretary of  Defense (Comptroller), proposed two legislative initiatives to Congress  intended to accomplish this goal.

The first of these proposals seeks to raise the legal threshold for ship  repair upward obligations requiring Navy and Defense Comptroller  approval from $4 million to $10 million. The proposal also provides for a  pilot ship availability with these new thresholds, which would allow the  Navy to determine the proposal’s effectiveness before fully implementing  the new threshold. According to Navy and DOD comptroller officials, this  proposal holds merit on several levels. First, the upward obligations  threshold has not changed since 1990, when the law implementing the  process first passed. The proposed increase to the threshold would  account for inflation and subsequent increases in the cost of ship repair  over the last 30 years. For example, the average maintenance availability  for a DDG 51 Arleigh Burke class destroyer cost $6 million in 1991, but  costs $36 million when the Navy proposed the legislative change.  Additionally, the scope of the Operations and Maintenance, Navy (O&M)  budget has increased by a factor of 2.5 since the law’s 1990 passage.  Navy officials believe that increasing the threshold to $10 million would  potentially raise this amount to a level corresponding to increases in Navy  ship repair budgets since that time.

The second proposal would permit Navy O&M funds—which the Navy  uses to fund ship repair, among other sustainment-related activities—to  be available for the Navy to obligate for up to 2 fiscal years following their  appropriation by Congress. Currently, these funds are available to be  obligated by the Navy for only 1 year. According to Navy financial  officials, since most ship repairs extend into a second year, this proposal  would allow ship availabilities to avoid using upward obligations. A senior  official with the Office of the Under Secretary of Defense (Comptroller)  said that the threshold change was more logical, as the thresholds are no  longer practical, and that the logistics of implementing 2-year funding  were likely to be more complicated because of the various DOD software  systems that would be affected.

In December 2019, Congress and the President enacted legislation that— although differing from the Navy’s legislative proposals—is responsive to  the Navy’s concerns relating to the process of approving upward  obligations more than $4 million in its MAC-MO availabilities. In the Fiscal  Year 2020 Consolidated Appropriations Act, Congress established a pilot  program that allows the Navy to use the Other Procurement, Navy (OPN)  account to fund Pacific fleet surface ship repair availabilities for 2020. Our  review of Navy budget documentation shows that the Navy plans to  execute 16 pilot availabilities using fiscal year 2020 OPN funds, and it has  requested funding for another 26 pilot availabilities in fiscal year 2021.  Unlike the Operations and Maintenance, Navy account, which the Navy  typically uses to fund ship repair availabilities in 1-year increments, the  OPN account provides the Navy with funding that will not expire for 3  years. Consequently, for availabilities the Navy funds through the pilot  program, any growth work that necessitates an availability stretching into  a second or even third year will avoid upward obligations and the related  approval processes, provided sufficient funding remains in the OPN  appropriation to cover the work.

The joint explanatory statement accompanying the enacted legislation  further stated that the Secretary of the Navy is to provide quarterly reports  to Congress on the execution of ship availabilities funded through the pilot  program in the OPN account. In these quarterly reports, the Navy is to  report on the estimated or actual start or end dates of pilot availabilities,  as well as the actual funded amount and estimate to complete.

The Navy already completes systematic, biennial assessments of MAC- MO implementation, in response to our November 2016 report. While the  Navy recognized upward obligations as an issue in its 2018 biennial  assessment, the Navy did not examine potential solutions to the schedule  delays that these obligations cause. Further, according to NAVSEA  officials, the Navy has yet to determine whether it will address schedule  outcomes and lessons learned from its pilot program availabilities within  future biennial assessments.

Our prior work identified leading practices for designing a well-developed  and documented pilot program. These leading practices include the  following:

Establish well-defined, appropriate, clear, and measurable objectives

Clearly articulate assessment methodology and data gathering  strategy that addresses all components of the pilot program and  includes key features of a sound plan  Identify criteria or standards for identifying lessons about the pilot to  inform decisions about scalability and whether, how, and when to  integrate pilot activities into overall efforts

Develop a detailed data-analysis plan to track the pilot program’s  implementation and performance and evaluate the final results of the  project and draw conclusions on whether, how, and when to integrate  pilot activities into overall efforts

Ensure appropriate two-way stakeholder communication and input at  all stages of the pilot project, including design, implementation, data  gathering, and assessment  These practices enhance the quality, credibility, and usefulness of  evaluations and help ensure that time and resources are used effectively.

As the Navy moves into implementation of the OPN-funded pilot program,  establishing a plan for analysis of the pilot program would provide a  means to identify opportunities to take the data on availability schedules,  which Congress directed, and compare it to the schedule performance  the Navy has attained in its other non-pilot, MAC-MO availabilities. Such  evaluations would provide information to the Navy and Congress to  determine if the pilot approach should be expanded to help address  persistent schedule challenges. In addition, similar to the lessons the  Navy has learned in implementing the MAC-MO strategy, the Navy is  likely to learn lessons from its OPN-funded pilot availabilities, including  ones that relate to schedule drivers currently overshadowed by delays  cast by the upward obligations process. Unless the Navy documents  within an analysis plan a process for evaluating lessons learned, it runs  the risk of missing opportunities to improve its overall performance  outcomes across availabilities executed under the MAC-MO strategy.

Navy Has Taken Action to Enhance the Predictability of Increasing Maintenance Workloads in Response to Contractor Concerns

Representatives of private ship repair contractors that the Navy relies on  to execute availabilities under the MAC-MO strategy told us that their  workforce and facilities investment decisions are driven by two key  considerations. First, the contractors seek visibility on planned workload  within a given port, which, under current law, the Navy must publicly  report on a quarterly basis. Second, the contractors assess that planned  workload to determine what share of the work they are most likely to  receive. This assessment affects whether a contractor hires more or  fewer people, recapitalizes or expands facilities, and, ultimately, elects to  remain part of the Navy’s industrial base for ship repair. In recognition of  these considerations, the Navy has taken recent steps to increase  predictability of workloads at each port, for example by bundling contracts  for both sequential and concurrent availabilities. The Navy anticipates that  these steps will help further increase contractors’ confidence in their  ability to forecast their share of future workloads.

Navy Forecasted Port Workloads Continue to Fluctuate but Are Expected to Exceed Ports’ Capacities in the Near- term

As we found in our November 2016 report, various factors regarding the  Navy’s level of demand for maintenance and repair work at each of the  three home ports implementing MAC-MO, including the deployment of  ships, can affect the demand for work in each of the home ports. Based  on our analysis of Navy data, this workload remains cyclical in nature,  and at times fluctuates above and below what port capacities ordinarily  support, as it was under the prior contracting strategy.

In May 2016, we found that wide swings in port workload can have a  negative effect on the private-sector industrial base, and various factors  can affect those workloads. Subsequent to that report, Congress  required the Navy to publicly release on a quarterly basis workload  projections covering the three ports implementing MAC-MO. Navy’s  forecasts indicate that ports implementing MAC-MO will, at times during  the next 3 years, be assigned workloads beyond their current capacity,  particularly for the Southeast Regional Maintenance Center in Mayport,  Florida. Figures 20, 21, and 22 identify the Navy’s port workload  projections for each of the three ports as of December 2019.

Large Contractors Offered Mixed Views on Workforce and Facilities Investment Planning under MAC-MO

Although the Navy projects that overall workload at the ports  implementing MAC-MO will fluctuate with periodic increases, lack of  certainty about company-specific workload is driving mixed views among  contractors on their willingness to make facility and workforce  investments. Multiple contractor representatives we interviewed stated  they have always worked within an environment of peaks and valleys of  workload regardless of the Navy’s contracting strategy. Representatives  of large ship repair contractors we interviewed commented on challenges  and changes they have made to remain competitive in the MAC-MO  strategy’s competitive, firm-fixed-price contracting environment. Under  MAC-MO, which requires competition for every availability within a home  port, large contractor representatives stated that they do not have a high  level of confidence or visibility into future work that the Navy will award to  their companies. They have noted that this uncertainty has affected their  planning for hiring and facilities investments. Specifically, contractor  representatives cited the following:

Of the eight large MAC-MO contractors in our review, four reported  that they have increased their full-time workforce and the other four  have reported a decreased workforce since 2015.

Representatives of three contractors selected in our review noted that  they have had to rely more heavily on temporary labor to conduct  work on Navy availabilities because of inability to predict workloads.  For example, a representative of one large contractor noted that their  company retains a permanent core workforce, which it then  supplements with temporary labor, as needed, depending on the  number of contracts it is awarded by the Navy. Representatives of  another large contractor noted that the company recently reinstated a  training program for new ship repair workers. A representative from  the third contractor stated that the company is considering reinstating  its equivalent training program based on workload forecasts and  confidence in their amount of workload, which underpins investments  in workforce training.

Representatives of multiple large contractors in our review also stated  that they increasingly rely on their subcontractors to execute ship  repair work. For example, a representative from one noted that  although the company reduced its full time workforce, it is still able to  execute availabilities through their use of subcontractor labor. A  representative of another large contractor noted that their company  staffed a recent availability with about 70 percent subcontracted labor,  in part to help the contractor work within the contract’s price as agreed  to with the Navy and to help the company make a profit.  Representatives of another large contractor stated their company’s  preference is to use subcontractors rather than to surge its permanent  staff, especially given the contractor’s uncertainty about its portion of  future Navy ship repair and maintenance workloads.

Representatives of three of the large contractors we interviewed also  stated that unstable workloads have limited their plans for significant  capital investments in new or expanded facilities. However,  representatives of two large contractors reported making new  investments in facilities due to high volume of work at their ports. For  example, representatives of one large contractor noted that their  parent company invested $100 million into building a new dry dock as  part of the company’s commitment to win new availabilities and  complete them on schedule. The company reported that it was willing  to make this investment, in part, because Navy forecasts show an  increase in ships being homeported at that location. These contractor  representatives further stated their company is considering additional  facilities investments.

Apart from the considerations that affect their hiring and facilities  investments, representatives from all of the large companies we  interviewed told us that they plan to continue competing for Navy ship  repair work under the MAC-MO strategy. For seven out of the eight of  these contractors, the Navy is their primary customer. A representative of  one large contractor noted their company’s preference for the MAC-MO  strategy, as compared to earlier Navy contracting strategies, especially as  a means to increase its ability to propose on and compete for  availabilities. Representatives of two additional large contractors also  echoed the positive effect of increased opportunities to propose on Navy  ship repair and maintenance contracts as a means to potentially grow  their workloads.

Representatives of Selected Small Business Contractors Report Expanded Hiring and Facility Investments under MAC-MO

Representatives of the three small business contractors we interviewed  told us that they have each increased their workforces since 2015, when  the Navy began implementing the MAC-MO strategy. Under this strategy,  small businesses are able to compete for noncomplex ship repair work as  prime contractors. Overall, these small business contractor  representatives stated they intend to further grow their workforces and  facilities, correspondent with the amounts of ship repair work they  receive. Specifically, representatives of these small business contractors  told us the following:

Representatives of one small business prime contractor reported that  their company grew its workforce from 625 to 982 between December  2015 and March 2019 as they stated that MAC-MO provided  additional opportunities to propose on ship repair contracts.

A representative of another small business prime contractor we  interviewed estimated that their company hired an additional 100  personnel at two locations because of new, increased workloads  related to MAC-MO’s implementation.

One small business prime contractor included in our review completed  a major facilities expansion, including the addition of a dry dock  intended to serve all lines of business, including commercial business  customers.

A representative of one small business prime contractor stated that  their company is considering significant infrastructure upgrades and  plans to aggressively compete for noncomplex Navy ship repair and  maintenance work.

The Navy Is Taking Steps to Increase Predictability of Future Workloads

The Navy has recently begun implementing two new contractual  approaches—horizontal and vertical contract bundling—within its MAC- MO strategy, but has not yet had sufficient time to collect or assess  results. These approaches are intended to increase contractors’ visibility  into and confidence regarding future ship repair workloads. Navy  leadership officials stated that by awarding multiple availabilities, industry  receives a body of work that creates confidence in hiring and retaining a  skilled workforce and investment in infrastructure. These approaches  provide for contractors to propose on multiple ship repair availabilities that  the Navy has bundled within a single request for proposal. Figure 23  illustrates these new contractual approaches.

Horizontal Contract Bundling: Navy leadership officials testified to  Congress in October 2019 that horizontal bundling helps them decide  where to direct ship repair and maintenance work, especially as a  means to not surpass capacity at a given port. A representative of one  large contractor told us the company anticipates positive effects from  horizontal bundling to include being awarded two availabilities from  one proposal process and guarantees of work for a longer period than  one availability. Another large business contractor representative  noted that horizontal bundling would help in stabilizing workloads over  a longer period of time, which would also help with its hiring planning.  The Navy awarded its first horizontally bundled availabilities in  September 2019, and the contractor is expected to complete work on  the two ships at its shipyard in Seattle, Washington in June 2021 and  May 2022, respectively. NAVSEA leadership officials noted that Navy  intends to implement horizontal contract bundling at all of its ports in  the future.

Vertical Contract Bundling: This contract bundling approach has the  potential to allow contractors to increase their workload through only  one proposal process, as they may then have the possibility to work  on two availabilities at one time. The Navy awarded its first vertically  bundled availabilities in February 2019 to three contractors. The  second award, in September 2019, resulted in one contractor  receiving two simultaneous availabilities.

Additionally, NAVSEA leadership officials state they are undertaking other  initiatives intended to avoid (1) large fluctuations in ship repair work at  individual ports, and (2) the need for contractor workforce layoffs and  surge hiring. These initiatives are outlined in further detail below:

Attempting to Level Port Workloads: Through its P2P initiative, the  Navy intends to use historical timelines from recent availabilities to  more accurately plan and forecast future availability time frames. This  effort is using computer modeling to avoid either underutilizing or  exceeding the available port loading capacity of the industrial base in  any given timeframe. On average, NAVSEA leadership stated that  they intend to lengthen planned availability timeframes by 56 days to  more accurately reflect completion times. The officials assessed that  this strategy will help ship repair contractors better manage their  workforce planning. They further stated that if contractors have  increased visibility in port loading, they will be more likely to hire an  increased number of permanent staff in key ship repair trades.  According to NAVSEA leadership officials, this could then allow for  increased workload capacity at a given port, as those permanent— rather than temporary—staff would become more skilled over time  and therefore would require less on-the-job training.

Contractor Workforce Capacity Reporting: NAVSEA leadership  officials also noted that the Navy is considering options for including  language in future ship repair contracts requiring contractors to  identify their workforce capacity, including by trade and skill set.  NAVSEA leadership officials noted that the intention of such an  initiative would be to obtain better workforce capacity data to better  plan future port workloads.

Although the MAC-MO strategy appears to have stabilized the cost and  quality components, completing maintenance availabilities within allotted  schedules continues to elude the Navy. The Navy has taken steps to  more readily accommodate growth work needs as they emerge, however  these likely cannot completely eliminate the Navy’s need for upward  obligations. The Navy has pointed to the low cost threshold and upward  obligations approval process, as provided for in statute, as not providing it  with the agility it needs to fund growth work on a schedule that minimizes  disruption to an availability. Recently, Congress enacted legislation,  signed into law by the President, which establishes an OPN-funded pilot  program and provides the Navy a platform to potentially demonstrate that  it can meet its MAC-MO schedule goals when freed from the time  intensive process of upward obligations. Nonetheless, every pilot program  should be thought out before it starts, including consideration of what data  need to be collected and how the data will be analyzed. Otherwise, the  pilot could be poorly run or could miss opportunities to gain information  and lessons learned. Such planning for the OPN-funded pilot could  enhance the quality, credibility, and usefulness of the pilot program.

The Secretary of the Navy should establish an analysis plan for the  evaluation of OPN-funded pilot program availabilities, based on the  leading practices for pilot programs. This analysis plan should identify  opportunities to evaluate schedule outcomes of pilot program availabilities  as compared to non-pilot program availabilities and document a process  for evaluating lessons learned from the pilot program (Recommendation  1).

We provided a draft of this report to the Navy for review and comment. In  written comments provided by the Navy (reproduced in appendix II), the  Navy concurred with our recommendation.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; and the Secretary of the Navy. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or at oakleys@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

In 2015, the Navy transitioned to the Multiple Award Contract, Multi Order  (MAC-MO) contract strategy for the maintenance and modernization of  surface ships. This report (1) examines competition, cost, schedule, and  quality outcomes under the strategy; (2) evaluates actions the Navy has  taken related to recent lessons learned; and (3) describes considerations  informing contractors’ plans for future hiring and facilities investments.

To examine the competition outcomes of the MAC-MO strategy, as well  as number of offers received, we analyzed delivery orders for all of the  MAC-MO availabilities in Norfolk, Mayport, and San Diego from the start  of the strategy in April 2015 through March 2019. Navy provided a list of  MAC-MO Indefinite Delivery, Indefinite Quantity contracts and identified  which were complex and noncomplex. We used the Federal Procurement  Data System – Next Generation (FPDS-NG) to identify the delivery orders  associated with these contracts, and the number of offers received for  each order. To assess the reliability of the FPDS-NG data, we reviewed  documentation, interviewed Navy officials, performed logic checks, and  compared the FPDS-NG data to contract documents. To confirm that we  had correctly identified orders related to MAC-MO availabilities, we  reviewed the order description in FPDS-NG to confirm that it was a valid  ship repair availability and the type of availability. For cases in which the  FPDS-NG description did not contain the availability type, we obtained  the contract to confirm that it was a valid ship repair availability. To  assess the reliability of the number of offers, we performed a logic check  to confirm the number of offers received for the delivery order was  generally different from the number of offers received for the base  contract. Documents reviewed included the FPDS-NG data dictionary,  FPDS-NG data validation rules, and Fiscal Year 2013-2018 Federal  Procurement Data Quality Summary, which contains results of agency  testing of selected fields in FPDS-NG. We determined the FPDS-NG data  were reliable for the purpose of assessing the competition outcomes of  the MAC-MO strategy.

To assess the quality outcomes of MAC-MO availabilities, we reviewed  Federal Acquisition Regulations to identify differences between fixed- price and cost reimbursement contract types, and interviewed Navy  officials regarding the steps the Navy takes to manage quality in a fixed- price environment.

To examine the cost and schedule outcomes of the MAC-MO strategy,  we collected ship maintenance availability data from NAVSEA and the  Commander, Navy Regional Maintenance Center (CNRMC). This data  contained the planned cost and schedule of Chief of Naval Operations  (CNO) availabilities, as well as the actual cost and schedule for the  availabilities that the Navy closed out between February 2, 2011 and  January 15, 2019. While we were directed to assess the MAC-MO  outcomes against the Multi-Ship, Multi-Option outcomes, differences in  how the availability cost and schedule are estimated between the two  strategies prevented us from comparing their cost and schedule  outcomes. To assess the reliability of the data, we (1) gathered  information from the Navy’s users of the data related to its reliability, (2)  compared different snapshots of the data over time to check the  consistency of completed entries, including the version that the Navy  used to publish its first assessment of the MAC-MO strategy, and (3)  compared availability documentation from our completed case study CNO  availabilities. We determined the data were reliable for the purpose of  assessing cost and schedule outcomes.

To narrow our sample, we filtered the data to the ship classes and  locations covered under the MAC-MO strategy and eliminated  availabilities that had yet to report final cost and schedule entries. This  yielded 41 closed out CNO availabilities since the start of the MAC-MO  strategy in April 2015. We then adjusted all dollar values for inflation to  fiscal year 2020 dollars by using the deflators for Operations and  Maintenance funding found in table 5-9 of the Department of Defense  budget estimates for fiscal year 2020. To calculate cost and schedule  change, we determined the difference between the final cost and  completion date, and the planned cost and completion date. The planned  cost and schedule represents the Navy’s estimate at the time the Navy  awarded the contract. We then calculated the average cost and schedule  change for all 41 availabilities, as well as the availabilities at each of the  three maintenance centers and classes of ships.

To help examine the cost, schedule, and quality outcomes of the MAC- MO strategy, as well as to identify lessons learned, we selected six  availabilities as non-generalizable case studies, four of which were  completed at the time of our review. To select the availabilities, we used a  list of MAC-MO Indefinite Delivery, Indefinite Quantity (IDIQ) contract  numbers provided by the Naval Sea Systems Command. We used the  Federal Procurement Data System, Next Generation (FPDS-NG) to  collect the descriptions of contract actions to determine the ship and  availability type, estimated cost, estimated completion dates, contractor,  and place of performance. We selected a combination of six availabilities  that provided a variety of the following characteristics:

We selected two availabilities of each class of ship under the MAC- MO strategy, including destroyers, cruisers, and amphibious ships.

We selected two availabilities from each maintenance center  executing the strategy: Mid-Atlantic Regional Maintenance Center,  Southeast Regional Maintenance Center, and Southwest Regional  Maintenance Center.

We selected availabilities awarded to a variety of ship repair  contractors, including two from BAE Systems, two from General  Dynamics NASSCO, one from Marine Hydraulics International, and  one from Huntington Ingalls Industries.

We selected a variety of availability types to describe different types of  ship repair work, including two Selected Restricted Availabilities, a  Special Selected Restricted Availability, Depot Modernization Period,  Phased Maintenance Availability, and a Continuous Maintenance  availability.

For each of the case study availabilities, we collected and reviewed Navy  availability documentation including the delivery order, correspondence  between the maintenance teams and contractors, availability completion  reports, weighted progress reports at the time of completion, and briefings  containing lessons learned following completion of the availability. We  reviewed the documents to: 1) confirm our selection criteria, 2) identify  any deficiencies in quality of work and contract changes as a result, 3)  identify the presence of growth work items, new work items, or deferred  work items, 4) corroborate interview statements, and 5) identify any other  issues during the availability and solutions that could be lessons learned  for future availabilities.

Actions Taken Related to Lessons Learned

To evaluate the actions the Navy has taken related to recent MAC-MO  strategy lessons learned, we analyzed Navy documentation containing  lessons learned that aim to improve the Navy’s implementation of MAC- MO. We identified a total of three lessons learned as key based on our  assessment of the Navy’s documentation of the MAC-MO contracting  strategy. These three lessons learned were also identified as such in one  or more interviews with NAVSEA officials knowledgeable about the  challenges associated with MAC-MO implementation and the steps the  Navy has taken to fix those issues. To evaluate the Navy’s progress in  taking actions to address potential challenges posed by the key lessons  learned, we reviewed Navy documents, including Navy assessments of  the contracting strategy’s effectiveness, documents implementing revised  planning milestones and contracting processes, strategy and planning  documents, documents from availability completion meetings, case study  contract file documents and other documentation related to lessons  learned. To assess the extent to which the Navy has taken actions, we  developed the following three-point scale:

Not Complete—The Navy has not taken any action to respond to  identified lessons learned.

Partially Complete—The Navy has taken some action to respond to  the identified lessons learned, but has not completed the action  needed to address the identified risk.

Complete—The Navy has completed the action needed to address  the identified lesson learned.

Considerations Informing Contractors’ Plans for Hiring and Facilities Investments

To describe considerations informing ship repair contractors’ plans for  future hiring and facilities investments under the MAC-MO strategy, we  conducted semi-structured interviews with and reviewed questionnaire  responses from 11 non-nuclear surface ship repair contractors. This  included all eight contractors responsible for executing major ship repair  work under this strategy at the three home ports implementing it,  including Mayport, Florida, Norfolk, Virginia and San Diego, California.  We randomly selected a non-generalizable sample of three small  business contractors performing noncomplex ship repair work at the three  home ports implementing MAC-MO, to obtain the views of small  businesses executing MAC-MO contracts. We used FPDS -NG data to  identify those small businesses that have been awarded MAC-MO  delivery orders.

Further, we used a data collection instrument to gather information from  each of the selected 11 contractors on their facilities, workforce, and  sources of revenue. For example, we collected contractor-reported  information on what types of facilities the contractor owned, such as a dry  dock or a pier, the number of the contractor’s full-time staff, and the  percentage of revenue from entities other than from the Navy.

To identify the Navy’s projected workload for non-nuclear surface ships  where the MAC-MO strategy is implemented, we obtained data from the  Navy from fiscal years 2019 through the end of 2023. Since the purpose  of our analysis was to show the Navy’s projections in anticipated port  workload, we did not conduct our own assessment of the accuracy of this  data.

We also interviewed key Navy officials and reviewed statements from  testimonies to Senate subcommittees, including of the NAVSEA  Commander and of the Assistant Secretary of the Navy for Research,  Development, and Acquisition, on their approaches to provide increased  visibility and avoid large fluctuations of workloads at Navy ports, including  the three home ports implementing the MAC-MO strategy. We collected  documentation on these approaches, such as for the Performance to Plan  initiative on how the Navy intends to use computer modeling to more  accurately plan and forecast future availability timeframes, leveraging  Navy historical datasets to provide more accurate and realistic planning  forecasts.

In addition, for all three objectives, we interviewed officials responsible for  overseeing, planning, administering, and funding the Navy’s ship repair  contracts, including representatives of the Office of the Under Secretary  of Defense (Comptroller); Office of the Assistant Secretary of the Navy  (Financial Management and Comptroller); the Office of the Chief of Naval  Operations; Commander, Navy Regional Maintenance Center (CNRMC)  and Deputy Commander, Surface Ship Maintenance and Modernization  (SEA 21); Surface Maintenance Engineering Planning Program  (SURFMEPP); Commander, Naval Surface Force, Atlantic; Commander,  Naval Surface Force, Pacific; Mid-Atlantic Regional Maintenance Center  (MARMC) in Norfolk, Virginia; the Southwest Regional Maintenance  Center (SWRMC) in San Diego, California; and the Southeast Regional  Maintenance Center (SERMC) in Mayport, Florida. We additionally  interviewed management representatives of 11 ship repair contractors  included in our review and the third party planning contractor.

We conducted this performance audit from November 2018 to May 2020  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Comments from the Department of the Navy

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, GAO staff who made key  contributions to this report include Christopher R. Durbin (Assistant  Director); Sean Seales (Analyst-in-Charge); Pete Anderson; Sonja  Bensen, Lorraine Ettaro; Lori Fields; Suellen Foth, Kurt Gurka; Cale  Jones; Ethan Kennedy; Sophia Payind; and Carol Petersen.

The Navy changed how it contracts for ship repair in 2015, in an effort to get this work done on time and at lower cost.

We found that the Navy has lowered costs from initial estimates for ship repair. But that work continues to run over schedule—due, in part, to the extra time it often takes to coordinate funding for additional repairs that may have been expected but could not be quantified when the contract began. The Navy is testing a potential solution via a pilot program, but doesn’t have plans to assess its impact.

We recommended that the Navy establish plans for evaluating lessons learned from this pilot program.