Navy Readiness: Actions Needed to Evaluate the Effectiveness of Changes to Surface Warfare Officer Training

In 2017, the Navy had four mishaps at sea including two collisions that resulted in the loss of 17 sailors' lives and hundreds of millions of dollars in damage to Navy ships. In the wake of those mishaps, the Navy identified deficiencies in SWO ship-driving training and related experience as contributing factors and has undertaken a number of efforts to improve these areas.

Senate Report 115-262, accompanying a bill for the Fiscal Year 2019 National Defense Authorization Act, contained a provision that GAO assess SWO training. This report (1) describes the changes the Navy has made to SWO ship-driving training since the 2017 collisions and (2) assesses the extent to which the Navy has taken actions to evaluate the effectiveness of changes made to SWO ship-driving training. GAO reviewed and analyzed changes made to Navy training and assessment practices and related investments; interviewed cognizant officials; and conducted discussions with SWOs aboard 12 ships.

Since 2017, the Navy has made numerous changes and plans additional changes to enhance Surface Warfare Officer (SWO) ship-driving training. The Navy plans for these changes to result in a threefold increase in the number of initial ship-driving training hours for SWOs by 2021, compared with the number of training hours prior to the 2017 collisions (see fig.). The Navy added classroom and simulator time to existing training courses to improve ship-driving skills and is developing two additional simulator-based ship-driving courses planned for 2021. These plans hinge on the completion of two new simulator-based training facilities, scheduled for completion in June 2021 and in January 2023.

The Navy has relied on added skill checks conducted throughout a SWO's career to ensure that each SWO has basic ship-driving skills, but has not put key processes and assessments in place to evaluate comprehensively the effectiveness of its changes to ship-driving training. Senior Navy officials stated that it could take 16 years or more to know if the planned changes to SWO training were effective in increasing Commanding Officer ship-driving proficiency across the fleet and stated that they intend to closely monitor the implementation of changes to the training. However, GAO found that in planning an approach for evaluating the changes, the Navy has not:

(1) identified a method to solicit fleet-wide feedback on the quality of the increased ship-driving training received by SWOs;

(2) planned to routinely conduct ship-driving competency “spot checks” that were instituted after the 2017 collisions despite Navy inspectors having found concerns with more than 80 percent of SWOs' ship-driving skills;

(3) provided standard criteria to ship Commanding Officers for qualifying SWOs to drive ships, contributing to significant variance in ship-driving experience and competency levels across the fleet; nor

(4) developed a specific plan to analyze and use information from logbooks in which SWOs are to document ship-driving and related experience.

Without addressing these challenges, the Navy cannot assess in the near term if the significant investments made to expand and enhance SWO ship-driving training are effective; further adjustments are necessary; and Navy ships are being operated safely at sea.

GAO is making four recommendations to the Navy to routinely evaluate SWO training, including that the Navy collect and evaluate fleet-wide feedback on the quality of training; routinely conduct ship-driving competency assessments; provide standard criteria for qualifying ship drivers; and develop a plan to analyze and use logbook information. The Navy concurred with GAO's recommendations.

In 2017, the Navy had four significant mishaps at sea, including two  collisions that resulted in the loss of 17 sailors’ lives and hundreds of  millions of dollars in damage to Navy ships. The Navy completed two  internal reviews to identify and correct the root causes of the mishaps,  ultimately compiling 111 recommendations to improve surface fleet  readiness. In the reviews, the Navy identified numerous contributing  factors to the mishaps, including deficiencies in Surface Warfare Officer  (SWO) training and related experience. SWOs are Navy officers whose  training and primary duties focus on the safe operation of Navy surface  ships at sea, management of various shipboard systems, and the  leadership of ships’ crews. The Navy’s identified deficiencies regarding  SWOs focused on “ship-driving training” and skills related to directing the  ship crew in safely navigating and handling a ship on its intended course.  In response to the internal review recommendations the Navy has  undertaken a number of efforts to improve SWO training.

The John S. McCain National Defense Authorization Act for Fiscal Year  2019 and Senate Armed Services Committee report 115-262 to  accompany a bill for the National Defense Authorization Act for Fiscal  Year 2019 contained provisions that we review SWO training and career  paths. For this report, we (1) describe the changes the Navy has made or  planned to make to SWO ship-driving training since the 2017 collisions  and (2) assess the extent to which the Navy has taken actions to evaluate  the effectiveness of those changes. We plan to issue a separate report on  SWO career paths in the future.

For objective one, we reviewed Navy documentation from Commander,  Naval Surface Forces and Surface Warfare Officers School Command  regarding the content, purpose, cost, and status of changes made to and  planned for ship-driving training since the 2017 collisions. We focused our  analysis on changes made to SWOs’ ship-driving training at the junior  officer level as they constitute the majority of changes made to date to  address the recommendations of the Navy’s two 2017 internal reviews to  ensure safe and effective operations at sea. See appendix II for a listing  of Comprehensive and Strategic Readiness Review recommendations  specific to SWO ship-driving training and their implementation status as of  August 2019. We analyzed planned investments from fiscal year 2018  through fiscal year 2025 for the construction of two ship-driving training  facilities and the development of three ship-driving training courses, which  includes the cost of purchasing new simulators, hiring new instructors,  conducting military construction, and developing course curriculum. We  discussed implementation plans—including the status of those efforts for  addressing the 2017 internal reviews’ recommendations—with the  Commander, Naval Surface Forces; officials from Surface Warfare  Officers School Command; Office of the Chief of Naval Operations  Surface Warfare Directorate; and officials from the Readiness and  Reform Oversight Council.

For objective two, we reviewed Navy documentation and interviewed  Navy officials on how they evaluate SWOs throughout their careers,  gather and use feedback from SWOs, assess the effectiveness of SWO  ship-driving training, and use available data to inform decisions regarding  SWO training. Specifically, we reviewed the implementation of the 10  career milestone checks outlined in Naval Surface Forces Instruction  1412.5 Surface Warfare Officer Milestone Mariner Skills Assessments,  Evaluations, and Competency Checks that are to be administered during  a SWO’s career; the Navy’s efforts to collect feedback from the surface  fleet on the quality of SWO ship-driving training and the health of the  SWO community; the Navy’s 2018 Officer of the Deck competency  assessment results, criteria, and plans to continue evaluating SWO ship- driving competency; the extent to which the Navy had provided  standardized criteria for ships’ Commanding Officers to use when  evaluating SWOs for ship-driving qualification; and the format of SWO  Mariner Skills Logbooks used to track SWO ship-driving experiences,  collected and reviewed Navy policies regarding the logbooks. To do this  we compared the Navy’s practices with relevant Navy reviews,  instructions, and guidance, Standards for Internal Control in the Federal  Government, and our prior work on assessing training efforts in the  federal government. We assessed the reliability of the results of the  Navy’s 2018 Officer of the Deck competency assessments by examining  them for missing values, comparing other sources that provide the same  types of data to ensure consistency, and by interviewing knowledgeable  agency officials regarding their accuracy and completeness. In addition,  we reviewed the Navy’s internal controls for performing the assessments,  such as grading criteria and use of independent inspectors to ensure  quality and consistency in the information. We determined that the results  of the Navy’s 2018 Officer of the Deck competency assessments were  sufficiently reliable for the purposes of reporting on the number and  percentage of the graded categories.

We also visited a total of 12 surface ships within the Pacific and Atlantic  fleets from January through April 2019, selected according to which ships  and crews were available at each of the sites we visited. Aboard these  ships we held 24 group discussions with SWOs at two levels—Division  Officers and Department Heads—and 12 interviews with Executive and  Commanding Officers. In these visits we met with approximately 225  SWOs and discussed the quality of SWO initial training and how the Navy  had implemented and assessed changes to training. The SWOs with  whom we held group discussions were from ships that Commander,  Naval Surface Forces, had identified as available for discussions during  our site visits, and do not represent a generalizable sample of SWOs.

We conducted an analysis of the discussion group responses to identify  common themes and provide illustrative examples in our report. Our  scope and methodology are discussed in greater detail in appendix III.

We conducted this performance audit from November 2018 to November  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

From January through August 2017, the Navy suffered four significant  mishaps at sea that resulted in the death of 17 sailors and hundreds of  millions of dollars in damage to Navy surface ships (see fig.1).

More recently, the Navy experienced two incidents during which Navy  surface ships collided. First, on February 5, 2019, a Ticonderoga-class  guided missile cruiser—USS Leyte Gulf (CG 55)—collided with a Navy  resupply ship—the USNS Robert E. Peary (T-AKE 5)—while conducting  an underway replenishment operation off the coast of Florida. Second, on  June 21, 2019, a Freedom-class Littoral Combat Ship—the USS Billings  (LCS 15)—struck a merchant ship while leaving a pier in Montreal,  Canada. According to Navy officials, these recent incidents did not result  in serious damage to the ships or injuries to the crews but they  demonstrate the need for continued focus and attention on safe ship  driving.

As of March 2019, the Navy had approximately 8,400 SWOs—Navy  officers whose training and primary duties focus on the operation of Navy  ships at sea and the management of various shipboard systems. The  Navy expects SWOs to progress over the course of their careers from  Division Officers driving ships, to Department Heads participating in  combat operations, to Executive Officers managing ship crews, and to  Commanding Officers overseeing operations. Figure 2 below outlines a  SWO’s career progression and associated shipboard duties.

Surface Warfare Officer Training and At-Sea Experience

The Commander, Naval Surface Forces, in coordination with Office of the  Chief of Naval Operations, Surface Warfare Officers School Command,  and Navy Personnel Command, manages and provides ship-driving  training to SWOs throughout their careers. Initially, the primary focus of a  Division Officer is on leading sailors and developing ship-driving  competency, ultimately working toward qualification as an Officer of the  Deck and Surface Warfare Officer. Therefore, a large part of initial SWO  training focuses on leading a division and developing the ship-driving  skills needed to qualify and perform as an Officer of the Deck. This  training is provided during a SWO’s initial training—in the Basic Division  Officer Course—offered in Norfolk, Virginia, and San Diego, California,  and is a mix of classroom and simulator-based training.

After completing the Basic Division Officer Course, SWO candidates  begin their first at-sea assignment as Division Officers. Division Officers  have three primary roles aboard a ship:  1.  They support ship-driving operations. New Division Officers gain ship- driving experience in pursuit of the qualification to stand as Officers of  the Deck. Once qualified as Officers of the Deck, they lead watch  teams in driving ships.  2.  They support ship department operations under the supervision of  Department Heads, and are responsible for a portion of ship  equipment and operations.  3.  They lead a division of approximately 12 to 50 enlisted personnel  within departments, and are responsible for the administrative and  supervisory duties for divisions.

In addition to their Division Officer responsibilities, new Division Officers  are expected to earn qualification as a SWO by completing required education and meeting experience standards, as well as gaining  watchstanding experience and demonstrating proficiency in the execution  of their duties, according to Navy officials. These qualifications include  Officer of the Deck and Combat Information Center Watch Officer, where  an officer assists in observing and analyzing information of importance for  combat, among others. Navy officials stated that after an officer  completes their qualifications, their Commanding Officer reviews the  officer’s ability and experiences, and can grant the candidate SWO  qualification.

In June 2010, we reviewed Navy policies for surface force training— including initial SWO ship-driving training—and found that the Navy had  reduced and altered initial SWO training as an efficiency measure, but  lacked performance measures and data necessary to evaluate the impact  of changes to training programs. We found that in 2003, the Navy had  replaced its 6-month Division Officer course consisting of classroom and  simulator training with software-based training where new Division  Officers were instead expected to learn SWO skills from computer-based  education software while onboard their first ship. The Navy said this  change saved about $50 million annually, but we found that the Navy  lacked outcome-based performance measures to evaluate the effects of  these changes to training on officer performance. We recommended that  the Navy develop metrics to measure the effects of training on SWO job  performance, knowledge, skills, and abilities. The Navy concurred with  this recommendation but did not implement the recommendation for the  software training or for subsequent training programs.

The Navy Has Enhanced Ship- Driving Training Following the 2017 Collisions and Plans to Triple Training Hours by 2021

The Navy has enhanced ship-driving training for SWOs at the early  stages of their careers following the 2017 collisions at sea, and by 2021  plans to triple the number of ship-driving training hours when compared  with the amount of training SWOs were required to receive prior to the  collisions. The Navy’s plans to increase ship-driving proficiency hinge on  the completion of two new simulator-based training facilities—the Mariner  Skills Training Centers—which are planned to be completed in June 2021  (San Diego, California) and in January 2023 (Norfolk, Virginia). Overall,  the Navy plans to invest more than $467 million to develop new ship- driving training courses, build simulator facilities, and deliver the training  through fiscal year 2025.

Prior to the 2017 ship collisions, SWOs were required to complete 174  hours of ship-driving training during their Division Officer assignment by  attending the Basic and Advanced Division Officer training courses.  Following the collisions, the Navy increased the amount of required ship- driving training in these two courses to 203 hours. In June 2019, the Navy  added a 4-week ship-driving course—the Junior Officer of the Deck  course—that focused exclusively on building ship-driving skills. This  course added 158 hours of required classroom and simulator ship-driving  training. In June 2021, the Navy plans to expand the curriculum of the  Junior Officer of the Deck course and rename it the Officer of the Deck  Phase I course, and add an additional 3-week Officer of the Deck Phase  II course. These two courses will add an additional 185 hours of required  ship-driving training for Division Officers in preparation for their first and  second at-sea assignments. Once these ship-driving training courses  are in place, Division Officers will be required to complete a total of 535  hours of training—triple (a threefold increase in) the number of ship- driving training hours SWOs were required to complete prior to the 2017  collisions (see fig. 3).

Below are detailed descriptions of the changes completed and planned to  enhance ship-driving training.

Basic Division Officer Course. From November 2017 through January  2019, the Surface Warfare Officers School Command changed the Basic  Division Officer Course—a 9-week course for new SWO candidates—by  increasing the required hours of classroom instruction and simulator  training by 12 percent, and broadening the course curriculum.  Specifically, prior to the 2017 collisions, SWO candidates were required  to spend 113 hours (81 hours of classroom instruction and 32 hours in  simulators) in this course to develop their ship-driving skills. After January  2019, however, SWO candidates were required to spend 126 hours (89  hours of classroom instruction and 37 hours in simulators) to develop  their ship-driving skills. Regarding added course content, the Surface  Warfare Officers School Command added subject matter including  additional training on the internationally accepted ship-driving standards  that govern ship maneuvers; radar navigation; and the tools used to aid  ship-driving.

Advanced Division Officer Course. From November 2017 through  January 2019, the Surface Warfare Officers School Command changed  the Advanced Division Officer Course—a 5-week course for SWOs  returning from their first at-sea assignment—to improve ship-driving skills  by increasing the required hours of simulator training from 24 to 36 hours.  Prior to the 2017 collisions, SWOs were required to spend 61 hours (37  hours in the classroom and 24 hours in simulators) refining their ship- driving skills in this course. As of January 2019, SWOs were required to  spend 77 hours (41 hours in the classroom and 36 hours in simulators)  developing and honing their ship-driving skills. Surface Warfare Officers  School Command officials also added subjects to classroom time to build  on the subject matter presented in the Basic Division Officer Course,  including more complex ship-driving techniques and advanced radar  navigation. Surface Warfare Officers School Command plans to reduce  the hours of training in this course once the Officer of the Deck Phase II  course comes online in 2021.

Junior Officer of the Deck course. In June 2019, Surface Warfare  Officers School Command provided this new 4-week course for the first  time—the course having been developed after the 2017 collisions and  focused predominately on building ship-driving skills. The Junior Officer  of the Deck course takes place after SWO candidates complete the Basic  Division Officer Course and before they begin their first at-sea  assignment. SWOs taking this course are required to complete 158 hours  of classroom and simulator training designed to increase their ship-driving  skills by exposing them to a variety of scenarios involving different  maneuvers, and varying sea and weather conditions. The Navy plans to  expand this course into a 6 week ship-driving training course (Officer of  the Deck Phase I), scheduled to begin in June 2021.

Officer of the Deck Phase I course. According to Commander, Naval  Surface Forces documentation, the Junior Officer of the Deck course will  expand into the Officer of the Deck Phase I course. Officer of the Deck  Phase I is under development and will be 6 weeks long (an additional 2  weeks longer than Junior Officer of the Deck), and will take place after  SWO candidates complete the Basic Division Officer Course and before  they begin their first at-sea assignment. Officer of the Deck Phase I is  intended to build on the Junior Officer of the Deck curriculum by  increasing the required number of ship-driving training hours from 158 to  241, and expanding the course content to include instruction on more  advanced radar navigation techniques. Surface Warfare Officers School  Command and Commander, Naval Surface Forces officials expect the  Officer of the Deck Phase I course to begin in June 2021.

Officer of the Deck Phase II course. According to Navy documentation,  the Officer of the Deck Phase II course that is under development will be  3 weeks long, and will take place after SWOs have completed their first  at-sea assignment and before they attend the Advanced Division Officer  Course. This course is intended to continue the development of ship- driving skills through an additional 102 hours of required classroom and  simulator training. Surface Warfare Officers School Command and  Commander, Naval Surface Forces officials stated that Officer of the  Deck Phase II course could begin as early as June 2021.

Mariner Skills Training Centers. According to Commander, Naval  Surface Forces and Surface Forces documentation, Surface Warfare  Officers School Command will provide the Officer of the Deck Phase I  and Phase II courses at the Mariner Skills Training Centers—new  simulator-based facilities expedited after the 2017 collisions. Officials  from the Office of the Chief of Naval Operations stated that these  facilities—including upgraded simulators, the instructors, classrooms, and  the curriculum development for the Officer of the Deck Phase I and Phase  II courses—will cost approximately $467.5 million through fiscal year  2025. According to Navy officials, construction on the

San Diego, California Mariner Skills Training Center will begin in early  fiscal year 2020 and will be complete by June 2021 and

Norfolk, Virginia Mariner Skills Training Center will begin in fiscal year  2021 and will be complete in January 2023.

The Mariner Skills Training Program is based upon the Littoral Combat  Ship ship-driving training program, which according to the Navy, provides  a balance of classroom, simulation, and shipboard experience.

According to Navy officials, since Littoral Combat Ship SWOs serve in  rotating crews and have less opportunity to train aboard their ships, the  Navy developed the Littoral Combat Ship Training Facility to support  SWOs’ training ashore (see fig. 4). The foundation of the Littoral Combat  Ship ship-driving program is repetitive training in sophisticated simulators  to build ship-driving proficiency. According to the Navy, the effectiveness  of this training has been validated over the last 10 years by the superior  ship-driving proficiency of Littoral Combat Ship officers during at-sea  operations and assessment performance when compared with non- Littoral Combat Ship officers, in many cases.

The Navy is Implementing Additional Skill Checks but Has Not Taken Other Actions Necessary to Evaluate the Effectiveness of Changes to SWO Ship-Driving Training

The Navy has relied on a series of added skill checks throughout a  SWO’s career to help validate that SWOs have necessary ship-driving  and other skills, but has not developed key processes and assessments  to evaluate the overall effectiveness of its existing and planned training  programs.

The Navy is Implementing Additional Skill Checks to Be Conducted throughout SWO Careers

The Navy is implementing a series of ten skill checks on ship-driving and  other mariner tasks at various career points—for example, before a SWO  begins leading a ship department and before the SWO takes command of  a ship. The Commander, Naval Surface Forces, issued an instruction in  September 2018 detailing ten skill checks to be conducted over the  course of a SWO’s career to periodically gauge SWOs’ ship-driving  skills. These checks, summarized in appendix IV, are to occur at  standardized points in a SWO’s career, either during training or at the  beginning or conclusion of certain at-sea assignments. Four of the ten  checks were already in place at the time the instruction was issued in  September 2018, with a preliminary version of a fifth check also in place.  According to Navy documentation, three more of the ten checks had also  been implemented as of August 2019, and Navy guidance states that the  remaining checks are scheduled to be in place by 2021 or earlier. Navy  officials stated they were making these checks more rigorous. For  example, according to Navy officials, previously Department Heads were  allowed to retake the Command Qualification Assessment ship-handling  test as many times as they needed to pass the assessment. According to  these officials, in 2018 Surface Warfare Officers School Command  allowed only three chances to take the test, leading to five of the 256  Department Heads assessed in 2018 to be disqualified from advancing  beyond the role of Department Head.

Navy officials report that these skill checks are intended to enhance the  development and sustainment of ship-driving proficiency across a SWO’s  career and to ensure that the changes in training are resulting in  competent SWOs at each level of their careers—essentially that SWOs  have the skills required to perform their duties. Surface Warfare Officers  School Command will administer checks during SWO training on ship- driving to better evaluate individual proficiency and target remediation for  those whose performance presents significant concerns. Ship  Commanding Officers will also observe and evaluate SWOs on a series  of ship-driving scenarios before the completion of their first Division  Officer assignment and later as a Department Head to certify that they  are prepared for more advanced ship-driving training and responsibilities.

The Navy Has Not Put Key Processes and Assessments in Place to Evaluate the Effectiveness of Changes to SWO Training

While the planned skill checks are designed to help ensure that SWOs  have the skills required to perform their duties, senior Navy officials stated  that it could take 16 years or more to know if the planned changes to  SWO training were effective in increasing Commanding Officer ship- driving proficiency across the fleet. These officials stated that they intend  to closely monitor the implementation of changes to the training; however,  we found a number of interrelated challenges that limit the Navy’s ability  to determine in the near term if the significant investments it is making to  expand and enhance SWO ship-driving training are effective. Specifically  and described in detail below, in planning an approach for evaluating its  efforts, the Navy has not (1) solicited fleet-wide feedback on the quality of  the increased ship-driving training, (2) planned to routinely conduct ship- driving competency assessments, (3) provided standard criteria for  qualifying Officer of the Deck candidates, and (4) determined how to  analyze and use information from logbooks that SWOs are required to  complete. The Navy’s Comprehensive Review of Recent Surface Force  Incidents—one of the internal reviews completed after the 2017  mishaps—notes the importance of assessing and monitoring performance  so that corrective actions can take place. In addition, federal  government internal control standards state that management should use  quality information and monitoring activities to ensure the agency’s  objectives are achieved. Moreover, our prior work on assessing training  efforts in the federal government states that an agency should evaluate  the effectiveness of its training and development efforts, to include  obtaining feedback, assessing competency, and analyzing relevant  data.

The Navy Has Not Solicited Fleet-Wide Feedback on the Quality of Increased Ship- Driving Training

We found that while the Navy collects feedback from certain groups of  SWOs, it did not have a formal fleet-wide process to solicit feedback from  SWOs on the quality of the increased amount of ship-driving training or to  gauge the health of the SWO community. In group discussions we held  as part of our review, SWOs identified challenges that Division Officers  experience in applying classroom and simulator training to their duties.  According to SWOs in 19 of 24 group discussions with Department Heads  and Division Officers, Division Officers have challenges in applying the  ship-driving training they receive, due to factors such as differences  between training curriculum and actual duties, extended lengths of time  elapsed between training and application, varying ship-driving  opportunities during Division Officer assignments, and difficulty retaining  the large volume of course material.

SWOs that participated in our discussion groups and interviews identified  positive aspects of ship-driving training, as well as concerns about  training material.

During five of 12 ship group discussions with Division Officers, those  Division Officers that had taken the Basic Division Officer Course  identified positive aspects of the training such as valuable practical  exercises and simulator time. However, SWOs in all 12 Division  Officer group discussions also identified challenges related to this  training, such as the information covered in training being too broad,  and a lack of connection to actual duties on their ship.

More experienced Division Officers in four of 12 Division Officer group  discussions identified challenges related to the Advanced Division  Officer Course, such as insufficient time in ship-driving simulators,  and too much time spent covering material that Division Officers were  already expected to learn during their first at-sea assignment.

Commanding Officers and Executive Officers in seven of 12  interviews, and Department Heads in four of 12 group discussions  likewise identified positive aspects of the Basic Division Officer  Course, such as improved knowledge of ship operations for Divisions  Officers that recently completed the course. However, Commanding  Officers and Executive Officers in three of 12 interviews and  Department Heads in seven of 12 group discussions identified  challenges with the course, including areas where they had to  compensate with on-the-job training for skills they felt should have  been addressed in initial training, such as ship-driving proficiency in  high-traffic environments.

Our prior work on assessing training efforts in the federal government  states that an agency should evaluate the effectiveness of its training and  development efforts, to include obtaining and analyzing feedback.  However, the Navy does not currently have a formal fleet-wide method of  soliciting feedback from SWOs to obtain input on the quality of their  classroom, simulator, and at-sea training on Division Officer performance  and evaluate trends in feedback, and instead uses more limited means to  assess training. For example:

According to Navy officials, Surface Warfare Officers School  Command conducts end-of-class surveys at the end of officer training,  but no follow-up is conducted by the command after SWOs have  assumed their ship duties or to obtain input from the trainees’ superior  officers on the value of the training.

The Navy had a survey for Division Officers and Department Heads in  the past, but this survey gave little helpful feedback on training and,  according to Navy officials, the Navy discontinued the survey after  2015.

Surface Warfare Officers School Command assembles a board of  officers from the fleet each year to review areas of its training  curriculum, but Navy officials stated that participants are invited based  on their expertise. As a result, only those selected to serve on the  board (not officers across the fleet) have the opportunity to provide  feedback.

The Navy’s current means to assess training do not allow for the full  range of junior and senior officers across the fleet to provide feedback on  how well training prepares SWOs for their ship duties. Senior Navy  officials acknowledged the value of conducting fleet-wide surveys of  SWOs to obtain feedback on how to improve SWO training and gauge the  health and morale of the SWO community. SWOs’ experiences in the  fleet are diverse, therefore fleet-wide data is of particular value as  centralized organizations like Naval Surface Forces, Surface Warfare  Officers School Command, and the Office of the Chief of Naval  Operations consider costly and consequential training investments.  Without a method to regularly collect and analyze information from SWOs  across the fleet, such as in a survey, regarding the quality of the  increased classroom, simulator, and at-sea training on Division Officer  performance, and evaluate trends in feedback received, Navy decision  makers lack valuable information that could help them to assess the  effects of training on SWO performance.

The Navy Developed a Ship- Driving Competency Assessment Conducted by Independent Inspectors, but Has Not Fully Planned to Routinely Conduct the Assessments

Navy Surface Warfare Officer School Command training experts  developed a ship-driving proficiency measurement system and used it in  fiscal year 2018 to conduct ship-driving competency assessments.  Specifically, from January through March 2018 Surface Warfare Officers  School Command conducted “spot check” ship-driving competency  assessments of 164 SWOs that had recently qualified as Officers of the  Deck during their first at-sea assignment. Each assessment was  conducted by three Navy inspectors that were independent of the  assessed SWOs’ chain of command. The independent Navy inspectors  found concerns in the ship-driving competency levels of more than 80  percent of these SWOs (see fig. 5). Specifically, Surface Warfare Officers  School Command found that 29 SWOs (18 percent) had significant  competency problems and 108 had some concerns (66 percent).  According to Surface Warfare Officers School Command officials, those  SWOs who experienced significant problems in their assessments likely  should not have been qualified as Officer of the Deck at the time of the  assessment because they violated fundamental ship-driving rules, among  other issues. Navy guidance to the fleet emphasizes that these  assessments performed by independent experts are valuable in  supporting impartial results and providing quality information for  analysis.

According to Navy documentation, the Navy also used the 2018  competency assessments to help validate its new Junior Officer of the  Deck and Officer of the Deck training curriculum. Specifically, Surface  Warfare Officers School Command used the same Officer of the Deck  competency assessment criteria to assess six officers in May 2018 and  12 in July 2018 that completed a pilot version of the Junior Officer of the  Deck course. Surface Warfare Officer School Command found that the  students with no at-sea experience that had completed a pilot of the new  Junior Officer of the Deck training course in some cases outperformed  qualified Officers of the Deck that had over a year of at-sea experience.  According to Navy officials, the ability to compare ship-driving proficiency  among populations and with earlier baselines using these competency  assessments was valuable to the Navy in identifying the effects of  changes to training, and could also be valuable in the future, as well.

However, when we visited Surface Warfare Officers School Command in  February 2019, officials told us they did not plan to conduct additional  competency assessments until 2020. In meetings with Surface Warfare  Officers School Command and senior Navy leaders, we noted that  delaying additional assessments could limit visibility over ship driving  proficiency trends and that small sample sizes could affect the Navy’s  ability to make comparisons over time. In response, the Navy accelerated  and expanded additional competency assessments. According to Navy  officials, in spring 2019, Surface Warfare Officers School Command  began to assess a sample of Division Officers using the Officer of the  Deck competency assessment at the beginning of each Advanced  Division Officer Course to collect and analyze performance data and  refine training curriculum. Further, as of July 2019, the Navy had  assessed 38 SWOs from three courses and found that the proficiency  level of the SWOs assessed had not improved from the proficiency levels  seen in the 2018 assessments.

Senior Navy officials we met with as part of this review stated that they  recognize the value in implementing periodic ship-driving competency  assessments by independent inspectors to identify trends in ship-driving  proficiency over time. However, we also found that the Navy has not  planned to routinely conduct these assessments in the future.  Specifically, in July 2019, Navy officials stated that they do not plan to  complete these Officer of the Deck competency assessments beyond  2021 and plan to replace them with a different assessment at the end of  the planned Officer of the Deck Phase II course. However, our analysis  shows that mid-fiscal year 2024 is the first time Officer of the Deck Phase  I course graduates will have completed their first at-sea assignment and  be available to have their ship-driving training assessed, resulting in a  multi-year gap in planned competency assessments. In order to measure  the effectiveness of the full complement of Navy’s new and enhanced  ship-driving training, the independent Navy inspectors will need to  continue administering the Officer of the Deck competency assessments  beyond 2021. In addition, an assessment performed at the end of  training, such as the planned Officer of the Deck Phase II assessment,  indicates the SWOs’ proficiency after additional training and may give a  less accurate indication of prior at-sea proficiency. According to federal  government internal control standards, management should use quality  information and monitoring activities to ensure the entity’s objectives are  achieved. Moreover, our prior work on assessing training efforts in the  federal government states that an agency should evaluate the  effectiveness of its training and development efforts, to include assessing  competency and analyzing relevant data. Without routinely conducting  Officer of the Deck competency assessments across the fleet using  samples of sufficient size and selection methods, the Navy will be  hindered in its ability to gauge fleet-wide ship driving proficiency trends  and determine the effectiveness of the changes made to training, and the  Navy may not know whether additional changes are needed.

The Navy Lacks Standard Criteria for Informing the Qualification of Officer of the Deck Candidates

We found that the Navy has not provided standard criteria to ship  Commanding Officers on fleet-wide ship driving proficiency expectations  to inform the qualification of Officer of the Deck candidates. Instead, the  Navy has determined that ship Commanding Officers should use their  individual judgment in granting this qualification based on a set of  required officer experiences, which the Navy refers to as Personnel  Qualification Standards. Following the 2017 collisions, Surface Warfare  Officers School Command developed proficiency standards to measure  and test Officer of the Deck ship-driving proficiency to implement the  Officer of the Deck competency assessments described above. The  proficiency standards require an Officer of the Deck to demonstrate  knowledge of navigation systems, rules of the road, and effective bridge  resource management and to demonstrate the ability to successfully  navigate high-traffic environments.

However, the varying at-sea experiences of officers and subjective nature  of some requirements have led to different experiences for SWO  candidates working to qualify as Officer of the Deck. SWOs must  complete a standard series of requirements in ship-driving and other  experience before they are eligible to qualify as Officer of the Deck, with  Commanding Officers granting qualification after their assessment of the  SWO’s performance and fitness. However, when we held group  discussions with SWOs on ships in the fleet, SWOs in nine of 12 group  discussions with Division Officers, eight of 12 group discussions with  Department Heads, and three of 12 interviews with Commanding Officers  and Executive Officers identified significant differences in opportunities,  experiences, and assessments that Division Officers experience in  earning their qualification as Officers of the Deck during their first Division  Officer assignment. For example:  In one group discussion, Division Officers reported being qualified as  Officers of the Deck without ever having stood watch at sea, with the  Commanding Officer granting qualifications based on their classroom  and simulator experience alone.

In five of 12 group discussions with Division Officers, Division Officers  stated that SWOs on ships in maintenance had few opportunities to  stand watch on the bridge at sea to build proficiency in difficult ship- driving operations, but still received their qualifications.

SWOs in 17 of 24 group discussions stated that some Division  Officers get more ship driving experience than others before earning  their Officer of the Deck qualifications. For example, Division Officers  assigned to ships with more time at sea or fewer Division Officers get  more experience to practice ship driving than those on ships with little  time at sea or that must divide ship-driving opportunities among  numerous Division Officers.

Commanding Officers in three of 12 interviews reported that they had  to temporarily place their Division Officers on other ships to gain  qualifying experience, and had to rely on the judgment of the other  ships’ Commanding Officers in determining their qualifications as  Officers of the Deck.

According to Navy officials, the Navy has not provided Officer of the Deck  assessment criteria based on the developed proficiency standards to ship  Commanding Officers, out of deference to their judgment in interpreting  an officer’s preparedness to drive their ship. Navy officials emphasized  the importance of allowing ship Commanding Officers to make their own  determination of an officer’s preparedness to drive a ship, due to their  knowledge of the ship’s operating conditions. Navy officials also stated  that they considered the Officer of the Deck assessment standards to be  a resource for use by Surface Warfare Officer Schools Command in  assessing training curriculum and had not considered using the standards  in the fleet for other purposes. However, the Navy’s 2018 and 2019  Officer of the Deck competency assessments identified significant  variance in the ship-driving competency levels of recently qualified  Officers of the Deck.

Since the Navy has developed fleet-wide standards for assessing Officer  of the Deck proficiency, the Navy could use these to provide standard  Officer of the Deck assessment criteria in guidance to ship Commanding  Officers. Federal government internal control standards state that  management should internally communicate the necessary quality  information to achieve the entity’s objectives and ensure decisions are  made based on consistent standards. A SWO’s assigned ship,  Commanding Officer, and operating conditions may change during a  career, so a standard set of criteria would help Commanding Officers to  determine what is expected of Officers of the Deck elsewhere in the fleet  as they determine a junior officer’s qualification. Without providing  standard Officer of the Deck assessment criteria and incorporating them  into surface fleet guidance to Commanding Officers, the Navy risks  creating uncertainty in Officer of the Deck qualification expectations— which can contribute to variations in ship-driving proficiency among  SWOs that could jeopardize safe operations at sea.

The Navy Has Not Determined How to Analyze and Use Surface Warfare Mariner Skills Logbook Data

In September 2018, the Commander, Naval Surface Forces, U.S. Pacific  Fleet and Commander, Naval Surface Forces Atlantic, began requiring  SWOs to document their ship-driving and related experience in a  handwritten logbook. The logbook—referred to as the Surface Warfare  Mariner Skills Logbook (see fig. 6)—captures an officer’s experience  gained during each watch aboard a ship, special evolution (e.g.,  underway replenishment, flight operations, and sea and anchor duty), and  simulator training session.

During our ship discussion groups, SWOs at the Division Officer and  Department Head levels reported that they had begun filling out their  logbooks and having them reviewed as required, but some acknowledged  that they are inconsistently filling them out or that they were not entering  any information in them. Specifically, SWOs in five of 24 discussion  groups reported that logbooks are completed with inconsistent quality or  not completed at all. Additionally, SWOs in four of 24 discussion groups  reported that they are unaware of any plans to use the logbook  information to identify any additional training needs and provide  opportunities for SWOs to improve their ship-driving proficiency. Navy  Personnel Command officials told us that, as of July 2019, they had  received 174 summaries of Surface Warfare Mariner Skills Logbook data from Commanding Officers. Navy officials stated that over time, as they  gather these data, they intend to examine the link between ship-driving  proficiency and SWO experience. However, officials did not have any  specific, measurable plans to analyze and use these data or to assess  the completeness of these data.

Federal internal control standards state that management should obtain  relevant data from reliable sources and process those data into quality  information to aid decision-making. Furthermore, Naval Surface Forces  guidance states that the surface warfare community should analyze and  use logbook data to link SWO experience with ship-driving proficiency.  Despite this guidance, the Navy does not yet have a plan that includes  specific steps to analyze and use logbook information to link SWO  experience with ship-driving proficiency. According to senior Navy  officials, while the Surface Warfare Mariner Skills Logbook is still  relatively new, developing a plan to use the information would be a logical  next step. Without a plan for analyzing and using Surface Warfare  Mariner Skills Logbook data, the Navy cannot determine the relationship  between SWO experience and ship-driving proficiency or use these data  to aid decision-making.

SWOs play a critical role in Navy surface fleet readiness, as they are  responsible for safely driving ships at sea and successfully leading ships  in Navy operations across the world. The Navy is making numerous  changes and investments to enhance Surface Warfare Officer ship- driving training following the 2017 collisions at sea—with plans to triple  initial training hours and spend nearly half a billion dollars to build  simulator capacity to deliver this training. The Navy’s oversight of these  efforts is centered on a series of added checks throughout SWOs’  careers to ensure that they have basic ship-driving and other skills. These  checks are steps in the right direction but may not provide adequate  assessment mechanisms in the near term and might lead to missed  opportunities going forward. For example, the Navy is expanding its ship- driving training but is not planning to collect fleet-wide feedback on  classroom, simulator, and at-sea training received. In addition, the Navy  developed standards for conducting spot checks on ship-driving  competency but is planning to stop those checks in 2021, missing an  opportunity for an outside assessment and to evaluate how well new and  updated training is working. Moreover, ship commanders are expected to  qualify SWOs on ship driving but have not been provided standard  guidance for how to do this, which can contribute to wide variations in  SWO competence. Finally, the Navy has developed detailed logbooks for  SWOs to track their experiences but the Navy has not developed a  specific plan to analyze and use the logbook data. Without actions to  address these challenges, the Navy cannot fully assess in the near term if  the significant investments it is making to expand and enhance SWO  ship-driving training are effective; further adjustments are necessary; and,  ultimately, Navy ships are being operated safely at sea.

We are making the following four recommendations to the Department of  Navy:  We recommend that the Secretary of the Navy ensure that the  Commander, Naval Surface Forces, in coordination with Surface Warfare  Officers School Command, develop a method to regularly collect  feedback from SWOs across the fleet, such as in a survey, regarding the  quality of their classroom, simulator, and at-sea training on Division  Officer performance; and evaluates trends in the feedback received for  the purpose of improving SWO training. (Recommendation 1)

We recommend that the Secretary of the Navy ensure that the  Commander, Naval Surface Forces, routinely conduct regular Officer of  the Deck competency assessments using samples of sufficient size and  using selection methods to gauge the level of fleet-wide ship-driving  proficiency trends following the implementation of the planned ship- driving training programs. (Recommendation 2)

We recommend that the Secretary of the Navy ensure that the  Commander, Naval Surface Forces, in coordination with Surface Warfare  Officers School Command, provide Commanding Officers with standard  criteria to inform their evaluation of candidates for their Officer of the Deck  qualification and incorporates these criteria into surface fleet guidance.  (Recommendation 3)

We recommend that the Secretary of the Navy ensure that the  Commander, Naval Surface Forces, in coordination with Surface Warfare  Officers School Command, develop a plan to analyze and use Mariner  Skills Logbook information to inform decision-making. (Recommendation  4)

We provided a draft of this report to DOD for review and comment. In  written comments provided by the Navy through DOD (reprinted in their  entirety in appendix V), the Navy concurred with all four of our  recommendations and identified actions it plans to take to evaluate the  effectiveness of changes to SWO training. The Navy also provided  additional information and context in its comments and provided technical  comments, which we incorporated as appropriate.

The Navy concurred with our first recommendation that the Commander,  Naval Surface Forces, in coordination with Surface Warfare Officers  School Command, develop a method to regularly collect feedback from  SWOs across the fleet, such as in a survey, regarding the quality of their  classroom, simulator, and at-sea training on Division Officer performance;  and evaluate trends in the feedback received for the purpose of improving  SWO training. The Navy stated that it plans to explore additional means  of garnering holistic SWO feedback regarding newly-implemented SWO  training and assessments as well as gathering additional targeted  feedback. However, the Navy stated that the use of performance data will  remain the primary focus of surface force training improvement efforts.  While using performance data is valuable, it will be important that the  Navy follow through to develop a holistic means of collecting feedback,  such as in a survey of SWOs across the fleet, on the effectiveness of  Division Officer training on SWO performance to ensure a variety of  perspectives are considered.

In its comments, the Navy noted that the SWOs who participated in our  ship visits and discussion groups had not experienced the changes made  or planned to SWO training. We agree that our ship visits did not include  officers who had experienced the expanded Division Officer training  courses, as they were first introduced to the fleet in June 2019, after we  had completed the majority of our work. While our discussion groups pre- date the implementation of new SWO training courses, the discussion  groups we conducted with over 200 SWOs reinforced the our finding that  the Navy needs to develop a method to regularly collect feedback from  SWOs across the fleet.  Also, the Navy plans to more than triple initial  training, so routinely soliciting and analyzing feedback from SWOs on  Division Officer training will be needed to determine the effectiveness of  the Navy’s investments in these training programs and inform the Navy’s  decisions on whether further adjustments are necessary.

The Navy acknowledged that only officers participating in the Surface  Warfare Officers School Command’s Board of Visitors provide direct  feedback on the training curriculum. The Navy noted, however, that all  available Surface Warfare units are invited to participate in Surface  Warfare Officer School Board of Visitors events and so could provide  feedback then. In addition, the Navy noted that Surface Warfare Officer  School Command also solicits feedback through visits to fleet  concentration areas and through semiannual symposiums of ship  Commanding Officers. While such targeted means of collecting  feedback may provide valuable information, officers may not be able to  participate due to their deployment status, position on shore duty, timing  of events during other personal responsibilities, or other factors. We  believe that developing a method to regularly collect feedback from  SWOs across the fleet would provide decision makers with valuable  information that could help them assess the effects of training on SWO  performance.

The Navy concurred with our second recommendation that the  Commander, Naval Surface Forces, routinely conduct regular Officer of  the Deck competency assessments using samples of sufficient size and  using selection methods to gauge the level of fleet-wide ship-driving  proficiency trends following the implementation of the planned ship- driving training programs. The Navy stated that it plans to routinely collect  and analyze standardized mariner skills performance data across an  officer’s career path. However, the Navy stated it will use training checks,  rather than the current Officer of the Deck competency assessment, to  evaluate SWO performance after 2020. This presents two problems in  meeting the intent of our recommendation. First, the Navy will need to  ensure that the training checks are sufficiently rigorous to assess  competency. Second, the Navy will not have valid data to compare the  effects of training changes on competency if it changes its assessment  approach.

In 2018, the Navy used the Officer of the Deck competency assessment  to establish a baseline of SWO ship-driving proficiency. We found that the  2018 competency assessments showed significant variation in ship- driving proficiency and the 2019 follow up assessments found that  competency had not improved. In currently documented plans, the Officer  of the Deck Phase II check after a Division Officer’s first assignment will  occur at the end of the course. Even if the Navy changes the Officer of  the Deck Phase II check to occur at the beginning of training as stated in  its comments, performance data from this check cannot be directly  compared with the results of the current competency assessment.  Differences in assessment content or difficulty, remediation attempts, and  the fact that the new check may have career implications for SWOs as a  go/no-go assessment may affect proficiency measurements and pass  rates. Due to these factors, we believe that a comparison between the  current Officer of the Deck competency assessment and the planned  Officer of the Deck Phase II check or another standard should not be  considered as valid means for demonstrating changes in ship-driving  proficiency over time. That is, adopting a new standard may affect the  Navy’s ability to determine the impact of training on ship-driving  proficiency compared with the 2018 baseline results.

The Navy also stated in its comments that the SWO training and  assessment continuum is designed to provide training and evaluation at  all career milestone levels. The planned system of additional skills checks  will provide the Navy with more insight into SWO proficiency levels over  the course of an officer’s career and help the Navy to understand the  effects of changes to training. As we stated in the report, we believe these  checks are significant steps in the right direction but may not provide  adequate assessment mechanisms in the near term. The more robust  Officer of the Deck competency assessments are necessary to gauge the  level of fleet-wide ship-driving proficiency trends following the  implementation of the planned ship-driving training programs.

Further, the Navy stated in its comments that while numerous means of  assessing SWO mariner skills proficiency at various milestone levels are  in place, the ultimate SWO career path goal is to develop the most  proficient, experienced, and confident Commanding Officers, which  occurs approximately 16 years into the SWO career path. While the  quality of ship Commanding Officers is a vital component of Navy  readiness and capability, the majority of SWOs do not remain in the Navy  long enough to advance beyond the position of Division Officer, according  to Navy documentation. Similarly few advance to the position of  Commanding Officer during their career as a SWO. Since Division  Officers constitute over one third of the SWO workforce and by design of  the SWO career path do most of the ship-driving, it is of utmost  importance to build and evaluate fundamental ship-driving skills for all  Division Officers to support excellence in the ship-driving proficiency  across the Navy.

Finally, the Navy stated in its comments that our report language implies  an absence of any Officer of the Deck assessments from 2021 through  2024. We acknowledge that the Navy will conduct assessments of  Officers of the Deck during this time period in line with its planned system  of ten checks over a SWO’s career. However, without maintaining the  current Officer of the Deck competency assessments through at least  2024, the Navy will be unable to demonstrate any proficiency  improvement, compared with the 2018 baseline, resulting from its new  training programs.  Further, the Navy stated in its comments that it is  important to clarify that the SWOs who received competency checks in  2019 had not benefitted from the new and expanded ship-driving training  courses. Our ship visits did not include officers who had experienced the  expanded Division Officer training courses, as they were first introduced  to the fleet in June 2019. Nonetheless, it is concerning that SWO  competency had not improved in the 2 years since the 2017 collisions  despite the fleet-wide attention to improving ship-driving skills.

The Navy concurred with our third recommendation that the Commander,  Naval Surface Forces, in coordination with Surface Warfare Officers  School Command, provide Commanding Officers with standard criteria to  inform their evaluation of candidates for their Officer of the Deck  qualification and incorporate these criteria into surface fleet guidance.  However, the Navy stated that such criteria are already in place.  Specifically, the Navy noted that existing Personnel Qualification  Standards provide the standard evaluation criteria for the Officer of the  Deck qualification. We agree that the Personnel Qualification Standards  are in place, but disagree that Qualification Standards provide standard  evaluation criteria. Unless the Navy provides additional guidance for  Commanding Officers to measure proficiency in addition to the list of  required experiences present in the Personnel Qualification Standards,  the actions the Navy identified as addressing our recommendation will not  meet the intent of our recommendation.

The Navy’s Officer of the Deck Personnel Qualification Standards provide  a list of required experiences; however, the 2018 and 2019 competency  assessments indicate that these existing criteria have not resulted in high  levels of proficiency among Officers of the Deck. The Navy’s Personnel  Qualification Standards do not require SWOs to demonstrate a standard  level of proficiency, but rather that SWOs participate in a required number  of ship-driving experiences at a level determined by his or her  Commanding Officer. The absence of a common proficiency standard  across the Navy may contribute to inconsistency in ship-driving skills  among SWOs. Since the Officer of the Deck competency assessment  provides a means to measure proficiency, communicating appropriate  standards in line with those used in the current assessments as  qualification criteria would help ensure a common understanding of  proficiency expectations.

In comments, the Navy stated that for junior officers whose ships  experience maintenance periods, it is an historic surface force-wide  practice for Commanding Officers to temporarily assign those officers to  similar ships whose operational schedule better support qualification. This  practice is understandable and may contribute to SWO career  development, but can lead to significant differences in opportunities,  experiences, and assessments that SWOs receive during their first  Division Officer assignment. For example, as noted in our report, some  Commanding Officers stated because of this temporary assignment, they  had to rely on the judgment of the other ships’ Commanding Officers to  determine their SWOs’ qualifications as Officers of the Deck.

The Navy concurred with our fourth recommendation that Commander,  Naval Surface Forces, in coordination with Surface Warfare Officer  School Command, develop a plan to analyze and use Mariner Skills  Logbook information to inform decision-making. The Navy noted that it  would comprehensively evaluate performance data relative to Mariner  Skills Logbook data in order to refine mariner skill milestone performance  and proficiency criteria. If Navy efforts result in a plan that includes  specific and measurable steps for analyzing and using Mariner Skills  Logbook data, the efforts will meet the intent of our recommendation.

In its comments the Navy stated that during the time we conducted our  group discussions (i.e. January through April 2019), Mariner Skills  Logbooks were still being introduced to the Fleet and recording practices  were still being established. While at the time of our discussion groups  the Mariner Skills Logbooks were relatively new, in September 2018, the  Navy issued an instruction that established guidance for the  implementation and use of the logbooks. In addition, all of the SWOs we  met with as part of our review had already received their Mariner Skills  Logbooks.

We are sending copies of this report to congressional committees, the  Secretary of Defense, the Secretary of the Navy, and other interested  parties. In addition, the report is available at no charge on the GAO  website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3489 or pendletonj@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix VI.

Appendix I: Overview of the Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard

Appendix II: Status of Navy Review Recommendations Related to Surface Warfare Officer Training

Following the four 2017 mishaps at sea, the Navy completed two internal  reviews on surface fleet readiness, ultimately compiling 111  recommendations for improvement. The Navy established a Readiness  Reform and Oversight Council under the leadership of the Vice Chief of  Naval Operations to oversee implementation of these recommendations.  The Readiness Reform and Oversight Council reported in February 2019  that it considered 91 of these recommendations to be implemented. We  reviewed the recommendations, identified 12 recommendations related to  Surface Warfare Officer (SWO) initial ship-driving training, and requested  the implementation status of each of these recommendations from the  Commander, Naval Surface Forces.

The Navy considers a recommendation to be “implemented” when there  is a policy in place or action has been taken to address a  recommendation. The Navy considers a recommendation to be  “transitioned” when the Readiness Reform and Oversight Council no  longer maintains regular oversight of a recommendation and has  transitioned oversight to another Navy organization. As of August 2019,  the Navy considered all 12 of the recommendations related to ship-driving  training as implemented with the final recommendation estimated to  transition by September 30, 2019. Table 1 lists the 12 recommendations  related to SWO initial ship-driving training, and our summary of the  Navy’s explanation for why they are considered to be implemented.

The John S. McCain National Defense Authorization Act for Fiscal Year  2019 and Senate Armed Services Committee report 115-262 to  accompany a bill for the National Defense Authorization Act for Fiscal  Year 2019 contained provisions that we review Surface Warfare Officer  (SWO) training and career paths. This report (1) describes the changes  the Navy has made or planned to SWO ship-driving training since the  2017 collisions and (2) assesses the extent to which the Navy has taken  actions to evaluate the effectiveness of those changes. We plan to issue  a separate report on SWO career paths in the future.

For objective one, we reviewed Navy documentation from Commander,  Naval Surface Forces, Surface Warfare Officers School Command, and  the Office of the Chief of Naval Operations on the content, purpose, cost,  and status of changes made and further changes planned to ship-driving  training since the 2017 collisions. We focused our analysis on changes  made to SWOs’ ship-driving training at the junior officer level as the Navy  prioritizes ship-driving training and ship-driving experience for junior  officers, and the Navy has identified actions it is taking to address  recommendations from the Navy’s two 2017 internal reviews to ensure  safe operations at sea through improvements to junior officer training.  We analyzed planned investments from fiscal year 2018 through fiscal  year 2025 for the construction of two ship-driving training facilities and the  development of three ship-driving training courses, which includes the  cost of purchasing new simulators, hiring new instructors, military  construction, and course curriculum development. We discussed  implementation plans for the 2017 internal reviews’ recommendations  with the Commander, Naval Surface Forces; officials from the Surface  Warfare Officers School Command; and officials from the Readiness and  Reform Oversight Council, a group within the Office of the Chief of Naval  Operations established to monitor the implementation of the internal  reviews’ recommendations.

Navy’s efforts to collect feedback from the surface fleet on the quality  of SWO ship-driving training and the health of the SWO community;

Navy’s 2018 Officer of the Deck competency assessment results,  criteria, and plans to continue evaluating SWO ship-driving  competency; extent to which the Navy had provided standardized criteria for ships’  Commanding Officers to use when evaluating SWO’s for ship-driving  qualification; and  format of SWO Mariner Skills Logbooks used to track SWO ship- driving experiences, and Navy policies regarding the logbooks.

To do this we compared the Navy’s practices with relevant Navy reviews,  instructions, and guidance, Standards for Internal Control in the Federal  Government, and our prior work on assessing training efforts in the  federal government. We assessed the reliability of the results of the  Navy’s 2018 Officer of the Deck competency assessments by examining  them for missing values, comparing other sources that provide the same  types of data to ensure consistency, and interviewing knowledgeable  agency officials regarding the assessments’ accuracy and completeness.  In addition, we reviewed the Navy’s internal controls for performing the  assessments, such as grading criteria and use of independent inspectors  to ensure quality and consistency in the information. We determined that  the results of the Navy’s 2018 Officer of the Deck competency  assessments were sufficiently reliable for the purposes of reporting on the  number and percentage of the graded categories.

In addition to meeting with Navy offices, we visited 12 surface ships in the  Pacific and Atlantic fleets from January through April 2019, selected  according to which ships and crews were available at each of the sites we visited. Aboard the ships we held group discussions and interviews with  approximately 225 SWOs to discuss their views on the sufficiency and  appropriateness of SWO training. Discussion group sizes ranged from  two to 20 SWOs. In conducting these group discussions, we held 24 group discussions, with two separate discussions for each of  the 12 ships—one with Department Heads and one with Division  Officers;  interviewed Commanding and Executive Officers aboard each of the  12 ships, where available; and  conducted each group discussion without the group’s supervisors or  subordinates present.

The ship crews we visited were those the Navy identified as available to  hold group discussions with us during site visits, and the results of these  group discussions are not generalizable to anyone outside these groups.  Due to the timing of our work, the interviews and group discussions did  not include SWOs that experienced changes made or planned for SWO  training beyond April 2019.

We asked each group a standard set of questions to obtain their views on  the following topics:  the sufficiency and appropriateness of SWO training programs in  preparing SWOs for their ship responsibilities, including ship driving;  the SWO career path, including the potential benefits and drawbacks  of more specialized career paths; and any opportunities to improve the SWO community.

We conducted an analysis of the discussion group responses to identify  common themes and provide illustrative examples in our report.  Specifically, we reviewed the responses received during discussion  groups, grouped the responses by themes, and counted how many  discussion groups and interviews provided similar feedback to our  questions. One GAO analyst conducted this analysis, coding the  information and entering it into a record of summary, and a different GAO  analyst checked the information for accuracy and agreement on themes.  Any initial disagreements in the coding were discussed and reconciled by  the analysts. The analysts then tallied the responses to determine the  extent to which the certain themes were covered during our discussion  groups and interviews.

We interviewed officials, or where appropriate, obtained documentation at  the organizations listed below:  Office of the Chief of Naval Operations

Surface Warfare (N96) Manpower and Training

Readiness Reform and Oversight Council  Commander, Naval Surface Forces, U.S. Pacific Fleet

Navigation, Seamanship, and Ship-handling Training facility

Commander, Naval Surface Forces, Atlantic

Basic Division Officer Course facilities—San Diego, California and

We conducted this performance audit from November 2018 to November  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Appendix IV: Current and Planned Ship- Driving Skill Checks to Be Conducted during a Surface Warfare Officer’s Career

The Commander, Naval Surface Forces, issued an instruction in  September 2018 listing ten ship-driving skill checks to be conducted  periodically over the course of a Surface Warfare Officer’s (SWO)  career. Failure to pass some of the checks can result in required  remediation or disqualification from career advancement. Table 2 lists the  ten current and planned checks as of August 2019, as well as information  on their timing and content as described in the instruction.

Appendix V: Comments from the Department of Defense

In addition to the contact named above, Chris Watson (Assistant  Director), Tobin McMurdie (Analyst-in-Charge), David Beardwood,  Vincent Buquicchio, Mae Jones, Amie Lesser, Shahrzad Nikoo, Michael  Silver, and Brandon Voss made key contributions to this report.

Following ship collisions in 2017 that resulted in the loss of 17 sailors’ lives and significant damage to Navy ships, the Navy has added classroom and simulator training for the Surface Warfare Officers who drive these ships. The Navy plans to triple ship-driving training hours by 2021.

However, the Navy does not yet have a plan to collect fleet-wide feedback on the quality of the new training or routinely test ship-driving skills. We recommended the Navy gather this feedback and take other actions to better measure training quality.