Professional Military Education: Programs Are Accredited, but Additional Information is Needed to Assess Effectiveness

DOD relies on PME and JPME to prepare its military personnel throughout their careers for the intellectual demands of complex contingences and major conflicts that typically involve more than a single military service. However, according to DOD's summary of the 2018 National Defense Strategy, PME “has stagnated, focused more on the accomplishment of mandatory credit at the expense of lethality and ingenuity.”

The Conference Report accompanying the John S. McCain National Defense Authorization Act for Fiscal Year 2019 included a provision for GAO to evaluate DOD PME and JPME institutions. This report examines the extent to which (1) the military services' PME programs have met civilian and JPME accreditation requirements, (2) OSD has assessed the effectiveness of the military services' PME programs, and (3) USD (Comptroller) has monitored the military services' PME program budget data. GAO analyzed applicable laws and policy, analyzed accreditation and budget information, and interviewed officials from the military services' intermediate- and senior-level resident PME programs.

All of the military services' intermediate- and senior-level officer Professional Military Education (PME) programs have met civilian and met or partially met Joint PME (JPME) accreditation requirements. However, not all of the military services' PME programs met the JPME seminar student mix requirement of at least one student from the nonhost military department. For example, the Army's intermediate-level PME program did not meet its Sea Service (i.e., Navy, Marine Corps, and, in certain instances, Coast Guard) requirement (see table). GAO's analysis found that the Navy could have assigned officers to Air Force and Army programs while not harming participation in its own seminars. Without taking steps to improve Sea Service participation, students lose opportunities to interact with students from other military departments, which officials have identified as critical to joint acculturation.

The Office of the Secretary of Defense (OSD) has taken steps to improve its oversight of the military services' PME programs, but is limited in its ability to assess their effectiveness. Department of Defense (DOD) guidance states that performance measurement is a means of evaluating efficiency, effectiveness, and results and that a balanced performance measurement scorecard includes nonfinancial and financial measures focusing on quality, cycle time, and costs. While OSD is in the process of developing some performance measures, it is not planning to require the military services to track program costs. Implementing its planned measures and establishing costs as a performance measure will better position OSD to assess the effectiveness of PME programs.

The Under Secretary of Defense (USD) (Comptroller's) ability to monitor the military services' PME programs is limited by incomplete and inconsistent reporting of service budget request data. DOD guidance does not require the Marine Corps to submit an annual budget request data exhibit for its senior-level PME program and existing guidance for programs that are reported does not specify how to uniformly account for costs. Without complete and uniform budget request data, USD(Comptroller) is challenged in monitoring these programs.

GAO is making seven recommendations, including that DOD take steps to determine its ability to assign Navy officers to PME programs of other services, implement performance measures–including tracking of costs, and issue guidance for service reporting of PME budget request data. DOD concurred with all of GAO's recommendations.

The Department of Defense (DOD) relies on professional military  education (PME) and joint professional military education (JPME) to  prepare its military personnel, throughout their careers, for the intellectual  demands of complex contingences and major conflicts that typically  involve more than a single military service. The military services are  responsible for overseeing PME at their respective staff and war colleges,  and for educating their personnel in service-specific core competencies.  For example, the Air Force focuses on air and space warfare, while the  Marine Corps focuses on maneuver warfare. JPME, a subset of PME  overseen by the Chairman of the Joint Chiefs of Staff (Chairman),  prepares leaders from all four military services to operate as a joint force,  such as at a combatant command. In practice, at the in-residence military  service colleges, JPME is embedded within the PME curricula. For  purposes of this report, we generally refer to each of these programs  collectively as the military services’ PME programs, unless otherwise  noted. In 2018, about 2,500 military personnel attended in-residence  military service PME and JPME programs, each of which generally last  about 10 months.

However, according to DOD’s summary of the 2018 National Defense  Strategy, PME “has stagnated, focused more on the accomplishment of  mandatory credit at the expense of lethality and ingenuity.” The  Conference Report accompanying the John S. McCain National Defense  Authorization Act for Fiscal Year 2019 included a provision for us to  evaluate DOD PME and JPME programs. In addition, we were also  requested to review DOD’s PME and JPME accreditation processes and  performance measures, among other areas. In this report, we assess the  extent to which (1) the military services’ PME programs have met civilian  and JPME accreditation requirements, (2) the Office of the Secretary of  Defense has assessed the effectiveness of the military services’ PME  programs, and (3) the Under Secretary of Defense (Comptroller) has  monitored the military services’ PME program budgets. Additionally, in  appendix I we describe the status of the Joint Special Operational  University (JSOU) pursuing additional civilian and JPME accreditation.

We focused our review on officer in-residence intermediate- and senior- level military service PME programs. Intermediate- and senior-level PME  and JPME programs are designed for officers at pay grades O-4 through  O-6. These programs include content on warfighting and leader  development; joint planning, doctrine, and joint force requirements;  national security and theater strategy; and civil-military relations.

For our first objective, we reviewed applicable civilian accreditation  regulations established by the Department of Education and the  standards, which civilian accreditation bodies apply in reviewing PME  programs. In addition, we reviewed and analyzed the most recent civilian  accreditation results. We also analyzed and compared the military  services’ student, faculty, and seminar data against Joint Staff  accreditation requirements for awarding JPME credit. Based on  responses to data reliability questionnaires from the Joint Staff and the  military services, as well as our examination of the data, we determined  that the student and seminar data for academic years 2014 – 2018 was  sufficiently reliable for the purpose of understanding service  representation in the seminars. We met with officials from the Office of  the Under Secretary of Defense for Personnel and Readiness  (OUSD(P&R)), the Joint Staff, each of the military services, and civilian  accreditation officials to discuss civilian and Joint Staff accreditation  requirements and challenges to meeting these requirements.

For our second objective, we reviewed and analyzed applicable DOD  policies and compared them to relevant federal statutes and DOD  guidance concerning PME and JPME. Where appropriate, we also  considered select training and development leading practices (which  include education), Standards for Internal Control in the Federal  Government, and the DOD Financial Management Regulation to assess  OUSD(P&R) oversight of these programs. We also met with officials  from OUSD(P&R), the Joint Staff, and each of the military services to  discuss the Office of the Secretary of Defense’s (OSD) PME roles and  responsibilities.

For our third objective, we analyzed relevant statutes and applicable  sections of DOD’s Financial Management Regulation. Further, we  analyzed the military services’ budget request data for fiscal years 2014 –  2020 submitted in support of DOD’s annual budget request to assess the  data for completeness and uniformity. We also met with officials from the  Offices of the Under Secretary of Defense(Comptroller), OUSD(P&R), the  Joint Staff, and each of the military services to discuss monitoring of the  military services’ PME program budget request data.

To address all three reporting objectives, we conducted site visits and  interviewed officials at each of the following military services’  intermediate- and senior-level PME and JPME programs:

College of Naval Command and Staff, Newport, Rhode Island;

College of Naval Warfare, Newport, Rhode Island;

Air Command and Staff College, Maxwell Air Force Base, Alabama;

Air War College, Maxwell Air Force Base, Alabama;

Marine Corps Command and Staff College, Marine Corps Base,

Marine Corp War College, Marine Corps Base Quantico, Virginia;

Army’s Command and General Staff College, Fort Leavenworth,

Army War College, Carlisle, Pennsylvania.

In addition, to describe the status of the Joint Special Operations  University (JSOU) pursuing additional accreditation, we reviewed the  most recent JSOU accreditation report and civilian accreditation  standards. We also met with JSOU and the Office of the Assistant  Secretary of Defense for Special Operations/Low-Intensity Conflict  officials to determine what actions, if any, JSOU had taken towards  additional accreditation. This information is presented in appendix I of this  report.

We conducted this performance audit from October 2018 to January 2020  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Background Overview of the Goldwater-Nichols Department of Defense Reorganization Act of 1986 and Relevant PME Statutes

The Goldwater-Nichols Department of Defense Reorganization Act of  1986, in part, was intended to improve joint officer management policies,  otherwise enhance the effectiveness of military operations, and improve  DOD’s management and administration. With the Goldwater-Nichols Act,  Congress also intended to, consistent with the congressional declaration  of policy in section 2 of the National Security Act of 1947 and among  other things, reorganize DOD and strengthen civilian authority in DOD.  The Goldwater-Nichols Act, as amended, also: established various joint officer management policies, including  requiring JPME for certain joint assignments and promotion  categories;  required officers to successfully complete an appropriate program at a  JPME school, among other things, to be designated as joint  qualified—a prerequisite for promotion to brigadier general or rear  admiral lower half rank except under certain circumstances; and  required the Secretary of Defense, with the advice and assistance of  the Chairman of the Joint Chiefs of Staff, to periodically review and  revise the curriculum of JPME schools to enhance the education and  training of officers in joint matters.

In addition, the Ronald W. Reagan National Defense Authorization Act for  Fiscal Year 2005 required the Secretary of Defense to implement a  comprehensive framework for officer JPME.

Overview of the Intermediate- and Senior- level Officer PME Continuum, Programs, and Locations

The PME continuum consists of five military educational levels that  correspond to the five phases of a military officer’s career: (1)  precommissioning, (2) primary, (3) intermediate, (4) senior, and (5)  general/flag officer. As figure 1 indicates, intermediate- and senior-level  PME and JPME programs—the focus of our review—are designed for  officers at pay grades O-4 through O-6.

As identified in figure 2 below, the military services’ intermediate- and  senior-level PME programs tailor curricula according to their respective  services’ needs. For example, the Army, Navy, and Marine Corps PME  programs focus on land, maritime, and maneuver warfare, respectively.  Further, the Chairman’s instruction concerning officer PME and JPME  (hereinafter referred to as the Officer Professional Military Education  Policy, or “OPMEP”) requires that JPME be integrated across a diverse  array of academic topics, including history and political science, and,  where appropriate, be offered in conjunction with PME. Collectively,  PME and JPME prepare officers, throughout their careers, to increase  their knowledge and develop the necessary skills to operate in joint  environments, such as a combatant command. PME and JPME also are  offered through distance learning and satellite education programs for  non-resident students.

Office of the Secretary of Defense, Chairman of the Joint Chiefs of Staff, and Military Service PME and JPME Oversight Responsibilities

The OSD, Chairman, and military services are responsible for overseeing  the services’ PME and JPME programs.

OSD: Within OSD, the Secretary of Defense has delegated  responsibility for, among other things, military readiness, total force  management, and military and civilian personnel training to the Under  Secretary of Defense for Personnel and Readiness. Under DOD  Directive 5124.02 the Under Secretary is responsible for, among other  things, developing education policies, plans, and programs for the  education of all DOD personnel, including PME and JPME  programs. Within OUSD(P&R), the Deputy Assistant Secretary of  Defense for Force Education and Training (DASD(FE&T)) was  established in 2015. The DASD(FE&T) is responsible for developing  policies, plans, programs, budgets, and other activities necessary to  develop, guide, measure, implement, assess, and oversee all aspects  of education and training for military personnel following basic officer  and enlisted training, which includes PME and JPME programs.

The USD(Comptroller) is the principal staff assistant and advisor to  the Secretary of Defense on budgetary and financial matters. The  USD(Comptroller) focuses on budgetary formulation and execution;  financial management and oversight; and accounting policy; among  other things. The USD(Comptroller), among other things, directs the  formulation and presentation of DOD budgets; and establishes and  supervises the execution of uniform DOD policies, principles, and  procedures, including terminologies and classifications, as necessary  for certain budgetary and financial matters.

Chairman: With the advice and assistance of the Chairman, the  Secretary of Defense periodically reviews and revises the JPME  curriculum to enhance the education and training of officers in joint  matters. The OPMEP outlines the Chairman’s roles and  responsibilities as they relate to PME and JPME. According to the  OPMEP, the Chairman formulates polices for coordinating military  education and advises and assists the Secretary of Defense through  the designation and certification/accreditation of JPME. The Chairman  accredits military service programs through periodic Process for the  Accreditation of Joint Education (PAJE) reviews. Further, the Joint  Staff Directorate for Joint Force Development is responsible for,  among other things, reviewing the Chairman’s PME policies,  overseeing the Military Education Coordination Council, and  coordinating PAJE reviews.

Military services: The military services provide PME to develop  officers with expertise and knowledge appropriate to their grade,  branch, and occupational specialty. Each military service is  responsible for funding, developing curriculum for, and administering  their respective PME programs. In addition, for programs accredited  to award JPME, each military service is responsible for meeting the  Chairman’s PAJE accreditation requirements and providing qualified  military students and faculty to the other military services’ PME  programs in accordance with the OPMEP. Membership on PAJE  teams, which accredit military services’ PME programs, will be tailored  to provide the appropriate balance of expertise in JPME learning  areas, objectives, criteria, and standards.

The Military Services’ PME Programs Are Accredited, but Not All Programs Met the JPME Seminar Student Mix Accreditation Requirement The Military Services’ Intermediate- and Senior- level PME Programs Are Accredited to Award Master’s Degrees

All of the military services’ intermediate- and senior-level PME programs  are accredited to award master’s degrees. Each program undergoes a  Department of Education-governed civilian accreditation process  generally every 10 years, depending on the accreditor and the program.  Civilian accreditation for the military services’ PME programs occurs at  the institution level and includes multiple programs. For example, the  civilian accreditation of Marine Corps University includes the Marine  Corps’ intermediate- and senior-level PME programs, as well as other  programs such as its School for Advanced Warfighting. According to PME  program and civilian accreditation officials, the civilian accreditation  process starts with the institution conducting a detailed self-evaluation of  its performance, and preparing and providing a self-evaluation report to  the accreditation officials. This is followed by a site visit by the  accreditation officials and a report describing the institution’s compliance  with applicable academic quality standards. The accreditation process  concludes with the accreditor’s decision on the institution’s accreditation  status. Table 1 shows when each of the military services’ intermediate-  and senior-level PME program was last accredited (at the institutional- level) for civilian accreditation.

Accreditation bodies assess academic quality by applying and enforcing  standards in the following areas required, generally, by the Department of  Education: (1) success with respect to student achievement; (2) curricula;  (3) faculty; (4) facilities, equipment, and supplies; (5) fiscal and  administrative capacity; (6) student support services; (7) recruiting and  admissions practices; (8) measures of program length and objectives of  the degrees or credentials offered; (9) record of student complaints  received by, or available to, the accreditation body; and (10) record of  compliance with certain federal student loan program responsibilities.  Within these areas, civilian accreditation bodies develop their own  accreditation standards, which can vary (see table 2). The military  services’ intermediate- and senior-level PME programs are assessed  against the applicable accreditation standards to enable the PME  programs to award master’s degrees.

There is no Chairman or OSD requirement for the military services’ PME  programs to have civilian accreditation status, but officials reported  several benefits related to civilian accreditation. Specifically, DOD and  civilian accreditation officials stated that civilian accreditation provides  additional assurance from a recognized external authority that the military  services’ PME programs are meeting educational standards required of  DOD and non-DOD programs alike. In addition, we previously reported  that the U.S. accreditation system’s use of peer review offers the relevant  expertise to assess academic quality and provides institutions with  feedback for improvement as a key strength of the system.  Furthermore, DOD officials said that the ability to award master’s degrees  from an accredited program helps the programs attract and retain high- quality faculty.

The Military Services’ Intermediate- and Senior- level PME Programs Are Accredited to Award JPME Credit, but Not All Programs Met the Seminar Student Mix Requirement All Military Service Intermediate- and Senior-level PME Programs Are Accredited to Award JPME Credit

All of the military services’ PME programs have been accredited by the  Chairman to award JPME credit. The OPMEP outlines the JPME program  accreditation requirements and processes that are to occur at least every  6 years. DOD’s process for accrediting the military services’ JPME  programs is through the Chairman’s PAJE. The PAJE is based on  accepted civilian accreditation standards and practices. According to the  OPMEP, the PAJE serves three purposes: (1) oversight, (2) assessment,  and (3) improvement. Once JPME programs are initially accredited,  accreditation is reaffirmed through subsequent PAJEs every 6 years. In  advance of a PAJE accreditation, the military service PME program  submits an OPMEP-required self-assessment, which the PAJE team  reviews prior to conducting the on-site accreditation. The PAJE team  prepares a report on its findings, and includes a full, conditional, or no  accreditation determination.

PME programs receiving a conditional accreditation or reaffirmation must  demonstrate improvements in particular areas within a specific timeframe  in order maintain their accreditation. Any program that fails to achieve  accreditation, reaffirmation, or conditional accreditation/reaffirmation is no  longer a JPME provider. According to the OPMEP, accreditation or  reaffirmation is awarded when programs are judged satisfactory overall  and have no significant weaknesses. Table 3 shows the date of the  most recent JPME accreditation for each of the military services’  intermediate- and senior-level PME programs.

Additionally, the military services’ PME programs have (1) met or partially  met all of the required joint learning areas, such as joint command and  control; and (2) met or partially met all required common educational  standards, such as periodically assessing their JPME programs. First, the  OPMEP requires intermediate- and senior-level PME programs to fulfill  the appropriate joint learning areas and objectives and common  educational standards, and generally have a curriculum that includes the  required JPME content prescribed in statute. The PAJE review of the  joint learning areas and common educational standards includes a  combination of objective and subjective assessment based on peer  expertise.

Specifically, the OPMEP requires intermediate-level PME programs to  fulfill the following six joint learning areas: (1) National military capabilities  strategy, (2) Joint doctrine and concepts, (3) Joint and multinational  forces at the operational level of war, (4) Joint planning and execution  processes, (5) Joint command and control, and (6) Joint operational  leadership and the profession of arms. The OPMEP requires senior-level  PME programs to fulfill the following five joint learning areas: (1) National  strategies; (2) Joint warfare, theater strategy and campaigning for  traditional and irregular warfare in a joint, interagency, intergovernmental  and multinational environment; (3) National and joint planning systems  and processes for the integration of joint, interagency, intergovernmental  and multinational capabilities; (4) Command, control and coordination;  and (5) Strategic leadership and the profession of arms. According to the  most recent Joint Staff PAJE accreditation reports, all of the military  services’ intermediate- and senior-level PME programs met all of these  mandatory joint learning areas, with the exception of the Marine Corps  intermediate-level PME program which received a partially meets in the  joint learning area for joint planning and execution processes.

Second, the OPMEP also requires intermediate- and senior-level PME  programs to meet seven common educational standards that the  Chairman considers essential in awarding JPME credit. Table 4 describes  these seven common educational standards.

The most recent Chairman’s accreditation review found that each of the  military services’ PME programs met or partially met all seven OPMEP- required common educational standards, as shown in table 5. According  to Joint Staff officials, to be assessed as “met,” the program must meet all  of the criteria for that common educational standard. On the other hand, if  a program does not meet all of the criteria then it “partially met” the  criteria for the accreditation standard. When a PAJE team determines that  a program “partially met” a standard, the team suggests corrective actions  for the program to consider. Receiving a “partially met” on a particular  standard does not exclude a program from being accredited, as  accreditation is based on the program being judged satisfactory overall  and having no significant weaknesses.

We identified the following examples of common educational standards  that were met or partially met by the military services’ intermediate- and  senior-level PME programs during our analysis of the Chairman’s most  recent accreditation reports for those programs.

Standard 2: Employ Predominantly Active and Highly Effective  Instructional Methods – The PAJE team found that the College of  Naval Warfare met this standard during its most recent review in May  2015. This standard states that instructional methods should be  appropriate to the subject matter and desired levels of learning, and  should employ active student learning whenever feasible. In addition,  the standard requires that the goals of the educational offerings be  rigorous and challenging, requiring that students engage in critical  thinking and active interaction. Specifically, the PAJE team found that  the College of Naval Warfare employed a preponderance of active  instructional methods to achieve desired learning outcomes. The team  found that the effective combination of Socratic discussion, case  studies, practical exercises, written assignments, and lectures  followed by seminar discussions, engaged students in critical thinking  and were appropriate to the desired levels of learning. The PAJE  team also found that active student discourse occurred both inside  and outside of seminars. Lastly, the team found that the effectiveness  of the curriculum in refining critical thinking skills was reflected in both  student and alumni surveys.

Standard 3: Assess Student Achievement – The PAJE team found  that the Marine Corps Command and Staff College met this standard  during its most recent review in September 2014. This standard states  that each college should aggressively assess its students’  performance, clearly state educational goals and objectives, and  measure students’ performance against defined standards using  direct and indirect assessment tools to identify whether desired  educational outcomes are being achieved. Specifically, the PAJE  team found that the Marine Corps Command and Staff College clearly  identified program outcomes, student learning outcomes, and lesson  educational objectives. The PAJE team also found that student  assessments were directly linked to student learning outcomes, joint  learning areas, and joint learning objectives. Additionally, the team  found that results were carefully tracked and used for educational  outcome achievement verification, curriculum improvement, and  faculty development feedback. Lastly, the PAJE team found that the  College used a variety of student assessments—including research  papers, exams, staff papers, oral presentations, exercises,  practicums, oral defenses, and seminar participation—to provide  feedback and verify learning outcome achievement.

Standard 4: Assess Program Effectiveness – The PAJE team  found that the Army’s Command and General Staff College partially  met this standard during its most recent review in February 2014. This  standard states that colleges should survey students, graduates, and  their supervisors to determine curricula and educational effectiveness  of their academic programs. The standard also states that leadership  should periodically assess the intended educational outcomes of  programs for currency, relevancy, and completeness, and the results  of these analyses should be used to refine or develop curricula that  continue to meet evolving mission requirements in the context of an  ever-changing world. Specifically, the PAJE team found that there is a  robust evaluation and assessment process for the common core  courses but that neither the electives nor the Command and General  Staff College-level outcomes were assessed. Additionally, the PAJE  team found that there did not appear to be a process for evaluating  the overall curriculum either directly or indirectly. The PAJE team  suggested that the Army’s Command and General Staff College  develop a capstone evaluation to assess outcomes of its common  core curriculum. Army’s Command and General Staff College officials  told us that in 2016 the college developed a capstone evaluation for  its common core curriculum, consisting of an online examination and  a faculty member oral examination.

Standard 5: Conduct Quality Faculty Recruitment: Selection,  Assignment, and Performance Assessment Program – The PAJE  team found that the Air War College partially met this standard during  its last review in October 2014. This standard states that faculty  should have the academic credentials, teaching skills, and experience  in joint and professional matters necessary to teach in the colleges.  This standard also states that faculty roles and responsibilities should  be clearly documented, and that colleges should hold faculty  accountable to clearly defined and measurable performance criteria  and standards. Specifically, the PAJE team found that the Air War  College did not meet the OPMEP standard for its student-to-faculty  ratio, but acknowledged that the college had a plan to meet this  requirement by the spring of 2015. The Air War College met the  student-to-faculty ratio in academic year 2015. The review also found  that delays in hiring presented challenges in maintaining the requisite  number of qualified faculty. The PAJE team suggested that the Air  War College continue its efforts to reduce the time to complete civilian  hiring actions. Air War College officials stated that as part of a wider  Air University effort to streamline the civilian hiring process they were  able to ameliorate this challenge by making the process more  transparent, predictable, and shorter.

Most Military Services’ Senior- level PME Programs Met the JPME Seminar Student Mix Accreditation Requirement, but Some Intermediate-level Programs Did Not

Most of the military services’ senior-level PME programs met the OPMEP  JPME seminar student mix accreditation requirement, which is part of the  develop joint awareness, perspective, and attitude common educational  standard (Standard 1) that pertains to joint acculturation. However, not all  of the military services’ intermediate-level PME programs met the seminar  student mix accreditation requirement. The OPMEP requires that each  intermediate- and senior-level JPME seminar contain at least one student  from each of the two non-host military departments: the Department of  the Army, the Department of the Navy (which includes the Marine Corps),  and the Department of the Air Force. DOD defines joint acculturation as  the process of understanding and appreciating the separate military  service cultures resulting in joint attitudes and perspectives, common  beliefs, and trust.

All but one of the military services’ senior-level PME programs met the  seminar student mix accreditation requirement from academic years 2014  through 2018. During that timeframe there were approximately 300  senior-level seminars, and only one did not meet the requirement.  Specifically, during academic year 2017, the Air Force’s senior-level PME  program lacked sufficient Navy representation for one seminar.

However, not all of the military services’ intermediate-level PME programs  met the seminar student mix accreditation requirement. Specifically, the  Air Force’s and the Army’s intermediate-level PME programs had less  than the required Sea Service representation for 3 years between  academic years 2014 and 2018. For academic years 2016 and 2018,  the Air Force’s intermediate-level PME program had less than the  OPMEP-required Sea Service representation for about 24 percent of its  seminars (totaling 288 students), as shown in table 6 below. During the 3- year timeframe, the Army’s intermediate-level PME program had less  than the required Sea Service representation for about 22 percent of its  seminars (totaling 664 students). On the other hand, the Navy’s and the  Marine Corps’ intermediate-level PME programs generally met their  respective seminar student mix accreditation requirement for each of the  last 5 academic years (2014 – 2018).

According to Navy officials and documentation, the Navy stated that it  was unable to provide the other military services’ intermediate-level PME  programs with the required numbers of officers during academic years  2016 – 2018 because of competing staffing priorities, such as its forward  presence mission. However, we found that the Navy provided sufficient  officers to its own intermediate-level PME program (College of Naval  Command and Staff) during each of these academic years so that it could  have instead assigned the required number of officers to the Air  Command and Staff College and the Army’s Command and General Staff  College to meet their respective Sea Service requirements. For example,  the Navy sent 121 Navy officers to the College of Naval Command and  Staff in academic year 2018 for 27 seminars when the Air Command and  Staff College and the Army’s Command and General Staff College  needed a cumulative total of 32 officers to meet their OPMEP seminar  student mix requirement. As a result, most of the College of Naval  Command and Staff’s seminars would have only been reduced by one  Navy Officer.

Officials from all of the military service PME programs told us that  students interacting with students from other military departments is  critical for joint acculturation. Officials from the Joint Staff Directorate for  Joint Force Development reinforced the importance of the seminar  student mix requirement, stating that satisfying the OPMEP common  educational standard of developing joint awareness, perspective, and  attitude (Standard 1) is dependent on time and intensity of student  interaction with students from other military departments. Military service  and Joint Staff officials stated it was difficult for Air Force and Army  officers to gain a full appreciation of the Navy’s contribution to joint  military operations when there were no Sea Service students in the  seminar. In the situations where a seminar did not have Sea Service  representation, Joint Staff officials told us that a decision was made to  award students JPME credit. Furthermore, officials told us that it was  decided to not “punish” military service PME programs for not meeting the  OPMEP’s JPME seminar student mix requirement as military services’  programs cannot control the number of in-bound students assigned by  the other military services.

Officials from the Air Force’s and the Army’s intermediate-level PME  programs told us that when they are unable to meet the OPMEP seminar  student mix requirement, they take steps to compensate for the lack of  Sea Service student representation, such as using faculty to provide Sea  Service perspectives. Similarly, a 2010 Congressional report noted the  value of in-residence officer PME programs because of the acculturation  opportunities that they offer.

Other than Joint Staff officials requesting that the Navy meet the  OPMEP’s JPME seminar student mix requirement, no other actions have  been taken by the Chairman, OSD, or the Navy to resolve the issue  concerning Navy participation in the Air Force’s and Army’s intermediate- level PME programs. Specifically, according to DASD(FE&T) officials, as  of November 2019, OSD has not been involved in addressing the Navy’s  failure to meet the OPMEP’s JPME seminar student mix requirement.

Additionally, Joint Staff officials told us that the Chairman cannot direct a  Secretary of a military department to comply with provisions of a  Chairman’s publication.

However, Standards for Internal Control in the Federal Government state  that management should identify, analyze, and respond to risks related to  achieving defined objectives. Given that joint acculturation is a key  component of intermediate-level PME programs, the lack of action to  resolve or mitigate the issues at hand has the potential to negatively  affect students’ opportunities to increase their knowledge and develop the  necessary skills to operate in joint environments. Without DOD taking  steps to determine whether the appropriate number of Navy officers can  be assigned to intermediate-level PME programs of the Air Force and  Army, the officers participating in these programs lack the perspectives of  Sea Service participants, which diminishes the quality of the educational  experience.

Furthermore, neither the Chairman nor OUSD(P&R) has evaluated or  approved the mitigation steps, either before or after-the-fact, when a PME  program lacks representation to meet the joint acculturation requirement.  Although the OPMEP requires that each intermediate- and senior-level  JPME seminar contain at least one student from each of the two non-host  military departments, the OPMEP does not contain guidance on what  PME programs should do when they do not meet this requirement.  Developing guidance concerning actions, if any, the military services can  take to mitigate JPME seminar student mix shortfalls and still meet the  intent of the OPEMP’s joint awareness common educational standard  could better position DOD and the military services to ensure that DOD’s  JPME programs are meeting their objectives.

OSD Is Taking Steps to Exercise Oversight of the Military Services’ PME Programs, but Its Ability to Assess the Effectiveness of These Programs Is Limited

OSD has had PME and JPME statutory oversight responsibilities for more  than 30 years; however, while it has taken some steps to strengthen its  oversight, it is not well-positioned to assess the effectiveness of the  military services’ PME programs. The Goldwater-Nichols Act, as  amended, states that the Secretary of Defense shall, with the advice and  assistance of the Chairman of the Joint Chiefs of Staff, periodically review  and revise the curriculum of JPME schools, and require that the PME  schools periodically review and revise their intermediate- and senior-level  PME curriculums to strengthen the focus on joint matters and preparing  officers for joint duty assignments. Moreover, DOD Directive 5124.02  requires the Under Secretary of Defense for Personnel and Readiness to  develop policies, plans, and programs for educating DOD personnel.

According to several DOD officials with whom we spoke, prior to the  establishment of DASD (FE&T), OUSD(P&R) unofficially relinquished its  responsibilities for PME and JPME to the Chairman, whose office issued  the first version of the OPMEP in 1996. As mentioned earlier, the OPMEP  outlines the Chairman’s process for meeting statutory responsibilities for  overseeing officer JPME, which is a subset of PME. For example, the  OPMEP states that JPME is a Chairman approved body of objectives,  outcomes, policies, procedures, and standards supporting the educational  requirements for joint officer management.

As recently as 2017, OUSD(P&R) reported to Congress that it had no  formal process for exercising its authority to periodically review and revise  the curricula of officer JPME. In the same report, OUSD(P&R) stated  that DOD was reviewing JPME and the DOD Joint Officer Management  Program. OUSD(P&R) also reported that with the reorganization of its  office to include a Deputy Assistant Secretary of Defense for Force  Education and Training (DASD(FE&T)) in 2015, OSD was now organized  to exercise its statutory authorities with respect to PME and JPME and  would do so in line with the Secretary of Defense’s direction in the  National Defense Strategy. According to the 2015 implementation plan  detailing the reorganization, the Deputy Assistant Secretary’s  responsibilities include measuring, assessing, and overseeing all aspects  of education and training, which includes PME and JPME. In 2019,  DOD issued guidance stating that the Assistant Secretary of Defense for  Readiness is the principal advisor to the Under Secretary of Defense for  Personnel and Readiness on all matters related to the readiness of the  Total Force, including by developing policies and plans, providing advice,  and making recommendations for PME to include alignment to the  National Defense and National Military Strategies and talent management  and utilization.

OUSD(P&R) is drafting its first DOD instruction (the draft instruction) that  covers PME and JPME, which DASD(FE&T) officials told us it plans to  issue in February 2020. According to DASD(FE&T) officials, once issued,  the DOD instruction will be the prevailing policy document for PME and  JPME at the OSD-level. While we believe these steps will improve OSD’s  oversight of the military services’ PME and JPME programs, we identified  areas that could continue to impede DOD’s ability to assess the  effectiveness of these programs. Specifically:

DOD lacks a mission statement and performance measures for  its PME and JPME programs. DASD(FE&T) officials stated that prior  to the draft policy OUSD(P&R) had not developed a mission  statement and performance measures for PME, but told us that the  draft instruction would include a mission statement and examples of  performance measures. However, we did not identify a mission  statement for PME that clearly defines the respective key purposes for  this program when we reviewed the draft instruction. According to  leading training and development practices, a mission statement is  important to an organization’s success because it explains the  organization’s purpose and goals and is the basis for goal-directed  performance measures. The draft instruction proposes the  performance measures the military services should track and assess  as part of their required annual program reviews, such as graduate  assignments and retention rates. Performance measures are  important because they assess an organization’s progress toward  achieving results that are aligned with its mission. However, without  a department-wide mission statement for PME and JPME,  OUSD(P&R) is not well-positioned to propose performance measures  for the military services to track and enable OUSD(P&R) to assess the  effectiveness of these programs.

Further, our review of the draft instruction found no examples of cost- related performance measures. DASD(FE&T) officials confirmed that  cost-related performance measures were not included in the draft  instruction, but told us that they planned to coordinate with officials  from the Joint Staff Directorate for Joint Force Development to refine  the performance measures sometime in the future. DOD’s Financial  Management Regulation states that performance measurement is a  means of evaluating efficiency, effectiveness, and results, and that a  balanced performance measurement scorecard includes nonfinancial  and financial measures focusing on quality, cycle time, and cost.  Moreover, leading training and development practices state that  performance measures should include both qualitative and  quantitative measures to assess training results, and include the  identification and tracking of costs. These same leading practices  state that organizations should compare associated costs and  monetized benefits of training programs to determine return on  investment. DASD(FE&T) officials told us that having cost information  on the military services’ PME and JPME programs to determine return  on investment would enable their office to compare and make well- informed decisions about these programs.

DOD lacks a requirement for the military services to report  periodically on PME and JPME programs. OUSD(P&R) has not  established a requirement for the military services’ to periodically  report information to its office on the military services’ respective PME  and JPME programs. For example, the Chairman’s PAJE reports that  document accreditation findings and include a full, conditional, or no  accreditation determination are not provided to OUSD(P&R).  According to the OPMEP, PAJE reports will be forwarded to the Chief  of the applicable military service, the Director of the Defense  Intelligence Agency, or the President of the National Defense  University for appropriate action. A Joint Staff official confirmed that  PAJE reports are not provided to OUSD(P&R).

Our review of the draft instruction found no requirement for the  Chairman to provide PAJE reports to OUSD(P&R), nor is there a  requirement for the military services to report information on their  PME and JPME programs—such as their annual program reviews—to  OUSD(P&R). According to DASD(FE&T) officials, reporting  requirements were omitted from the draft instruction because their  office lacks the personnel to review and assess the information the  military services would be required to collect and report. However,  without a requirement for the military services’ to periodically report  information on their PME and JPME programs, OUSD(P&R)’s ability  to assess the effectiveness of these programs and perform  meaningful oversight will continue to be limited.

Leading training and development practices state that organizations  should collect appropriate performance data during implementation  and establish accountability for the results of these efforts.  Additionally, Standards for Internal Control in the Federal Government  state that management relies on quality information to make informed  decisions and evaluate the entity’s performance in achieving key  objectives and addressing risks. These same standards state that  management should receive quality information about the entity’s  operational processes to help management achieve the entity’s  objectives. Because OUSD(P&R) does not require the military  services to periodically report information on their respective PME  programs, it does not have information that would help it assess the  effectiveness of these programs.

We believe that addressing these limitations will enhance the ability of  OUSD(P&R) and its subordinate office (i.e., DASD(FE&T)) to oversee  and assess the effectiveness of the military services’ PME programs.

USD(Comptroller’s) Ability to Monitor the Military Services’ PME Program Budgets Is Limited

USD(Comptroller’s) ability to monitor the military services’ PME programs  is limited because the military services’ budget request data are  incomplete and lack uniformity. DOD’s Financial Management Regulation  requires the military services to submit separate budget request data on  PME programs in support of DOD’s annual budget request, and this data  is included in DOD’s annual congressional budget justification exhibits.  While the Financial Management Regulation requires the military services  to submit separate annual budget request data exhibits for most of their  intermediate- and senior-level PME programs, it does not require the  Marine Corps to submit an exhibit for its senior-level PME program, the  Marine Corps War College. Based on our review of the Marine Corps’  fiscal years 2014 through 2020 budget request data exhibits and  according to the USD(Comptroller) and Marine Corps officials, the Marine  Corps did not submit a budget request data exhibit for the Marine Corps  War College during this 7-year period. USD(Comptroller) and Marine  Corps officials could not explain why the Marine Corps War College was  omitted from the DOD Financial Management Regulation, where DOD  last updated the chapter requiring this submission in September 2008.

In addition, the data the military services include in their annual budget  requests varies. DOD Directive 5118.03 outlines USD(Comptroller)  responsibilities, requiring the Comptroller to, among other things: (1)  direct the formulation and presentation of DOD budgets; and (2) establish  and supervise the execution of uniform DOD policies, principles, and  procedures, including terminologies and classifications, as necessary, for  budget formulation, presentation, and execution, and certain other  topics. Additionally, section 2162 of title 10, U.S. Code, requires the  Secretary of Defense, with the advice and assistance of the Chairman of  the Joint Chiefs of Staff, to promulgate a uniform cost accounting system  for use by the Secretaries of the military departments in preparing budget  requests for the operation of PME schools. However, the DOD Financial  Management Regulation does not specify how the military services  should account for the data required for the military services’ budget  request data submissions. Consequently, the budget request data  reported by the military services varies. For example, in their fiscal year  2020 budget request data submissions the Army and the Air Force  combined distance education and in-residence education programs, the  Navy reported this data in separate exhibits, and the Marine Corps  omitted distance education costs for its intermediate-level PME program.  Additionally, according to DOD officials, the extent to which the military  services accounted for costs to operate and maintain their PME  colleges—such as security, facility maintenance, and information  technology support—varies.

In 1987, the year following the passage of the Goldwater-Nichols Act, the  House Armed Services Committee established a panel on PME led by  Representative Ike Skelton (the Skelton Panel). The Skelton Panel  undertook a comprehensive congressional review of PME, and published  its findings and recommendations in a 1989 report (the Skelton Report).  Although the Skelton Panel did not take a comprehensive look at how  well PME institutions were funded to accomplish their mission, the panel  inquired into cost per student at each school and reported receiving from  OSD raw data submitted by each PME institution, which included  considerable differences in scope and cost methodology used by the  PME institutions. The Skelton Report recommended that DOD establish a  uniform cost accounting system for the PME schools, and that the annual  report of the Secretary of Defense provide data on PME costs beginning  in 1990. A 2010 congressional report focused on PME developments  since the Skelton Panel’s review, investigated whether a uniform cost  accounting system existed, among other things. The congressional  report found that DOD did not have a uniform cost accounting method for  PME schools, and that it had not provided cost data to support useful  comparisons among PME schools. The report included a  recommendation for DOD to report its PME funding to Congress using a  standardized accounting method for cost per student at each of the PME  institutions, as recommended by the Skelton Panel in 1989. According to  DASD(FE&T) and Joint Staff officials, the department has not collected or  reported PME program cost information to Congress as the 1989 Skelton  Report and the 2010 congressional report both recommended.

Without complete and uniform budget request data, USD(Comptroller)’s  ability to monitor the military services’ PME programs, identify program  trends within the Marine Corps and among the other military services’  PME programs, and formulate meaningful inter-service comparisons is  limited.

DOD relies on PME to prepare its military personnel for the intellectual  demands of complex contingences and major conflicts that typically  involve more than a single military service. While all the military services’  intermediate- and senior-level PME programs have met or partially met  the accreditation requirements established by civilian accreditation bodies  and the Chairman to award master’s degrees and JPME credit,  respectively, not all service programs have met the seminar student mix  requirement. The Navy, for example, has not provided the requisite  representation of officers in Army and Air Force intermediate-level  seminars during the 2016 – 2018 academic years. Requiring DOD to  determine whether the requisite number of Navy officers can be assigned  to the military department’s JPME programs and to develop policy to  mitigate student mix shortfalls would address persistent student mix  imbalances and align with the joint acculturation goal of JPME.

OUSD(P&R)’s draft DOD instruction, expected to be finalized in February  2020, will be the prevailing policy document for PME and could improve  OSD’s oversight of the military services’ PME and JPME programs.  However, OUSD(P&R)’s ability to assess the effectiveness of the military  services’ PME programs is limited by the absence of a department-wide  mission statement that explains the purpose and goals of PME that aligns  with the proposed performance measures in the draft instruction; the  absence of a requirement for the military services to track program costs  as a performance measure; and the absence of a requirement for the  military services to report data on their PME and JPME programs—such  as their annual reviews of PME programs. Addressing these limitations  would better position OUSD(P&R) to oversee and assess the  effectiveness of the military services’ PME and JPME programs.

Finally, USD(Comptroller)’s ability to monitor the military services’ PME  programs is limited because the services’ budget request data are  incomplete and lack uniformity. Although the military services are required  to submit separate budget request data exhibits for most PME institutions,  the Financial Management Regulation does not require the Marine Corps  to submit an annual budget request data exhibit for its senior-level PME  program. Moreover, the data the military services include in their annual  budget requests vary because the Financial Management Regulation  does not specify how to account for costs. Requiring the Marine Corps to  report budget request data on its senior-level PME program annually, and  specifying how to account for costs in the exhibits would enhance the  USD(Comptroller)’s ability to monitor the military services’ PME programs  and also enhance Congress’s ability to identify trends among these  programs.

We are making a total of seven recommendations to the Secretary of  Defense. Specifically:   The Secretary of Defense should ensure that the Under Secretary of  Defense for Personnel and Readiness, in coordination with the Chairman  of the Joint Chiefs of Staff and the Secretary of the Navy, determine  whether it can assign the required number of Navy officers to the other  military departments’ JPME programs, consistent with Chairman of the  Joint Chiefs of Staff guidance. (Recommendation 1)

The Secretary of Defense should ensure that the Chairman of the Joint  Chiefs of Staff, in coordination with the Under Secretary of Defense for  Personnel and Readiness and the military services, develop policy  concerning actions, if any, the military services can take to mitigate JPME  seminar student mix shortfalls and still meet the intent of the OPMEP’s  joint acculturation requirement. (Recommendation 2)

The Secretary of Defense should ensure that the Under Secretary of  Defense for Personnel and Readiness, in coordination with the Chairman  of the Chiefs of Staff, develop and issue a department-wide mission  statement for PME that will explain the program’s purpose and goals, and  serve as a basis for performance measures. (Recommendation 3)

The Secretary of Defense should ensure that the Under Secretary of  Defense for Personnel and Readiness, in coordination with the Chairman  of the Joint Chiefs of Staff, issue and implement performance  measures—to include the tracking of costs—that align with the  department-wide mission statement for PME. (Recommendation 4)

The Secretary of Defense should ensure that the Under Secretary of  Defense for Personnel and Readiness, in coordination with the Chairman  of the Joint Chiefs of Staff, require the military services to periodically  report information to its office about the military services’ PME and JPME  programs—such as results of program reviews. (Recommendation 5)

The Secretary of Defense should ensure that the Under Secretary of  Defense(Comptroller) updates the DOD Financial Management  Regulation to require the Marine Corps to include a budget request data  exhibit for the Marine Corps War College in support of DOD’s annual  budget request. (Recommendation 6)

The Secretary of Defense should ensure that the Under Secretary of  Defense (Comptroller), in coordination with the military services and the  Chairman of the Joint Chiefs of Staff, issue guidance to standardize the  cost data that the military services should include in their annual PME  budget request data submissions. (Recommendation 7)

We provided a draft of this product to DOD for comment. In its comments,  reproduced in Appendix II, DOD concurred with all of our  recommendations and stated that it will be implementing our  recommendations by issuing policy, among other actions.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, the Under Secretary of Defense  for Personnel and Readiness, the Chairman of the Joint Chiefs of Staff,  and the Secretaries of the Army, Navy, and Air Force. In addition, the  report will be available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix III.

Appendix I: Status of the Joint Special Operations University Pursuing Additional Accreditation

The Joint Special Operations University (JSOU) was established in  September 2000 and is located at MacDill Air Force Base, Florida. The  mission of JSOU is to prepare special operations forces to shape the  future strategic environment by providing specialized Joint Professional  Military Education (JPME); developing applicable undergraduate- and  postgraduate-level equivalent curriculum; and fostering special operations  research, analysis, and outreach in support of Special Operations  Command objectives. JSOU staff and faculty include active duty, active  reserve, and temporary duty reserve military personnel; government  civilians; civilian contractors; private consultants; and guest lecturers and  speakers. JSOU’s active duty military personnel are assigned to the  university by Special Operations Command and the military services.  JSOU’s professional military education vision is to prepare warfighters to  solve ambiguous, complex problems across the spectrum of conflict by  providing dynamic and adaptive professional education opportunities.

In August 2015, the Accrediting Council for Continuing Education and  Training accredited JSOU through December 2019. As of January 2020,  officials stated that they are currently undergoing reaccreditation and  expect reaffirmation notification by the end of February 2020. While JSOU  offers a number of courses, seminars, and programs, officials from JSOU  and the Office of the Assistant Secretary of Defense for Special  Operations/Low-Intensity Conflict stated the university has no near-term  plans to award master’s degrees; therefore, no additional civilian  accreditation is necessary. JSOU officials said that they are  contemplating offering senior-level JPME in the future, but stated that  such an endeavor would take approximately at least 10 years to  accomplish.

Consistent with its mission of preparing special operations forces to  shape the future strategic environment, JSOU laid out the following seven  goals in its 2019 academic guidance:  1.  Continue to refine target audiences in all courses, assuring the right  curricula is provided to the right student at the right time.  2.  Implement a title 10, U.S. Code, civilian faculty hiring process that  leverages the DOD professional military education community, fully  supports the JSOU vision, and retains control to rapidly hire faculty  with expertise in required disciplines.  3.  Establish and complete a comprehensive building improvement plan  that provides a quality learning environment conducive to educational  excellence and student success.  4.  Establish and complete a comprehensive education technology plan  that brings all classrooms and auditoriums up to planned capability  inherent in a state-of-the-art learning institution.  5.  Facilitate the Technology Review Committee to define and develop  the JSOU advanced classroom concept, capable of a wide variety of  innovative teaching methodologies.  6.  Develop and sustain academic programs in the emerging mission  areas of artificial intelligence/machine learning, countering weapons of  mass destruction, cyberspace, sensitive activities, and joint  unconventional warfare that directly support special operations.  7.  Develop highly effective academic instructors and distinguished  experts in their individual fields of knowledge. Remain sensitive to  individual needs and career development as JSOU embarks on new  hiring processes and classroom innovations.

According to the JSOU Fact Book for 2018, the newly authorized title 10,  U.S. Code, civilian faculty hiring authorities will allow JSOU faculty to  attain new heights of excellence with expertise not normally found within  the military or civil service communities. The handbook states that the title  10, U.S. Code, faculty hiring authority will have a major impact on shaping  JSOU’s curriculum, and will directly add to special operations forces’  readiness and capability.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

Brenda S. Farrell, (202) 512 -3604 or farrellb@gao.gov.

In addition to the contact named above, Marc Schwartz (Assistant  Director), Norris “Traye” Smith (Analyst in Charge), Rebecca Guerrero,  Edward Malone, Stephanie Moriarty, Patricia Powell, Carter Stevens, and  Lillian M. Yob made significant contributions to this report.

Higher Education: Expert Views of U.S. Accreditation. GAO-18-5.  (Washington, D.C.: December. 22, 2017).

Higher Education: Education Should Strengthen Oversight of Schools and  Accreditors. GAO-15-59. Washington, D.C.: December 22, 2014.

Joint Professional Military Education: Opportunities Exist for Greater  Oversight and Coordination of Associated Research Institutions.  GAO-14-216. Washington, D.C.: March 10, 2014.

Joint Military Education: Actions Needed to Implement DOD  Recommendations for Enhancing Leadership Development. GAO-14-29.  Washington, D.C.: October 23, 2013.

Joint Professional Military Education programs bring together officers from across the military and prepare them to work together in complex operations and major conflicts. However, according to one DOD document, these programs have “stagnated” into a credit-earning exercise.

We found the programs met accreditation standards. However, they did not always include the required mix of students. For example, an Air Force program lacked Navy/Marine Corps/Coast Guard participation in about a fourth of its seminars. We made 7 recommendations, including that DOD address the student mix and establish better ways to measure program effectiveness.