Improving Program Management: Key Actions Taken, but Further Efforts Needed to Strengthen Standards, Expand Reviews, and Address High-Risk Areas

PMIAA requires OMB to adopt program management standards and guidelines government-wide; OPM is to establish new—or revise existing—occupational standards for program and project management. PMIAA includes a provision for GAO, no later than 3 years after the enactment of the act, to issue a report examining the implementation and effectiveness of certain provisions of the act on federal program and project management.

This report (1) describes steps taken by OMB, OPM, and agencies to implement PMIAA; (2) assesses OMB's efforts to address issues on GAO's High-Risk List using PMIAA; and (3) examines the extent to which OMB provided methods for agencies to measure and assess the results of PMIAA.

GAO reviewed documents from and conducted interviews with OMB and OPM. GAO surveyed all 24 CFO Act agencies, and selected five agencies to illustrate implementation efforts. GAO also interviewed subject matter specialists from academia and the private sector regarding their views on how program and project management practices applied to PMIAA.

The Office of Management and Budget (OMB) has begun to implement all requirements of the Program Management Improvement Accountabilitiy Act of 2016 (PMIAA), but further efforts are needed to fully implement the law. OMB released its 5-year strategic plan for PMIAA and developed program management standards. However, the standards are not detailed compared with accepted program and project management standards, and OMB's governance structure is insufficient for developing and maintaining these standards over time. In 2019, OMB conducted ten reviews of agency program portfolios—organized groupings of programs whose coordination in implementation enables agencies to achieve their objectives. Each review addressed one or two portfolios per agency. Further, OMB's required portfolio reviews of high-risk areas were limited to only five out of 35 areas on GAO's High-Risk List. OMB could establish measures to track agencies' progress. Although not required by PMIAA, this is a good practice for demonstrating improvement.

As required by PMIAA, the Office of Personnel Management (OPM) developed competencies for program and project managers and updated the program management job series. Further, OPM is developing a career path for program and project managers by the end of 2019. OPM also plans to create a unique job identifier code in 2020 so that agencies can more completely identify their program management workforce.

The Program Management Policy Council (PMPC), established by PMIAA and chaired by OMB's Deputy Director for Management, met for the first time in September 2018 and met twice in 2019 to discuss PMIAA implementation with Chief Financial Officers (CFO) Act agencies. All CFO Act agencies designated a Program Management Improvement Officer to participate in the PMPC. However, the PMPC has neither addressed GAO high-risk areas nor advised OMB on how to address high-risk areas, as required by the PMIAA.

GAO is making eight recommendations that OMB further develop the standards to include more detail, create a governance structure for program management standards, hold meetings on all High-Risk List areas, and establish measures to track agencies' progress in program management. OMB neither agreed nor disagreed with the recommendations and stated that it would consider them when making future updates to its program management policies and guidance.

The Program Management Improvement Accountability Act of 2016  (PMIAA) is intended to improve program and project management  practices within the federal government by requiring the development of  government-wide standards, policies, and guidelines for program  management; establishing an inter-agency council to focus on improving  program management; and requiring clearer identification of skills and  competencies necessary for effective program management, according to  the Senate Committee on Homeland Security and Governmental Affairs.  Both prior to and following the enactment of the law, we have reported on  many federal programs with weaknesses in management capacity, both  government-wide and in individual agencies, that impaired efficient and  effective government operations. Effective program and project  management could improve the likelihood that a given program or project  meets its intended purpose, remains on schedule, and is managed  efficiently.

Effective implementation of PMIAA could also enhance progress on high- risk areas by helping to focus leadership attention on these areas. Since  the early 1990s we have issued the High-Risk List, a list of programs and  operations that are ‘high risk’ due to their vulnerabilities to fraud, waste,  abuse, and mismanagement, or that need transformation. The list of high- risk areas is issued every 2 years at the start of each new session of  Congress. Billions of dollars are at stake. For example, financial benefits  to the federal government due to progress in addressing high-risk areas  over the past 13 years (fiscal year 2006 through fiscal year 2018) totaled  nearly $350 billion, or an average of about $27 billion per year. In fiscal  year 2018, financial benefits were the highest we ever reported at nearly  $47 billion. A number of high-risk areas have long-standing or significant  program and project management concerns. These and other programs  can benefit, and billions of dollars can potentially be saved, from  improving program and project management.

The Office of Management and Budget (OMB) has worked to develop a  standardized, federal government-wide approach to building agencies’  capacity for improved program and project management. OMB has put in  place methods for reviewing portfolios of agency programs, assessing  their effectiveness, and offering guidance to agencies for improved  program management in concert with the management tools provided  through the GPRA Modernization Act of 2010 (GPRAMA). For its part, in  partnership with OMB, the Office of Personnel Management (OPM) has  focused on providing agencies with the workforce support needed to  improve federal employees’ program and project management  capabilities.

PMIAA includes a provision that no later than 3 years after the enactment  of the act, we issue a report examining the implementation and  effectiveness of certain provisions of the act on improving federal  program and project management. This report examines: (1) the steps  taken by OMB, OPM, and agencies listed in the Chief Financial Officers  Act of 1990, as amended (CFO Act), to implement PMIAA, (2) the extent  to which OMB is using or planning to use portfolio reviews required by  PMIAA to address issues on our High-Risk List, and (3) the extent to  which OMB provided methods for agencies to assess the results of  PMIAA. To examine the steps taken by OMB, OPM, and agencies to  implement PMIAA, we reviewed agency documents and conducted  interviews with OMB and OPM. We evaluated PMIAA implementation  plans from 22 out of 24 CFO Act agencies on whether the plans met,  partially met, or did not meet the requirements provided in the OMB  implementation guidance.

We also disseminated a questionnaire to all CFO Act agencies to collect  information on PMIAA implementation. We selected the CFO act  agencies as our unit of analysis because PMIAA’s requirement to  designate a Program Management Improvement Officer applies to the  CFO Act Agencies. All 24 CFO Act agencies completed their  questionnaires between February and April 2019.

Additionally, we analyzed OPM’s Enterprise Human Resources  Integration data from fiscal year 2018, the most recent data at the time of  our review, to identify employees in the program management job series  (0340 job series). Finally, we interviewed outside subject matter  specialists to obtain their views on federal program and project  management. Specifically, we met with staff from the Project  Management Institute and Professor Janet Weiss from the University of  Michigan, who had conducted a study on how to improve federal program  management as she had been recommended by the Congressional  Research Service, OMB, and the IBM Center for the Business of  Government.

To determine the extent to which OMB is using, or is planning to use,  portfolio reviews required in PMIAA to address areas on our High-Risk  List, we interviewed officials at OMB about their efforts to address their  high-risk areas. Additionally, our questionnaire included requests for  information from OMB and all CFO Act agencies about high-risk program  portfolio reviews.

To examine the extent to which OMB provided methods for agencies to  assess the results of PMIAA, we evaluated OMB’s PMIAA 5-year  strategic plan for program and project management (PMIAA strategic  plan). We used leading practices for strategic planning to evaluate OMB’s  PMIAA strategic plan. To determine the extent to which the leading  practice criteria was included in the strategic plan, we assessed  documentary evidence from the OMB strategic plan and testimonial  evidence from OMB officials. We have reported in the past that, taken  together, the strategic planning elements established under the  Government Performance and Results Act of 1993 and GPRAMA,  associated OMB guidance, and practices we identified provide a  framework to evaluate federal strategic plans.

We also applied all leading practices we identified from our previous work  on data governance standards to assess the governance process OMB  used to develop, maintain, and monitor program management standards.  In addition, we assessed the pilot of the required PMIAA program portfolio  reviews against the five leading practices we identified from our work on  designing pilots.

We selected five agencies to illustrate PMIAA implementation efforts: the  Departments of Commerce, Energy, Veterans Affairs, and the Treasury,  and the National Aeronautics and Space Administration (NASA). We  selected these five agencies based on five criteria. We assessed  whether: agencies had responsibility for a program, function, or operation on  our 2019 High-Risk List,

OMB considered them further along in PMIAA implementation  compared to other agencies, agencies participated in OMB’s pilot of portfolio reviews, agencies reported taking action to direct internal workforce  development regarding program management, and agencies reported taking action to implement PMIAA beyond  guidance provided by OMB.

To achieve of a range of PMIAA experiences, we selected five agencies  that met varying numbers of the criteria. We chose the Department of  Commerce because it met all four selection criteria, the Department of  Energy met three, the Department of Veterans Affairs met two, and the  Department of the Treasury and NASA each met one.

To examine the steps taken to implement PMIAA, we reviewed agency  documents and conducted interviews with the five selected agencies.

We reviewed documents and interviewed officials at the selected  agencies about steps taken to implement PMIAA, their efforts to address  their high-risk areas, and any measures agencies had developed, or  planned to develop, to evaluate PMIAA implementation. We also asked  agency officials from the selected five agencies for their perspectives  about evaluative measures that would be useful to monitor the successful  implementation of PMIAA.

We conducted this performance audit from June 2018 to December 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

According to the Senate Committee on Homeland Security and  Governmental Affairs report concerning PMIAA, the purpose of PMIAA is  to improve program and project management in certain larger federal  agencies. The act includes requirements for OMB, OPM, and the 24  agencies listed in the CFO Act.

PMIAA requires OMB’s Deputy Director for Management or the designee  to, among other things: adopt government-wide standards, policies, and guidelines for  program and project management for executive agencies; engage with the private sector to identify best practices in program  and project management that would improve federal program and  project management;  conduct portfolio reviews of agency programs not less than annually,  to assess the quality and effectiveness of program management, in  coordination with Program Management Improvement Officers  (PMIO); establish a 5-year strategic plan for program and project  conduct portfolio reviews of programs on our High-Risk List.

The two types of portfolio reviews required by PMIAA—the portfolio  reviews of agency programs and the portfolio reviews of programs  identified as high risk on our High-Risk List—are separate requirements.

For purposes of this report, we define programs, projects, and portfolios  consistent with how those terms are defined in OMB’s PMIAA strategic  plan. OMB defines program as the functions or activities which agencies  are authorized and funded by statute to administer and enforce.  Programs typically involve broad objectives. OMB views projects as  temporary efforts with defined scopes to create products or services to  improve the efficient and effective implementation of programs. Because  programs are comprised of projects, programs inherently address the  projects subsumed within them. Consequently, our discussions of  programs throughout this report also pertain to projects. Finally, OMB  defines portfolios as organized groupings of programs whose  coordination in implementation enables agencies to achieve their  objectives.

The act also established the Program Management Policy Council  (PMPC), an interagency forum for improving agency practices related to  program management. OMB’s Deputy Director for Management chairs  the PMPC. The PMPC responsibilities include advising OMB on the  development and applicability of government-wide standards for program  management transparency. Furthermore, the act requires PMPC  members “to discuss topics of importance to the workforce,” such as  workforce development needs and major challenges across agencies in  managing programs. As chair of the PMPC, OMB’s Deputy Director is  required to preside at meetings, determine agendas, direct the work, and  establish and direct its subgroups, as appropriate. The act requires the  PMPC to meet not less than twice per fiscal year.

Additionally, OPM’s Director, in consultation with OMB’s Director, is  required to issue regulations that:  identify key skills and competencies needed for a program and a  project manager in an agency; establish a new job series, or update and improve an existing job  series, for program and project management within agencies; and establish a new career path for program and project managers within  an agency.

Overall, OPM’s role in implementing PMIAA is to establish a new job  series or update an existing job series by providing the occupational  standards that agencies will need to develop a trained and competent  workforce with the program and project management experience,  knowledge, and expertise to solve management challenges and support  agency decision-making. The act requires OPM to establish new—or  revise existing—occupational standards in consultation with OMB.  Occupational standards are included within OPM’s classification  guidance, which is provided to agencies to assist in classifying positions.  This guidance helps agencies to determine the proper occupational  series, position title, and grade of each position.

The act requires OMB’s Deputy Director of Management to oversee  implementation of the standards, policies, and guidelines for executive  agencies. OMB implemented some PMIAA requirements using existing  processes put in place to implement GPRAMA. We previously reported  that GPRAMA provides important tools that can help decision makers  address challenges facing the federal government, such as the annual  reviews of progress on agency strategic objectives conducted during  strategic reviews and the implementation of federal government priority  goals. Federal government priority goals, also known as cross-agency  priority (CAP) goals, are written by OMB in partnership with agencies.  GPRAMA requires OMB to coordinate with agencies to develop CAP  goals, which are 4-year outcome-oriented goals covering a number of  complex or high-risk management and mission issues. For example,  OMB directed agencies to align their noninformation technology major  acquisition programs with relevant strategic objectives so they could  assess progress for the PMIAA required program portfolio reviews  concurrent with required GPRAMA strategic reviews.

GPRAMA also requires OMB to present a program inventory of all federal  programs by making information available about each federal program on  a website. Finally, GPRAMA required OMB to establish a number of  CAP goals intended to cover areas where increased cross-agency  collaboration is needed to improve progress towards shared, complex  policy or management objectives across the federal government. OMB  uses CAP goals to address issues outlined in the President’s  Management Agenda. For example, OMB wrote a CAP goal to improve  management of major acquisitions across the government which  complements PMIAA and its required activities.

PMIAA requires the OMB Deputy Director, as chair of the PMPC, to  conduct portfolio reviews of programs from our High-Risk List. The  PMPC is also required to review programs we identify as high risk and to  make recommendations for actions to be taken by the Deputy Director for Management of OMB or a designee. See figure 1 below for an overview  of roles and responsibilities of OMB, OPM, the PMPC, and agencies.

OMB, OPM, and CFO Act Agencies Have Taken Steps to Implement PMIAA, but Some Program and Project Management Capacity Limitations Exist

Agencies responsible for PMIAA implementation have taken steps to  complete some requirements, but actions remain to fully implement the  law (see Table 1).

OMB’s PMIAA Strategic Plan Incorporated Leading Practices

OMB met the PMIAA requirement “to establish a five-year strategic plan  for program and project management.” The plan OMB developed details  three key strategies to implement PMIAA: (1) coordinated governance, (2)  regular OMB and agency engagement reviews, and (3) strengthening  program management capacity to build a capable program management  workforce. The three strategies focus on areas such as clarifying key  roles and responsibilities, identifying principles-based standards, and  identifying plans for enhancing workforce capabilities. The plan describes  the roles and functions of the PMIOs, the PMPC, and the requirements of  the agency implementation plans. It outlines a phased approach for  implementing PMIAA actions with milestones occurring throughout the 5- year period.

We found that OMB followed several strategic planning leading practices  in the creation of the PMIAA strategic plan. First, the plan incorporates  general goals and objectives for agencies’ implementation of PMIAA with  three corresponding strategies explaining OMB’s overall approach. OMB  followed a second leading practice by gathering input from stakeholders.  OMB staff told us they solicited input from congressional staff, and  members of external organizations like the Federal Program and Project  Management Community of Practice (FedPM CoP). Agencies’ staff also  confirmed to us that they had input into the OMB plan. Third, OMB  demonstrated interagency collaboration in its efforts to establish and lead  the PMPC and its efforts to work with the FedPM CoP to address any  issues identified by agencies. Finally, the plan included a timeline with  quarterly milestones to track completion of PMIAA’s activities and to  gauge progress toward achieving the desired results of PMIAA.

OMB’s Program and Project Management Standards Are Less Detailed Compared with Accepted Program and Project Management Standards

PMIAA required OMB to establish standards and policies for executive  agencies consistent with widely accepted standards for program and  project management planning and delivery. A consistent set of  government-wide program management standards and policies is  important because it helps ensure that agencies utilize key program  management practices to improve the outcomes of government  programs.

OMB published in June 2018 a set of standards for program and project  management as part of OMB’s PMIAA strategic plan. OMB’s strategic  plan directed agencies to apply these 15 standards to internal  management processes for planning, implementing, and reviewing the  performance of programs and activities.

OMB staff told us they decided to develop this set of standards rather  than adopt an existing set of consensus-based standards, such as the  widely accepted standards for program and project management from the  Project Management Institute (PMI). PMI is a not-for-profit association  that provides global standards for, among other things, project and  program management. The PMI standards are utilized worldwide and  provide guidance on how to manage various aspects of projects,  programs, and portfolios and are approved by the American National  Standards Institute (ANSI).

OMB staff told us that they decided not to specifically adopt the PMI  standards because they wanted to allow agencies to use a range of  standards that agencies had already developed and were using to  manage their programs, such as standards developed in-house by NASA  for their space flight programs. OMB further directed CFO Act agencies  that the 15 standards and application of them should be incorporated or  aligned with existing agency-specific program management policies and  practices, and tailored to reflect program characteristics. OMB staff told  us that they chose the approach to provide more principle-based  standards, as opposed to specific standards, to be flexible enough for a  range of government agencies to apply them.

OMB’s standards are similar in definition to PMI standards, but they are  less detailed by comparison. Our analysis of OMB’s standards shows that  OMB uses similar definitions for all 10 of PMI’s program management  standards and nine out of 10 of PMI’s project management standards,  such as risk management and change management. However, OMB  program and project management standards are less detailed when  compared to PMI’s standards in the following ways:

OMB standards do not provide a minimum threshold against which  agencies can gauge to what extent they have met each standard.  PMI’s Standard for Program Management provides the definition of a  standard but also what components are required for an entity to  confirm that the standard has been met. For example, meeting the  program financial management standard in PMI requires a financial  management plan to be developed, along with its related activities.

This plan allows entities applying the standard to confirm whether they  have met the standard for program financial management or not.

OMB’s standards do not distinguish between how the standards apply  differently to programs and projects while PMI has separate detailed  standards for program management and for project management. The  project management standards from PMI provide details on how the  standards apply to more granular tasks, such as establishing a quality  management or communication plan for a specific project.

OMB’s standards do not distinguish between how the standards relate  to each other during a program or project while PMI’s Standard for  Program Management details how project standards help build on  each other during a program. For example, a program scope  management plan is needed to determine the type of schedule  management planning that is necessary to accomplish the delivery of  the program’s outputs and benefits.

OMB provides minimal guidance on how standards apply differently  across the life cycle of a program or project while PMI’s Standard for  Program Management provides information detailing when a specific  standard should be utilized in different ways during the life cycle of a  program. For example, in the beginning of a program, risk  management should be planned and an initial risk assessment  created. Later, during program implementation, risk management  tasks focus on monitoring, analyzing risk, and responding to risk.

If the standards had the additional detail, it would be possible to  determine if agencies are meeting them and properly applying them to  programs and projects.

OMB Does Not Have a Detailed Governance Structure for Further Developing Program Management Standards

Our work on the Digital Accountability and Transparency Act of 2014  (DATA Act) standards has emphasized the necessity for a governance  structure with a clear set of policies and procedures for developing and  maintaining standards over time that are consistent with leading  practices. A governance structure is important because it helps ensure  that the standards are developed, maintained, adjusted, and monitored  over time. The DATA Act is similar to PMIAA because PMIAA gives OMB  responsibility to develop standards for program management, and the  DATA Act gives OMB and the Department of the Treasury responsibility for establishing data standards for the reporting of federal funds. These  standards specify the data to be reported under the DATA Act and define  and describe what is to be included in each element with the aim of  ensuring that information will be consistent and comparable.

Several governance models exist that could inform OMB’s efforts to help  ensure that the standards are developed, maintained, adjusted, and  monitored over time. These models define governance as an  institutionalized system of decision rights and accountabilities for  planning, overseeing, and managing standards. Many of these models  promote having a common set of key practices that include establishing  clear policies and procedures for developing, managing, and enforcing  standards. A common set of key practices endorsed by standards setting  organizations including the National Institute of Standards and  Technology, ANSI, and the American Institute of Certified Public  Accountants recommend that governance structures should include the  key practices shown in the text box below.

Key Practices for Governance Structures 1.  Delineating roles and responsibilities for decision-making and accountability,  including roles and responsibilities for stakeholder input on key decisions.  2.  Obtaining input from stakeholders and involving them in key decisions, as  appropriate.  3.  Developing and approving standards.  4.  Making decisions about changes to existing standards and resolving conflicts  related to the application of standards.  5.  Managing, controlling, monitoring, and enforcing consistent application of standards.

OMB staff told us they did not have any additional documentation about  the governance structure used to develop the program management  standards and how OMB will further develop and maintain them. We  compared available information about OMB’s governance structure for  developing and maintaining program management standards to the five  key practices on governance structures and found OMB’s governance  structure is incomplete in each of the five key practices.

OMB has not delineated roles and responsibilities for decision-making  and accountability, including responsibilities for stakeholder input on key  decisions. OMB’s strategic plan notes that one role of the PMPC is to  help further develop the program management standards. However, OMB  has not provided information on how roles and responsibilities will be  assigned to continue developing standards in the future. Without clearly  delineated roles and responsibilities, there is a risk of confusion which  could impede action and accountability for future improvements to  program management standards. Further, having clearly delineated roles  and responsibilities is particularly important during periods of transition  when administrations change.

OMB has an incomplete plan for how it will obtain input from stakeholders  and involve them in decision-making. OMB received input from  stakeholders on the standards it developed in 2018, though the strategic  plan states that standards will be further developed with the PMPC in the  fourth quarter of fiscal year 2020. However, the strategic plan does not  give details on how the PMPC and others will further develop standards.  Without robust and comprehensive outreach to individuals who will use or  otherwise be affected by the standards, the opportunity to learn from  stakeholder experience and perspectives, or anyone who will use or  otherwise be affected by the standards, may be diminished.

OMB has an incomplete process for developing and approving program  management standards. OMB developed and approved the existing  standards by obtaining stakeholder input and releasing their approved  standards in its strategic plan. However, the strategic plan does not  provide documentation on how that process was structured and how it will  function in the future. Thus, it is unclear how OMB plans to further  develop the standards and what responsibilities and resources will be  required from OMB, the PMPC, and agencies under the leadership of the  agency PMIOs.

OMB has not defined a process for making decisions about changes to  existing standards and describing how conflicts related to the application  of standards would be resolved. Therefore, it is unclear if or how the  standards will be periodically reassessed and updated as circumstances  change and leading practices in program and project management are  identified. Also, lack of consensus on standards and conflict over how to  use them can lead to weakened acceptance and inconsistent application.

OMB has not defined a process for managing, controlling, monitoring, and  enforcing consistent application of standards. OMB has not developed or  directed any type of review or oversight process to determine the  adequacy of existing or newly developed standards agency use to  manage programs. Having such a process could help agencies to  achieve a balance between consistent application of standards and  flexible application to account for differences in programs, agency  missions, and other factors. However, OMB staff told us that they  consider the PMIAA program portfolio review process as a way to help  monitor and enforce program standards, as they have a view into how  each agency is applying standards for their particular portfolio of  programs. Additionally, OMB has given agencies flexibility in using  existing agency standards and flexibility to adopt or develop new ones.  Without a review mechanism, OMB lacks reasonable assurance that  agencies’ efforts to use existing standards or develop new ones will align  with government-wide efforts to improve program and project  management. Also, establishing an approach to monitoring agencies’  efforts would help identify opportunities to improve program management  standards.

Without having a governance structure for the program standards, the  potential exists that standards will develop in an ad hoc manner, may be  applied inconsistently or not at all, and may not be updated to reflect new  developments in program management. Further, having a governance  structure for managing efforts going forward better positions OMB to  sustain progress on program standards as they change over time.

OMB Leveraged Existing Performance Reviews, but Reviews Are Limited to Major Acquisitions

PMIAA requires agencies and OMB to regularly review portfolios of  programs to assess the quality and effectiveness of program  management and identify opportunities for performance improvement.  To conduct these portfolio reviews, OMB Circular A-11 notes that  agencies and OMB are to use a set of broadly applicable program  management principles, practices, and standards associated with  successful program outcomes, in addition to more specific standards  based on the type of program under review.

As a way to help agencies acclimate to the requirements of PMIAA, OMB  leveraged two components of the GPRA Modernization Act of 2010  (GPRAMA): the strategic review and a cross-agency priority (CAP) goal.  OMB guidance stated that agencies’ portfolio reviews of programs would  be conducted and integrated to the extent practical with strategic reviews.  Furthermore, OMB staff told us that the implementation of PMIAA and the  CAP goal for improving management of major acquisitions (CAP Goal 11)  shared complementary goals and strategies. For example, the CAP Goal  11 action plan includes the routine monitoring of federal program  management progress. Consequently, OMB staff said they decided that  the first PMIAA program portfolio reviews would focus on major  acquisitions.

Excerpt from OMB Cross-agency Priority Goal 11 from 2018 President’s  Management Agenda: Improve Management of Major Acquisitions  Federal agencies will ensure that contracts supporting transformative and priority  projects meet or beat delivery schedules, provide exceptional customer service, and  achieve savings or cost avoidance for the taxpayer.   The Challenge: Major acquisitions—which vary in size by agency but often exceed $50  million—account for approximately one-third of annual federal spend on contracts.  These large contracts frequently support projects meant to transform areas of critical  need. Yet major acquisitions often fail to achieve their goals because many federal  managers lack the program management and acquisition skills required to successfully  manage and integrate large and complex acquisitions into their projects. These short- comings are compounded by complex acquisition rules that reward compliance over  creativity and results.  The Strategies: Agencies will pursue three strategies: 1) strengthen program  management capabilities in the acquisition workforce; 2) use modern and innovative  acquisition flexibilities; and 3) track investments using portfolio, program, and project  management principles.

OMB Reported Lessons Learned from Pilot, but Did Not Follow Most Leading Practices for Pilot Design

In 2018, OMB conducted a pilot project involving program portfolio review  focused on noninformation technology (IT) major acquisition programs.  According to OMB staff, the pilot project gave agencies the opportunity to  complete “dry runs” for the PMIAA-required portfolio reviews and to  provide lessons learned in anticipation of the fiscal year 2019 portfolio  reviews. OMB planned for the results from the pilot to provide information  for internal dialogue and decision-making about subsequent portfolio  reviews. Further, according to OMB’s strategic plan, the purpose of the  pilot was (1) to determine how well agency program portfolios of non-IT  major acquisitions were performing throughout the life cycle of the  investment using a set of standards and practices, and (2) to refine the  process of coordinating program portfolio reviews as a component of  OMB agency strategic reviews.

For the pilot, OMB staff directed agencies to assess the cost, schedule,  and performance of agency-selected acquisition portfolios. One result  from the pilot was that agencies demonstrated a range of maturity in their  abilities to collect data for these required program portfolio measures from  their various departments and program types. OMB staff told us pilot  agencies found it easier to compile data on major construction projects  compared to service contracts. Consequently, an agency doing many of  these projects might be more advanced than an agency for which major  acquisitions focus on services. Department of Veterans Affairs (VA) staff  shared their lessons learned from their participation in pilot portfolio  reviews, as seen in the text box below. OMB staff said that they  determined that the portfolio review process worked sufficiently well for  the pilot agencies and continued their planned strategy of focusing solely  on non-IT major acquisition programs for fiscal year 2019 portfolio  reviews.

Example of Department of Veteran Affairs (VA) Lessons Learned from Pilot  Portfolio Review The VA looked at the effectiveness of portfolio management during the Office of  Management and Budget noninformation technology major acquisition pilot portfolio  review by focusing on the agency’s adherence to best practices in assessing project  performance and progress. VA officials said this pilot informed their decision-making  and was successful in the following ways:   1.  The pilot helped VA determine logical ways to manage a portfolio by showing what  data were helpful to make impactful decisions.   2.  VA learned how best to display the data on cost, schedule, scope, and quality of  outcomes on a dashboard to make it accessible and comparable across the  agency.   3.  VA learned that it needs to collect better quality data so that project management  principles can be instituted and aligned across the agency.

A well-developed and documented pilot program can help ensure that  agency assessments produce information needed to make effective  program and policy decisions. Such a process enhances the quality,  credibility, and usefulness of evaluations in addition to helping to ensure  the effective use of time and resources. We have identified five leading  practices that, taken together, form a framework for effective pilot design,  as seen in the text box below.

OMB fulfilled the first leading practice of establishing objectives in its  design of the PMIAA pilot program portfolio review. OMB’s PMIAA  strategic plan and the CAP Goal 11 Action Plan stated the objectives of  the pilot. In addition to the two objectives listed in the PMIAA strategic  plan, the CAP Goal 11 Action Plan lists seven pilot objectives, as seen in  the text box below.

PMIAA Pilot Program Portfolio Review Objectives 1.  Perform portfolio management preparation activities  2.  Identify first portfolio of major acquisitions  3.  Align portfolio with agency strategic goals  4.  Collect performance data for each item in the portfolio  5.

OMB officials said that they did not structure the pilot to follow the  remaining four leading practices for effective pilot design. However, OMB  said that it learned that the pilot agencies demonstrated several program  management capabilities. They also learned that it would be important to  tailor portfolio reviews to the agency and the program to account for  significant differences in the types of acquisitions and the level of program  management maturity.

Despite identifying lessons learned from its pilot program portfolio review,  in neglecting to fully follow leading practices, OMB may have missed  opportunities to make additional improvements for fiscal year 2019  portfolio reviews. Going forward, as OMB expands the portfolio reviews to  other types of program areas beyond non-IT major acquisitions, it has the  opportunity to develop and learn from additional pilots. Although OMB  staff have not yet determined if they will do additional pilots for program  management in the future, they could decide to pilot the portfolio reviews  of grants that they plan to initiate in fiscal year 2020.

OMB Limited Its Portfolio Reviews to Non-IT Major Acquisition Programs

For fiscal year 2019, OMB directed all agencies to select portfolios of  non-IT acquisition programs and align them with relevant strategic  objectives as part of their internal agency strategic review processes. In  spring 2019, OMB expected agencies to discuss one to two of these  major-acquisition portfolio reviews during their strategic reviews with  OMB. OMB expected agencies to track the cost, schedule, and  performance of their selected major acquisition programs. However, OMB  reports that not all agency program portfolio reviews were completed  because OMB was behind in scheduling the reviews due to the partial  government shutdown. According to documents we reviewed and what  OMB staff told us, in October 2019 OMB completed agency program  portfolio reviews with ten agencies: the Departments of Commerce,  Homeland Security, Housing and Urban Development, Labor, and  Transportation; the General Services Administration, the Social Security  Administration, NASA, the National Science Foundation, and the US  Agency for International Development. OMB staff also told us that they  also held preparatory meetings with agencies to set expectations for  future portfolio reviews. OMB reported that these one-on-one meetings  were held with 12 agencies as of October 2019 to discuss their initial  portfolio structures and other transformative initiatives.

Portfolio reviews in 2020 are to expand in scope to include grants, and  also will continue acquisition portfolio reviews as part of the agency’s  routine management processes. However, OMB has not yet identified  other program areas, such as research and development or benefit  programs, to be included in future portfolio reviews. Standards for Internal  Control in the Federal Government states that effective information and  communication are vital for an entity to achieve its objectives. Specifically,  management should externally communicate necessary quality  information to achieve its objectives. Increasing communication to  agencies about specific program areas, portfolio review procedures, and  expectations beyond 2020 could help ensure continued progress to  implement PMIAA more broadly. Furthermore, communicating such  procedures with specific time frames could help agencies better direct  their efforts to improve the portfolio review processes.

OMB Has Not Fully Implemented an Inventory of All Federal Programs

GPRAMA requires OMB to make a list of all federal programs identified  by agencies publicly available, on a central government-wide website.  The implementation of the program inventory is a critical tool to help  decision makers better identify and manage programs across the federal  government. Among other things, the completion of the program inventory  would provide agencies and Congress with a comprehensive list of  programs, so it would be clear how many programs agencies are  managing and how they relate to their strategic objectives and portfolios  of programs at each agency. Having a program inventory could also help  ensure a match between the number of agency programs and needed  program manager resources.

Agencies continue to struggle with challenges defining their programs.  Officials from three of the five selected agencies we spoke with told us  that they have not yet identified all of their programs and projects. In our  first report on the program inventory in October 2014, we noted similar  issues. For example, agencies were not using the same program  definition approach across their subcomponents or offices, which limited  comparability of their own programs. We made eight recommendations  in that report to the Director of OMB to update relevant guidance to help  develop a more coherent picture of all federal programs and to better  ensure information is useful for decision makers. As of October 2019,  OMB had not taken any actions in response to the eight  recommendations. While OMB has provided a timetable for action in its  June 2019 A-11 guidance, this does not complete the recommendation.

In September 2017, we made two recommendations to OMB to make  progress on the federal program inventory. First, we recommended that  OMB consider using a systematic approach for the program inventory,  such as the one we developed from principles of information  architecture. Information architecture—a discipline focused on  organizing and structuring information—offers an approach for developing  a program inventory to support a variety of uses, including increased  transparency for federal programs. OMB staff told us that they considered  our information architecture approach and noted that a structured  information architecture format is used on USASpending.gov. However,  OMB staff told us they had not yet determined how the information  architecture format of USASpending.gov—which is focused on spending  data—could be used to meet additional information reporting  requirements and our past recommendations related to the inventory.

We made a second recommendation that OMB should revise and publicly  issue OMB guidance—through an update to its Circular A-11, a  memorandum, or other means—to provide time frames and associated  milestones for implementing the federal program inventory. As mentioned  above, OMB did provide a timetable but it does not have milestones.  According to the timetable, beginning with the 2021 budget cycle,  agencies’ program activities will be used for the inventory’s program-level  reporting requirements. This will allow OMB and agencies to present  program-level spending data by leveraging what is reported on  USASpending.gov as required by the DATA Act. However, OMB’s  guidance does not cover other inventory information reporting  requirements, or the actions we recommended in October 2014. We will  continue to monitor progress.

We continue to believe it is important for OMB to implement our program  inventory recommendations. Such an inventory could be a critical tool to  help decision makers better identify and manage fragmentation, overlap,  and duplication across the federal government. Additionally, fully taking  action on these recommendations would assist agencies in identifying  programs, better prepare for future PMIAA portfolio reviews, and help  match resources to agencies’ program management needs.

Further, OMB developed three different definitions for what constitutes a  “program” or “program activity” that it provided to agencies in its PMIAA,  GPRAMA, and DATA Act guidance, respectively. OMB developed each of  these definitions independently and in response to three different  statutory requirements. OMB staff told us that these three requirements  differ in their legislative intent. The definitions and their associated  guidance are in the table below.

OMB has not reconciled these overlapping, yet divergent, definitions of  what constitutes a “program” or “program activity.” According to  Standards for Internal Control in the Federal Government, management  should ensure that specific terms are fully and clearly set forth so they  can be easily understood. Standards for Internal Control in the Federal  Government also states that management should design processes that  use entities’ objectives and related risks to identify information  requirements needed to achieve objectives and address risks. OMB has  defined what constitutes a “program” or “program activity” in PMIAA,  GPRAMA, and the DATA Act each, but its three different program  definitions and approaches to determining what is a “program,” could  cause confusion for agencies.

Agency officials from the Department of Energy told us they are already  experiencing confusion over how to appropriately apply the applicable  program definition to identify their programs for PMIAA. Agency officials  from Treasury told us that different definitions for programs could  contribute to confusion as they work to implement PMIAA within the  Department. The inconsistent approaches may increase the burden on  agencies as they work to identify, maintain, and report on three sets of  differently defined programs. Conversely, clarifying the definitions could  help agencies and OMB identify synergies across the three laws and  increase transparency. For example, providing explanations of how the  term “program” or “program activity” is used across the three statutory  definitions and developing a crosswalk to show similarities and  differences could provide more clarity for agencies. Then, spending and  performance data can be aligned with agency strategic goals, which could  be monitored, reviewed, and reported in a streamlined manner.

OPM Meeting Workforce Requirements of PMIAA

OPM followed PMIAA requirements to create policy and guidance.  Specifically, according to documents we reviewed, OPM (1) worked with  subject matter experts to develop program and project management skills  and competencies, (2) updated the program management 0340 job series  and created guidance for identifying project management positions, (3)  plans to release a career path for program and project managers by the  end of calendar year 2019, and (4) plans to create a unique job identifier  code that can be used to pinpoint program and project managers in any  job series. These efforts will form the foundation needed by agencies to  strengthen resource and talent management.

Competency modeling. Since enactment of PMIAA, OPM identified  skills and competencies which will be required for program and project  managers. According to documents we reviewed, OPM met with subject  matter experts and human capital staff in agencies to help identify the  skills needed to develop the competency model. OPM also conducted a  literature review looking at prior competency studies and industry  practices to help identify and support program and project management  competencies. OPM also drew from Project Management Institute  resources, such as the Project Management Body of Knowledge and the  Standard for Program Management, as part of identifying its  competencies. The resulting competencies are in two categories: general  and technical. General competencies focus on interpersonal or general  on-the-job skills such as teamwork and problem solving. Technical  competencies more narrowly focus on particular skills needed to run  programs and projects, such as risk management and cost-benefit  analysis.

OPM documents stated that agencies will need to determine the  applicability of these competencies to positions within their agency.  Agencies must determine if staff meet the competencies, and if not, staff  will have the opportunity to develop them or must move to a different job  series, according to OPM staff. OPM staff also said additional  competency assessment steps are needed to finalize the model.  Agencies will be given time to consider the competency model. In  addition, OPM will use subject matter expert panels to further develop the  model, according to OPM documents we reviewed.

Updated job series. To implement job series requirements in PMIAA,  OPM staff conducted an occupational study and determined that pre- existing classification policy was sufficient for classifying program  management work rather than creating a new job series classifying  program management positions, according to OPM staff.

Prior to OPM updating the program management 0340 job series for  PMIAA, the classification standard was not developed, as it did not  contain competencies describing what qualifications staff were required to  meet as a program manager. In May 2019, OPM released the updated  job series classification guidance designed to assist agencies in  determining which employees fit in the job series. OPM also released  guidance for classifying project managers to help agencies specifically  identify project managers in any occupational job series. According to the  memorandum sent by the Acting Director of OPM to agencies with the  OPM classification guidance, agencies are required to implement the  policy and guidance to covered positions by May 1, 2020.

Career path. OPM staff told us that they have developed a career path  for program and project managers that is currently in internal review.  They said that the value of the updated career path is that it will highlight  training and skills needed to progress in a program management career.  According to the presentation given by OPM at the 2019 April PMPC  meeting, the career path will contain: (1) a career progression outline for  employees to move among and across jobs in program and project  management, (2) help for employees and supervisors to plan and  sequence appropriate career training and development for each general  and technical competency, and (3) a list of common degrees and  certifications completed by program and project managers, among other  things. Staff told us they plan to release the program and project  management career path for agency comment by the end of calendar  year 2019.

Job identifier for program managers and project managers. Because  program and project managers are found in other job series outside the  0340 program management series, OPM is developing a job identifier  code that can be attached to any job series for the purposes of identifying  program and project managers. OPM staff told us that program managers  classified to the 0340 series means that the position does not have a  specialization. If the position requires specialized expertise, the position  would be classified to a specialized occupational series but would also  have a program management job identifier code. For example, since a  grants managers is also a program manager, “grants manager (program  management)” would be his or her official title. Project management  positions will also use a job identifier to identify project managers in any  occupational series. The job identifier will allow employees with a  specialization to be designated program and project managers, while still  maintaining their original career path. OPM staff told us they plan to  complete this project in 2020.

OPM and Agencies in Early Stages of Workforce Planning

Our analysis of OPM Enterprise Human Resources Integration data  shows that the 0340 job series included about 15,000 employees across  all 24 CFO Act agencies in fiscal year 2018. However, OPM reported  that not all employees in this job series are actually program and project  managers; conversely, many program and project managers are working  outside of the 0340 job series.

Selected agencies reported varying degrees of difficulty identifying  program and project managers. For example, NASA staff reported that  they were able to identify almost all their program and project managers.  In contrast, the Department of the Treasury reported that it faces  challenges identifying the number of program and project managers  outside of the program management job series, as this would require a  resource-intensive manual effort, made more challenging by the agency’s  large, complex, and decentralized structure. The Department of Energy  (DOE) staff said they have not completed the count of their program  managers. The Departments of Commerce and Veterans Affairs also  report they do not know the number of program and project managers in  their departments, respectively. The Department of Commerce staff told  us that they cannot accurately identify the number of program and project  managers until they can use the job identifier that they expect OPM to  release in 2020. Further, Commerce officials told us they are also  continuing to work to identify program managers and engaged the Project  Management Institute (PMI) to request a list of those within Commerce  who have the Project Management Professional (PMP) certification.   PMI was able to provide Commerce details about the numbers of PMPs  at Commerce, but PMI declined to share the names of those individuals  with the PMP certification.

In OPM’s 2018 Federal Workforce Priorities report, OPM recognizes that  not all agencies have adequately analyzed workload demands, staffing  levels, or current and future skills needs—all steps in workforce planning.  As part of the OPM human capital framework, agencies are required to  develop a human capital operating plan which is an agency’s human  capital implementation document. These plans are to describe how  agencies will execute the human capital strategies needed to implement  the agency’s strategic plan and Annual Performance Plan (APP).  Agencies are also required to include program specific strategies (e.g.,  hiring, closing skills gaps, etc.) in the APPs as appropriate.

Effective workforce planning can help agencies focus on determining how  many program and project managers they have, how many they may  need, what skills gaps exist, and what training and other strategies can  help address skills gaps. OPM’s workforce planning model is comprised  of five steps:  1.  Set strategic direction;  2.  Analyze workforce, identify skills gaps, and conduct workforce  3.  Develop action plan;  4.  Implement action plans; and  5.  Monitor, evaluate, and revise.

The discussion below describes how OPM and agencies are working to  strengthen the program management workforce in the context of OPM’s  workforce planning model. Some activities may span more than one  phase of workforce planning.

Set strategic direction. The PMIAA strategic plan establishes direction  for agencies to build its program management capacity and capability  with its third strategy, “Strengthening Program Management Capacity to  Build a Capable Program Management Workforce.” Setting strategic  direction also involves linking work activities to the objectives of a  strategic plan. OPM’s planned activities, such as updating the  classification standards and creating a job identifier, are critical to  executing this strategy so agencies can identify their workforce and build  program management capacity through training, career paths, and  mentorship opportunities.

Analyze workforce, identify skills gaps, and conduct workforce  analysis. OPM and agencies are in the early stages of identifying who  their program and project managers are and what human capital  strategies might be needed to address agencies’ needs. Documents we  reviewed showed that OPM also worked with the Chief Human Capital  Officers Council, the Chief Administrative Officers Council and others to  develop competencies. These competencies provide a foundation for the  subsequent assessment of program and project manager skills.

Develop action plan. In their PMIAA implementation plans, some  agencies have identified available training and possible recruitment and  hiring strategies. In OPM’s model, agencies need to complete their  workforce analysis before they can develop their action plans.

Implement plan. This step is dependent on agencies developing action  plans. However, OPM and agencies have already started to develop staff  in the absence of plans. For example, OPM is working with agencies to  identify program management training matching desired competencies to  be placed in an online training repository that will be accessible to all  agencies. OPM staff told us that agencies would provide the trainings  from their learning management systems and offer them for interagency  access. OPM is developing this training and development repository  which will house agency-owned courses and also identify mentors in  project and program management, according to OPM staff. OPM will  house the repository on its training and development policy wiki at  https://www.opm.gov/wiki/training/index.aspx.

Each PMIO is to also establish a website with agency-specific program  management tools and resources. Additionally, OMB recognized that the  Federal Program and Project Management Community of Practice  (FedPM CoP), scaled up from a community of practice housed in DOE,  could be an important partner in supporting PMIAA implementation. As of  April 2019, more than 1,000 managers had joined the FedPM CoP as  indicated in its briefing to the PMPC. The FedPM CoP has identified  several project management-related documents that are now available on  the PMIAA portal.

To further develop program managers, OMB is working with agencies to  improve mentoring and recognition efforts. To improve mentoring  government-wide, OMB reports that PMIOs will work with agency chief  human capital offices to develop and implement a mentoring strategy for  agency program managers. OMB also plans to take existing mentorship  programs established in more functionally aligned-management fields  (e.g., information technology, acquisition) and expand them to include a  broader range of management career paths. To improve recognition  efforts in acquisitions, the Chief Acquisition Officer Council plans to  establish an annual award to recognize federal program manager  excellence.

Monitor, evaluate, and revise. This step cannot begin until agencies  develop and implement their workforce action plans. As agencies begin to  monitor their implementation of these plans, they will need to determine if  any skills gaps exist in the program and project manager occupational  series. OPM regulations require agencies to describe in their human  capital operating plans agency-specific skills and competency gaps that  must be closed through the use of agency selected human capital  strategies. Agencies must also have policies and programs that monitor  and address skills gaps within government wide and agency-specific  mission-critical occupations. OPM has not yet determined if program  and project management occupations are experiencing mission-critical  skills gaps across the government, and OPM staff noted that agencies  are not specifically required to report program and project manager skills  gaps in their annual human capital operating plans.

OMB and OPM Completed Some PMIAA Requirements Late

OMB and OPM both missed statutory deadlines to fulfill requirements in  PMIAA. In June 2018, OMB issued the required PMIAA agency  implementation guidance in the PMIAA strategic plan, 6 months after the  statutory deadline of December 2017. According to OMB staff, this  delay was due to their own research project to (1) build sufficient  knowledge in program and project management; and (2) increase  stakeholder support in Congress and with agencies for its approach.  Specifically, OMB met with experts from PMI, academics, consulting  firms, federal chief senior level officer (CXO) councils, and other agency  officials to increase its own understanding of program and project  management principles.

OMB staff told us that they used the collected information to draft initial  guidance, which they then shared with congressional stakeholders and  executive branch agency officials to obtain feedback and incorporate  changes. OMB staff also told us that it was a transition year from one  administration to another administration, and this transition was an  additional factor in delaying completion of the guidance. None of the  selected agencies’ staff identified an impact from the delayed guidance.

OPM officials told us they missed the statutory deadline to complete their  required activities after the issuance of OMB guidance. The release of the  policy and guidance was due to the partial government shutdown from  December 22, 2018 to January 25, 2019, along with a 3-month delay due  to OPM’s own internal review and clearance process. As a result, OPM  released the key skills and competencies needed for program and project  management on April 5, 2019, and the classification guidance for the  program manager job series 0340 and project manager interpretative  guidance on May 2, 2019. OPM officials told us that agencies have 1 year  from the date of issuance to comment on any language in the guidance.  None of the selected agencies’ staff identified an impact from OPM’s  delays, although one agency expressed concern that the pace of their  efforts to identify program and project managers is dependent on OPM  completing the job identifier. Figure 2 shows the delays in releasing OMB  and OPM guidance.

PMPC Has Met Three Times and CFO Act Agencies Have Started to Implement PMIAA Requirements

OMB officials established the PMPC in 2018 and fulfilled requirements  that it meet at least twice per year. By September 2018, the 24 CFO Act  agencies had all appointed a PMIO and held three PMPC meetings, in  September 2018, April 2019, and September 2019. Selected agenda  items for these PMPC meetings included:  status updates on OPM completing program and project manager  competencies, job series, and career path; breakout sessions to discuss PMIAA implementation approaches with discussion of PMPC priorities and focus for 2020.

At the April 2019 PMPC meeting, for example, staff from the Department  of Veterans Affairs and the National Science Foundation shared some  best practices, such as how to improve the tracking performance of  portfolios, programs, and projects.

According to OMB documents we reviewed, OMB plans to:  convene the PMPC in the first quarter of each calendar year to  prepare for upcoming OMB and agency strategic review meetings; use the PMPC meeting in the third quarter of the calendar year to  review findings and outcomes from the most recent strategic review; update program and project management standards based on its  findings and feedback at the PMPC meeting in the fourth quarter of  2020; use the PMPC to develop revised strategies, initiatives, and priorities  to be reflected in an updated 5-year strategic plan at the PMPC  meeting in the fourth quarter of 2021; and use the PMPC to focus on improving our high-risk areas at some  future point.

At the September 2019 PMPC meeting, OMB informed agencies of  PMIAA implementation resources placed on OMB’s online portal for  PMIAA and discussed OMB’s observations on portfolio reviews  completed in 2019. One observation was the need to reinforce better  visualization of performance data. In addition, OPM updated the PMPC  on the status of its required PMIAA workforce efforts. The PMPC decided  its primary focus for the year 2020 should be on the third strategy of the  PMIAA strategic plan to build a capable workforce.

Officials from the selected agencies that we interviewed provided us  some suggestions on how OMB can improve the functionality of the  PMPC. Table 3 illustrates the range of these suggestions:  The PMPC met twice in 2019, as required by PMIAA, and has not  established any working groups to help execute its significant  responsibilities to share leading practices, develop standards, and help  improve the workforce.

Agencies have taken initial steps to incorporate requirements into  program efforts. According to OMB guidance, agencies were to report in  implementation plans how they are institutionalizing PMIAA efforts— especially PMIO responsibilities—into existing program and project  management practices. OMB requested that agencies include 10 specific  elements in their implementation plans, such as:  identification of the agency PMIO,  identification of major acquisition portfolios, and  strategies and actions for enhancing training and improving  recruitment and retention of program and project managers.

These plans were due to OMB by November 30, 2018.

We reviewed PMIAA draft implementation plans for 22 of the 24 CFO Act  agencies and determined the extent to which agencies included the  required elements in their plans. In its PMPC meeting in April 2019,  OMB reported that a majority of agencies only partially included OMB  requirements in their draft implementation plans. OMB told us they have  not directed agencies to address missing requirements nor have they  required agencies to finalize their draft implementation plans. They told us  that they view the implementation plans as an opportunity for each  agency to engage with OMB and discuss how they will implement PMIAA.

OMB staff told us that their view is that if implementation plans provide  value to agencies, they may stay in draft form and do not need to be final.

Overall, draft implementation plans for these agencies provided some but  not all information required to fully meet the directives from OMB. Our  analysis of the plans shows that on average, agencies fully met six out of  10 requirements for their implementation plans. For example, almost all  agencies met the requirements for identifying the PMIO (21 out of 22).  However, 11 out of 22 agencies did not provide complete information on  major acquisition portfolios. Table 4 shows how agencies’ implementation  plans varied in meeting the requirements.

Seven of 24 agencies reported in our questionnaire that they were  creating either task forces or new or restructured offices to direct PMIAA  implementation within their agencies. For example, DOE reported  establishing a new office to support its PMIO. The Department of the  Treasury and NASA reported creating an intra-agency cross-functional  core team to discuss and design PMIAA implementation strategies. OPM  reported establishing an enterprise program management office to drive  the standardization of program and project management processes  internally.

Agencies selected PMIOs in existing leadership positions to leverage  resources and agency processes to implement PMIAA. All agency PMIOs  reported having additional leadership responsibilities beyond their PMIO  roles. OMB documentation and information gathered from CFO Act  agencies shows: every PMIO has at least one additional CXO role within its agency;  thirty-eight percent of PMIOs have an additional performance  management role; eight of 24 PMIOs have an additional budgetary role; and  four of the 24 PMIOs have an explicit additional program or  acquisition role.

OMB Has Taken Limited Steps to Address Areas on Our High-Risk List

In the past, we have met with senior management officials from OMB and  applicable agencies to discuss where additional management attention  could be beneficial to addressing high-risk areas identified on our High- Risk List. We also reported that these trilateral meetings, which began in  2007 and pre-dated PMIAA’s 2016 enactment, have continued across  administrations and have been critical for progress that has been made in  addressing high-risk areas.

According to PMIAA, OMB’s Deputy Director of Management is to  conduct annual portfolio reviews of the most at-risk agency programs, as  designated by our High-Risk List. OMB officials view the trilateral  meetings as their method for holding the portfolio review meetings for  high-risk areas as required under PMIAA. Our High-Risk List is comprised  of programs as well as functions and operations. Consequently, in our  assessment of OMB’s implementation of PMIAA, we consider programs,  functions, and operations on our High-Risk List as relevant for OMB’s  portfolio review of areas on our High-Risk List. OMB used three strategies  intended to meet PMIAA’s high-risk requirements. OMB (1) expanded its  strategic reviews in 2018 to include a review of some high-risk areas, (2)  continued to use the long-standing trilateral meetings to review high-risk  areas with agency leaders and with us, and (3) held ad hoc meetings with  agencies outside of the strategic review and trilateral meetings.

OMB Discussed High-Risk Areas with Some Agencies during Strategic Review Meetings

In preparation for the 2018 strategic reviews, OMB issued Memorandum  M-18-15 directing agencies to provide several items in advance of their  strategic review meetings with OMB. Requested items included updates  from agencies on areas identified on our High-Risk List in which agencies  disagreed with our recommendations or faced implementation barriers  preventing progress. These materials were to be discussed during  strategic review meetings. Thirteen CFO Act agencies reported  submitting high-risk updates to OMB prior to these meetings, and eight  agencies reported discussing their high-risk areas with OMB during the  meetings.

OMB guidance from June 2019, communicated in OMB’s Circular No. A- 11, did not include the statement from Memorandum M-18-15 that high- risk areas would be discussed during strategic review meetings. OMB  staff felt that a broader approach could yield better results for addressing  high-risk areas. Guidance in Circular No. A-11 maintained that agencies  should submit updates about high-risk programs to OMB for the Deputy  Director’s high-risk portfolio review, but it did not specify what should  comprise agency updates about high-risk programs. Also, OMB staff told  us that they requested that agencies provide topics for discussion at  strategic review meetings, and that agencies could provide agenda items  related to our High-Risk List.

OMB staff said they addressed only a few of the high-risk issues during  strategic reviews, both during the review process and the strategic review  meetings. Discussions about high-risk issues during strategic review  meetings generally focused on government-wide high-risk areas, if  relevant, such as “Ensuring the Cybersecurity of the Nation” and  “Improving the Management of Information Technology (IT) Acquisitions  and Operations.” However, OMB and agencies also discussed high-risk  areas in instances when agencies provided strategic review meeting  agenda topics related to our High-Risk List. For example, Treasury staff  told us they spoke with OMB this year about high-risk areas as part of the  strategic review process. Treasury is directly responsible for the  Enforcement of Tax Laws high-risk area and shares responsibility with  other agencies for other high-risk areas, such as the government-wide  areas on cybersecurity and strategic human capital.

OMB Held Trilateral Meetings on Five of 35 High-Risk Areas

OMB has held a limited number of trilateral meetings with agencies and  us about high-risk areas as part of the high-risk portfolio reviews.  Between March 2018 and October 2019, OMB addressed the following  five high-risk areas in trilateral meetings with applicable agencies and us:

Government-wide Personnel Security Clearance Process,

Ensuring the Cybersecurity of the Nation, and

OMB has not held meetings to address the remaining 30 high-risk areas  on our High-Risk List. OMB staff told us they plan to hold additional  meetings in the next year but that they are unlikely to be able to schedule  all remaining meetings within our 2-year cycle for updating the High-Risk  List. OMB staff said that it is sometimes challenging to coordinate and  convene trilateral meetings given the high-ranking officials who must  attend and finding available times across schedules. OMB also told us  that they plan to meet with agencies for all high-risk areas eventually, but  that they prioritize meetings aligned with our priority areas and the  President’s Management Agenda.

We evaluate progress made on high-risk areas every 2 years to  determine if new areas should be added to our High-Risk List and if areas  on the list should be removed due to progress to address the risks. Top  leadership commitment is one of the five criteria we use to assess  whether progress is being made to address and ultimately remove areas  from our high-risk list. As we have reported in our March 2019 High-Risk  Series report, leadership commitment is the critical element for initiating  and sustaining progress, and leaders provide needed support and  accountability for managing risks. Leadership commitment is vital if  agencies are to adequately address high-risk areas, and trilateral  meetings have been critical in focusing leadership attention in the past.  Because OMB officials have met on only five of 35 high-risk areas, it  remains to be seen if they will meet on all high-risk areas in the future.  Convening the trilateral meetings on all high-risk areas in the 2-year  reporting cycle, would better position OMB to enhance the leadership  commitment needed to make greater progress on the remaining high-risk  areas.

OMB Occasionally Discussed High-Risk Areas with Some Agencies throughout 2018 and 2019 beyond Trilateral and Strategic Review Meetings

Staff from OMB said that they sometimes have briefings related to  agencies’ high-risk areas separate from the annual strategic review  meetings and high-risk trilateral meetings. These meetings happen on an  ad hoc basis and are typically initiated by agency officials. Officials from  some of our selected agencies corroborated that the discussion at the  strategic review meetings and trilateral meetings is not the full extent of  OMB’s interaction with agencies about high-risk areas throughout the  year. For example, VA officials said that high-risk areas are frequently  agenda items in meetings with OMB. NASA officials said they spoke with  OMB about NASA’s high-risk areas after submitting material as part of the  strategic review process.

Program Management Policy Council Has Not Made Recommendations to Address High-Risk Areas

The PMPC, chaired by the Deputy Director for Management of OMB, did  not address our High-Risk List during its three meetings nor did it make  recommendations to OMB about addressing high-risk areas, as  required. The PMPC meetings have lasted 60 to 90 minutes each and  the High-Risk List has not appeared as an item on any of the PMPC  meeting agendas. OMB staff said PMPC meetings at this point in PMIAA  implementation primarily act as forums in which agencies can share  program management practices. Rather than focusing meeting time on  high-risk areas, OMB staff asserted that the best use of the PMPC is  primarily as a forum for agencies to share program and project  management best practices. Consequently, the PMPC has not satisfied  all PMPC requirements as delineated in PMIAA, including for high-risk  areas to be addressed.

OMB Identified Measures to Assess Results of Portfolio Reviews, but Has Been Limited by Agency Data Quality

OMB Established a Prototype Dashboard to Help Track Portfolio Program Management Measures of Cost, Schedule, and Performance

OMB created a dashboard to identify measures of cost, schedule, and  performance that agencies should use to track their selected non-IT major  acquisition programs for the first PMIAA program portfolio review. OMB  partnered with the General Services Administration to complete a  prototype of a dashboard to show cost, schedule, and performance data  from each program or project within a portfolio of programs. The  dashboard also provides a short description of each program or project  and its strategic alignment to the agency’s relevant strategic goal. Staff  from OMB’s Office of Federal Procurement Policy said the dashboard  could provide them with some visibility and improved transparency for  major acquisitions programs. According to the PMIAA strategic plan, the  dashboard would display the agency portfolio and summarize  performance for each item in the portfolio, similar to the portfolio reviews  of IT programs required by the Federal Information Technology  Acquisition Reform Act. Initially, according to OMB, it plans to request  summary information for each portfolio, and restrict the dashboard to  authorized government employees. Moving forward, OMB staff said that  as the portfolio management process matures, a portion of the dashboard  may be available to the public, similar to the IT dashboard.

OMB staff told us they are in conversation with agencies about how to  overcome difficulties in collecting data for the dashboard. According to  OMB, the results from the pilot portfolio review showed that agencies  experienced challenges with collecting high-quality data. OMB staff said  there will likely be more metrics for large construction projects because  management practices for them are more mature than for other types of  programs, such as services. OMB is working with agencies to see how  they can retrieve cost, schedule, and performance data that could provide  early warning indicators of potential problems with programs.

Agencies Plan a Range of Ways to Measure PMIAA

Agencies reported in our questionnaire they are considering various ways  to measure implementation of PMIAA. A little more than half of agencies  responding to our PMIAA questionnaire provided ideas on how to  measure implementation of PMIAA, such as tracking completion of their  identified PMIAA milestones, developing their own survey as a baseline  measure, or using their agency implementation plan outcomes to  measure results. Six agencies’ questionnaire responses noted that they  are planning to use existing metrics to assess program performance,  either through internal processes or their annual strategic review process.  For example, Treasury plans to focus in the near term on tracking  completion of milestones of PMIAA implementation, such as major  program and project alignment to department strategic objectives,  development of an information-sharing site for program and project  management resources, and workforce capabilities, among other things.  VA anticipates developing outcome measures associated with successful  program execution and is leveraging measures from existing plans, such  as their Acquisition Human Capital Plan.

OMB staff told us that they have no plans to identify measures to assess  outcomes of PMIAA because it is too early and agencies are in the early  stages of implementation. Rather than tracking anything specific, they told  us that OMB looks at whether agencies’ PMIOs are engaged, if agencies  are using training material and mentorship programs, the involvement of  chief senior level officers, and if there is funding in the budget for program  management certificate programs. However, OMB has not identified  specific measures to track any of these areas.

In collaboration with OMB, VA developed a program management  maturity model survey identify capability gaps, obtain insights, and enable  benchmarking of program management capabilities. It surveyed agencies’  level of maturity on a range of program management capabilities, such as  talent management, governance, and portfolio management. Maturity  assessment surveys can be useful tools for measuring progress to  develop capacity in areas such as program management, according to  subject matter specialists. Periodically measuring maturity can help  agencies institutionalize continuous assessment and improvement. PMI  also supports using such tools to identify trends that can help pinpoint  actions needed and opportunities to learn from more mature  organizations.

We have found that ongoing performance measurement can serve as an  early warning system to management and as a vehicle for improving  accountability to the public. We have previously reported that providing  baseline and trend data can help to assess an agency’s performance  more fully because the data show progress over time and decision  makers can use historical data to assess performance.

As OMB and agencies move forward with PMIAA implementation, it will  be critical to measure how agencies are maturing or building their  capacity in the areas of program and project management. Such  measures could include showing how OMB’s program management  standards and principles are integrated into agencies’ programs and  policies, the improvement of data quality used to track agency program  outcomes in the program portfolio reviews, and improvement in program  manager skills. Although not required by PMIAA, it is a good practice for  OMB and agencies to consider ways to measure the effects of the act.  Without establishing such measures to assess PMIAA outcomes, it will be  challenging to gauge how agencies are making progress to identify  trends, or to help agencies improve data quality.

The program and project management standards OMB developed are  less detailed than accepted standards and are missing several elements  that would have made them more useful. For example, the OMB  standards do not provide a minimum threshold against which agencies  can gauge to what extent they have met each standard. Further, OMB’s  current governance structure is insufficient for further developing and  maintaining program management standards. Although OMB received  input from stakeholders to develop the standards and plans to update  them in partnership with the PMPC in 2020, OMB does not have a  governance structure that assigns roles and responsibilities to further  develop, approve, maintain, or monitor standards. Having such a  governance structure for managing efforts going forward could help  sustain the program standards as they change over time.

OMB did not follow most leading practices for designing pilots and may  have missed opportunities to make improvements for fiscal year 2019  portfolio reviews. OMB has not determined if it plans to conduct additional  pilot efforts. Going forward, as OMB expands the portfolio reviews to  other types of program areas beyond non-IT major acquisitions, it has the  opportunity to develop and learn from additional pilots. Although OMB  staff have not yet determined if they will do additional pilots for program  management in the future, they could decide to pilot the portfolio reviews  of grants that they plan to initiate in fiscal year 2020.

OMB has not identified other program areas beyond non-IT major  acquisitions and grants to be included in future portfolio reviews.  Communicating to agencies about specific program areas, portfolio  review procedures, time frames, and expectations beyond 2020 could  help agencies better direct their efforts to improve the portfolio review  processes and help ensure continued progress to implement PMIAA  more broadly.

As of October 2019, OMB had not taken any actions in response to the  recommendations in our September 2017 report and has not yet fully  established an inventory of federal programs. Such an inventory of  programs could be a critical tool to help agency officials identify and  manage programs across the federal government. Furthermore, if OMB  were to fully implement our recommendations and complete the required  inventory of federal programs, it would assist agencies to match  resources to agencies’ program management needs and assist agencies  in preparing for future PMIAA portfolio reviews. Furthermore, OMB  provides three different definitions for a “program” in its guidance for  PMIAA, GPRAMA, and the DATA Act. Having different definitions of what  constitutes a program could lead to confusion among agencies. It could  also cause increased burden on agencies as they work to identify,  maintain, and report on three sets of differently defined programs.

Meetings between OMB, relevant agencies, and us have been critical for  past progress on high-risk areas. However, OMB has held these trilateral  meetings to address only five of 35 high-risk areas since it began  implementing PMIAA. These meetings could both demonstrate and  improve the commitment of agency leadership to high-risk areas across  the federal government. As we have reported, leadership commitment is  a key tenet in agencies’ ability to address high-risk areas. Without  convening trilateral meetings on each high-risk area, OMB might miss  opportunities to make progress toward addressing high-risk areas by  improving leadership commitment to addressing them.

The PMPC did not address our High-Risk List during its meetings nor has  it made recommendations to OMB about high-risk areas. The High-Risk  List has not appeared as an item on any of the PMPC meeting agendas.  OMB staff asserted that the best use of the PMPC’s limited meeting time  is as a forum for agencies to share program management best practices.  In choosing to focus on program management practices rather than high- risk areas, the PMPC has not satisfied all PMPC requirements as  delineated in PMIAA.

Having measures to assess outcomes of PMIAA, such as establishing a  baseline of information on programs or collecting trend data, can help  OMB ensure that it has established a framework to effectively guide and  assess PMIAA’s implementation. Assessment measures would also allow  OMB to better target efforts to improve project management and the  capabilities of managers.

We are making a total of eight recommendations to OMB. Specifically:  The Deputy Director for Management of OMB, in conjunction with the  PMPC, should develop program and project management standards to  include (1) a minimum threshold for determining the extent to which  agencies have met the standards, (2) how standards apply differently at  the program and project levels, (3) how standards are interrelated to work  in a synchronized way, and (4) how standards should be applied across  the life cycle of a program or project. (Recommendation 1)

The Deputy Director for Management of OMB, in conjunction with the  PMPC, should create a governance structure to further develop and  maintain program and project management standards that fully aligns  with key practices for governance structures. (Recommendation 2)

The Deputy Director for Management of OMB should, when expanding  PMIAA to additional program types, design pilot efforts to follow leading  practices so that OMB can optimize its efforts to improve and broaden  portfolio reviews across a full range of program types. (Recommendation  3)

The Deputy Director for Management of OMB should communicate  program areas and timeframes, and expectations pertinent to annual  program portfolio reviews, to be reviewed in future program portfolio  reviews. (Recommendation 4)

The Deputy Director for Management of OMB should clarify for agencies  how the different definitions of a “program” relate to each other in OMB  guidance. (Recommendation 5)

The Deputy Director for Management of OMB should convene trilateral  meetings between OMB, relevant agencies, and us for addressing all  high-risk areas during each two-year high-risk cycle (Recommendation 6).

The Deputy Director for Management of OMB, in conjunction with PMPC,  should ensure PMPC meeting agendas include time for discussing high- risk areas during meetings and provide time for the PMPC to make  recommendations to OMB about addressing high-risk areas.  (Recommendation 7)

The Deputy Director for Management of OMB, in conjunction with PMPC,  should establish measures to assess outcomes of PMIAA, such as  establishing a baseline of information on programs or collecting trend  data. (Recommendation 8)

We provided a draft of this product for comment to OMB, OPM, and the  five selected agencies. OMB neither agreed nor disagreed with the  recommendations and stated that it would take them into consideration  when making future updates to its policies and guidance for agencies for  improving program and service delivery. In addition, OMB, OPM,  Commerce, NASA, Treasury, and Veterans Affairs provided technical  comments which we incorporated as appropriate. Energy responded that  it had no comments.

We are sending copies of this report to congressional committees, the  Acting Director of OMB and Director of OPM, The Secretaries of the  Departments of Commerce, Energy, Treasury, and Veterans Affairs, the  Administrator of NASA, and other interested parties. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-6806 or Jonesy@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

Appendix I: Objective, Scope, and Methodology

This report examines: (1) the steps taken by the Office of Management  and Budget (OMB), the Office of Personnel Management (OPM), and the  Chief Financial Officer Act of 1990 (CFO Act) agencies to implement the  Program Management Improvement Accountability Act (PMIAA); (2) the  extent to which OMB is using or planning to use portfolio reviews required  in PMIAA to address issues on our High-Risk List; and (3) the extent to  which OMB provided methods for agencies to assess the results of  PMIAA.

To examine the steps taken by OMB, OPM, and CFO Act agencies to  implement PMIAA, we reviewed agency documents, designed and  disseminated a questionnaire to the 24 CFO Act agencies, and analyzed  their responses. We also selected five PMIAA CFO Act agencies as case  studies. We reviewed documentation from OMB, including the OMB  PMIAA strategic plan and actions taken, as well as Cross Agency Priority  goal 11 quarterly reports, and screen shots of PMIAA documents on OMB  Max portal. We interviewed OMB staff to gain insight into their approach  to implementing PMIAA. To examine the OMB standards for program and  project management, we used criteria from the Project Management  Institute (PMI) for Standard for Program Management and the Project  Management Body of Knowledge. In addition, we reviewed  documentation from OPM regarding their PMIAA plans and documents  for the update of the 0340 job series. We further analyzed Enterprise  Human Resources Integration (EHRI) data from fiscal year 2018 from  OPM to identify employees in current program management 0340  occupational series. We also interviewed OPM officials regarding their  role in implementing PMIAA.

We interviewed outside subject matter specialists to provide their views  on federal program and project management. Specifically, we met with  staff from PMI and Professor Janet Weiss from the University of  Michigan—who had conducted a study on how to improve federal  program management—as she had been recommended by the  Congressional Research Service, OMB, and the IBM Center for the  Business of Government.

To examine the steps agencies had taken, we requested PMIAA  implementation plans from all 24 CFO Act agencies. CFO Act agencies  were to submit PMIAA implementation plans to OMB by November 30,  2018. We collected implementation plans between November 29, 2018,  and April 16, 2019. We received 22 out of 24 implementation plans. We  did not review plans from the Department of Health and Human Services  or the Environmental Protection Agency because they had not completed  their plans at the time of our review. Two analysts independently  reviewed separate implementation plans. These reviews were then  verified by another analyst. Implementation plans were evaluated on  whether they fully met, partially met, or did not meet the 10 requirements  provided in the OMB implementation guidance, such as how the major  acquisition portfolios aligned to relevant strategic objectives, or whether  the agency had existing training for program and project managers.

We also disseminated a questionnaire to all CFO Act agencies to collect  information on PMIAA implementation. This questionnaire was pre-tested  by two CFO Act agencies and two members of the Federal Program and  Project Management Community of Practice and revised for clarity. The  questionnaire was sent to all 24 CFO Act agencies on February 4, 2019,  and responses collected between February 11 and April 22, 2019. All 24  agencies responded to the questionnaire. Agency officials were asked  questions on:  1.  the steps their agency has taken to implement PMIAA,  2.  the challenges their agency faces in implementing PMIAA,  3.  efforts to address high-risk issues, and  4.  plans to measure PMIAA outcomes, if any.

We selected five agencies for case studies and analyzed further  documentation and interviewed agency officials to provide illustrative  examples of PMIAA implementation at the agency level. We assessed  whether: agencies had responsibility for a program, function, or operation on  our 2019 High-Risk List;

OMB considered them further along in PMIAA implementation  compared to other agencies;  the agency reported it was selected for the OMB pilot of  noninformation technology acquisition program portfolio reviews; agency officials reported actions taken to direct internal program  management training or workforce development in their questionnaire  responses or OMB required implementation plans; and agency officials reported any actions to implement PMIAA beyond the  requirements listed in the OMB PMIAA strategic plan.

To achieve of a range of PMIAA experiences, we selected five agencies  that met varying numbers of the criteria. The Department of Commerce  was chosen because all four selection criteria were met, the Department  of Energy met three, the Department of Veterans Affairs met two, and the  Department of the Treasury and the National Aeronautics and Space  Administration each met one. We interviewed and reviewed documents  from each of the agencies. We asked questions about steps agencies  were taking and their interactions with OMB and OPM to help them  implement PMIAA. We also asked these agencies to suggest any ways in  which OMB and OPM could improve implementation.

To assess the OMB PMIAA strategic plan, we reviewed leading practices  on strategic planning from our body of work. We also considered  testimonial evidence from OMB staff. Specifically, we reviewed prior  reports on leading strategic planning practices and requirements for  agencies to use in strategic planning. We selected relevant criteria from  the Government Performance and Results Act of 1993 (GPRA) and the  GPRA Modernization Act, that not only pertained to agency strategic  plans, but also were relevant as for strategic planning principles.  Specifically, we selected criteria from the following categories: (1) mission  statement; (2) general goals and objectives; (3) strategies for  accomplishing goals and objectives; (4) input from stakeholders; (5)  interagency collaboration; 6) milestones and metrics to gauge progress.

To determine the extent to which the leading practice was included in the  strategic plan, we assessed documentary evidence from the PMIAA  strategic plan and testimonial evidence from OMB staff as defined below:

A practice was categorized as fully met if the evidence fulfilled all  aspects of the definition.

A practice was categorized as partially met if the evidence fulfilled  some, but not all, aspects of the definition, or if the evidence was  judged to fulfill the general meaning of the definition, while not  technically meeting it fully.

A practice was categorized as not met if no evidence was found  relevant to the criterion, or if evidence did not fulfill any aspects of the  definition.

In addition, we reviewed documents from and interviewed selected  agencies on what measures OMB was developing for evaluating PMIAA  implementation. We also asked these agency officials what kinds of  evaluative measures would be useful to monitor the successful  implementation of PMIAA from their perspective.

In addition, we assessed the pilot of the required PMIAA program portfolio  reviews against the five leading practices we identified from our work on  designing pilots. We determined that the design fully met the criteria  when we saw evidence that all aspects of a leading practice were met.  When we were unable to assess whether all aspects of a leading practice  were met without additional information, we determined that the design  partially met the criteria. Finally, when we saw no evidence of a leading  practice, we determined that the criteria were not met.

To examine OMB’s standards for program and project management, we  selected two sets of criteria for program and project management criteria  from PMI. PMI standards are generally recognized as leading practices  for program and project management. To select program management  standards, we identified 10 PMI program management activities. To  select project management standards, we identified 10 project  management knowledge areas. Further, PMI’s leading practices were  selected to explain how program and project management standards  apply differently, and how both set of standards relate to the lifecycle of a  program or project. We then compared the definition of these 10 PMI  program and 10 PMI project management standards to the definition of  OMB’s initial 15 program and project standards released for PMIAA  implementation. In addition, OMB’s initial standards were compared to  PMI leading practices that distinguish the relationship between programs and projects and leading practices on applying standards across the life  cycle of a program or project.

We also applied leading practices we identified from our previous work on  data governance standards to assess the governance process OMB used  to develop, maintain, and monitor program management standards. Our  past work identified common key practices for establishing effective data  governance structures. This work selected a range of organizations,  including domestic and international standards-setting organizations,  industry groups or associations, and federal agencies, to ensure we had  comprehensive perspectives of data governance key practices across  several domains. Two analysts compared the five key practices on the  data governance structures to OMB plans and documented practices.

We assessed the reliability of OPM’s EHRI data through electronic testing  to identify missing data, out of range values, and logical inconsistencies  for employees classified as 0340s. We believe the EHRI data we used  are sufficiently reliable for the purpose of this report.

To examine the extent to which OMB is using or planning to use portfolio  reviews to address our High-Risk-List, we reviewed documentation from  OMB and 24 CFO Act Agencies. As part of our questionnaire, we asked  24 CFO Act agencies to provide any of our High-Risk List summary and  detailed analyses that the agencies were required to submit to OMB as  part of the 2018 strategic review process. We analyzed this information to  determine the extent to which agencies provided information to OMB  during their 2018 strategic review process.

We also selected criteria from the Standards for Internal Control in the  Federal Government on maintaining documentation of the internal control  system to assess steps that OMB had taken related to its responsibilities  for conducting high-risk portfolio reviews and the management of the  Program Management Policy Council. Specifically, we selected  information and communication which states that management should  externally communicate the necessary quality information that an entity  needs to achieve its objectives.

We conducted this performance audit from June 2018 to December 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: GAO Contact and Staff Acknowledgments

Yvonne D. Jones, (202) 512-6806, or jonesy@gao.gov.

In addition to the contact named above, William Reinsberg (Assistant  Director), Carole J. Cimitile (Analyst in Charge), Jacqueline Chapin,  Martin J. De Alteriis, Emily Gamelin, Jaeyung Kim, Matthew L. McKnight,  Robert Robinson, Dylan Stagner, Andrew J. Stephens, and John Villecco  made key contributions to this report.

The Program Management Improvement Accountability Act of 2016 requires the Office of Management and Budget to develop government-wide standards for program management.

We found that OMB has begun to do so, but further efforts are needed to fully implement this act. For example, OMB does not have an adequate governance structure for this effort—i.e., a system to develop, oversee, and maintain these standards over time.

We made 8 recommendations to OMB, including that the agency develop a governance structure for program management standards.