Human Capital: Improving Federal Recruiting and Hiring Efforts

Strategic human capital management plays a critical role in maximizing the government's performance and assuring its accountability to Congress and to the nation as a whole.

GAO designated strategic human capital management as a government-wide, high-risk area in 2001. Since then, important progress has been made. However, retirements and the potential loss of leadership and institutional knowledge, coupled with fiscal pressures, underscore the importance of a strategic and efficient approach to acquiring and retaining individuals with critical skills. As a result, strategic human capital management remains on GAO's High-Risk List.

This testimony is based on a large body of GAO work issued from May 2008 through May 2019. This testimony, among other things, focuses on key human capital areas where some actions have been taken but attention is still needed by OPM and federal agencies on issues including: (1) addressing critical skills gaps and (2) recruiting and hiring talented employees.

GAO, along with the Office of Personnel Management (OPM) and individual agencies, has identified skills gaps in numerous government-wide occupations. According to GAO's 2019 analysis of federal high-risk areas, skills gaps played a role in 17 of the 35 high-risk areas. Causes vary but these skills gaps often occur due to shortfalls in one or more talent management activities such as robust workforce planning. Staffing shortages and the lack of skills among current staff not only affect individual agencies but also cut across the entire federal workforce in areas such as cybersecurity and acquisition management. Additionally, the changing nature of federal work and the high percentage of employees eligible for retirement could produce gaps in leadership and institutional knowledge, and threatens to aggravate the problems created from existing skills gaps. For example, 31.6 percent of permanent federal employees who were on board as of September 30, 2017, will be eligible to retire in the next 5 years with some agencies having particularly high levels of employees eligible to retire.

GAO's work has identified a range of problems and challenges with federal recruitment and hiring efforts. Some of these problems and challenges include unclear job announcements and a lengthy hiring process. Further, the federal workforce has changed since the government's system of current employment policies and practices were designed. Strategies that can help agencies better manage the current and future workforces include:

Manage the timing of recruitment . To address issues of funding uncertainty at the beginning of the fiscal year, agencies should recruit continuously, starting the hiring process early in the school year.

Write user-friendly vacancy announcements . GAO has reported that some federal job announcements were unclear. This can confuse applicants and delay hiring. OPM stated that when hiring managers partner with human resources staff, agencies can develop more effective vacancy announcements.

Leverage available hiring and pay flexibilities . To help ensure agencies have the talent they need, they should explore and use all existing hiring authorities. A variety of special pay authorities can help agencies compete in the labor market for top talent, but GAO has found that agencies only use them for a small number of employees.

Increase support for an inclusive work environment . An increasingly diverse workforce can help provide agencies with the requisite talent and multidisciplinary knowledge to accomplish their missions.

Encourage rotations and other mobility opportunities. Upward and lateral mobility opportunities are important for retaining employees, but few employees move horizontally because managers are sometimes reluctant to lose employees.

Without these measures, the federal government's ability to address the complex social, economic, and security challenges facing the country may be compromised.

Over the years, GAO has made numerous recommendations to agencies and OPM to improve their strategic human capital management efforts. Agencies have taken actions to implement some of these recommendations, but many remain open. GAO encourages OPM and the agencies to fully implement the recommendations.

Thank you for the opportunity to be here today to discuss the state of the  21st century federal civil service and what can be done to improve federal  hiring and close mission-critical skills gaps. The government’s system of  current employment policies was designed generations ago for a  workforce and type of work that largely no longer exist. Much has  changed since the Civil Service Reform Act of 1978 and the Classification  Act of 1949 laid the foundation of much of today’s federal personnel  system. Agencies’ missions have evolved and employees’ expectations  of work and the workplace are changing. Further, an increasingly volatile  world makes understanding and responding to evolving trends and issues  essential. As we have long reported, strategic human capital  management plays a critical role in maximizing the government’s  performance and assuring its accountability to Congress and to the nation  as a whole.

Strategic human capital management has been a GAO high-risk area  since 2001. Since then, Congress, the Office of Personnel Management  (OPM), and some individual agencies have made progress towards  addressing the government’s human capital challenges. As we reported  in March 2019, although progress has been made toward improving the  government’s human capital efforts, the area remains high-risk because  more work is needed to address government-wide mission critical skills  gaps. As our 2019 analysis of all federal high-risk areas shows, skills  gaps played a role in 17 of the 35 governmentwide high-risk areas we  have identified.

In December 2016, OPM finalized revisions to its strategic human capital  management regulation that include the new Human Capital Framework.

This framework is to be used by agencies to plan, implement, evaluate,  and improve human capital policies and programs. Additionally, the  revised regulation provides that agency human capital policies and  programs must monitor and address skills gaps within government-wide  and agency-specific mission-critical occupations by using comprehensive  data analytic methods and gap closure strategies. The revised regulation,  which became effective on April 11, 2017, also requires that agencies  conduct a quarterly, data-driven review process known as HRStat,  which, as we reported in 2015, could be an important tool in reviewing  key performance metrics related to closing skills gaps.

This statement among other things focuses on key human capital areas  where some actions have been taken but attention is still needed by OPM  and federal agencies on issues including (1) addressing critical skills  gaps, and (2) recruiting and hiring talented employees. The statement is  based on our large body of work on federal human capital management  issued primarily between May 2008 and May 2019. More detailed  information on our objectives, scope, and methodology for that work can  be found in the issued reports. We conducted the work on which this  statement is based in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.

OPM and Agencies Need to Strengthen Efforts to Identify and Close Mission-Critical Skills Gaps

The federal government faces long-standing challenges in strategically  managing its workforce. We first added federal strategic human capital  management to our list of high-risk government programs and operations  in 2001. Because skills gaps within individual federal agencies—as well  as across the federal workforce—can lead to costly, less-efficient  government, the issue has been identified as the focus of the Strategic  Human Capital Management GAO high-risk area since February 2011.  Our experience has shown that the key elements needed to make  progress in high-risk areas are top-level attention by the administration  and agency leaders grounded in the five criteria for removal from the  High-Risk List, as well as any needed congressional action. The five  criteria for removal are: (1) leadership commitment, (2) agency capacity,  (3) existence of a corrective action plan, (4) program monitoring, and (5)  demonstrated progress.

Although Congress, OPM, and individual agencies have made  improvements since 2001, federal human capital management remains a  high-risk area because mission-critical skills gaps within the federal  workforce pose a high risk to the nation. GAO, along with OPM and  individual agencies, has identified mission critical skills gaps in numerous  government-wide occupations. These skills gaps both within federal  agencies and across the federal workforce impede the government from  cost-effectively serving the public and achieving results. For example, the  difficulties in recruiting and retaining skilled health care providers and  human resource staff at Veterans Health Administration’s (VHA) medical  centers make it difficult to meet the health care needs of more than 9  million veterans. As a result, VHA’s 168 medical centers have large  staffing shortages, including physicians, registered nurses, physician  assistants, psychologists, physical therapists, as well as human resource  specialists and assistants.

In October 2017, we reported that the VHA, within the Department of  Veterans Affairs (VA), has opportunities to improve staffing, recruitment,  and retention strategies for physicians that it identified as a priority for  staffing, or mission-critical. For 2016, the top five physician mission- critical occupations were primary care, mental health, gastroenterology,  orthopedic surgery, and emergency medicine. However, VHA was unable  to accurately count the total number of physicians who provide care in its  VA medical centers (VAMC). Additionally, VHA lacked data on the  number of contract physicians and physician trainees. Five of the six  VAMCs in our review used contract physicians or physician trainees to  meet their staffing needs, but VHA had no information on the extent to  which VAMCs nationwide use these arrangements. We also reported that  VHA had not evaluated the effectiveness of its physician recruitment and  retention strategies. One such strategy—hiring physician trainees—was  weakened by ineffectual hiring practices, such as delaying employment  offers until graduation.

In February 2018, we reported that the Department of Homeland Security  (DHS) had taken actions to identify, categorize, and assign employment  codes to its cybersecurity positions, as required by the Homeland  Security Cybersecurity Workforce Assessment Act of 2014; however, its  actions were not timely and complete. While DHS has implemented four  of our six recommendations from this report, two recommendations  remain open. For example, DHS has not yet completed its efforts to  identify all of the department’s cybersecurity positions and accurately  assign codes to all filled and vacant cybersecurity positions. Further, it  has not yet fully developed guidance to assist DHS components in  identifying their cybersecurity work categories and specialty areas of  critical need that align to the National Initiative for Cybersecurity  Education framework. Without ensuring that its progress in identifying and  assigning codes to its positions is accurately reported and it has guidance  to fully assist components, DHS will not be positioned to effectively  examine its cybersecurity workforce, identify its critical skill gaps, or  improve its workforce planning.

In March 2019, we reported that 24 federal agencies generally assigned  work roles to filled and vacant positions that performed information  technology, cybersecurity, or cyber-related functions as required by the  Federal Cybersecurity Workforce Assessment Act of 2015. However,  most agencies had likely miscategorized the work roles of many IT  positions. Until agencies accurately categorize their positions, the  agencies may not have reliable information to form a basis for effectively  examining their cybersecurity workforce, improving workforce planning,  and identifying their workforce roles of critical need.

Skills gaps caused by insufficient number of staff, inadequate workforce  planning, and a lack of training in critical skills are contributing to our designating strategic human capital management and other areas as high  risk. (See table 1.) Skills gaps affect individual agencies but also cut  across the entire federal workforce in areas such as cybersecurity and  acquisition management. As our 2019 analysis of federal high-risk areas  shows, in addition to Strategic Human Capital Management, skills gaps  played a role in 16 of the other 34 high-risk areas we have identified.

Insufficient numbers of staff with critical skills can be related to staff  retirements as well as to inadequate recruitment and hiring. Moreover, if  not carefully managed, anticipated retirements could widen skills gaps or  open new ones, adversely affecting agencies’ capabilities. As shown in  figure 1, more than 31 percent of federal employees on board by the end  of fiscal year 2017 will be eligible to retire in the next 5 years.

Key Strategies and Practices for Recruiting, Incentivizing and Engaging the Current and Future Federal Workforce

In March 2019, we reported on key talent management strategies that  can help agencies better manage the current and future workforce.  Below we focus on nine selected practices we identified related to  recruiting, incentivizing, and engaging the federal workforce:  Cultivate a diverse talent pipeline. In our prior work, we have noted the  importance of active campus recruiting that goes beyond infrequent  outreach to college campuses. Active campus recruiting includes  developing long-term institutional relationships with faculty,  administrators, and students. In addition, OPM guidance emphasizes that  agencies should develop an inclusive approach to their talent acquisition  strategies. This includes developing strategic partnerships with a diverse  range of colleges and universities, trade schools, apprentice programs,  and affinity organizations from across the country.

Recruit continuously and start the hiring process early in the school  year. The ability to hire students is critical to ensuring that agencies have  a range of experience levels for succession planning and a talent pipeline  to meet mission requirements. One of the key challenges agencies face  in recruiting students is managing the timing of recruitment. The federal  fiscal year begins on October 1—about when private sector firms we  interviewed start recruiting on campus. Frequently, however, federal  agencies have been unable to hire at this time of year because of the  limitations of budget uncertainty. Yet if agencies wait to start the recruiting  and hiring process until they receive funding, many graduates will have  taken other job opportunities.

Agencies can overcome these timing challenges by recruiting  continuously and starting the hiring process early in the school year. To  recruit continuously, Chief Human Capital Officers (CHCOs) from the U.S.  Departments of Agriculture and Homeland Security said that they  advertise funding-conditional positions throughout the year.

Write user-friendly vacancy announcements. We previously reported  that some federal job announcements were unclear. This can confuse  applicants and delay hiring. In July 2018, OPM officials stated that  agencies can develop more effective vacancy announcements when  hiring managers partner with human resource (HR) staff. According to  OPM, hiring managers can work with HR staff to identify the critical  competencies needed in the job, develop a recruiting strategy, and  ensure the job announcement accurately and clearly describes the  required competencies and experience. To promote collaboration  between hiring managers and HR staff, OPM is training agencies on the  role of hiring managers in writing vacancy announcements, according to  OPM officials.

Strategically leverage available hiring and pay flexibilities. To help  ensure agencies have the talent they need to meet their missions, we  have found that federal agencies should have a hiring process that is  simultaneously applicant friendly, sufficiently flexible to enable agencies  to meet their needs, and consistent with statutory requirements, such as  hiring on the basis of merit. Key to achieving this is the hiring authority  used to bring applicants onboard.

In March 2019, we reported that CHCOs cited the complex competitive  examining process as a cause of the lengthy hiring time. This has been  a long-standing concern. In our 2002 report on human capital flexibilities,  we noted that for many years prior, federal managers had complained  that competitive examining procedures were rigid and complex.  Agencies can use a number of additional hiring authorities beyond  competitive examining. These authorities can add flexibility to the process  and CHCOs have expressed a desire for more. However, we previously  found that agencies relied on only a small number of available  authorities. In fiscal year 2014, of the 105 hiring authority codes used in  total, agencies relied on 20 hiring authority codes to make around 90  percent of the new appointments.

We recommended in 2016 that OPM use information from its reviews of  agencies’ use of certain hiring authorities to determine whether  opportunities exist to refine, consolidate, or expand agency-specific  authorities, and implement changes where OPM is authorized, including  seeking presidential authorization or developing legislative proposals if  necessary. OPM agreed with our recommendation and has made  progress in these areas, although more work is needed to follow through  on planned actions to streamline authorities.

For example, in December 2018, OPM said that it continues to research  and examine streamlining opportunities, such as those identified in its  July 2018 study on excepted service hiring authorities. However, OPM did  not provide a time frame for implementation. In addition, in its March 2019  Congressional Justification for the Fiscal Year 2020 Budget Request,  OPM included legislative proposals for new hiring authorities, such as  authority for short-term appointments to allow agencies to appoint and  compensate highly qualified experts to help agencies meet critical needs  as well as a change to the criteria for granting direct hire authority.

A variety of special pay authorities can help agencies compete in the  labor market for top talent, but agencies only use them for a small number  of employees. In fiscal year 2016, these incentives were used for less  than 6 percent of employees. In December 2017, we reported that  agencies can tap an array of special payments when they need to recruit  or retain experts in engineering, cybersecurity, or other in-demand  fields. These payments include, for example, incentives for recruitment  or retention, or higher rates of pay for critical positions. We found that  agencies reported that these payments were helpful, but few documented  their effects, and OPM had not assessed their effectiveness. Further, in  our March 2019 report, we found that less than 5 percent of employees  received payments for recruitment or retention annually in the past 10  years.

In December 2017, we made three recommendations to OPM, including  for it to track the effectiveness of special payment authorities. OPM  partially concurred with this recommendation, saying that agencies are in  the best position to take this action. In December 2018, OPM stated that it  established a baseline to measure changes in the use of special payment  authorities over time, and that it is focused on government-wide, mission- critical occupations to help identify trends where there may be recruitment  and retention difficulties. However, documents OPM provided gave no  information on actions taken on this recommendation. We will continue to  monitor OPM’s actions to implement this recommendation. This is one of  18 priority recommendations in GAO’s Priority Recommendations letter to  OPM.

Use relevant assessment methods and share hiring lists. In March  2019, we reported that CHCOs and OPM officials we interviewed stated  that roadblocks to hiring the right skills include issues with assessment  methods. Specifically, agencies may use methods that are less relevant  for assessing the desired skills or agencies may experience issues  incorporating multiple assessments in the hiring process. For example,  one CHCO we interviewed said that her agency uses multiple-choice  questions to assess candidates, but essay questions more effectively  assess the skills she seeks. OPM issued guidance to agencies on how to  use additional assessment methods, including how to rank applicants.

Additionally, federal employee and management group representatives  we spoke with said agencies could reduce the time of the assessment  process by sharing hiring lists. The Competitive Service Act of 2015  allows agencies to share hiring lists, but agencies have only started to  pilot the practice within departments, according to OPM officials we spoke  with for our March 2019 report. OPM and agencies discussed sharing  hiring certificates with the CHCO Council, and OPM is planning virtual  training sessions on this topic. However, one federal employee group  representative noted that to be consistent with merit principles, agencies  may need to refresh the list every 2 to 3 months to give new candidates  the opportunity to enter the application pool.

Highlight agency mission and link to employees’ work. Agencies can  help counter negative perceptions of federal work by promoting their  missions and innovative work, according to experts and CHCOs we  interviewed for our March 2019 report. For example, DHS’s CHCO told  us that DHS provides “Day in the Life” information on its work to promote  public awareness of how its everyday tasks tie in with its mission of  protecting the United States.

In addition, we have previously reported that high-performing  organizations create a “line of sight” between individual performance and  organizational results by aligning employees’ daily activities with broader  results. Agencies can motivate and retain employees by connecting  them to their agency’s mission, according to human capital experts and  federal employee and management group representatives we  interviewed. Employee responses to Federal Employee Viewpoint Survey  (FEVS) indicate the federal government appears to be performing well in  this area. In 2017, 84 percent of employees knew how their work related  to the agency goals and priorities.

Increase awareness of benefits and incentives, such as work-life  programs. As shown in figure 2, the majority of federal employees were  satisfied with compensation, and employees who participated in work-life  programs were satisfied with those incentives. However, OPM’s 2018 Federal Work-Life Survey Governmentwide Report found that one of the  most commonly reported reasons employees do not participate in work- life programs is lack of program awareness among employees and  supervisors.

Increase support for an inclusive work environment. An increasingly  diverse workforce can help provide agencies with the requisite talent and  multidisciplinary knowledge to accomplish their missions. In January  2005, we reported fostering a diverse and inclusive workplace could help  organizations reduce costs by reducing turnover, increasing employee  retention across demographic groups, and improving morale. We also  reported that top management commitment is a fundamental element in  the implementation of diversity management initiatives.

Encourage details, rotations, and other mobility opportunities. In  March 2019, we stated that CHCOs, human capital experts, and federal  management groups said upward and lateral mobility opportunities are  important for retaining employees. CHCOs also said that in some cases,  lateral mobility opportunities such as rotations, details, and opportunities  to gain experience in other sectors can help employees gain new skills  more cost-effectively than training, particularly for rapidly changing skill  sets such as those related to the sciences. Further, we previously  reported that effective interagency rotational assignments can develop  participants’ collaboration skills and build interagency networks.

However, according to OPM data, few employees in 2017 moved  horizontally because, according to federal manager group representatives  and our previous work, managers are sometimes reluctant to lose  employees. (See table 2.)

We have previously made recommendations that could help address  these challenges. For example in 2015, we recommended that OPM  determine if promising practices, such as providing detail opportunities or  rotational assignments to managerial candidates prior to promotion,  should be more widely used across government. OPM partially  concurred with this recommendation and agreed to work with the CHCO  Council to explore more government-wide use of rotational assignments.  However, OPM noted that agencies already have authority to take these  actions. In June 2019, OPM officials told us they had discussed the  scalability of promising practices for supervisors—specifically, details and  rotational assignments and a dual career ladder—with members of the  CHCO Council. OPM found these practices were being used at some  agencies, but has not determined if these practices may be beneficial to  other agencies.

In conclusion, OPM has instituted numerous efforts to assist agencies’ in  addressing mission-critical skills gaps within their workforces. This  includes providing guidance, training and on-going support for agencies  on the use of comprehensive data analytic methods for identifying skills  gaps and the development of strategies to address these gaps. However,  as of December 2018, OPM had not fully implemented 29 of our  recommendations made since 2012 relating to this high-risk area. We will  continue to monitor OPM’s efforts to implement our recommendations.  Further, we have reported on numerous talent management strategies  that can help agencies better manage the current and future workforce.  Without these measures, the federal government’s ability to address the  complex social, economic, and security challenges facing the country  may be compromised.

Chairman Lankford, Ranking Member Sinema, and Members of the  Subcommittee, this completes my prepared statement. I would be  pleased to respond to any questions you may have at this time.

If you or your staff have any questions about this testimony, please  contact Yvonne D. Jones, Director, Strategic Issues, at (202) 512-6806 or  jonesy@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this  statement. GAO staff who made key contributions to this testimony are  Clifton Douglas, Jr., Assistant Director; Christopher Falcone; Karin  Fangman; Cindy Saunders, Alan Rozzi and Katherine Wulff.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

Technology, demographics, and attitudes toward work are evolving in the private and public sectors. But federal government employment policies were designed decades ago. As a result, the government may struggle to compete for talented workers—which is one reason why federal human capital management is an issue on our High Risk list.

We testified about the state of the federal civil service and recruiting and retaining workers with critical skills.

For example, agencies have long said competitive hiring procedures are rigid, complex, and lengthy. We found more flexible hiring options were available to agencies but only a small number were used.


VA Health Care: Opportunities Exist for VA to Better Identify and Address Racial and Ethnic Disparities

According to VA, in 2016, racial and ethnic minority veterans represented about 22 percent of the total veteran population of 18.6 million. VA projects racial and ethnic minority veterans will make up 36 percent of its total veteran population by 2040. VA has identified racial and ethnic disparities in its health care outcomes, mirroring trends seen across the United States.

House Report 115-188 included a provision for GAO to review whether VHA provides quality, equitable care for minority veterans. GAO's report examines, among other issues, (1) the extent to which VA has taken steps to advance health equity for racial and ethnic minority veterans, and (2) VA's efforts to use race and ethnicity data to identify and address disparities in health care outcomes involving minority veterans. GAO reviewed relevant documents, such as strategic and operational plans and peer-reviewed research studies; assessed VA's health equity action plan against criteria identified in GAO's body of work on effectively managing performance; and interviewed VA officials familiar with VA's health equity efforts, as well as race and ethnicity data.

The Department of Veterans Affairs (VA) has taken steps to reduce disparities in health care outcomes linked to race and ethnicity, but lacks mechanisms to measure progress and ensure accountability for results. In 2012, VA established the Office of Health Equity to identify and address health care outcome disparities and to develop an action plan to achieve health equity. This office issued an action plan in 2014 that identified activities to make improvements in five focus areas, such as increasing awareness of the significance of disparities and strengthening leadership for addressing them. However, GAO found that the extent of VA's progress in implementing the action plan and advancing health equity is unknown because the action plan lacked performance measures and clear lines of accountability for specific offices. For example, although VA's action plan included a list of “success criteria” for each of the five focus areas, these criteria were not measurable, and were not linked to specific activities or to offices responsible for implementation.

VA funds research efforts that have identified disparities in health care outcomes involving minority veterans, but rely on data that VA officials and researchers noted have weaknesses in completeness and accuracy. One concern is that race and ethnicity information can be labeled incorrectly in VA patients' electronic health records as ”self-reported”, a highly reliable method of collection, when data were actually collected based on the less reliable method of VA staff observation. Other reported concerns include missing values on patients' race and conflicting race and ethnicity information. VA researchers told GAO they account for some of these concerns by using data from other sources, such as Medicare, but such work-arounds are time intensive. Further, VA officials reported that data weaknesses limit their ability to identify and address disparities in health care outcomes in their medical centers. Despite recognizing weaknesses related to the quality of race and ethnicity data, VA has not implemented corrective actions to address them. Without doing so, VA medical center officials cannot readily identify and address disparities in health care outcomes by race and ethnicity.

Note: Concerns about the completeness and accuracy of race and ethnicity information were raised by officials from VA's Office of Health Equity, Veterans Experience Office, and Health Services Research & Development.

GAO is making two recommendations to VA to (1) ensure that any health equity action plan includes performance measures to assess progress, and clear lines of accountability designating responsibility to specific offices, and (2) conduct an assessment to determine how to address weaknesses identified with the completeness and accuracy of race and ethnicity data in the electronic health record, and implement corrective actions as necessary. VA agreed with GAO's recommendations.

Within the Department of Veterans Affairs (VA), the Veterans Health  Administration (VHA) operates one of the nation’s largest health care  systems, providing care to veterans at its 172 VA medical centers. In  2016, racial and ethnic minority veterans—hereafter referred to as  minority veterans—comprised about 22 percent of the total veteran  population of 18.6 million; VA projects minority veterans will make up 36  percent of its total veteran population by 2040. Additionally, minority  veterans’ use of VA’s health care services increased about 9 percent from  2005 to 2014, according to the most recent VA data available.

As its veterans’ population is becoming increasingly more diverse, VA has  acknowledged the importance of ensuring health equity—the ability for all  veterans to attain their highest possible levels of health. However, VA has  identified racial and ethnic disparities in its health care outcomes,  mirroring trends seen across the United States. For example, in 2011,  VA found outcomes for controlling blood pressure, blood glucose, and  cholesterol levels were significantly worse for Black or African American  veterans than they were for White veterans. Additionally, VA has noted  racial and ethnic disparities in reported patient experiences. For example,  in 2013, VA researchers found higher rates of negative experiences at VA  medical centers reported by Asian, Native Hawaiian/Pacific Islander,  American Indian/Alaska Native and multiracial veterans compared with  other minority veterans and White veterans, particularly in areas of doctor  and nurse communication, speed of receipt of care, and overall rating of  health care. More recently, in 2019, VA reported evidence of disparities  in health care outcomes within VA medical centers in the form of lower  survival rates for African American veterans with cancer and  cardiovascular-related illnesses compared with other minority veterans  and White veterans.

A report accompanying the Military Construction, Veterans Affairs, and  Related Agencies Appropriations Bill, 2018, included a provision for us to  review whether VA provides quality, equitable health care for minority  veterans, among other things. In our report, we  1.  examine the extent to which VA has taken steps to advance health  equity for minority veterans;  2.  examine VA’s efforts to use race and ethnicity data to research and  address disparities in health care outcomes involving minority  veterans; and  3.  describe the feedback VA collects from minority veterans on their  patient experiences.

For the first two objectives, we reviewed agency documents such as the  VHA Strategic Plan, VHA operational plan, and reports by the Advisory  Committee on Minority Veterans. We also reviewed the Commission on  Care report, a 2016 report that included recommendations for improving  VA’s health care system, including its efforts to address health equity and  disparities. We reviewed VA-funded research related to identifying and  reducing disparities in health care outcomes. Based on this review, we  interviewed researchers about their efforts to identify disparities in health  care outcomes involving minority veterans and achieving health equity.  We also spoke with VA Central Office officials, including officials from the  Quality Enhancement Research Initiative, the Center for Health Equity,  Research, and Promotion, and the Office of Patient Centered Care and  Cultural Transformation. In addition, we reviewed written responses to  questions we provided to the Advisory Committee on Minority Veterans,  in lieu of an interview, since the Committee meets only twice per year.

To examine the steps taken to reduce disparities in health care outcomes  involving minority veterans to advance equity, we reviewed relevant  documents from VA that describe the efforts of the Office of Health Equity  (OHE), which is responsible for advancing VA health equity goals across  the agency. These documents included information on OHE’s budget  from fiscal years 2013 through 2019, and OHE staffing levels during this  time. We also assessed the efforts of OHE to implement the health equity  action plan, VA’s action plan to address health equity across the agency.  We assessed the health equity action plan against relevant criteria from  GAO’s body of work on effectively managing performance under the  Government Performance and Results Act of 1993 (GPRA), as enhanced  by the GPRA Modernization Act of 2010. We interviewed OHE officials  about these topics, including the extent to which they had assessed  implementation of the action plan, as well as future plans for the office.

In addition, to examine VA’s efforts to use race and ethnicity data to  research and address disparities in health care outcomes involving  minority veterans, we compared VA’s efforts to the recommendations  made by the 2016 Commission on Care report, which advised VA to  increase the availability, quality, and use of race, ethnicity, and language  data. We reviewed VA webinar presentations on race and ethnicity data.  To determine how race and ethnicity data are collected and to obtain  perspectives on the completeness and accuracy of such data, we  conducted interviews with five VA researchers, who are also data experts,  selected based on their publications in peer-reviewed journals on  disparities in health care outcomes and knowledge of race and ethnicity  research at VA medical centers. We interviewed officials from two  Veterans Integrated Service Networks (VISN) in order to obtain their  perspectives on using race and ethnicity data to assess health care  disparities in their networks. We selected these officials because of their  prior efforts working with VA researchers to assess race and ethnicity  data among their minority veteran populations. The perspectives of these  officials are not generalizable. We also interviewed officials from two  private health care systems, selected because of their efforts to address  health care disparities. We asked them about the methods they use to  collect race and ethnicity data, and how they have used these data to  assess and address disparities in health care outcomes and patient  experience. We interviewed both VA and Department of Defense officials  regarding their collaborative efforts and respective plans to implement  new electronic health record (EHR) systems and how these systems will  capture race and ethnicity data.

To describe what feedback VA collects from minority veterans on their  patient experiences, we reviewed how patient feedback survey results are  collected and reported. We also interviewed officials from VA’s Office of  Reporting, Analytics, Performance, Improvement, and Deployment, the  Veterans Experience Office, and OHE to obtain information on how the  surveys are administered, the type of information they capture, and how  results are reported. To obtain information on the types and nature of  patient feedback received by patient advocates at VA medical centers, we  interviewed VA officials with the Office of Patient Advocacy and  conducted interviews with four patient advocate coordinators across four  VISNs, and small group interviews with 21 VA medical center patient  advocates located in 12 medical centers within the four VISNs. We  selected these medical centers to provide diversity in geographic regions,  number of enrollees’ by race per VISN, facility complexity, and racial and  ethnic demographics of the veteran population. The perspectives  obtained from staff located within the 12 VA medical centers and four  VISNs cannot be generalized. In addition, we also interviewed  representatives from five veterans service organizations, and received  written responses from another to obtain information about the  mechanisms minority veterans may use to provide feedback to VA on  patient experiences.

We conducted this performance audit from August 2018 to December  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

VA Established a Program Office and Issued an Action Plan to Advance Health Equity, but Lacks Mechanisms to Measure and Ensure Accountability for Implementation

VA has taken several steps in recent years to reduce health care  disparities for minority veterans and advance health equity, but lacks  mechanisms to ensure accountability for advancing health equity using  these or other actions. In particular, VA established a program office and  a dedicated steering committee to draft VA’s first action plan designed to  achieve health equity. However, this initial action plan lacked  performance measures and clear lines of accountability; as such, the  extent to which it has been implemented and the progress made in  achieving its goals is unknown.

VA Established a Dedicated Program Office to Lead VA’s Efforts to Achieve Health Equity

In 2012, VA established OHE to lead the department’s efforts to advance  health equity and reduce health disparities throughout VA’s health care  system. At that time, the Principal Deputy Under Secretary for Health— who reports to the Under Secretary for Health—identified health equity as  a cross-cutting issue of the highest priority, and as such, he required the  Director of OHE to report directly to him. OHE is responsible for several  efforts, including providing education, training, research, communications  and information; promoting common awareness about health care  disparities and working to improve health care outcomes; and  representing VA and serving as a liaison to other governmental and non- governmental organizations working to achieve health equity. OHE was  also tasked with developing and maintaining a comprehensive action plan  to achieve health equity in VA medical centers and improve VA’s overall  quality of care.

Also in 2012, VA’s Principal Deputy Under Secretary for Health created  the Health Equity Coalition (HEC). Chaired by the Director of OHE, the  HEC is a VA-wide steering committee that is comprised of officials from  several VA program offices dedicated to areas such as patient care  services, communications, research and development, and minority  veteran issues. The HEC advises and assists OHE in developing and  implementing plans, sets milestones to review progress to ensure timely  completion of initiatives, and ensures that program offices commit  appropriate organizational resources needed to meet these goals.

Since its establishment, OHE has experienced changes in budget and  staff levels from year to year. OHE’s core budget, which VA officials told  us is spent on salaries, travel, and supplies, fluctuated between 2013 and  2019; staffing levels began decreasing in 2014 and subsequently  increased in 2019. (See fig. 1.) VA officials told us that staff reductions  were due to transfers and reassignments of OHE staff to other offices and  positions, which coincided with a shifted focus from equity issues to other  issues such as veteran wait times and modernization efforts. In addition,  OHE was also repositioned to a lower level within the organization from  VHA’s Office of the Principal Deputy Under Secretary for Health to the  Office of the Deputy Under Secretary for Health for Organizational  Excellence.

VA’s Health Equity Action Plan Set Goals to Eliminate Health Disparities, but Extent of Progress Is Unknown Due to a Lack of Performance Measures and Clear Lines of Accountability

In February 2014, VA released its first Health Equity Action Plan (action  plan), drafted by OHE and the HEC to document VA’s approach for  eliminating health disparities and achieving health equity. The action plan  included five focus areas, or goals, in which VA intended to direct its  efforts to improve the overall quality of care for all veterans, including  minority veterans:

Awareness: increase awareness of the significance of health  disparities, their impact on the nation, and the actions necessary to  improve health care outcomes for racial, ethnic, and underserved  populations.

Leadership: strengthen and broaden leadership for addressing health  disparities at all levels.

Health system and life experience: improve health and health care  outcomes for racial, ethnic, and underserved populations.

Cultural and linguistic competency: improve cultural and linguistic  competency, and the diversity of the health-related workforce.

Data, research, and evaluation: improve data availability; and  coordination, utilization, and diffusion of research and evaluation  outcomes.

OHE and the HEC included between two to 14 “implementation activities”  in each of the five focus areas to describe specific plans and tasks VA  could undertake to advance health equity. For example, under the “data,  research, and evaluation” focus area, examples of implementation  activities included, “identify limitations of existing data, barriers to access  to data, and data collection methodologies that affect VA’s ability to  describe disparities,” and “develop a strategy for prioritizing identified  disparities.” In addition, all five focus areas also included “success  criteria” to measure success, a list of resources needed in order  implement the activities, and identified stakeholders.

However, despite documenting elements needed to make improvements  in the five focus areas, the extent of VA’s progress in implementing the  action plan and advancing health equity is unknown because the action  plan lacks performance measures and clear lines of accountability.  Performance measures and clear lines of accountability are among the  criteria identified in GAO’s body of work on effectively managing  performance under GPRA. Without such mechanisms, VA cannot be  assured that the plan has been implemented or will ultimately be effective  in addressing health equity. In particular, we found that VA’s plan did not  include the following mechanisms:

Performance measures. Our past work on effectively managing  performance shows that performance measures should demonstrate  how well the organization is meeting its goals and should be linked  directly to offices that have responsibility for the program or activity.

As previously noted, although VA’s action plan included a list of  “success criteria”, such criteria were not measurable, and were not  directly linked to the specific implementation activities or to the  responsible lead office for any of the five focus areas. For example, under the “leadership” focus area, the action plan identified  “development of process tools for monitoring in FY 2014” as one  of the success criteria, but it was not clearly linked to one of this  focus area’s specific implementation activities and did not identify  who among the list of lead offices and stakeholders was  responsible for it. under the “data, research, and evaluation” focus area, the action  plan identified “developed standards for collecting data used to  understand disparities” and “improved on-going data sharing  between programs” as two of the success criteria, but they were  not linked to one of this focus area’s implementation activities, nor  were they specifically assigned to one or more of the lead offices  and stakeholders listed as responsible for achieving them.

Clear lines of accountability: Our past work on effectively managing  performance also shows that designating a lead official or office to be  responsible for coordinating efforts to achieve results for each goal or  action creates clear lines of accountability. This is critical to  implementing change to achieve goals and marshaling resources  needed to improve management. In contrast, VA’s action plan listed  for each of the five focus areas: a broad group of lead offices and stakeholders responsible for the  entire focus area, in general, (for example, HEC members and  their respective offices, VISN officials, and VA medical center  directors) but did not designate specific offices or officials  responsible for coordinating efforts to complete specific  implementation activities.  vaguely described resources—such as leadership support, time,  money, and travel—needed to execute all of the implementation  activities under each of the five focus areas, but did not designate  specific lead offices or stakeholders responsible for committing  specific resources needed to implement each activity.

Without such performance measures or lines of accountability, VA lacked  the means to measure specific progress in implementing and achieving  the action plan’s goals. Moreover, according to VA officials, following the  release of the action plan and the reduction in number of OHE staff, the  frequency of HEC meetings decreased and the last regular meeting  before it reconvened in January 2019 occurred in early 2015. As such, VA  officials told us that there was no formal involvement or oversight  following the release of the action plan to ensure that coalition members  were meeting their responsibilities, including committing the  organizational resources needed to ensure implementation.

In recent years, there have been several recommendations from  stakeholders, urging VA to provide OHE with the resources needed to  fully implement its action plan. Specifically, the 2016 Commission on Care  report recommended that VA commit additional resources to address the  causes of the problem and ensure the action plan is fully implemented.  The Secretary of VA at the time concurred with the Commission’s  recommendation and said that VA would identify health equity leaders  and clinical champions in each VISN and VA medical center who could  catalyze and monitor actions to implement the action plan and further  advance the elimination of health disparities.

More recently, VA has signaled renewed interest in supporting the  advancement of health equity by increasing OHE’s budget and staffing  levels in fiscal year 2019, and reconvening the HEC in January 2019.  According to OHE officials, the reconvened HEC has held regular  meetings and approved an updated action plan in September 2019. In  October 2019, OHE officials told us that the action plan had been sent to  VHA leadership for review, which they anticipated would be completed  within the first fiscal quarter of 2020.

VA Funds Research to Identify Disparities in Health Care Outcomes, but Weaknesses in Race and Ethnicity Data Impede Medical Centers’ Ability to Address Them

Both OHE and other VA programs fund research conducted by VA  clinicians and staff to identify disparities in health care outcomes.  However, VA officials and researchers have noted problems with the  completeness and accuracy of the data on veterans’ race and ethnicity.  These weaknesses, in turn, limit VA’s ability to assess and address health  care disparities at the VA medical center level.

VA Funds Research Aimed at Identifying Disparities in Health Care Outcomes Involving Minority Veterans

VA funds research aimed at identifying health care outcome disparities  involving minority veterans. According to OHE officials, annually OHE  receives a research budget separate from its core budget, and can apply  monies from this separate budget to any appropriate research activities it  wishes to support. As an example, in fiscal year 2019, OHE officials told  us it provided funds to the Quality Enhancement Research Initiative, VA’s  Center for Health Equity, Research, and Promotion, and two VISNs. In  addition to OHE-funded research, VA’s Health Services Research &  Development (HSR&D) has spent about $12 million to fund research  studies related to identifying and reducing disparities in health care  outcomes between minority and other veterans since 2014. This  research has identified disparities in health care outcomes for minority  veterans. Research funded by HSR&D includes the following studies:

A 2017 report focused on whether changes in the way VA delivered  primary care were effective in addressing racial and ethnic disparities  in health care outcomes. Using VA data from 2009 and 2014, the  study found lower rates of control of hypertension and diabetes  among veterans who were African American, Hispanic, American  Indian/Alaska Native, and Native Hawaiian and other Pacific Islanders  compared with White veterans.

A 2016 report examined why minority veterans with mental health and  substance abuse disorders are less likely to use mental health and  substance abuse services, and to complete mental health and  substance abuse treatment. The study, which used 2013 data, found  health disparities between White veterans and Black, Hispanic, and  American Indian or Alaskan Native veterans with mental health and  substance abuse disorders on several quality measures, including  access to care. The study also found disparities by race and ethnicity  in patients’ experiences communicating with providers and office  staff.

A 2017 systematic review of 351 studies published between 2006 and  February 2016 examined the prevalence of disparities in health care  outcomes experienced by veterans, including health disparities based  on race and ethnicity. This systematic review concluded that a large  proportion of the research conducted has focused on differences  between Black or African American and White veterans and  suggested that future targeted research is needed to capture the  unique characteristics of American Indian or Alaska Natives and  Native Hawaiian or other Pacific Islanders.

Despite VA’s funding of numerous studies to identify health disparities  and to explore interventions to potentially reduce or eliminate them,  health disparities continue to persist among VA’s patient population.  HSR&D officials told us that VA has faced difficulties translating research  into practice in clinical settings, including their research findings about  disparities in health care outcomes. HSR&D officials told us that they  have recently undertaken new efforts aimed at implementing research  findings, including those focused on disparities in health care outcomes.  Among these efforts is the development of a new program to provide  additional funding (for up to two years) for research projects that are  completed or close to completion so that researchers can: 1) develop tool  kits that others can adopt, and 2) implement research in additional VA  medical centers in order to facilitate the sharing of information about  successes and failures to make the impact of research more effective.

VA Officials and Others Have Identified Weaknesses in Veterans’ Race and Ethnicity Data

Generally, VA collects and records race and ethnicity information in  veterans’ EHRs when they enroll in VA health care online, by mail, fax, or  telephone applications, or through self-service touch-screen kiosks at VA  medical centers. Intake clerks may also collect and record race and  ethnicity information when assisting veterans with enrollment, as well as  when checking a veteran in at a clinic for an outpatient appointment, or  registering a veteran for an inpatient hospital admission. However, VA  researchers and officials have identified weaknesses in the completeness  and accuracy of VA’s patient data on race and ethnicity, which has raised  data reliability concerns. (See fig. 2)

VA cannot ensure that race and ethnicity information labeled in  the EHR as self-reported is accurate. VA follows standards outlined  by the Office of Management Budget, which state that self-reported  information is the preferred method for obtaining an individual’s race  and ethnicity, because it is more accurate than data collected by  observation of a third party. However, a VA data expert with  HSR&D’s Center for Health Equity Research and Promotion, and  officials at one of the VISNs in our review told us that they are aware  that intake clerks sometimes enter information based on observation,  which may be inaccurate, because they feel uncomfortable asking  veterans for their race and ethnicity information in case the veterans  find it offensive. Adding further to potential inaccuracies, because  VA’s EHR default setting automatically records all race and ethnicity  information as self-reported, observational data are only accurately  labeled as such if a clerk manually changes the default setting to  ‘observational’. The VA data expert from HSR&D’s Center for Health  Equity Research and Promotion told us that, based on her research,  almost all of the information collected electronically in the EHR is  automatically assigned as self-reported, the default setting, even  when it is collected by observation of VA staff. This expert also told us  that research efforts at VA medical centers have indicated that the  default setting is rarely changed and that some clerks had never  changed the setting because they do not know how. VA research has  indicated that observational data is more accurate for Blacks or  African Americans and Whites than other racial groups, and that  studies focused on other racial groups may be especially vulnerable  to misclassification bias. As such, VA lacks reasonable assurance  that the identification of race and ethnicity as “self-reported” is  accurate.

Data on veterans’ race are often incomplete. Race and ethnicity  information is collected as two separate categories in the EHR, and as  previously stated, is generally obtained when a patient enrolls in VA  health care, or seeks care at a VA medical center or clinic. Two VA  researchers told us that ethnicity data—based on veterans’  designation of whether they are Hispanic or non-Hispanic—are often  more complete than race data. They said that one reason for this is  that the order in which the questions are asked may be problematic;  specifically, the ethnicity question is asked first, followed by a second  question to request a race designation. Veterans may self-report that  they are “Hispanic” upon enrollment or check-in for an inpatient  admission or outpatient medical visit, and then leave the race field  empty because they believe that they have already provided this  information. Missing data on race impedes VA’s ability to identify  potential disparities in health care outcomes.

Conflicting race and ethnicity information in a veteran’s medical  records makes it difficult to determine which information is  accurate. According to VA researchers we spoke with, because a  patient’s race and ethnicity information is uploaded from his or her  EHR after each inpatient admission and outpatient appointment, there  can be multiple records for each patient’s race and ethnicity data in  VA databases. These patient records often conflict with one another,  and may result from the use of both observational and self-reported  data. As such, officials stated that it can be difficult to determine which  of the multiple race or ethnicity records are accurate.

To account for the issues with completeness and accuracy, VA  researchers have used various approaches. VA researchers we spoke  with told us that while they use data entered into VA’s EHR, which are  then uploaded into various databases, they also must use a variety of  methods, often time-intensive, to enable the use of race and ethnicity  data due to concerns about its completeness and accuracy. These  methods include using veterans’ patient records that may be several  years old and from multiple VA health care settings, and looking at patient  race and ethnicity information captured across multiple years and VA  facilities. In addition, researchers also described using multiple non-VA  data sources to supplement VA’s race and ethnicity information, such as  Medicare data, and data from the Department of Defense’s roster of  veterans from recent military operations.

VA officials and other stakeholders representing veterans’ interests have  recognized the weaknesses in VA’s race and ethnicity data and the  importance of improving those data in order to address disparities and  improve health equity. For example,

VA’s first action plan included a goal to improve data availability,  supported by implementation activities to “identify limitations of  existing data, barriers to access to data, and data collection  methodologies that affect VA’s ability to describe disparities” and  “identify strategies for capturing data on race, ethnicity,  language…needed to stratify the results for all quality measures and  to address disparities.”   in 2016, the Commission on Care recommended that VA increase the  availability, quality, and use of race, ethnicity, and language data to  improve the health of minority veterans, as well as utilize systems that  monitor trends in health status, patient satisfaction, and quality  measures.  in its 2015 annual report, the Advisory Committee on Minority  Veterans recommended that VA enhance its existing data collection  processes to include the reporting of race and ethnicity data for all  benefits and utilization programs to ensure the identification of  delivery gaps and potential disparate levels of service. Furthermore, in  its 2017 annual report, the Committee again highlighted ongoing  concerns with VA’s inconsistency in collecting race and ethnicity data  and stated that it impedes VA’s ability to adequately identify health  disparities and to ensure minority veterans are receiving quality care  and services throughout VA’s system.

VA is currently collaborating with the Department of Defense to  implement a new EHR system. As yet, they have not yet addressed how  the EHR will store race and ethnicity information. The new EHR system is  to provide both departments with a common EHR platform that is  intended to support the provision of seamless care and create a single  health record for service members and veterans. VA officials from the  Office of EHR Modernization told us that this collaboration is still in the  very early stages and that while race and ethnicity information will be  included in the new EHR system, the new EHR will take 10 years to fully  implement.

VA Officials Have Reported that Data Weaknesses Limit the Ability to Identify and Address Disparities in Health Care Outcomes at VA Medical Centers

Data weaknesses, including incomplete and inaccurate data have limited  VA’s ability to advance health equity and patient care in its medical  centers, according to VA officials. Unlike VA researchers, who report  being able to account for missing and inaccurate race and ethnicity data,  most VA medical centers do not have the research staff and data  specialists needed for these efforts, according to a VA researcher from  HSR&D’s Center for Health Equity Research and Promotion and officials  from a VISN included in our review. As a result, the inaccurate and  missing data have limited the ability of VISN and VA medical center staff  to identify and address disparities in health care outcomes by race and  ethnicity at the medical center level.

VISN officials we spoke to discussed the challenges they encountered  when trying to obtain complete and accurate data on health care  outcomes by race and ethnicity to identify disparities involving their  minority veteran populations. For example, one VISN official told us that she began an effort to analyze disparities  in health care outcomes by race and ethnicity in fiscal year 2018, but  encountered challenges in obtaining complete and accurate data for  minority veterans. She said she contacted OHE officials for  assistance, who provided data for diabetes and hypertension by race  and ethnicity, but these data were 2 years old and available only at  the national level. According to the VISN official, complete and  accurate health care outcomes data by race and ethnicity were not  available for minority veterans that received care in her region. The  official told us that she subsequently contacted both VISN-level and  national data specialists, but was still unable to obtain the data to  assess health care outcomes by veterans’ race and ethnicity at the  regional or local level. other VISN officials we spoke to explained that they had a similar  experience when they explored using race and ethnicity data to  design a dashboard for a VISN-funded project to track efforts to  address disparities in ambulatory care readmissions involving minority  veterans. They also told us they too contacted regional and national  data specialists, but were told that the readmissions data were  missing and inaccurate by race and ethnicity and therefore not  useable for their efforts.

VA officials told us they are taking steps to provide VA medical centers  with data on health care outcomes at their facilities. These officials told us  that they are currently developing two health equity dashboards that will  use VA’s data on race and ethnicity to provide information on health care  outcomes, which would allow VA staff to identify any disparities in these  outcomes at the VISN and VA medical center levels. The two health  equity dashboards are in different stages of development and, as of  September 2019, VA did not have a timeline for completion and  implementation across VA medical centers for either dashboard.  According to VA officials, the development of these dashboards will not  address the accuracy and completeness of the race and ethnicity data in  the VA’s EHR. In order to maximize the effectiveness of these  dashboards, VA needs to ensure that underlying data weaknesses are  addressed by ensuring that race and ethnicity data in the EHR are  complete and accurate.

VA Collects Patient Experience Feedback from Veterans, including Minority Veterans, through Surveys and VA Medical Center Patient Advocates

VA collects patient experience feedback from veterans, including minority  veterans, through the following surveys:

The Survey of Healthcare Experiences of Patients (SHEP) is VA’s  national, standardized, and publically reported patient experience  survey that comprises up to 83 questions that are used to collect  information about patients’ experiences in various inpatient care  settings. The SHEP covers topics to assess patients’ perceptions of  their experience using the Consumer Assessment of Health Providers  and Systems Survey, which is the health care industry standard.  According to VA officials, the response rate for the SHEP is just under  40 percent, and on average, 95 percent of respondents complete  survey questions about their race and ethnicity. SHEP survey results  are reported publically at the national, VISN, and VA medical center  level.

VA does not report survey data for specific racial and ethnic groups  because, according to VA officials, the number of minority veterans  responding to the SHEP is too small to report. In 2012, a memo  establishing the OHE recommended that VA send the SHEP survey to  a higher number of veterans from racial and ethnic minority groups so  there are enough responses to report survey results by those minority  groups. However, VA officials told us that they were not aware of this  recommendation and had not addressed it. Currently, VA officials told  us that VA staff can access SHEP data by race and ethnicity in four  broad categories: Hispanic, White, African American or Black, and  other, which includes American Indian or Alaskan Native, Asian, and  Native Hawaiian or other Pacific Islander racial groups. VA officials  told us these data can be accessed on VA’s intranet and are updated  on a monthly basis.

The Survey of Veteran Enrollees’ Health and Use of Health Care  comprises questions about a range of issues, such as enrollee’s  health status, insurance, VA and community health care use, and  attitudes and perceptions of VA services. The survey is generally  conducted on an annual basis, and achieved a 32 percent response  rate in 2018. VA publically reports these survey results by race and  ethnicity at the national level, and also provides survey results by race  and ethnicity for each VISN. For example, one indicator in the 2018  survey results showed that Native Hawaiians were far less satisfied  with their ability to get referrals compared to other minority groups.  According to VA survey documentation, VA requires a minimum  number of survey responses to draw conclusions across the VA  enrollee population; the number of survey responses must be  aggregated at the VISN level to meet this minimum number.

Veterans Signals is a VA survey intended to collect immediate  targeted feedback on veterans’ experiences with outpatient services  on an ongoing basis. VA officials told us that about one to two million  survey invitations are sent out via email each week to veterans who  recently received outpatient services, and have provided their email  addresses to VA. These short surveys include eight to nine questions  and focus on a particular area related to veterans’ recent experiences  with VA health care services, such as scheduling appointments,  pharmacy wait times, and proficiency of provider communication  about veteran concerns during appointments. VA officials told us that  the surveys have a response rate of about 20 percent, and of the  responses received, 44 percent of respondents provide their race and  ethnicity information.  VA officials told us that VA medical center  staff have access to survey results in real-time and can review results  by race and ethnicity for their individual medical centers.

In addition to these surveys, VA collects patient experience feedback  from veterans, including minority veterans, through its patient advocates  located at its medical centers. Each of VA’s 172 VA medical centers is  responsible for making at least one patient advocate available to respond  to veterans’ feedback and for ensuring feedback is recorded in its Patient  Advocate Tracking System (PATS)—an electronic system used to  describe and track the resolution of veterans’ feedback across VA  medical centers. Patient advocates enter veterans’ feedback in PATS and  assign one or more issue codes that generally describe the nature of the  feedback. Of the 21 patient advocates we interviewed across 12 VA  medical centers, most said they generally do not include race and  ethnicity information in PATS when filing a veteran’s complaint, but a few  patient advocates said they will include such information if it pertains  directly to the complaint. For example, a patient advocate told us she may  include race and ethnicity information in PATS in the case of a concern  that discrimination occurred.

We found that some veteran complaints may not be consistently coded  and reported under the correct PATS issue codes in a manner similar to  inconsistencies we have identified in prior work. According to VA  officials from the Office of Patient Advocacy, two PATS codes were  created in 2017 that, in particular, may specifically apply to issues  affecting minority veterans: (1) discrimination concerns, and (2) diversity  concerns. Of the 21 patient advocates we interviewed about these two  specific issue codes, nine were not familiar with or had never used them.  VA officials told us that they expected to see patient advocates use these  codes more often in 2019 as a result of updates to their patient advocate  training curriculum, required for newly hired patient advocates and  available to all others.

Patient advocates that we interviewed often told us that they review PATS  data to report systemic issues to their VA medical center leadership, and  a few said they report on a weekly or monthly basis. Additionally, VISN  patient advocate coordinators use the PATS data to determine whether  there are any trends in PATS data across the medical centers in their  networks.

As one of the nation’s largest health care systems, VA has a unique  opportunity to gain a better understanding of the reasons that disparities  in health care outcomes occur. VA signaled its commitment to reducing  disparities for racial and ethnic minorities and achieving health equity by  establishing a responsible program office, creating an action plan, and  funding research toward this goal. As the number of minority veterans  receiving VA health care services continues to increase, it is important  that VA enhances and strengthens its efforts to identify and address  disparities in health care outcomes to ensure that all veterans receive  equitable care.

Despite these efforts, however, without including performance measures  or lines of accountability, VA lacks the means to ensure any action plan  will be fully implemented to achieve its goals. Further, weaknesses in  race and ethnicity data due to problems with the completeness and  accuracy continue to limit VA’s ability to identify and address disparities in  health care outcomes at the VA medical center level. Although VA is  developing equity dashboards to provide health care outcomes data by  race and ethnicity at the VA medical center level, these efforts will not  improve the completeness and accuracy of the race and ethnicity data in  VA’s EHR. Until VA resolves known weaknesses with the completeness  and accuracy of its race and ethnicity data, it will be limited in its ability to  assess health equity for veterans receiving care at its facilities.

We make the following two recommendations to VA:  The Under Secretary for Health should ensure that any action plan for  achieving health equity includes key elements for successful  implementation by consistently applying criteria identified in GAO’s past  work on effectively managing performance, including developing  performance measures to assess progress and creating clear lines of  accountability by designating specific offices or officials with responsibility  for coordinating efforts to implement actions and committing resources  necessary for achieving its goals and objectives. (Recommendation 1)

To ensure the availability of information on health care outcomes by race  and ethnicity throughout the VA health care system, the Secretary of  Veterans Affairs should conduct an assessment to determine the  completeness and accuracy of race and ethnicity data captured in VA’s  electronic health record, and implement corrective actions as necessary  to resolve any identified deficiencies. (Recommendation 2)

We provided a draft of this report to the Department of Veterans Affairs  and the Department of Defense for review and comment. The Department  of Defense did not have comments.

VA provided written comments, which are reprinted in appendix I. VA  concurred with both of our recommendations—that any health equity  action plan should include performance measures to assess progress and  that VA should resolve weaknesses identified with the completeness and  accuracy of race and ethnicity data. VA further provided information on  how the agency intends to address our recommendations, with targeted  completion dates of December 2020 and June 2021, respectively.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Veterans Affairs and the Under Secretary  for Health, and the Secretary of the Department of Defense. In addition,  the report is also available at no charge on GAO’s website at  http://www.gao.gov.

If you or your staff has any questions regarding this report, please contact  me at (202) 512-7114 or draperd@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

Appendix I: Comments from the Department of Veterans Affairs

Appendix II: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Ann Tynan, Assistant Director;  Michelle Paluga, Analyst-in-Charge; Jennie Apter, Romonda McKinney  Bumpus, and Phil Steinberg made key contributions to this report. Also  contributing were Kaitlin Farquharson and Ethiene Salgado-Rodriguez.

Racial and ethnic minority veterans made up about 22% of the total veteran population of 18.6 million in 2016, according to VA.

Mirroring trends seen across the United States, VA has identified worse health care outcomes for some diseases among racial and ethnic minority veterans. For example, VA reported lower survival rates for African American veterans with cancer and cardiovascular-related illnesses than for other minority and white veterans.

We reviewed VA efforts to address these disparities. Among other things, we recommended VA ensure that any plans to address the issue include measures of progress and clear lines of accountability.