FEMA Disaster Workforce: Actions Needed to Address Deployment and Staff Development Challenges

During the 2017 and 2018 disaster seasons, several large-scale disasters created an unprecedented demand for FEMA's workforce. FEMA deployed 14,684 and 10,328 personnel at the peak of each of these seasons and reported staffing shortages during the disasters. GAO was asked to review issues related to the federal response to the 2017 disaster season.

This report addresses (1) how FEMA's disaster workforce is qualified and deployed, (2) how effective FEMA's qualification and deployment processes were during the 2017 and 2018 disaster seasons in ensuring workforce needs were met in the field, and (3) the extent to which FEMA's disaster workforce receives staff development to enhance skills and competencies. GAO analyzed documentation and data on incident workforce qualification and deployment; conducted 17 focus groups with 129 staff members; and interviewed FEMA officials in headquarters, field, and regional offices.

The Federal Emergency Management Agency (FEMA) has established mechanisms to qualify and deploy staff to disasters. For example, the FEMA Qualification System tracks training and task performance requirements for disaster workforce positions and has a process to designate staff as qualified in their positions once they have completed these requirements. FEMA's deployment process uses an automated system to deploy staff members to disasters that match field requests for positions and proficiency levels. The process depends on the agency's qualification and deployment systems to identify staff qualification status and skillsets to meet field needs.

However, FEMA's qualification and deployment processes did not provide reliable and complete staffing information to field officials to ensure its workforce was effectively deployed and used during the 2017 and 2018 disaster seasons. Specifically, GAO's focus groups with over 100 incident staff members and interviews with field and regional officials indicate that disaster personnel experienced significant limitations with qualification status matching performance in the field, due in part to challenges with how staff are evaluated through the qualification process. In all focus groups with applicable incident personnel, participants cited issues with staff members who were qualified in the FEMA Qualification System not having the skills or experience to effectively perform their positions. For example, one participant described supervising staff members who were qualified in the system but did not know the eligibility requirements for applicants to receive housing assistance, or what information needed to be included in the applicant's file. In addition, participants in the majority of the focus groups reported challenges with using FEMA's deployment processes to fully identify staff responsibilities, specialized skillsets, and experience. FEMA headquarters officials acknowledged the identified information challenges but said they have not developed a plan to address them in part because of competing priorities. Developing a plan to address identified challenges with providing reliable staffing information to field officials would enhance FEMA's ability to use staff as flexibly and effectively as possible to meet disaster needs.

Further, FEMA's disaster workforce experienced challenges with receiving staff development through the agency's existing methods to enhance the skills and competencies needed during disaster deployments—challenges FEMA headquarters officials acknowledged. Specifically, GAO's focus groups and interviews indicate that disaster personnel encountered challenges related to the availability of courses, providing and receiving on-the-job training and mentoring, and consistently receiving performance evaluations. For example, in 10 of 17 focus groups, participants cited barriers to taking courses that in their view would help them better perform their jobs. In addition, participants in seven focus groups stated that they did not receive coaching or feedback on the job. Relatedly, FEMA data show that at the start of deployments during the 2017 and 2018 disaster seasons, 36 percent of staff did not have an official assigned to coach and evaluate task performance—the primary mechanism the agency depends on for coaching. Creating a staff development program would help better ensure FEMA's disaster workforce develops the skills and competencies needed to meet mission needs in the field.

GAO is making three recommendations, including that FEMA develop (1) a plan to address identified challenges that have hindered its ability to provide reliable information to field officials about staff skills and abilities and (2) a staff development program for its disaster workforce that addresses training access, delivery of on-the-job training, and other development methods. The Department of Homeland Security concurred with GAO's recommendations.

During the 2017 and 2018 disaster seasons, several sequential, large- scale disasters created an unprecedented demand for federal resources.  In 2017, weather and climate-related disasters, including hurricanes  Harvey, Irma, and Maria, caused over $300 billion in damages in the  United States. Along with the severe 2017 California wildfires, these  disasters collectively affected 47 million people—nearly 15 percent of the  nation’s population. In 2018, hurricanes Florence and Matthew and  another severe California wildfire season again necessitated a major  federal response. We have previously reported that the rising number and  costs of disasters and the increasing reliance on the federal government  for disaster assistance will likely continue to rise as the climate changes.  Figure 1 shows the time line for key major disasters during the 2017 and  2018 disaster seasons.

The October 8, 2017 wildfires included the Tubbs, Nuns, Atlas, and Pocket fires, among others.

The December 4, 2017 wildfires included the Thomas, Creek, and Rye fires, among others.

The November 8, 2018 wildfires included the Camp, Woolsey, and Hill fires.

The Federal Emergency Management Agency (FEMA) coordinated the  federal response to these disasters and relied heavily on its workforce to  meet its mission. The agency deployed 14,684 personnel at the peak of  the 2017 disaster season and 10,328 personnel at the peak of the 2018  season. The numbers for each of these seasons are more than double  the number deployed at the peak of the 2016 disaster season, which was  about 5,000 personnel. The concurrent nature of the disasters in both the  2017 and 2018 disaster seasons highlighted the complex challenges  facing FEMA’s workforce. The 2017 and 2018 disaster seasons required  FEMA management to redeploy response personnel from one disaster to  the next, and the agency reported facing staffing shortfalls throughout the  response to these disasters. Additionally, a large influx of new employees  added to challenges with providing timely, program-specific training.  FEMA’s disaster workforce is expected to be in high demand for the  foreseeable future. According to FEMA, at the end of fiscal year 2019,  there were 64 open presidentially declared disasters that required federal  assistance.

In recent years, we have reported on long-standing workforce  management challenges within FEMA. For example, in September 2018,  we reported on the workforce capacity and training challenges FEMA  faced in the wake of the 2017 disasters. In November and December  2017, we reported on staffing challenges in FEMA’s Public Assistance  grant program, which provides funding to state and local governments,  among others, to help them respond to and recover from disasters. In  our March 2019 report on the status of recovery efforts in Puerto Rico, we  also reported Puerto Rico officials’ concerns about FEMA staff turnover  and lack of knowledge among FEMA staff about how disaster assistance  grant procedures are to be applied in Puerto Rico. While FEMA has  taken actions to address several of our workforce management-related  recommendations since 2016, a number of recommendations remained  open at the start of the 2019 hurricane season.

You asked us to review a broad range of issues related to disaster  response and recovery following the 2017 disaster season, including the  response and recovery to hurricanes Harvey, Irma, and Maria, and the  California wildfires. Because of the importance of addressing workforce  issues to meet future disaster response and recovery needs, this report  addresses  1.  how FEMA’s disaster workforce is qualified and deployed, and  workforce staffing levels during the 2017 and 2018 disaster seasons;  2.  how effective FEMA’s qualification and deployment processes were  during the 2017 and 2018 disaster seasons in helping ensure  workforce needs were met in the field; and  3.  the extent to which FEMA’s disaster workforce receives staff  development to enhance skills and competencies to support the  agency’s disaster missions.

To address all three objectives, we focused on FEMA staff members who  deploy to disaster sites. We analyzed documentation and data on disaster  workforce qualification, deployment, staffing levels, and development. For  example, we reviewed FEMA’s 2018-2022 Strategic Plan, 2017 Incident  Management Handbook, 2015 and 2019 FEMA Qualification System  guides, 2019 Deployment Guide, and documentation on FEMA’s staffing  targets. We also analyzed data from FEMA’s Deployment Tracking  System to determine staffing levels and to evaluate efforts FEMA has  taken to develop its staff, and human capital data to determine the  number of new staff FEMA hired. To assess the reliability of the data, we  interviewed officials at FEMA headquarters about their data quality control  procedures and reviewed documentation about these data systems. For  the Deployment Tracking System, we also conducted electronic testing  and reviewed the data for obvious errors and omissions. We found these  data sufficiently reliable for the purposes of this report.

In addition, we conducted focus groups with members of FEMA’s disaster  workforce and interviewed FEMA officials in the agency’s field and  regional offices to obtain perspectives on, and experiences with, the level  of staffing and skillsets of personnel deployed to disasters, FEMA’s  processes to qualify and deploy staff, and how staff were trained and  developed. Specifically, we conducted 17 focus groups with a total of 129  participants at FEMA joint field offices in Florida and Puerto Rico, FEMA’s  regional office in Texas, and headquarters. We selected these locations  based on where staff members who were deployed during the 2017  disaster season were located at the time of our review and to obtain  variation in geographic location to the extent possible, among other  things. Participants were selected using a stratified random sample from  those who had been deployed to a disaster during the 2017 disaster  season. Participants in each focus group were of the same employee  type, and we conducted separate groups with participants in supervisory  and nonsupervisory positions so they could speak more freely. We also  selected participants to obtain a mix of staff from different program areas  and qualification designations.

For our interviews with FEMA field and regional officials, we spoke with  FEMA leadership and managers who worked in various programmatic  areas in joint field offices in Florida, North Carolina, South Carolina, and  Puerto Rico and regional offices in Colorado, Texas, and Washington.  We selected the joint field offices and regions as locations for interviews  based on our focus group locations and to obtain variation in geographic  location and disaster activity. We evaluated the interviews with field and  regional officials and transcripts from audio-recordings of the focus  groups using systematic content analysis to identify key themes  concerning the topics our report addresses. The results from our analyses  of the focus groups and interviews are not generalizable, but provide  important perspectives on how effectively FEMA qualifies, deploys, and  develops staff for disasters.

Finally, we interviewed senior officials in FEMA headquarters to obtain  their perspectives on the staffing levels of the disaster workforce and how  the workforce is qualified, deployed, and developed. In addition, we  obtained information from these officials on the actions FEMA has taken  to address the challenges we identified through our focus groups,  interviews with field and regional officials, and data analysis. We  compared the results of our analysis and the information we gathered  with Standards for Internal Control in the Federal Government, The  Standard for Program Management, FEMA strategic documents and  guidance, and our prior reports on strategic human capital  management. Appendix I provides further details about our scope and  methodology.

We conducted this performance audit from June 2018 to May 2020 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The federal disaster workforce is designed to scale up or down depending  on the timing and magnitude of disasters. Specifically, FEMA has the  authority to augment its permanent full-time staff with temporary  personnel and deploy non-FEMA staff members when needed. FEMA  has historically relied on both permanent and temporary staff members to  respond to presidentially declared disasters. FEMA’s disaster workforce is  organized according to position categories, employee types, functions,  and job titles.

Every FEMA employee is assigned to one or more of four position  categories. Staff assigned to incident management positions deploy to  disaster sites to administer federal emergency response and recovery  programs. Staff assigned to the other three position categories—incident  support, ancillary support, and mission essential—provide support  services to deployed incident management staff, as well as to FEMA  more generally. For example, incident support staff assist with disaster  operations at the regional or national level, while mission essential staff  maintain basic agency operations, such as payroll and information  technology.

FEMA has several different employee types that operate out of the  agency’s national headquarters, regional offices, and joint field offices at  specific disaster locations. Each of the different employee types hold one  or more of the four position categories described above.

Permanent full-time employees are steady-state federal employees  that support FEMA’s mission areas and operations on a daily basis.

Cadre of On-Call Response/Recovery Employees (CORE) are a type  of temporary full-time employee hired to directly support response and  recovery efforts related to disasters for a 2- to 4-year term. These  positions may be renewed if there is ongoing disaster work and  funding is available.

Incident Management COREs are a type of CORE employee that  maintain a regular state of readiness to provide emergency-state  support and can be deployed up to 300 days a year in mission  areas.

Incident Management Assistance Teams are rapid-response  teams of CORE employees that deploy to disaster sites with little  to no notice and remain at disaster sites for unspecified amounts  of time, depending on mission needs. Members generally receive  4-year appointments.

Reservists are on-call employees who work intermittently as required  during incident management operations. Reservists must be available  to deploy as needed on 24 hours’ notice at all times during their 24  month appointment.

FEMA also has the authority to augment its disaster workforce with  temporary employees. This includes local hires, Surge Capacity Force  volunteers, and FEMA Corps members. FEMA further augments its  workforce with technical assistance contractors, who are specialized contractors hired to perform specific responsibilities. See figure 2 for  more information on FEMA’s employee types.

As shown in figure 3, reservists made up the largest portion of FEMA’s  deployed workforce during peak deployments for the 2017 and 2018  disaster seasons. In 2017, reservists made up about 32 percent of  FEMA’s deployed workforce, followed by COREs, permanent full-time  staff, and local hires. In 2018, reservists made up about 44 percent of  FEMA’s deployed workforce, followed by local hires, COREs, and  permanent full-time staff.

Organizational Structures for Incident Management Staff

FEMA’s incident management workforce is organized into 23 cadres.  Cadres are groups organized by operational or programmatic functions.  They are composed of full-time equivalent and intermittent staff members  who perform incident-related duties during disaster response. For  example, the Public Assistance cadre administers financial assistance to  state, local, tribal, and territorial communities for debris removal,  implementation of emergency protective measures, and permanent  restoration of infrastructure.

FEMA’s incident management workforce performs functions to support its  response, recovery, and mitigation missions. Each cadre supports at  least one of these missions, and some cadres perform functions across  more than one. Cadres also generally deploy to an incident at varying  points in the response and recovery phases, depending on their  functions. For example, FEMA officials said that the Logistics cadre,  which coordinates and monitors all aspects of resource planning,  movement, and order tracking, typically deploys staff to an incident before  the Hazard Mitigation cadre, which supports risk reduction activities later  during the recovery phase. See figure 4 for an example of peak  deployment by cadre during Hurricane Florence and appendix II for a  description of each cadre and their primary duties.

FEMA manages the staffing, training, and deployment of its cadres at the  national level. FEMA employees whose primary responsibilities are  incident management and disaster response, such as Incident  Management CORE and reservists, are generally considered national  assets and may be deployed to a disaster anywhere in the country,  regardless of permanent duty station.

FEMA organizes incident management positions into four tiers denoted  by increasing levels of leadership managerial responsibilities and further  categorizes senior leaders and officers by level of disaster complexity  experience. See figure 5 for more information on FEMA’s position tiers  and titles.

All FEMA incident management employees have a primary title, which  specifies their principal roles and responsibilities, and may also hold  subordinate titles for additional roles and responsibilities that the  employee can perform. Incident management staff members have one  primary incident management title and may have multiple subordinate  titles. FEMA may also assign specialties—categories used to identify a  specific measured (documented or credentialed) skill, task, experience, or  certification that may enhance performance of an associated position—to  certain staff. For example, a full-time equivalent staff member who works  day-to-day in FEMA’s Office of Policy and Program Analysis could hold a  primary incident management title as a Facilities Manager in FEMA’s  Logistics cadre and a subordinate title of Logistics Specialist in the same  cadre, and may be certified to operate certain types of forklifts.

FEMA Has Mechanisms in Place to Qualify and Deploy Staff to Disasters and Faced Staffing Shortages during the 2017 and 2018 Disaster Seasons

FEMA Designed and Implemented a System to Ensure Standards for Disaster Workforce Qualifications and Capabilities

FEMA designed and implemented the FEMA Qualification System in  2012 to standardize and manage the agency’s incident workforce  capabilities through prerequisite experience, training, and demonstrated  performance. FEMA uses the system to track requirements for incident  management positions and the proficiency level of staff members in those  positions.

According to the 2019 FEMA Qualification System Guide, training and  demonstrated performance are foundational elements of the system.

Required qualification system training consists of courses designed to  build competency in specific position responsibilities and is offered in a  variety of settings, such as the Department of Homeland Security (DHS)  Center for Domestic Preparedness or at a joint field office, and through a  variety of mediums, such as in a classroom, online, or on the job.  Demonstrated performance involves validation of the ability to  successfully and independently perform specific tasks. According to  FEMA, employees conduct required training concurrently with  demonstrated performance so that training builds on previous experience  and coursework. After FEMA assigns an incident management position to  staff, they are issued an electronic position task book, which lists and  tracks the training and demonstrated performance requirements for that  position.

Tasks in the position task book need to be signed off by a coach-and- evaluator—an individual that is trained and designated as a subject  matter expert by their cadre to evaluate one or more FEMA Qualification  System positions. After staff members have worked with a coach-and- evaluator to complete the tasks and trainings included in their task book,  they may submit it to cadre management as part of their qualification  application package.

Submitted qualification packages go through a number of rounds of  review before a final decision is made. First, FEMA’s Qualification System  Branch conducts an initial review to validate qualification package  completion and requirement waivers, among other things. The branch  then forwards the qualification package to cadre management for review.  Cadre management reviews employees’ applications for all positions,  including specialists and first-line supervisors, and a Qualification Review  Board also reviews employees’ applications for leadership, upper  management, and middle management positions. See figure 6 for an  overview of FEMA’s qualification system process.

FEMA Has a Process to Deploy Its Workforce to Disasters

A regional or national Incident Management Assistance Team is generally  among the first FEMA units to arrive on the ground at a disaster site to,  among other things, set up federal facilities, establish a joint field office,  and coordinate with officials from the impacted region and other relevant  federal, state, tribal, territorial, or local partners. If there are staffing  shortages among regional full-time equivalent staff members, FEMA  leadership in the region where the disaster occurs may submit a  deployment request for additional incident management staff members  through the Deployment Tracking System. After the Incident  Management Assistance Team stands up a joint field office, the Federal  Coordinating Officer assumes authority over all emergency response and  recovery efforts for the disaster, which includes filling staffing needs.

To determine the number and type of incident management personnel  needed in each position to meet FEMA requirements for any given  disaster, the Federal Coordinating Officer coordinates with regional  leadership, the joint field office’s Chief of Staff, and cadre management.  The basis of this determination is a variety of factors related to the nature  and scope of the disaster. For example, Individual Assistance and Public  Assistance needs are based in part on preliminary damage assessments  to determine the level of program assistance that may be required. To fill  identified staffing needs, field leadership uses a standard process to  request specific FEMA Qualification System titles and proficiency levels.

Once a standard deployment request is approved, the Deployment  Tracking System identifies staff members that match the requested  positions, skill sets, and qualification status using a preprogrammed,  automated process. The Deployment Tracking System then notifies staff  members selected in a rotational order based on length of time since their  last deployment, among other things. If an employee declines a  deployment request, the Deployment Tracking System automatically  sends a request to the next staff member with that incident management  position title on the deployment order list. Standard deployment  requests are filled by deploying employee types in the following order:  1.  Incident Management COREs  2.  Reservists  3.  Full-time equivalent employees other than Incident Management  At the incident, the Federal Coordinating Officer and other field leadership  staff are responsible for overseeing coordinating the responders working  for FEMA. Generally, after response operations and programs are  initiated, staffing needs may change. At this point, field leadership may  decide to demobilize some personnel deployed in certain cadres. The  decision to do so is based on workload, complexity of operations, and  disaster needs.

FEMA Faced Staffing Shortages in Key Cadres during the 2017 and 2018 Disaster Seasons

According to FEMA’s 2017 Hurricane Season After-Action Report, FEMA  did not meet its annual staffing target for disaster personnel during the  2017 hurricane season. FEMA uses force structure and force strength  targets to estimate staffing requirements for incidents and analyze the  number of staff the agency has available against these targets. FEMA  establishes a longer-term target for the number of incident management  staff needed to meet mission needs, called force structure, and tracks the  actual number of incident management staff who can deploy at a point in  time, which it calls force strength. FEMA uses its force strength  measure to track progress towards meeting staffing goals set out in the  force structure target and also sets interim targets each fiscal year for  reaching the longer-term force structure target.

In 2015, FEMA’s Workforce Management Division conducted a review of  FEMA’s workforce in coordination with the 23 cadres and adopted a force  structure target of 16,305. According to FEMA, this target was  established based on a number of considerations, including potential  incident levels and historical staffing data for incident management staff  deployed to different level events. The agency’s force strength at the end  of fiscal year 2017 was 11,656. On average, reservists had the largest  gap between force strength and established annual targets. For example,  at the end of fiscal year 2017, FEMA’s force strength for reservists was  6,793, which was 15 percent short of its target of 7,982 for that year. In  2019, FEMA’s Workforce Management Division completed a similar  review of its incident management workforce and updated its force  structure target to 17,670 incident management personnel, which it aims  to achieve by 2025. This new target was established using an updated  methodology based on information on historical disasters and deployed  incident management staff, along with input from each cadre’s  management and other officials with expertise on staffing patterns  throughout disasters.

According to FEMA’s 2017 Hurricane Season After-Action Report, FEMA  faced shortages across over half of its cadres when disasters made  landfall or began during the 2017 season, and we found that FEMA faced  similar shortages during the 2018 disaster season. For instance,  according to FEMA’s deployment data, 18 of 23 cadres operated with 25  percent or fewer staff available to deploy when Hurricane Maria made  landfall shortly after Hurricane Irma hit Florida, including the Individual  Assistance, Logistics, and Hazard Mitigation cadres. See figure 7 for  more information on cadre staffing levels at the start of major disasters  during the 2017 and 2018 disaster seasons.

In addition, many staff members who showed availability to deploy  declined when requested to do so. For example, according to FEMA  officials, the austere conditions in Puerto Rico and fatigue from previous  deployments to hurricanes Harvey and Irma contributed to the high  declination rate for Hurricane Maria deployment requests. In addition,  FEMA officials stated that permanent full-time employees may not  consistently update their availability or may be unavailable to deploy  because of their steady-state responsibilities. Further, reservists may  decline deployment requests because of the lack of employment  protections, which can create difficulties with leaving their jobs to work  intermittently in disasters. See table 1 for the declination rates for eight  major disasters during the 2017 and 2018 disaster seasons.

According to FEMA officials, the agency’s staffing shortages may have  originated in part from policy changes in 2012. Specifically, officials said  that a large number of incident management staff left the agency after  changes were made to the agency’s reservist program and qualification  system for disaster personnel in 2012. For instance, officials told us  many reservists with years of experience and technical skills left FEMA  when the reservist program increased the number of days they were  required to deploy or when many were assigned to positions in the  qualification system with lower levels of responsibility than what they  previously held in order to meet force structure targets.

FEMA took various actions to address the staffing shortages during the  2017 and 2018 disaster seasons and used new approaches to augment  its workforce. For example, in 2017, FEMA reported that it coordinated  the deployment of 2,740 Surge Capacity Force volunteers from eight DHS  components. DHS also expanded the Surge Capacity Force to other  federal agencies outside DHS for the first time in 2017, including 34  federal departments and agencies in the program, increasing the Surge  Capacity Force by 1,323 employees. Additionally, FEMA used local hires  to augment its workforce and expedited the local hiring process in  response to hurricanes Harvey, Irma, and Maria, resulting in the  onboarding of 4,095 local hires from August to November 2017. The  Federal Coordinating Officer who initially managed the Puerto Rico joint  field office instituted a goal of having a staff composed of 90 percent local  hires for recovery efforts. According to the official, investing heavily in  local hires and converting them to COREs will help build FEMA’s disaster  workforce for long-term Puerto Rico recovery efforts and any future  disasters that may occur in the region.

As mentioned previously, FEMA also conducted a review of its incident  management workforce in 2018 to determine the force structure needed  to effectively respond to disasters moving forward. FEMA officials we  spoke with said the agency has taken several steps to meet its new force  structure, such as establishing a program management office that is dedicated to achieving the agency’s staffing targets. Cadre management  officials we spoke with said that FEMA has hiring initiatives in place or  planned to help meet the staffing needs established from the review and  noted that it will take time for new staff to develop the skills and  experience to meet mission needs in the field.

FEMA Did Not Provide Reliable and Complete Staffing Information to Field Officials during Disasters and Lacks Mechanisms to Assess How Effectively It Deployed Staff

FEMA’s qualification and deployment processes did not provide reliable  and complete information on staff skills and abilities to ensure its  workforce was effectively deployed and used to meet field needs during  the 2017 and 2018 disaster seasons. In addition, FEMA lacks  mechanisms to assess deployment outcomes or the extent to which it  deployed the right mix of staff at the right time to meet mission needs.

FEMA’s Qualification and Deployment Processes Did Not Provide Reliable Staffing Information to Ensure Its Workforce Was Effectively Deployed and Used in the Field

FEMA field officials in our focus groups and interviews said they  experienced a number of challenges with the reliability of information from  FEMA’s qualification and deployment processes and systems during the  2017 and 2018 disaster seasons. Specifically, these officials reported that  qualification status was not an accurate indicator of ability to perform,  which affected disaster assistance delivery and created difficulties with  ensuring the right mix of staff were deployed and effectively assigning  responsibilities at disaster sites. Officials also reported other challenges  with identifying and leveraging staff skills, including lack of information  about specialized abilities and expertise. In response to its experience  with recent disaster seasons, FEMA has taken or planned some actions  to improve its qualification and deployment processes. However, these  actions have not been fully implemented and do not fully address the  information shortcomings that field officials experienced, as discussed  below.

Field Officials Reported Qualification Status Was Not a Reliable Indicator of Staff’s Ability to Perform Their Positions in the Field

FEMA’s qualification and deployment processes and systems do not  provide accurate and complete information about staff members’ abilities  to ensure field leadership and managers get staff with the right skills at  the right time or to most effectively employ and leverage the staff that are  deployed to support FEMA’s missions. As discussed earlier in this report,  field leadership use the Deployment Tracking System to request staff  based on mission needs. The system uses an automated process to  select who to deploy from a list of available staff by position and  qualification status, and relies on the FEMA Qualification System to  identify staff members who are qualified in their positions and those who  are trainees. Qualified staff members are expected to be able to  successfully and independently perform the duties of their position.

However, as shown in table 2, our focus groups with incident  management staff and interviews with field and regional officials indicate  that disaster personnel experienced significant limitations with  qualification status in the FEMA Qualification System matching  performance in the field. Very few found that it was a good indicator of  staff abilities. For example, participants in two of 14 focus groups  described positive experiences with qualification status as an indicator of  staff abilities; while, in all 14 groups, participants stated that staff  members who were designated as qualified in the system did not have  the skills or experience to perform effectively in their positions. Officials  in 15 of our 29 field and regional office interviews had similar  experiences. For example, Individual Assistance managers in one of the  joint field offices we visited said that they had 20 staff members who were  designated as qualified but not capable of performing basic tasks, such  as knowing how to access the program’s information system. Hazard  Mitigation managers from the same joint field office said that about half of  their staff who were designated as qualified could not proficiently perform  their job duties.

Participants in our focus groups and field leadership and managers we  interviewed cited numerous operational challenges that resulted from  qualification status not being an accurate indicator of staff abilities.  Specifically, they stated that (1) staff designated as qualified who lacked  the skills and knowledge to perform their positions negatively affected  disaster assistance delivery, staff workload, and morale and (2) the  unreliability of qualification designations hindered their cadre’s ability to  staff disasters with the right mix of staff at the right time and effectively  assign responsibilities. Table 3 provides examples of the challenges they  experienced.

Participants in our focus groups also cited a range of challenges with  position task books and the qualification process that in their view  contributed to qualification status not being an accurate indicator of staff  proficiency. For example:  Position task book tasks. In 12 of our 14 focus groups with FEMA  incident management staff, participants said the tasks in the position task  books did not fully reflect the skills or competencies needed to perform  positions. For example, a participant in one focus group said that the  tasks in her book did not incorporate sufficient soft skills, such as the  ability to communicate with sensitivity and empathy and other  interpersonal skills, which are important because staff in her cadre often  interact with disaster survivors who have suffered great losses.

Coach-and-evaluator process. Participants in 12 of our 14 focus groups  also raised concerns with how coach-and-evaluators endorsed tasks,  such as lack of consistency and objectivity with signing off on tasks.  These issues included coach-and-evaluators signing off on large numbers  of tasks too quickly or easily, which participants in 12 focus groups said  occurred. Some participants who functioned as coach-and-evaluators  said they felt pressure from staff to endorse tasks because reservists  receive salary increases when they get qualified. Participants also told us  that cadre management may push for staff to be qualified to meet  qualification rate targets. A participant in one of our supervisory-level  focus groups said he felt pressure from both these sources and admitted  to signing off on tasks even though he did not feel the staff member could  proficiently perform them. He said that the staff member was qualified in  the FEMA Qualification System and later deployed to a smaller disaster,  where she was the sole person responsible for her functional area and  unable to do the job.

Difficulties completing position task books. Participants in all 14 of  our focus groups also raised various challenges with completing their task  books. These challenges include a lack of available coach-and-evaluators  to sign-off on tasks; lack of opportunities to deploy or perform certain  tasks; and being unable to complete all the training courses in their task  books because classes were unavailable, full, or conflicted with mission  needs; among others. As a result, staff members who are able to perform  their positions may not be designated as qualified in FEMA’s qualification  system.

Field Officials Cited Challenges with Using FEMA’s Qualification and Deployment Processes to Fully Identify and Use Staff Skills and Experience

Participants in our focus groups and leadership and managers in our field  and regional office interviews reported other challenges with being able to  fully identify and use staff skills and experience during disasters. For  example:  Position titles not fully reflecting staff abilities. FEMA allows staff to  have one primary position title in which they are qualified or have an open  task book (trainee or candidate status). Officials in 14 of our 29 field and  regional interviews and participants in eight of our 14 focus groups with  incident management staff raised concerns with this policy. Specifically,  officials noted that many employees have experience and expertise in  multiple cadres or programs within a cadre, but their full abilities are not  reflected in FEMA’s qualification and deployment systems. As a result,  field leadership and managers may not be able to fully identify and use  the available skills and experience of FEMA’s workforce during disasters,  which can limit FEMA’s capacity and flexibility to meet disaster needs,  especially when demand is high. For example, one regional official said  the Deployment Tracking System has Operations Section Chief as her  position title but does not capture her ability to deploy as an Individual  Assistance Branch Director, another position in which she has  considerable experience.

Overly broad position titles and lack of information on specialized  skills. In addition, participants in our focus groups told us that some  cadre position titles are overly broad (five of 14 groups) and that FEMA’s  qualification and deployment systems do not include information on  specialized skillsets and experience that would be useful for making  deployment and staffing decisions (10 of 14 groups). Officials in 14 of our  29 field and regional interviews raised one or more of these same issues.  For example, Logistics managers in one of the joint field offices we visited  said that the Logistics Specialist title is too general and captures the  majority of warehouse personnel without specifying the actual  responsibilities they are able to perform. They noted that, as a result,  management needs to query staff members when they arrive to help  determine their skills and, in many cases, assign responsibilities by trial  and error. According to officials, this can create a safety hazard because  some responsibilities require specific skills, such as the ability to operate  a certain type of forklift. They also noted that while the Deployment  Tracking System allows cadres to input specific skillsets, such as forklift  certification, this field has not been consistently filled in.

Limitations with fully capturing permanent full-time employee and  CORE qualifications. In seven of our eight focus groups with permanent  full-time employees and COREs, participants stated that it is not a priority  for them to complete their task books because they have little or no  incentives to be designated as qualified in the FEMA Qualification  System. For example, some participants noted that unlike reservists, their  pay and professional development is not directly tied to their qualification  status or position. Another participant said that he has been deployed to  many disasters and does not have any tasks in his task book endorsed  because he is focused on meeting mission needs and does not care  enough about being qualified in the system to take the time to complete  his task book. Some regional and field officials also raised the same  issues. For example, Response Division managers in one of the regions  we selected for interviews said that some of the best talent at FEMA,  such as staff members with years of experience who work full-time in  positions similar to their incident management titles, have never opened  or completed a task book because there is no incentive for them to do so.  As a result, FEMA may not be fully capturing the qualifications and skills  of permanent full-time employees and COREs.

FEMA Has Taken Actions to Help Improve Its Qualification and Deployment Processes, but These Actions Do Not Fully Address the Key Challenges Field Officials Identified

FEMA has taken a number of actions intended to help address some of  the challenges with its qualification and deployment processes that  hindered its ability to provide accurate and complete staffing information  to field officials. FEMA headquarters officials acknowledged the  challenges we identified with the FEMA Qualification System and noted  that the system is still evolving. Key efforts to improve the reliability of  qualification designations include:  Qualifying staff members who could proficiently perform their  positions. During the 2017 hurricane season, FEMA took steps to qualify  staff members who were not designated as qualified in the FEMA  Qualification System but could proficiently perform the duties of their  position. For example, according to the agency’s after-action report for  the hurricane season, FEMA temporarily changed qualification  procedures during the season to more rapidly qualify employees who had  demonstrated their skills outside the traditional process. FEMA  headquarters officials stated that this helped qualification designations  better reflect the skills and abilities of these staff members. Other actions  that FEMA has taken to help qualify staff include allowing cadre  management to waive certain tasks or training, allowing specified tasks to  be signed-off on during training exercises, and, as discussed later in this  report, conducting a pilot on deploying staff to specifically serve as coach- and-evaluators during disasters.

Revising position task books. FEMA headquarters officials said they  began reviewing task books in spring 2017 to help ensure that tasks are  measurable and better align with the knowledge, skills, and abilities  needed to perform positions. Officials said this project was completed in  November 2018 and the revised task books have been implemented.  They noted that this will help streamline the qualification process, allow  for more objective evaluation, and help ensure tasks better reflect the  skills needed on the job. According to FEMA officials, they plan to  continue to work with the cadres to ensure task books align with the skills  and competencies required to complete disaster missions.

Enhanced coach-and-evaluator training. FEMA revised its training for  coach-and-evaluators to provide more guidance and tools for how to  accurately evaluate staff and improve the integrity of the evaluation  process. Specifically, in October 2017, FEMA updated the coach-and- evaluator training class and added material on, for example, techniques  for evaluating with integrity, types of observation, and documenting task  performance by including comments in the task books. All staff members  must pass the class by performing a capstone exercise and taking a  written exam before being able to serve as a coach-and-evaluator.

Additional controls in the qualification process. Since 2017, FEMA  has established additional controls to provide more oversight on the  qualification process. For example, headquarters officials said that as part  of the qualification review process, officials may review the qualification  packages to check how many tasks were endorsed during a given period  of time. If the number is unusually large, they may note it for cadre  management to consider when making qualification decisions. This step  was incorporated in the new FEMA Qualification System Guide that was  issued in August 2019. The guide also includes changes to the  Qualification Review Board process, such as requiring candidates for  leadership and upper-level management positions to attend the review in  person and answer questions about their deployments, training history,  and task book completion.

FEMA has also taken some initial actions and considered options to  better identify and use staff skills and experience in the field. For  example, FEMA headquarters officials said they are aware that limiting  staff to one primary position or one open task book may not fully capture  their experience and abilities and are exploring ways to address it.  However, they stated that this is a complex issue and allowing staff to  hold multiple primary positions could affect the time it takes for staff to  complete task books and, on a broader level, pay scales, career  progression paths, and training budgets. They also noted that this could  create complications with how to deploy staff if multiple cadres need  positions filled during times of scarce resources. FEMA headquarters  officials told us that staff can be deployed in positions other than their  FEMA Qualification System positions but acknowledged that because  these other positions are not systematically recorded in the Deployment  Tracking System, leadership would need to be aware of staff skills and  abilities through informal means. Further, FEMA headquarters officials  said that as part of its review of the incident management workforce, they  have revised the position titles for certain cadres, which they noted could  potentially result in the titles better reflecting staff roles and  responsibilities. Officials added that they need to balance the enhanced  staffing information that more specific titles provide with the flexibility of  broader titles, particularly for entry-level positions. FEMA has also  included information on assigning specialized skills to staff in the  Deployment Tracking System in its new FEMA Qualification System and  deployment guides.

While FEMA has taken some steps to improve its qualification and  deployment systems, its efforts primarily affect the qualification process  moving forward and do not fully address field officials’ experiences with  not having reliable information on staff qualifications and skills to  effectively use the available workforce to meet mission needs. For  example, the changes to the position task books, coach-and-evaluator  program, and FEMA Qualification System guide do not affect staff  members who have already been qualified in the system but cannot  perform their duties, and as discussed later in this report, FEMA currently  does not have an effective performance evaluation system in place to  identify and address skill deficiencies. In addition, the agency has not  taken actions to address the challenges with identifying staff who can  serve multiple incident management positions and fully capturing the  qualifications of permanent full-time employees and COREs. Also,  headquarters officials stated that FEMA has revised some of its position  titles, but not all the cadres that reported challenges with overly broad  titles had adjustments made to their positions. Further, this initiative is in  the early stages of implementation and it is too soon to assess whether  the revised positions will provide better information to field officials on  staff members’ specific responsibilities.

Further, the lack of reliability of qualification status as an accurate  indicator of staff skills and abilities has been a persistent issue with the  FEMA Qualification System. For example, we stated in our 2015 report on  FEMA workforce management that in five of 11 focus groups with  permanent full-time employees and COREs, participants cited concerns  about the implementation of the FEMA Qualification System, and some  observed employees whose training and experience did not reflect the  position and qualification level to which they were assigned. Also, in a  2016 report on the reservist workforce, the DHS Office of the Inspector  General stated that in five of the seven disaster deployments included in  their review, joint field office staff encountered problems obtaining  capable reservists with position titles under the FEMA Qualification  System. These officials said that reservists sometimes lacked the  experience and training to perform their duties, and as a result, were  reassigned to positions outside their system titles.

One of the purposes of the FEMA Qualification System is to ensure  consistency in skill identification and deployable assets for positions  across the agency. In addition, FEMA’s 2018-2022 Strategic Plan states  that the agency should guarantee that the FEMA Qualification System  maximizes the ability of FEMA staff to use their skills and talents to  deliver the best outcomes possible. However, as discussed above, FEMA  experienced challenges with achieving these objectives. In addition,  Standards for Internal Control in the Federal Government directs  management to use quality information to achieve the agency’s objectives. It states that, as part of designing control activities for human  capital management, management should continually assess the  knowledge, skills, and ability needs of the agency to help achieve  organizational goals. According to the standards, only when the right  personnel for the job are on board and are provided the right  responsibilities, among other things, is operational success possible. In  addition, according to The Standard for Program Management, program  monitoring, reporting, and controls include the development of plans to  respond to identified issues. It also states that program management  should include timeframes and milestones for achieving program benefits  and obtaining feedback from stakeholders to better understand the  concerns related to the program and impact of the program.

Given the complexity of FEMA’s workforce and the persistent issues with  the reliability of qualification designations and other challenges with  identifying the knowledge, skills, and abilities of its staff, FEMA would  benefit from developing a comprehensive plan—with timeframes and  milestones—to address issues with the quality of information its  qualification and deployment processes and systems provide to field  officials. Such a plan would also benefit from the inclusion of perspectives  from field leadership who depend on the information. FEMA officials  acknowledged the staffing information challenges we identified and noted  that they have not developed a plan to address them because the issues  are multifaceted—changes in policy can potentially affect numerous areas  of the workforce—and they had been focused on other initiatives, such as  revising force structure targets and streamlining the qualification process.  However, they said that such a plan would be useful. Developing a plan  to address the challenges that hindered FEMA’s ability to provide reliable  and complete information about staff skills to field leaders and managers  would better enable the agency to use its disaster workforce as flexibly  and effectively as possible to meet mission needs in the field.

FEMA Does Not Have Mechanisms to Assess How Effectively Its Disaster Workforce Was Deployed to Meet Field Needs

FEMA does not have mechanisms to assess the extent to which its  deployment strategies met mission needs in the field during disasters.  FEMA’s Deployment Guide states that for the agency to fulfill its  preparedness, response, recovery, and mitigation missions, it must be  able to effectively and efficiently deploy its responders through a process  that sends the right people to the right place at the right time with the right  qualifications. FEMA has measures and collects data related to staffing  levels and availability, such as comparing cadre force strength to annual  targets, comparing staff qualification rates to targets, determining the  percent of staff in each cadre that show availability in the Deployment  Tracking System, and tracking the number of staff deployed to disasters.  However, none of these measures or data directly demonstrate  deployment outcomes or how effectively FEMA deployed available staff to  meet mission needs.

Headquarters officials said that, among other things, they generally have  looked at the number of staff members that were deployed to disasters,  as well as declinations, to assess the extent to which they were able to  meet staffing needs. They noted that this assumed the number, type, and  timing of staff deployments matched field needs. However, our focus  groups and interviews with field officials indicate that this was not  generally the case. For example, in all 17 of our focus groups,  participants experienced challenges with the staffing, skill, or experience  levels of the deployed workforce, such as having too few staff members  with the right technical skills to perform their missions efficiently and  effectively. Further, in 12 of the 17 focus groups we conducted,  participants said that there were challenges with the timing of  deployments, such as staff from certain cadres being deployed too early  or redeploying staff from key positions when the mission need was still  high. In most of our interviews with field leadership and managers,  officials described similar challenges with the number, skill level, or timing  of staff deployments. Participants in our focus groups and field officials  we interviewed said they make every effort to meet mission needs despite  challenges with staff deployment, but noted that these challenges with  deployment outcomes not meeting field needs can increase staff  workload and delay disaster assistance, among other impacts and  inefficiencies.

Our work on strategic human capital management states that effective  geographic and organizational deployment strategies can enable an  organization to have the right people, with the right skills, doing the right  jobs, in the right place, at the right time by making flexible use of its  internal workforce. Additionally, Standards for Internal Controls in the  Federal Government states that management should establish and  operate monitoring activities to continually monitor the internal control  system, evaluate results, and remediate any deficiencies identified on a  timely basis. As part of remediating deficiencies, the standards advise  management to report and evaluate issues that were identified as a result  of the monitoring and take corrective actions to address them.

As discussed earlier in this report, field leadership request staff based on  cadres’ anticipated needs using estimates of the severity of damage and  the nature and scope of the disaster, among other factors. However,  FEMA headquarters officials told us their data systems cannot determine  the extent to which field deployment requests were met during  disasters. In addition, these officials noted that they have not  established other mechanisms to assess deployment outcomes because  this is extremely complex and they are considering how best to do so.  They noted that they have been working with in-house data science  experts to consider what kinds of measures and metrics they could  design to assess deployment outcomes, but they did not have any  concrete proposals or time frames for when this might be completed.  Without mechanisms to assess deployment outcomes, FEMA officials in  headquarters lack critical information to monitor and evaluate the extent  to which its deployment policies and strategies effectively placed staff  with the right skills in the right place at the right time to meet mission  needs in the field. As a result, FEMA may miss opportunities to identify  when corrective actions are required to better deploy its workforce to  meet field needs, such as adjusting the timing and staging of  deployments, and the amount of staff deployed.

FEMA Staff and Managers Experienced Challenges with Staff Development Efforts Intended to Enhance the Skills and Competencies Needed During Deployments

We found significant shortcomings in FEMA’s ability to ensure staff  development—which consists of courses, on-the-job-learning, and  coaching and mentoring—for the skills and abilities needed in the field.  Specifically, although the current approach to developing staff includes  efforts to provide training courses, opportunities for on-the-job training  and mentoring, and a performance evaluation system, each of these  elements has limitations as implemented, and they are not effectively  coordinated to help ensure systematic and comprehensive staff  development. Staff and managers cited certain recurrent challenges with  staff development in focus groups and interviews, such as (1) limitations  on the ability to take useful classroom training, (2) challenges providing or  receiving on-the-job training and mentoring, (3) inconsistent use of  performance evaluations, and (4) difficulty with ongoing development  when not deployed to a disaster.

Some Staff and Managers Cited Challenges with the Ability to Take Useful Classroom Training

One way staff members develop skills and competencies is through  completing required courses in their position task books. However, in 10  of our 17 focus groups, participants discussed barriers to taking courses  through FEMA’s qualification system that in their view would help them  better perform their jobs, such as being unable to take courses that are  not in their position task books or if they are already qualified in their  positions. Officials in 11 of the 29 field and regional interviews we  conducted raised the same issue. FEMA headquarters officials stated that  staff are generally required to obtain cadre management approval before  they can register for incident management-related courses that are not  specifically listed in their position task books, but staff told us it can be  difficult to receive approval because of funding limitations. For example, a  Hazard Mitigation official at one joint field office we visited described a  situation where a staff member wanted to take a course on mitigation and  engineering techniques for coastal construction that would have  benefitted the work the person was doing, but was not able to get  approval. Participants in our focus groups also told us that staff deployed  to a position other than their FEMA Qualification System title had been  unable to take courses related to the work they were doing.

Moreover, staff members said the FEMA Qualification System limits  training opportunities for those already qualified in their positions. For  example, some staff members said that once they had completed their  position task book, they were sometimes unable to get training that  included new information on updated policies or procedures specific to  their work. An official in one of the FEMA regions we selected for  interviews said that some staff members in the region who were qualified  would have preferred to be designated as trainees in the FEMA  Qualification System because it would allow them to take relevant  courses. In March 2020, FEMA officials told us the agency has recently  taken actions to make it easier for cadres to send staff to courses that are  not required in their position task book or for positions where the person  is qualified.

Finally, participants in our focus groups with permanent full-time staff  members reported challenges with being able to take courses to develop  their incident management competencies. These participants told us it is  challenging for them to take disaster-related courses while performing  their steady-state work. They said this is because there is no budget for  localized disaster-related courses in their offices and it can be difficult to  get approval and take time from their duties to travel for this type of  training.

Some Staff and Managers Cited Challenges with Providing and Receiving On-the-Job Training and Mentoring during Disasters

Focus group participants frequently said developing skills on the job was  the most useful type of training they receive. Specifically, participants in  12 of our 17 focus groups said on-the-job training was the most useful  kind of training and participants in 13 of the 17 focus groups said this is  how they received most of their training. In addition, headquarters officials  in the Individual Assistance cadre said one of the benefits of on-the-job  training during deployments is that it provides an opportunity for staff to  learn and practice their craft in a setting that is difficult to simulate during  training.

The FEMA Qualification System Guide states that FEMA uses coach- and-evaluators as the primary mechanism for staff to learn the specific  skills needed for each position. However, staff members we spoke with  said they have difficulties developing their skills through the qualification  process. Specifically, in seven of the 17 focus groups, participants told us  they did not get feedback or coaching on the job. According to staff in our  focus groups, the coach-and-evaluator aspect of the qualification system  is not the ideal mechanism to support on-the-job training and  development because it often emphasizes the evaluation role over the  coaching role. In nine of 14 focus groups, participants told us the position  task book process focuses more on completing tasks than on  performance, development, or building competencies. Officials in eight  of our 29 field and regional interviews reported similar experiences. Some  staff who did receive coaching said it was often based on the interest  level and time that an individual who was willing to invest and was not  done in a systematic or consistent way.

Moreover, a commonly cited challenge—in 11 of our 14 focus groups— was the lack of coach-and-evaluators to sign off on position task books.  Officials in 16 of our 29 field and regional interviews raised the same  issue. Participants in our focus groups said they had difficulties finding  available coach-and-evaluators at disaster sites. For example, our  analysis of FEMA data found that 36 percent of FEMA’s incident  management workforce did not have a coach-and-evaluator at the start of  their deployment during the 2017 and 2018 disaster seasons. In addition,  according to staff in our focus groups and interviews, coach-and- evaluators at the disaster often do not have time to coach staff. For  example, officials at one of the joint field offices we visited said mission  needs always come first and coaching and evaluating responsibilities are  often not anyone’s priority.

In addition to on-the-job training challenges related to the FEMA  Qualification System, focus group participants also reported more general  challenges with on-the-job training. For instance, multiple supervisors in  the Logistics cadre at one joint field office said that in addition to doing  their own work, experienced staff members need to spend significant time  training others, which competes with performing their mission.  Furthermore, participants in seven of the 17 focus groups said providing  on-the-job training was particularly challenging at the beginning of a  disaster, when the disaster is often hectic and at its busiest. Recovery  Division officials in a FEMA regional office said a challenge at the start of  the disaster is finding staff members who know what to do and have the  time to train those who do not. Staff members also described difficulties  with providing and receiving on-the-job training in later phases of a  disaster. In one focus group with supervisors, a participant said that once  the disaster has reached a pace where they have time to train, staff  members are often redeployed.

Finally, in 16 of our 29 field and regional interviews, officials said there  was a lack of mentoring and sustained staff development across  disasters. For example, officials at one joint field office told us that once  staff members complete their position task book, they generally do not  receive any additional coaching or mentoring in that position. This official  stated that reservists have a more difficult time identifying mentors than  other employee types because they deploy intermittently and likely have  different supervisors and coach-and-evaluators each time they deploy. In  addition, FEMA officials said coach-and-evaluators are not meant to  serve as mentors. FEMA human capital officials said that different offices  can develop their own mentoring programs but these may not be  available to all employee types. As a result, not all staff members know to  ask for, or expect to receive, mentoring.

FEMA headquarters officials acknowledged some of these staff  development challenges and described actions they have planned, or are  underway, to help address some of them. Specifically, FEMA revised the  coach-and-evaluator course in 2017 to place a greater emphasis on the  coaching responsibilities of the coach-and-evaluator role. For example,  the revised course teaches effective coaching strategies, including how to  give effective, actionable feedback. Also, in summer 2019, FEMA  conducted a pilot with the National Disaster Recovery Support cadre to  deploy a single coach-and-evaluator solely in that position and  communicated to cadre management that this individual was not to be  used for other disaster-related responsibilities. FEMA officials said this  pilot was a success. In evaluating the pilot, FEMA said the coach-and- evaluator was able to devote time to proper training and answering any  questions presented. Finally, the agency revised the FEMA Qualification  System Guide in August 2019, which included clarifying differences  between coaching and evaluating. The revised guide states that, as part  of the position task book process, a coach explains, demonstrates, trains,  assesses, and documents an individual’s task performance while an  evaluator observes, assesses, documents, and endorses an employee’s  independent performance of specific tasks.

FEMA Officials Reported Inconsistent Use of Performance Evaluations at Disasters

Headquarters officials told us that during the 2017 and 2018 disaster  seasons, disaster workforce employees inconsistently received  performance evaluations when deployed. Performance evaluations at  disasters are to be completed on a paper form by a temporary duty  supervisor. If the staff member has a coach-and-evaluator, the temporary  supervisor may request input regarding progress toward mastering the  skills covered by the position task book. The temporary supervisor is  supposed to provide that evaluation to cadre management if an  evaluation was completed. However, FEMA officials told us there are no  mechanisms in place to ensure these steps occur or that the evaluations  will be used to help develop staff competencies, and it is not something  FEMA officials monitor.

Further, FEMA headquarters officials stated there are no controls in place  to ensure supervisors rate staff consistently from supervisor to supervisor.  These officials told us they are aware of some problems with how the  agency conducts performance evaluations for the disaster workforce and  are developing changes to address them. For example, in the months  prior to the 2017 disasters, the agency began revising its performance  evaluation system, but suspended its efforts when that year’s disasters  occurred. In 2019, FEMA resumed this initiative and agency officials told  us they expect it will be implemented by June 2020. They said the new  system will include replacing the paper form with an electronic program  that will be integrated into FEMA’s other personnel systems, such as the  Deployment Tracking System. Further, in March 2020, FEMA officials told  us they are finalizing a directive intended to provide guidance to  supervisors at disasters on how they are to provide deployment  performance evaluations. In addition, in April 2020, FEMA issued  guidance for the administration, implementation, and oversight of a  performance management process that will provide reservists with annual  performance appraisals. FEMA officials told us this will help ensure that  reservist performance appraisals accurately reflect their job performance  and assist them in maintaining and improving performance in the future.  The agency’s reservist performance management initiative is expected to  be completed by January 2021, but officials have not provided specific  interim milestones or target dates.

Staff and Managers Cited Difficulties with Receiving Staff Development When Not Deployed to a Disaster

Many disaster workforce staff members are not likely to get ongoing  development directly from their cadre management when they are not  deployed. According to data from FEMA, there was one cadre supervisor  of record for every 128 reservists and Incident Management CORE staff  as of June 1, 2019. During the 2017 and 2018 disaster seasons, this  ratio was higher in certain cadres. For example, there was one supervisor  of record for every 807 reservists and Incident Management CORE staff  as of June 1, 2017 in the Individual Assistance cadre. FEMA  headquarters officials told us they are assessing what the right mix of  supervisors to reservists should be across the cadres.

Further, staff members told us they have difficulties getting ongoing  development through hands-on training outside of a disaster. While  FEMA headquarters officials told us that cadres periodically conduct  mission rehearsal trainings each year to prepare their staff for disasters,  they also said not all staff can attend them because cadre management  determines which staff to invite. These trainings are designed for staff  members to simulate a potential disaster scenario while in a training  environment.

Finally, FEMA headquarters officials stated that receiving ongoing  development for staff who do not deploy frequently, such as reservists,  can be a challenge. The only instances when reservists are paid while not  deployed occur when they complete 40 hours a year of mandatory  training or 32 hours a year coordinating with their cadre. In addition, an  individual in one of our focus groups with permanent full-time employees  said reservists had difficulties accessing online mandatory training  because they did not have a FEMA laptop. A recovery manager in a  FEMA regional office told us that it can be challenging to provide staff  development for reservists because they are generally sent to the field to  do a discrete job and have limited opportunities to develop their skills and  competencies when not deployed.

As discussed above, FEMA’s disaster workforce reported challenges  receiving staff development through the agency’s existing methods, which  consists primarily of classroom training, on-the-job training and  mentoring, and performance evaluations. While FEMA has taken actions  to address some of the challenges staff experienced, opportunities  remain to ensure more effective and consistent staff development.  Specifically, FEMA does not have a staff development program in place  to provide assurance of effective and comprehensive staff development of  the skills and abilities needed during deployments. Further, FEMA  headquarters officials said it is primarily the responsibility of staff  members to find available coach-and-evaluators at disaster sites and the  agency has not developed a mechanism to help ensure deployed staff  are consistently paired with coach-and-evaluators. In addition, FEMA  headquarters has not taken actions to address the challenges we  identified with the lack of mentoring for staff deployed to disasters.  Further, given that FEMA’s performance evaluation initiatives are not yet  implemented, it is too early to assess how effective they will be in  enhancing staff development, including whether they will have  mechanisms in place to ensure employees receive useful evaluations or  the extent to which they will be coordinated with other development  activities, such as coaching through on-the-job training.

Standards for Internal Control in the Federal Government states that  management recruits, develops, and retains competent personnel to  achieve the entity’s objectives. This includes enabling individuals to  develop competencies appropriate for key roles, reinforcing standards of  conduct, and tailoring training based on the needs of the role. It also  includes mentoring to develop individual performance based on standards  of conduct and expectations of competence that align the individual’s  skills and expertise with the entity’s objectives and help personnel adapt  to an evolving environment. In addition, we have previously reported that  identifying where an agency’s development process is lacking can help  address barriers that hinder its ability to achieve meaningful results. We  also reported that it is important for agencies to treat continuous learning  as an investment in success as it can address employees’ career  development issues, skill-specific training needs, and provide managers  with opportunities to identify where training and development is  appropriate.

Effective and consistent staff development is particularly important  because FEMA has hired a large number of reservists over the past few  years. Our analysis of FEMA data shows that from June 1, 2017 to May  31, 2019, the agency hired over 3,200 reservists, which was 40 percent of  the agency’s entire reservist workforce as of June 1, 2019. Creating a  staff development program that systematically and comprehensively  addresses staff development through courses, on-the-job training and  mentoring, performance evaluation, and ongoing developmental  opportunities would provide better assurance that staff develop the skills  and competencies needed to meet mission needs during field operations  and help ensure the best results for disaster survivors.

The large-scale and concurrent disasters during the 2017 and 2018  disaster seasons highlighted the complex challenges facing FEMA’s  workforce. The agency deployed 14,684 and 10,328 personnel,  respectively, at the peak of each of these disaster seasons, and the  increased demand for its workforce is expected to continue. Without  accurate and complete information on the knowledge, skills, and abilities  of these staff members, field officials face challenges with efficiently  providing disaster assistance, managing staff workload, and assigning  responsibilities. FEMA has taken some initial actions to improve the  information provided by its qualification and deployment systems, such as  establishing additional controls in its qualification process. However,  developing a plan to address the information challenges experienced  during the 2017 and 2018 disaster seasons would be beneficial to  enhance field leadership’s ability to identify and leverage staff skills and,  given the persistence of some of these challenges, help ensure they do  not continue to affect FEMA’s ability to support mission needs in future  disasters. Further, in light of the staffing constraints that FEMA faces, it is  important that the agency be able to assess how effectively it deploys  available staff to disasters to meet field needs. Developing a mechanism  to assess FEMA’s deployment outcomes would provide officials in  headquarters with critical information to monitor and evaluate the extent  to which its deployment policies and strategies effectively place staff with  the right skills in the right place at the right time to meet mission needs  and take corrective actions if needed. Finally, creating a staff  development program for its disaster workforce that addresses access to  training, delivery of on-the-job training and mentoring, use of performance  evaluations, and developmental opportunities when not deployed would  help FEMA ensure more consistent and comprehensive development of  the skills and abilities needed during deployments. Consistent and  effective staff development is particularly important to help build the skills  of staff who are qualified in the FEMA Qualification System but unable to  proficiently perform their duties and develop the large number of staff that  FEMA has recently hired to meet its new disaster workforce targets.

We are making the following three recommendations to FEMA:  The FEMA Administrator should develop a plan—with time frames and  milestones and input from field leadership—to address identified  challenges that have hindered FEMA’s ability to provide reliable and  complete information to field leaders and managers about staff  knowledge, skills, and abilities. (Recommendation 1)

The FEMA Administrator should develop mechanisms, including  collecting relevant data, to assess how effectively FEMA’s disaster  workforce was deployed to meet mission needs in the field.  (Recommendation 2)

The FEMA Administrator should create a staff development program for  FEMA’s disaster workforce that, at a minimum, addresses access to  training, delivery of on-the-job training and mentoring, use of performance  evaluations, and consistent developmental opportunities regardless of  deployment status. (Recommendation 3)

We provided a draft of this report to DHS for review and comment. DHS  provided written comments, which are reprinted in appendix III and  summarized below. In its comments, DHS concurred with our three  recommendations and provided a number of ongoing and planned actions  that it intends to leverage in addressing them. DHS also provided  technical comments, which we incorporated as appropriate.

With regard to our first recommendation for FEMA to develop a plan to  address identified challenges with providing reliable and complete staffing  information to the field, DHS reiterated some of the steps described in this  report that FEMA has taken to improve the coach-and-evaluator program.  DHS noted that FEMA plans to engage field leaders on these initiatives to  develop a plan to address identified challenges. DHS also reported that  FEMA plans to increase training offerings and align its curriculum so that  FEMA Qualification System status matches workforce capability. DHS  anticipates these efforts will be completed by March 31, 2021. While  these are positive initial steps, they focus solely on the coach-and- evaluator program and staff training. Our report identified a number of  complex and interrelated challenges with the agency’s qualification and  deployment processes that hindered FEMA’s ability to provide reliable  information to field officials about staff members’ skills and abilities,  including their qualifications, specialized skillsets, and experience within  and across program areas. As such, in developing the plan we  recommended, it will be important for FEMA to take a comprehensive  approach and consider solutions that may cut across multiple systems  and processes. We will monitor DHS’s and FEMA’s efforts in this area to  assess the extent to which they fully implement our recommendation.

With regard to our second recommendation for FEMA to develop  mechanisms to assess how effectively FEMA’s disaster workforce was  deployed to meet mission needs in the field, DHS reiterated the actions  described in this report that FEMA took to establish new force structure  targets for its incident management workforce. DHS also reported that  FEMA plans to convene subject matter experts to develop mechanisms  that demonstrate how effectively FEMA’s disaster workforce deploys to  meet mission needs in the field, which are expected to be completed by  March 31, 2021. When they are complete, we will assess the  mechanisms to determine the extent to which they address our  recommendation.

Regarding our third recommendation for FEMA to create a staff  development program, DHS reiterated some of the actions FEMA has  taken to develop its disaster workforce that were described in this report.  Our report identified recurrent challenges with FEMA’s efforts to develop  staff through training courses, on-the-job training and mentoring, and  performance evaluations and noted that the agency’s current and planned  efforts do not fully address these challenges. In creating the staff  development program we recommended, it is important for FEMA to  consider how its overall control environment and the initiatives it puts in  place are coordinated to ensure staff receive comprehensive and  consistent development to build the skills needed during disaster field  operations. DHS anticipates that FEMA’s efforts to implement our  recommendation will be completed by March 31, 2021. At that time, we  will assess the agency’s actions to determine the extent to which they  address the intent of our recommendation.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Homeland Security, the FEMA  Administrator, and other interested parties. If you or your staff have any  questions about this report, please contact me at (404) 679-1875 or  curriec@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made key contributions to this report are listed in appendix  IV.

Appendix I: Objectives, Scope, and Methodology

This report addresses (1) how the Federal Emergency Management  Agency’s (FEMA) disaster workforce is qualified and deployed, and  workforce staffing levels during the 2017 and 2018 disaster seasons; (2)  how effective FEMA’s qualification and deployment processes were  during the 2017 and 2018 disaster seasons in helping ensure workforce  needs were met in the field; and (3) the extent to which FEMA’s disaster  workforce receives staff development to enhance skills and competencies  to support the agency’s disaster missions.

Our review focused on FEMA’s incident management workforce, which is  composed of FEMA staff who deploy to disaster sites. We defined the  2017 and 2018 disaster seasons as the time periods from August 23,  2017 through January 31, 2018, and September 7, 2018 through  November 25, 2018. The 2017 dates represent the start of the FEMA  incident period for Hurricane Harvey through the end of the incident  period for the California wildfire season. The 2018 dates represent the  start of the FEMA incident period for Hurricane Florence through the end  of the incident period for the California wildfires.

To address all three objectives, we (1) analyzed documentation and data  on incident management workforce qualification, deployment, staffing  levels, and development; (2) conducted focus groups with members of  FEMA’s incident workforce across a range of employee types— permanent full-time employees, Cadre of On-Call Response/Recovery  Employees (CORE), Incident Management CORE, reservists and local  hires; and (3) interviewed FEMA officials in headquarters and field and  regional offices. We compared the results of our analysis and the  information we gathered with Standards for Internal Control in the Federal  Government, The Standard for Program Management, FEMA strategic  documents and guidance, and our prior reports on strategic human  capital management and strategic training and development.

Analysis of FEMA Workforce Documents and Data

We analyzed documentation on how FEMA’s incident management  workforce is qualified, deployed, and developed. Documentation included  the agency’s 2017 Incident Management Handbook, 2015 CORE  Program Manual, 2017 Reservist Program Directive, 2015 and 2019  FEMA Qualification System guides, 2019 Coach-and-Evaluator Program  Directive, coach-and-evaluator training materials, 2014 Incident  Workforce Deployment Directive, and 2019 Deployment Guide. In  addition, we analyzed FEMA’s 2018-2022 Strategic Plan, 2017 Hurricane  Season After-Action Report, and documentation on FEMA’s staffing  targets for its incident management workforce.

We analyzed data from FEMA’s Deployment Tracking System to  determine incident management staffing levels, the number of staff  deployed to a disaster, the number of incident management staff that had  a coach-and-evaluator assigned, and the ratio of managers to incident  management staff. We also analyzed data FEMA provides to the National  Finance Center to determine the number of new staff the agency hired.  To assess the reliability of the data, we interviewed officials at FEMA  headquarters about their data quality control procedures and reviewed  documentation about these data systems. For the Deployment Tracking  System, we also conducted electronic testing and reviewed the data for  obvious errors and omissions. We found the data to be sufficiently reliable  for the purposes of this report.

Focus Groups with Incident Management Staff Members

As shown in table 4, to obtain perspectives on how effectively FEMA  qualifies, deploys, and develops its disaster workforce, we conducted 17  focus groups with a total of 129 participants who were deployed in  incident management positions during the 2017 disaster season, and in  some cases, the 2018 disaster season.

The focus group locations were selected based on where staff members  who were deployed during the 2017 disaster season were located at the  time of our review. We also selected these locations to reflect where the  2017 disasters occurred and to obtain variation in geographic location to  the extent possible. Participants were selected using a stratified random  sample from a universe of incident management staff members who were  deployed to a federally declared disaster during the 2017 hurricane and  wildfire season. For each employee type, we conducted separate focus  groups with participants in supervisory and nonsupervisory positions so  they could speak more freely. We also selected participants to obtain a  mix of staff from different cadres and a mix of staff that were qualified and  not qualified in the FEMA Qualification System. If selected staff members  indicated they could not attend, we replaced them with the next individual  on our randomized list who had similar attributes. There were between  three to 11 participants in each focus group, with an average of eight in  each. These focus group discussions were guided by a moderator who  used a structured list of discussion topics. The topics focused on staff  members’ perspectives on, and experiences with, the level of staffing and  skill sets their team had, how they were trained and developed, and the  FEMA Qualification System and its qualification determinations.  Supervisors were also asked about their staff’s skill sets, training, and  qualification status.

Focus group sessions were audio recorded and transcribed. We  evaluated the transcripts using systematic content analysis to identify key  themes on how effective FEMA’s qualification and deployment processes  were in helping to meet field needs and the extent to which staff members  received staff development to enhance their skills and competencies. An  analyst coded the transcripts and a second analyst validated the coding.  Any discrepancies were resolved by both analysts agreeing on the coding  of the associated statement by a participant. If needed, a third analyst  adjudicated any continued disagreement between coders. The results of  our focus group analysis are not generalizable to all incident management  staff members. However, they provided valuable first-hand experiences  with staffing levels and skill sets during disasters, FEMA’s deployment  processes, the FEMA Qualification System and the reliability of its  qualification designations, and how well staff were trained and developed.

Interviews with FEMA Officials in Field and Regional Offices and Headquarters

We conducted site visits to FEMA’s joint field offices in Columbia, South  Carolina; Durham, North Carolina; Guaynabo, Puerto Rico; and  Tallahassee, Florida, to obtain officials’ perspectives on staffing levels  and skill sets, the effectiveness of FEMA’s qualification and deployment  processes and systems in meeting field needs, and the extent to which  FEMA’s deployed staff receive coaching and development to enhance  their skills and competencies. Officials we interviewed at the joint field  offices included federal coordinating officers; chiefs of staff; training  managers; and managers in the Individual Assistance, Public Assistance,  Hazard Mitigation, and Logistics cadres, among others. We also  interviewed an official who was previously a federal coordinating officer at  a federally-declared wildfire in California. In addition, we interviewed  leadership and managers for FEMA regions VI, VIII, and X to obtain the  perspectives of regional officials on the topics above. In each of the  regions, we interviewed the regional administrator and managers in both  the response and recovery divisions, among others.

We selected the joint field offices and regions to conduct interviews based  on our focus group locations and to obtain variation in geographic location  and disaster activity. We conducted systematic content analysis of this  work using the same approach we used to analyze the focus groups. The  results from this analysis are not generalizable to all field and regional  officials, but provide important perspectives from leadership and  managers on FEMA’s mechanisms to qualify, deploy, and develop  incident management staff.

In addition, we conducted interviews with multiple senior officials in FEMA  headquarters. For example, we interviewed officials in the Field  Operations Directorate and management in the Individual Assistance,  Public Assistance, and Hazard Mitigation cadres to obtain information  about how FEMA’s incident management workforce and staff in their  cadres are qualified, deployed, and developed, and how the Deployment  Tracking System and the FEMA Qualification System are used for these  purposes. We also interviewed officials in the Office of the Chief  Component Human Capital Officer to learn how FEMA trains and  develops this workforce. We obtained information from these officials on  the actions FEMA has taken to address the challenges we identified  through our focus groups, interviews with field and regional officials, and  data analysis.

We conducted this performance audit from June 2018 to May 2020 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Federal Emergency Management Agency (FEMA) Cadre List and Descriptions

Appendix III: Comments from the U.S. Department of Homeland Security

Appendix IV: GAO Contact and Staff Acknowledgments

Chris Currie, (404) 679-1875 or curriec@gao.gov  In addition to the contact named above, Kathryn Godfrey (Assistant  Director), Johanna Wong (Analyst-in-Charge), James Cook, Lawrence  Crockett, Elizabeth Dretsch, Ricki Gaber, Eric Hauswirth, Tracey King,  Ronald La Due Lake, Rebecca Mendelsohn, Amanda Miller, and Adam  Vogt made key contributions to this report.

FEMA deployed thousands of staff to respond to multiple large-scale disasters in 2017 and 2018.

FEMA’s processes are intended to identify staff qualifications and skills, and use this information to deploy personnel. However, we found that FEMA didn’t provide reliable staffing information to the field during disasters. For example, staff in all 14 of our focus groups cited issues with personnel who were deemed “qualified” but didn’t have the skills to effectively perform their jobs, which affected disaster assistance.

We recommended that FEMA develop a plan to provide reliable information to field officials about staff skills and abilities.