Surface Transportation: DHS Is Developing and Testing Security Technologies, but Could Better Share Test Results

Since 2016, bombings of subways and bus systems in foreign cities and attempted attacks in U.S. cities demonstrate continued security threats to mass transit and other surface transportation systems. S&T and TSA are the primary federal entities responsible for researching, developing, and testing technologies designed to address threats to these systems. GAO has previously identified challenges with S&T's oversight of R&D projects.

GAO was asked to review S&T and TSA's roles in developing and testing surface transportation security technologies. This report, among other objectives, (1) assesses the extent to which S&T is developing technologies to secure surface transportation systems and progress made, and (2) identifies the key mechanisms that S&T, TSA, and stakeholders use to collaborate and share information on identifying capability gaps and security technologies, and analyzes the extent to which they are effective.

GAO assessed S&T's mass transit program because it was the only active R&D effort for surface transportation security. GAO interviewed officials from S&T, TSA, and nine mass transit operators; observed technologies; reviewed documentation; and analyzed budget information from fiscal years 2013 to 2018. GAO also used GAO's leading collaboration practices to assess collaboration on security technologies.

The Department of Homeland Security's (DHS) Science and Technology Directorate (S&T) has one research and development (R&D) effort focused on surface transportation, the Surface Transportation Explosive Threat Detection (STETD) program, which is developing technologies to secure mass transit systems (see figure). DHS guidance requires S&T to develop results-oriented milestones to track progress. GAO found, however, that S&T has not used milestones that fully adhered to DHS guidance. For example, most STETD program milestones did not clearly link to key activities described in program plans. As a result, DHS may not have the information needed to determine whether the STETD program is meeting its goals.

S&T, TSA, and stakeholders effectively collaborate, but TSA could better share test results with mass transit stakeholders. For example, S&T, TSA, and mass transit operators regularly collaborate on issues related to identifying mass transit capability gaps and testing security technologies to address those gaps. Nevertheless, GAO found TSA's efforts to share information on existing technologies to secure mass transit could be improved. Specifically, TSA regularly assesses commercially available technologies, but does not routinely or comprehensively share its results with mass transit operators. For example, TSA's reports on its testing of commercially available products would provide mass transit operators with technical assessment information. However, seven of the nine mass transit operators GAO spoke with asked for more technical assessment information on existing commercial technologies, indicating that they may not be receiving the TSA products that would provide this information. Sharing this information more routinely and comprehensively with mass transit operators would allow TSA to better inform them about the capabilities of technologies that could be acquired to secure thteir systems.

GAO is making two recommendations: that S&T incorporate DHS milestone guidance for its STETD program, and that TSA develop a mechanism to routinely and comprehensively share security technology information with mass transit operators. DHS concurred with both recommendations.

Since 2016, bombings of subway and bus systems in foreign cities such  as Brussels and St. Petersburg, as well as planned attacks in the New  York City area and other U.S. cities, demonstrate the persistence of  security threats to surface transportation systems. Surface transportation  systems generally rely on an open infrastructure that is difficult to monitor  and secure due to its multiple access points, hubs serving multiple  carriers, and in some cases, lack of access barriers. Mass transit systems  are one component of the nation’s surface transportation system, which  also includes freight rail, highways, and pipelines, among other modes.  Given the inherent difficulty of securing mass transit and other surface (or  land) transportation systems, federal, state, local, and mass transit  officials rely on research and development (R&D) to identify or produce  potential technology solutions to address security vulnerabilities.

Within the federal government, the U.S. Department of Homeland  Security’s (DHS) Science and Technology Directorate (S&T) and  Transportation Security Administration (TSA) are the primary entities  responsible for researching, developing, and testing technologies  designed to address risks facing surface transportation systems.  Specifically, S&T is the primary component within the department  responsible for R&D of security technologies, as well as coordinating and  integrating all such activities of the department. TSA is responsible for  ensuring the security of all modes of transportation in coordination with  other federal entities, state and local governments, and the private sector,  including surface transportation systems. We have previously identified  challenges with S&T’s coordination and oversight of R&D projects.  Specifically, in March 2019, we reported that R&D coordination across  DHS had improved, but additional actions were needed to better track  and evaluate S&T R&D projects.

Given DHS’s past challenges managing R&D projects and the inherent  difficulties associated with monitoring and securing surface transportation  systems, you asked us to review federal R&D efforts to develop  technologies that strengthen security within the surface transportation  environment. This report (1) assesses the extent to which S&T is  developing technologies to secure surface transportation and what  progress it has made; (2) describes the extent to which TSA is testing  technologies to secure surface transportation; and (3) identifies the key  mechanisms that S&T, TSA, and stakeholders use to collaborate on  surface transportation capability gaps and share information on relevant  security technologies, and analyzes the extent to which they are effective.

To assess the extent to which S&T is developing technologies to secure  surface transportation and its progress to date, we identified all S&T  related efforts since 2010, when S&T began a research initiative focused  on surface transportation security technologies. We focused this review  on mass transit security because it was the only active S&T R&D effort  related to surface transportation since 2010. To collect information on  technologies developed we reviewed S&T documentation, including  technology performance reviews and project developmental milestones,  interviewed agency officials, and visited one of the nation’s largest mass  transit operators that was testing an S&T-developed technology. We  reviewed prior GAO reports on S&T’s management of R&D projects to  identify prior findings applying to S&T’s broader R&D portfolio that may  also apply to S&T’s management of its mass transit R&D program. In  addition, to better assess how S&T has managed challenges related to its  R&D program for mass transit security, we assessed S&T’s program  milestones that were reported to Congress for each fiscal year from 2013  through 2018 against DHS budget guidance for developing milestones.

To describe the extent to which TSA has tested surface transportation  security technologies, we reviewed agency documentation pertaining to  tested technologies, TSA budget information for fiscal years 2013 through  2018 (the most recent 5 year period when we initiated our work), and  interviewed agency officials. We also conducted two site visits to mass  transit operators to observe the testing and use of security products. We  focused our site visits on mass transit operators because TSA officials  told us they prioritize this mode over other surface transportation modes  when determining what to test. We selected mass transit operators for  site visits to obtain a variation in the type of commercial technologies  being tested.

To analyze the key mechanisms S&T, TSA, and stakeholders use to  collaborate on surface transportation capability gaps and share  information with each other on relevant security technologies, we focused  on mass transit operators because S&T and TSA have focused their  respective R&D and testing efforts on technologies designed to enhance  the security of mass transit systems. We identified mechanisms used by  S&T, TSA, and mass transit stakeholders (including operators and  industry associations) to promote collaboration on security-related issues.  We reviewed documentation related to these mechanisms, and  interviewed agency officials and mass transit stakeholders that held  memberships in these groups or participated in these processes. We  evaluated these mechanisms against leading collaboration practices. To  determine the extent to which information is shared on security  technologies, we assessed TSA’s efforts to share information because it  has responsibility for securing surface transportation and, thus has  established relationships with mass transit operators. We did not assess  S&T’s information sharing with mass transit operators because S&T  primarily interacts with operators based on their participation in TSA’s  testing program for surface transportation technology products. We  conducted semi-structured interviews with nine mass transit operators on  TSA’s efforts to share information. For these interviews, we selected  operators to reflect a variation in the size of transit systems, geographic  location, and membership in certain groups that promote collaboration on  mass transit security issues. Further, we reviewed documentation and  interviewed TSA officials on their efforts to share information on security  technologies with mass transit operators. We assessed these information  sharing efforts against federal internal control standards and criteria  within DHS’s National Infrastructure Protection Plan.

We conducted this performance audit from April 2018 to September 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Mass transit rail operators have primary responsibility for securing their  own systems. Unlike the aviation environment, where TSA has  operational responsibility for screening passengers and baggage for  prohibited items prior to boarding a commercial aircraft, TSA has no  operational role for securing mass transit, such as employing screeners  or purchasing and acquiring security equipment. Rather, TSA regularly  partners with mass transit operators to address their security needs by  conducting vulnerability assessments, sharing intelligence information  and best practices, and working to mitigate security risks to their systems  by assessing commercially available security technologies, among other  measures. Mass transit operators which can be public or private  entities, administer and manage all transit activities and services,  including acquiring and operating any technologies designed to augment  their existing security infrastructure.

Securing transit systems presents inherent challenges for mass transit  operators for numerous reasons. In general, mass transit systems are  designed to expedite the movement of large numbers of people through  multiple stations, situated along extended routes, and technologies used  in the mass transit environment should not disrupt the efficiency of these  operations. In addition, individual stations within these systems frequently  include multiple points of entrance and exit that vary in the extent to which  they may be accessed by passengers. For example, open systems  include walk-up platforms with little to no barrier to entry, while other,  more closed systems, typically include dedicated points of entry and exit  that allow or prohibit entry access through various mechanisms. Given  the size and complexity of these systems, it can be difficult for operator  personnel to comprehensively monitor them for security threats. Finally,  the large number of riders that pass through these systems during peak  hours generally makes the sustained use of some security measures and  technologies (e.g., metal detectors) difficult because such measures  could result in long lines that would disrupt scheduled service.

Though mass transit systems are difficult to secure, mass transit  operators commonly employ a number of standard security measures for  the transit environment, including: public awareness announcements and signage (e.g., reminders to  report unattended baggage or suspicious activities to operator  personnel immediately); use of canine teams; access controls, such as the use of lasers for intrusion detection;  station design (e.g., designing transit stations to limit recess areas  where bombs could be hidden, such as under a bench or a ticket  machine); and  video surveillance, which includes video cameras that transmit a  signal to a set of television monitors to display real-time footage of  transit system platforms, entrances, exits, etc. (see figure 1).

Requirements, Roles, and Responsibilities for Mass Transit R&D

Federal requirements for mass transit R&D efforts guide department and  agency efforts. Specifically, the Implementing Recommendations of the  9/11 Commission Act of 2007 requires that S&T, in consultation with TSA  and the Federal Transit Administration, carry out an R&D program to  improve the security of public transportation systems. Additionally,  Executive Order 13416, Strengthening Surface Transportation Security,  requires the Secretary of Homeland Security to coordinate research,  development, testing, and evaluation of technologies related to the  protection of surface transportation, including commercial off-the-shelf  products. To implement these requirements and other related activities  to help secure mass transit systems, DHS, S&T, and TSA, in coordination  with mass transit operators, have assumed the following roles and  responsibilities:  DHS. DHS carries out its requirement to coordinate department-wide  R&D (including that for mass transit and other surface transportation  modes) through its Integrated Product Team (IPT) process. The IPTs  themselves are comprised of officials across DHS components, and are  tasked with identifying DHS technology capability gaps, which are defined  as differences between a department or agency’s current capabilities and  those capabilities needed to perform its mission. Each IPT is responsible  for identifying capability gaps related to a broad security area (such as  preventing terrorism) and potential R&D efforts to address those gaps.  Capability gaps relevant to surface transportation, including mass transit,  fall under the Prevent Terrorism IPT and its Explosive Screening sub-IPT.  S&T’s Research Council receives IPT information on security gaps and is  responsible for prioritizing them across all IPTs. The results are used to  inform which R&D research projects S&T and DHS components will  undertake (see figure 2).

S&T. Once capability gaps are agreed upon and prioritized by DHS  components through the IPT process, S&T undertakes R&D projects  intended to address the highest prioritized capability gaps. S&T can  undertake R&D projects on behalf of any of the department’s  components, including TSA, and will either initiate development of a new  technology solution or coordinate or adapt existing technologies to meet  the project’s needs.

Once S&T’s R&D efforts result in a preliminary technology, or prototype,  S&T will begin the testing process. S&T’s technology development  process includes developmental and operational testing phases that are  carried out by staff at laboratories S&T contracts with. Developmental  testing is typically conducted in simulated environments, such as  laboratories, test facilities, or engineering centers, which can sometimes  be representative of the complex operational environment (i.e., an actual  subway station). Operational testing includes field tests performed under  realistic conditions by actual users (i.e., the transit operators testing in a  subway or rail station) and overseen by S&T in order to determine the  effectiveness and suitability of a prototype technology. Once testing is  complete, S&T contracts with staff at partner laboratories and works with  private sector industry partners on further developing the product for the  commercial market, where it can be purchased by mass transit operators  to help secure their systems.

TSA. While TSA does not conduct R&D for mass transit security, it does  sponsor testing of commercially available security technologies for the  mass transit environment. This testing can take place in both laboratory  environments and in operational, real-world environments. Regarding the  latter, TSA uses its Surface Transportation Operational Test Bed Program  (Operational Test Bed Program) to place commercially available security  technologies in surface transportation environments, such as mass transit  systems, for performance testing and evaluation. TSA established the  program to assess the effectiveness of emerging and existing security  technologies in real-world environments; verify prior laboratory testing  performance results versus performance in a TSA mass transit system (or  other surface transportation mode serving as a test bed); and develop  recommendations for use of certain technologies in surface  transportation. As part of the program, TSA establishes memorandums of  agreement with surface transportation entities that participate in the  program and also provides them with logistical support, such as installing  technology and providing personnel to help operators with technology  training and operating needs. As of 2019, there are nine mass transit  operators participating as test beds in the program (test beds), and TSA  officials reported working to add two more mass transit agencies as test  beds. In addition, there are five other surface transportation test beds,  including two pipeline and two freight rail test beds.

Mass Transit Operators. Mass transit operators generally do not have  dedicated portions of their budgets for, and therefore do not conduct,  R&D. However, selected operators work with TSA to test commercially  developed technology solutions intended to enhance their system  security. In general, mass transit operators must assume security-related  expenses for their systems, including the purchase or acquisition of  surface security technologies.

S&T Has One Surface Transportation R&D Program Under Way, but Is Not Following DHS Guidance to Track Its Progress

S&T Is Developing Technologies to Address Explosive Threats within Mass Transit Stations

In 2010, S&T’s Explosives Division began work on the Surface  Transportation Explosive Threat Detection (STETD) program to address  the threat of improvised explosive devices (IED) on persons or in objects  within a mass transit station. S&T officials stated that, as of fiscal year  2019, the STETD program remains S&T’s sole R&D program related to  surface transportation. In addition, it is the only DHS technology  development program focused on developing products to address the  threat of IEDs in a mass transit security environment.

The STETD program consists of four separate technologies designed to  address explosive threats within a mass transit station, each of which is in  a different stage of development and maturation. Specifically, the four  technologies are known as Forensic Video Exploitation and Analysis  (FOVEA), Standoff Detection, Real-Time Threat Detection Agent, and  Layered Architecture. The technologies are designed to address unique  aspects of the mass transit environment (i.e., multiple access points, lack  of access barriers, etc.) while also working together to provide IED  detection coverage from the point at which the passenger enters a mass  transit station, boards a train, and then finally exits the system. These  technologies would allow mass transit operators to scan large  unstructured crowds to detect concealed explosives worn or carried on a  person (person-borne IED), or placed in stationary objects such as baggage, or intentionally deposited in an unnoticed location for detonation  by a timer or remote control (leave-behind IED). S&T intends for STETD  technologies to perform without requiring checkpoint baggage screening  or other measures that could impede the traveling public moving through  mass transit systems during periods of high passenger volume (e.g., rush  hour).

FOVEA is a software suite designed to interface with video management  systems already installed in mass transit systems, and is intended to help  operators use recently recorded camera footage to quickly determine a  person’s movement through the system. S&T began developing the  technology in fiscal year 2013, and it is generally directed at helping  operators identify responsible parties when objects are left behind in a  mass transit system. Specifically, FOVEA includes a number of tools to  enable its video analysis (see figure 3).

As of December 2018, the FOVEA video suite has been installed in the  Washington Metropolitan Area Transit Authority’s Security Operations  Control Center. During our site visit to the control center to view the use  of the FOVEA suite, officials told us that FOVEA has enhanced the ability  of personnel to analyze video footage for active law enforcement  investigations. Officials also told us that because of this functionality,  FOVEA is a valuable tool. S&T anticipates it will transition the technology  to commercial development in fiscal year 2020.

S&T is developing a set of imaging sensors designed to scan  unstructured crowds to detect hidden potential threat items (e.g., a  person-borne IED) on travelers without requiring passengers to open  bags or remove outerwear. According to S&T, the sensor technology  would be placed in walls, near platforms, or other structures. These  screening devices are designed to unobtrusively scan for detailed  information on possible person-borne threat objects, such as wires  connected to a pressure cooker, and provide alerts to operators via the  Real-Time Threat Detection Agent (described below). Development of  these sensors began in fiscal year 2014, and since 2017, S&T officials  have been using the Massachusetts Bay Transportation Authority training  facility to test prototypes of the technology. S&T expects to transition the  technology to commercial development in fiscal year 2023.

The Real-Time Threat Detection Agent technology is intended to  automatically detect abandoned objects that could be potential threats,  and notify transit operators of their existence. Specifically, the system is  to analyze live video footage to identify, tag (i.e., mark on the video  footage), and track left-behind objects (i.e., baggage possibly containing  IEDs), as well as individuals associated with the object, without the need  for continuous human monitoring of video footage. To track potential IEDs  without human monitoring, the system is intended to have the capability  to identify abandoned objects that could be potential threats and compare  them against defined criteria for person-borne, as well as leave-behind,  IED threats. Based on its analysis, the system would then create alerts  and send them to operators, as well as to video review software, such as  FOVEA. Development of this system began in fiscal year 2013, and S&T  began developmental testing of the technology at Washington  Metropolitan Area Transit Authority facilities in October 2018. S&T  expects to transition the equipment to commercial development in fiscal  year 2021.

Layered architecture, the final component of the STETD program, is  intended to have the capability to integrate information from the various  existing security technologies utilized by a mass transit system to enable  more accurate threat identification. The goal of this component is to  gather input from multiple pieces of technology, such as distributed  sensors and tools used across a mass transit environment (to include  existing sensors and other STETD technologies), to present a  consolidated threat profile to operators in a command center. This  technological component is the least developed of the STETD program  technologies, with ongoing work focused on experimentation with different  prototypes. S&T expects to transition the technology to commercial  development in fiscal year 2023.

Since 2013, S&T’s funding for mass transit R&D has decreased, delaying  the development of associated technologies. Specifically, during fiscal  years 2013 through 2017 funding for the STETD program—the only DHS  R&D program focused solely on mass transit security—decreased by 78  percent, but then increased again in fiscal year 2018 (see figure 4).

According to S&T officials, one reason funding was reduced during this  period was to direct additional funds to R&D for a newly-identified threat.  Specifically, in fiscal years 2015 and 2016, following the landing of a  gyrocopter on the U.S. Capitol grounds and other incidents, unmanned  aerial systems became a significant and emerging threat, and a top DHS  priority. At the time, S&T had no funding allocated for a related R&D  effort, so S&T leadership subsequently redirected funding from the  STETD program toward R&D on unmanned aerial systems.

S&T officials told us that, due to fluctuations in funding, in addition to  other factors, the program’s completion date has shifted from 2017 to  2023, which has delayed efforts to make the technologies commercially  available to mass transit operators. Although S&T began increasing funds  for the STETD effort in 2018, according to program officials, decreases in  program funding have delayed program deliverables and pushed  timelines out. For example, according to S&T officials, lower levels of  program funding have made it difficult to employ highly-skilled contract  staff at the laboratories S&T partners with to carry out STETD R&D,  which has slowed the pace of development. S&T officials also stated  that a lack of funding and changes in funding slow down what is already a  technically challenging development effort. As S&T officials explained, the  STETD program is pushing the performance boundaries and capabilities  of existing technologies, and in some cases, inventing entirely new  technologies for screening highly trafficked environments.

S&T Has Not Developed Milestones to Effectively Track the Program’s Progress

S&T is not using milestones that fully adhere to DHS guidance for  milestone descriptions to track its progress on developing STETD  technologies. Specifically, DHS budget development guidance directs  DHS components to develop program milestones that are specific,  measurable, results-oriented and relevant, and time-bound. To be  results-oriented and relevant, milestones must clearly link to activities in  program strategy, budget, or other planning documents. Linking  milestones to such activities allows parties reviewing the milestones, such  as DHS leadership and Congress, to understand how achieving the  milestones move the development process forward overall.

We assessed all 22 STETD milestones that S&T has used to report  progress on the program from fiscal years 2013 through fiscal year 2018.  We found that 17 of the 22 milestones were not results-oriented as  required by DHS guidance because they did not clearly link to any key  activities described by STETD program documents. Specifically, one  STETD program document identified several key activities for completing work on the technologies, including a requirements development phase,  developmental testing, and operational testing, but STETD milestones did  not clearly link to these activities. For example,

One fiscal year 2018 STETD milestone for the layered architecture  technology was to conduct a simulation and analysis of layered  sensing configurations to optimize sensor placement and system  performance, to be completed by the end of fiscal year 2018. While  this milestone was specific, measurable, and time-bound, it did not  clearly link to a key activity (e.g., developmental or operational testing)  identified in the STETD program’s plan, and thus was not results- oriented.

Another milestone from 2013 was to demonstrate advanced leave- behind detection software in a mass transit system. The milestone  was to be met by the second quarter of fiscal year 2014, and  remained unmet as of March 2019. Because the milestone was not  results-oriented (i.e., it did not link back to activities in program  documents), it was unclear how failing to achieve the milestone  impacted, or potentially delayed, the overall technology development  process.

S&T officials explained because they are dealing with technology  innovation and invention, they plan to develop milestones that closely  align to program plans after they develop a potential technology solution  that is ready for developmental or operational testing. However, according  to STETD program plans, it can take several years for STETD  technologies to begin developmental testing. For example, one STETD  technology (layered architecture) is not expected to begin developmental  testing until the fiscal year 2021-2022 time frame. Therefore, under S&T’s  current practice, the program would not begin using results-oriented  milestones, clearly linked to program plans, for this STETD technology  until more than ten years after work on the program was initiated.  Furthermore, we found that for one STETD technology (FOVEA), the  program did not consistently use results-oriented milestones after the  technology began developmental testing. Specifically, in fiscal year 2015  S&T began developmental testing for FOVEA, but, two of three FOVEA  milestones reported in fiscal years 2017 and 2018 were not results- oriented.

Without milestones for the STETD program that reflect DHS guidance to  clearly link the milestone to key events in program planning documents,  Congress and DHS decision makers cannot fully assess whether the  STETD program is meeting its goals within identified time frames.

Additionally, DHS decision makers are not positioned to identify  adjustments that may be needed to facilitate the achievement of program  goals.

We previously recommended in March 2019 that S&T take steps to more  fully incorporate DHS’s budget development guidance, to include more  results-oriented milestones, for its R&D programs. DHS concurred with  this broader recommendation, and as of June 2019, is taking initial steps  to ensure its implementation.

TSA Prioritizes Tests of Technology Products, but Projected Funding Shortfalls May Reduce the Scope of Future Testing

TSA Prioritizes the Testing of Technology Products to Secure Mass Transit Systems through Its Operational Test Bed Program

TSA sponsors tests of commercial products at contracted partner  laboratories, as well as at mass transit stations and other surface  transportation venues through its Operational Test Bed Program. The  purpose of these tests is to inform surface transportation operators about  different technological products that could address their security needs  and to confirm whether the products will operate effectively. As part of  the testing process, TSA officials assist in product installation, hold  technical demonstrations, and provide training for mass transit officials.  Once product testing concludes, TSA officials document test results in  written assessments, which they make available to mass transit and other  surface transportation stakeholders to review upon request.

TSA officials told us they use a number of methods to identify products  that are currently available in the commercial marketplace and could be  tested. Officials stated they conduct market research on vendors currently  making technology products that could potentially meet the needs of  mass transit operators, such as portable screening devices used to detect  potential person-borne IEDs. To do so, officials maintain and utilize an  existing list of vendors, conduct market research on their products, attend  relevant symposiums and university conferences on technological  advancements, and solicit information on these products and their  capabilities from both vendors and operators who have used them. TSA  officials also work with national laboratories to assist with relevant  research on specific technological products and their capabilities that  could be good candidates for testing.

To determine which technology products to test, TSA prioritizes  technologies that can be used in the mass transit environment. TSA  officials told us that because of the level of risk facing mass transit  systems, they generally try to ensure that the commercial products that  are tested address capability gaps relevant to the mass transit  environment. In addition, TSA officials stated that while they try to  address all identified surface transportation capability gaps, due to limited  resources, they tend to select, on an annual basis, products for testing  related to the following gaps: anomaly and explosive detection, high  throughput threat detection, intrusion detection, infrastructure protection,  and chemical and biological threat security, all of which have applicability  to the mass transit environment. Moreover, TSA officials told us that  anomaly and explosive detection and high throughput threat detection are  the technology gaps that are critical to securing mass transit systems.

TSA officials also identify other criteria used for testing, including  performance requirements and vendor claims about their products’ ability  to address specific capability gaps.

Regarding performance requirements, TSA officials told us any products  selected for testing must meet a set of minimum performance  requirements in order to be considered appropriate for addressing the  unique security needs for different surface transportation modalities. For  example, for an explosive screening product used in mass transit, TSA  officials established requirements for probability of detection and  probability of false alarm. TSA officials stated they work with mass transit  operators and others to identify these requirements to ensure that IED  detection technologies can effectively identify threats without disrupting  mass transit operations. Lastly, TSA officials said they also try to select  technologies that can be used to secure multiple surface transportation  modes. For example, officials said they may select an intrusion detection  sensor for testing that could be adapted for securing pipelines or freight  rail yards.

TSA Tested Approximately 110 Existing Technology Products to Secure Surface Transportation Systems from Fiscal Years 2013 through 2018, but May Have to Reduce the Scope of Future Testing

From fiscal years 2013 through 2018, TSA sponsored laboratory and field  tests of approximately 110 commercial products that are designed to  address identified surface transportation capability gaps (such as  intrusion detection and explosive detection). These tests take place in  either of two environments—laboratory or field (i.e., within a mass transit  venue)—and are designed to address surface transportation security  capability gaps. Since 2013, 67 percent of the technology products (72  products) assessed by TSA focused on detection-related gaps, most of  which were related to intrusion detection. The remainder of products  tested addressed interoperable information systems or a combination of  capability gaps, such as anomaly and explosive detection and chemical  and biological threat security (see figure 5).

Of the products that addressed more than one capability gap, 78 percent  (14 of 18) of these products could be used for anomaly and explosive  detection, as well as high throughput threat detection. TSA officials told  us that because anomaly and explosive detection and high throughput  threat detection technologies can be easily transported to different  locations within a station, they are of particular interest to mass transit  operators.

On our site visits to two mass transit operators that TSA utilizes to test  technology products, we observed the testing and use of commercial  products. These products were designed to detect anomalies on the  underside of railcars as well as among persons traversing transit  platforms, terminals, and stations. The products were being used to  support both normal operations and a national security special event (see  figure 6).

Mass transit officials told us during our site visits that they considered  these commercial technologies to be useful additions to their existing  security measures. They also said the test results, which TSA made  available to them, were helpful in determining whether to invest in  purchasing the products for long-term use. In addition to allowing TSA to  perform technology assessments, the program also gives transit  operators hands-on experience with technologies they are unfamiliar with.  For example, an official from one mass transit test bed told us that  equipment often performs differently when the manufacturer’s employees  are operating it due to their prior experience with, and dedicated training  on, the equipment. This official noted that transit system employees, who  often do not have similar experience and training with a particular  technology, can sometimes have different performance results when  operating this equipment. The official told us that TSA’s Operational Test  Bed Program gives transit employees an opportunity to develop  structured, hands-on experience with certain products with TSA’s  assistance, allowing them to understand the full potential and capabilities  of the technology. This official said that, in her case, observing fellow  transit employees using a particular technology for several hours  convinced her of the product’s application in a real-world environment,  and subsequently was an important factor in her decision to recommend  its purchase for use by her transit agency.

Although mass transit operators we spoke with valued the Operational  Test Bed Program, TSA has decreased funding for the program since  fiscal year 2013. Specifically, our analysis of program funding showed  that the program experienced an approximately 70 percent decrease in  funding from fiscal years 2013 through 2018 (see figure 7).

TSA officials stated that recent decreases in program funding, coupled  with projected funding shortfalls for the Operational Test Bed Program for  2019 through 2024, will limit the program’s capacity to conduct testing  and assessments of technologies. Specifically, the TSA program  manager for the Operational Test Bed Program told us that the recent  decreases in funding for the program to its current level will materially  impact the operation of the program moving forward. Furthermore, a TSA  May 2019 budget planning document shows that, to fully meet project  requirements, the program will require approximately $20 million in  additional funding for fiscal years 2019 through 2024. Should the program  not receive this funding, TSA officials stated they would not be able to test  as many products or address as many surface transportation capability  gaps through the program. They also stated that the funding shortfalls  would limit TSA’s analysis of technology performance. Program  managers are in the process of identifying additional funding  requirements for the program through TSA’s internal budget review  process.

Mass Transit Stakeholders Effectively Collaborate to Identify Capability Gaps and Test Security Technologies, but TSA Does Not Comprehensively Share Technology Assessments

S&T, TSA, and Mass Transit Operators Have Effectively Collaborated to Identify Mass Transit Capability Gaps and Test Technology Solutions through Four Key Mechanisms

S&T, TSA, and mass transit operators regularly collaborate on issues  related to identifying mass transit capability gaps and testing security  technologies to address those gaps. During the course of our review, we  identified four key mechanisms that S&T, TSA, and mass transit  operators use to collaborate on mass transit security issues—the DHS  IPT process’s sub-IPT focusing on surface transportation capability gaps  (including those pertaining to mass transit); the Intermodal Transportation  Systems Research and Development Working Group (RDWG); TSA’s  Operational Test Bed Program; and the Transit Policing and Security  Peer Advisory Group (PAG) (see table 1).

We assessed the effectiveness of collaboration in these four mechanisms  using our leading collaboration practices (see side bar) and found that  each of them generally followed these practices. We reported in March  2019 that DHS-wide R&D collaboration has improved through the IPT  process but some challenges remain, such as ensuring all components  participate in the process, among other things. However, as discussed  below, we found that S&T and TSA collaborate effectively through the IPT  process for identifying mass transit security capability gaps and security  technologies. sustained over the long-term? If  leadership is shared, have roles and  responsibilities been clearly identified and  agreed upon?

Have participating agencies clarified roles  and responsibilities?  participants been included? Do they have  the ability to commit resources for their  agency?  mechanism be funded and staffed? Have  online collaboration tools been  developed?

Written Guidance and Agreements: If  appropriate, have participating agencies  documented their agreement regarding  how they will be collaborating? Have they  developed ways to continually update and  monitor these agreements?

Prevent Terrorism sub-IPT on Explosive Screening. S&T and TSA  collaborate on identifying surface transportation (including mass transit)  capability gaps through the Prevent Terrorism’s sub-IPT on Explosive  Screening. At the federal level, we found that TSA and S&T’s ongoing  use of this mechanism met several leading collaboration practices,  including bridging organizational cultures, leadership, clarity of roles and  responsibilities, participants, and written guidance and agreements.  Specifically, the Explosive Screening sub-IPT has a formal structure that  is outlined in the Prevent Terrorism IPT’s charter. This written agreement  establishes leadership and clarifies the roles and responsibilities of each  of the participants. As key participants and voting members, S&T and  TSA possess the necessary expertise to identify capability gaps for mass  transit (i.e., S&T has expertise in technology research and development,  and TSA has in-depth knowledge of the mass transit and other surface  transportation sectors). Moreover, by requiring S&T and TSA to work  together to prioritize capability gaps, the process allows them to operate  across agency boundaries (i.e., to bridge their respective organizational  structures). In 2012, S&T officials stated that they collaborated with TSA  as members of the Explosive Screening sub-IPT to identify anomaly and  explosive detection and high-throughput threat detection as the highest  priority capability gaps for surface transportation. These gaps were the  basis for S&T’s STETD program.

RDWG. We also found that the RDWG facilitates effective collaboration  between S&T, TSA, and surface transportation stakeholders, including 30  mass transit operators. The RDWG generally follows leading  collaboration practices related to bridging organizational cultures,  leadership, clarity of roles and responsibilities, participants, resources,  and written guidance and agreements. Specifically, the RDWG is a  working group with an established charter that brings together federal and  surface transportation representatives to operate across agency and  sector boundaries to identify surface transportation-related capability  gaps, including those for mass transit. The RDWG charter clarifies the  participant roles and responsibilities and establishes a framework for  nominating new members to ensure all relevant participants are included.  TSA, as the designated chair, funds the working group and ensures its  continuation. In addition to identifying surface transportation security  gaps, an S&T official told us that the members of the RDWG review the  prior year’s security capability gaps to determine whether they are still  relevant and if there are commercially available technologies to address  them. The S&T official explained that they then use the results of these  reviews to inform their work on the Explosive Screening sub-IPT,  specifically using them as basis for R&D project requirements. For  example, S&T officials said that anomaly and explosive and high- throughput threat detection were the gaps identified by the participants of  the RDWG and then reported through the DHS IPT process, which  ultimately helped inform the scope of S&T’s STETD program. In addition,  S&T officials told us that they use the RDWG annual meetings to  communicate to surface transportation stakeholders the progress they  have made to address these gaps through the STETD program.

Operational Test Bed Program. TSA’s Operational Test Bed Program  facilitates collaboration between S&T, TSA, and mass transit operators on  the testing and evaluation of security technologies. The program, through  its memorandums of agreement with mass transit operators, generally  follows leading collaboration practices related to collaboration criteria for  bridging organizational cultures, clarity of roles and responsibilities,  participants, resources, and written guidance and agreements.  Specifically, memorandums of agreement serve as a mechanism for TSA  and S&T to operate outside of their agency boundaries to test  technologies in real-world environments (i.e., mass transit systems). The  agreements also clarify the roles and responsibilities and serve as  guidance for TSA, S&T, and mass transit operators on how testing is to  be carried out. For example, the agreements clarify responsibilities for  installing and operating test equipment. The program is funded and  managed by TSA, and S&T and mass transit operators leverage TSA’s  resources through their participation. Collaboration through the program  has led to numerous benefits for TSA, S&T, and mass transit operators.  According to TSA officials, the program allows TSA to fulfill federal  requirements to test and evaluate mass transit security technologies and  to expand the market for these products. Additionally, S&T has used the  program’s established agreements with transit systems to facilitate the  testing of STETD program prototypes. Lastly, transit operators use the  program to obtain first-hand information on the performance of security  technologies within their system. For example, an official from one mass  transit operator participating in the test bed program told us they  purchased two different types of passive millimeter wave scanners that  they tested through the program and found to be effective.

Transit Policing and Security Peer Advisory Group (PAG). The PAG  facilitates collaboration among mass transit stakeholders to share  information and meets the bridging organizational cultures, leadership,  participants, resources, and written guidance and agreements  collaboration key features criteria. Specifically, the PAG follows several  practices through its charter, which designates a transit police chief as the  chair of the group. The charter also outlines the participating mass transit  stakeholders and allows them to share information across agency  boundaries on security-related issues, including information on incident  response, emerging threats, and other best practices on mitigating  security issues. Officials from all nine of the mass transit operators we  spoke with are members of the PAG, and seven of them stated that the  PAG fosters collaboration between mass transit operators by providing a  forum for transit officials to connect and share information. Particularly,  one mass transit operator stated that the PAG was beneficial to the safety  and security of her system because it has allowed mass transit officials to  share experiences and disseminate best practices for responding to  threats. Another official said that the PAG helped him stay informed about  the current risks facing all of the mass transit operators, and without the  group, collaboration probably would not happen.

TSA Engages in Collaborative Activities to Share Information on Security Technologies, but Does Not Comprehensively Share Technology Assessment Information

TSA engages in a number of collaborative activities to share information  on security technologies with mass transit stakeholders to help improve  their technology investments. Specifically, TSA disseminates a quarterly  newsletter to surface transportation stakeholders (including mass transit  operators) which summarizes TSA’s efforts to address various surface  transportation security issues. Among other things, the newsletter shares  information on the technologies for which TSA has sponsored testing in  different mass transit operator systems. Additionally, TSA maintains a  collective email account that was created for all mass transit operators to  send TSA suggestions for technologies or products to test, among other  things. Further, TSA officials stated that they notify and communicate to  regional mass transit stakeholders any information on upcoming test bed  demonstrations, so these stakeholders can attend in person if they prefer.  Finally, according to the TSA program manager, TSA officials utilize the  American Public Transportation Association’s annual conference, industry  symposiums, and security roundtables to engage with mass transit  stakeholders to share information on security threats, capability gaps, and  technology.

In addition, to assist transit operators, TSA produces a number of  assessments and reports (products) that include performance information  on a range of technologies. These products include:

TSA’s Market Survey. TSA officials regularly update a market  survey, which contains a list of commercial vendors who develop  technological products capable of addressing surface transportation  security issues. This list, which does not contain sensitive information  and may be readily shared with operators, includes vendors and their  associated products that TSA believes may be applicable to mass  transit security; it does not catalog the list of products TSA has  sponsored testing for. TSA officials populate this list by attending  relevant symposiums and university conferences, as well as soliciting  input from partner laboratories.

Surface Transportation Sensor Catalog. The Surface  Transportation Sensor Catalog documents the technology  assessments performed by TSA and contains summaries of various  security technologies evaluated since 2007. In addition to evaluated  products, it also contains summaries of the vendor product  demonstrations received by TSA since fiscal year 2016. TSA  officials stated that the catalog is updated each year with new entries  for recently evaluated products, and is intended as a resource for both  TSA and its stakeholders to have greater awareness of technologies  and help them make more informed technology investment decisions.

State of Technology Reports. TSA publishes a State of Technology  report that provides a detailed overview of a specific challenge to  surface transportation stakeholders (such as person-borne IEDs) and  gives a high-level summary of available products that could address it,  and a technology maturation roadmap (with objectives) that needs to  be implemented in order to meet surface transportation stakeholders’  operational and security needs to address those specific threats.

Although these TSA products contain technology assessment and other  information that would benefit mass transit operators seeking to purchase  and implement security technologies, mass transit operators may not be  receiving them. Specifically, TSA shares these products with transit  operators upon request. However, officials from seven of the nine mass  transit operators that we spoke with said they wanted more technical  assessment information on commercially available security technologies,  indicating that they may not be routinely requesting, and therefore not  receiving, the TSA products that would provide this information. In  addition, four of the nine said they would like TSA’s assessment  information on technologies to be more accessible. Finally, an official  from one mass transit system who previously worked for TSA on mass  transit issues, and thus has knowledge of the broader mass transit  community, stated that many operators would benefit from the Surface  Transportation Sensor Catalog, but smaller operators are not aware of  this resource and therefore do not know how to request it.

TSA officials stated they do not routinely share information on security  technologies with mass operators for two reasons. First, TSA officials  explained that many of the in-depth reports that result from its testing of  security technologies contain sensitive information and cannot be  distributed without first assessing whether the requester is eligible to  receive it. However, officials from most of the mass transit operators we  spoke with said they would like more technical assessment information.  Therefore it could be useful for mass transit operators to know when TSA  publishes these reports so they can request the full report for review. This  notification could consist of non-sensitive, high-level information on  technologies assessed so that mass transit operators could request the  information in full. Second, the TSA program manager responsible for  these assessments told us that his office does not have sufficient  resources to develop and maintain a centralized, web-based repository  that would allow mass transit operators to search and retrieve sensitive  information independently, such as the sensor catalog and technology  assessment reports.

Despite these limitations, in the past, TSA has shared information related  to technology assessments routinely with mass transit operators. For  example, TSA used to post a verified technology list on a Federal  Emergency Management Agency web page. A TSA official stated that the  information posted on the website included summary information on  technology evaluations and other technology information. Further, TSA  received feedback from surface transportation (including mass transit)  stakeholders that the information posted was useful. TSA no longer posts  information on the web page because the Federal Emergency  Management Agency no longer maintains the website. In addition, TSA  officials stated that they had plans to include more comprehensive  information about TSA’s technology assessments within an online  information resource known as the DHS Responder Knowledge Base—a  department-wide database previously developed to house information for  first responders. In April 2019 officials from DHS’s Countering Weapons  of Mass Destruction Directorate said they had plans to reach out to TSA  and other DHS components on how to utilize the Responder Knowledge  Base as a repository for their reports and other sensitive information.  Furthermore, officials from that directorate told us that the database is  about 2 years from being launched, and they do not have a specific  completion date.

Standards for Internal Control in the Federal Government states that  management should communicate externally to their stakeholders  through the appropriate means. Further, the 2013 National  Infrastructure Protection Plan states that in order to ensure that situational  awareness capabilities keep pace with the evolving risk environment,  officials should improve practices for sharing information that will improve  security and resilience. Until the Responder Knowledge Base is  operational, mass transit operators could benefit from TSA routinely  sharing appropriate information on the technology assessments and other  performance information at its disposal. For example, TSA could leverage  the resources of existing coordination mechanisms, like the PAG, or  develop a listserv to automatically notify a more comprehensive group of  mass transit operators of the existence of a new technology assessment  or sensor catalog. Notifying more mass transit operators on an ongoing  basis that this information is available would help ensure they have the  benefit of all relevant TSA information when making strategic security  technology investments. Further, doing so would help mass transit  operators to better use their limited resources to acquire proven  technologies that could enhance the overall security posture of their  systems.

Monitoring and securing surface transportation systems continues to  present unique challenges. With respect to mass transit systems, for  example, operators must balance the need to efficiently move passengers  through the system with the need to screen for explosives and other  threats. Since 2010, S&T’s STETD program has been the only DHS R&D  program that has developed technologies to address these challenges.  Although S&T has made progress, as of fiscal year 2019, none of the  technologies associated with the STETD program have matured enough  to undergo commercial development, and the program’s completion date  has been extended from fiscal year 2017 to fiscal year 2023. While  fluctuations in the program’s funding have contributed to delays, S&T has  not followed DHS guidance for developing milestones that would help  officials understand whether the program is achieving key activities  identified in planning documents when faced with funding and other  challenges. Without milestones that clearly convey an understanding of  the program’s progress, DHS decision makers are not positioned to  identify any adjustments that may be needed to facilitate the achievement  of program goals.

S&T, TSA, and mass transit operators effectively collaborate through a  number of stakeholder groups to identify mass transit security gaps and  to test possible technology solutions that could address them. TSA also  supports greater awareness of available technologies by publishing key  information on commercially available products (such as technology  assessment results) and making it available to mass transit operators  upon request. However, TSA does not comprehensively or routinely  share this information, and seven of the nine mass transit operators we  spoke with stated they wanted more technology assessment information.  Without a mechanism to share technology assessments and related  information with more mass transit operators and on a routine basis, TSA  cannot ensure that mass transit operators will be fully informed about  available technologies they could use to secure their systems. Moreover,  without this information, mass transit operators may not be positioned to  make the best possible use of the limited funding available for purchasing  these technologies.

We are making two recommendations, one to DHS and one to TSA.

The Deputy Secretary of Homeland Security should ensure that S&T take  steps to more fully incorporate practices for developing milestones within  DHS’s budget preparation guidance, into the Surface Transportation  Explosive Threat Detection program. (Recommendation 1)

The Administrator of TSA should develop a mechanism to more routinely  and comprehensively share appropriate information on the performance  of mass transit security technologies (such as the annual sensor catalog  and security technology assessments) with mass transit operators and  stakeholders until DHS completes work on a more permanent information  sharing resource. (Recommendation 2)

We provided a draft of this report to S&T and TSA for review and  comment. DHS provided written comments which are reprinted in  appendix I. In its comments, DHS concurred with both recommendations  and described actions planned to address them. S&T and TSA also  provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees and the acting Secretary of Homeland Security. In addition,  the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact William Russell at (202) 512-8777 or russellw@gao.gov. Contact  points for our Office of Congressional Relations and Public Affairs may be  found on the last page of this report. Key contributors to this report are  listed in appendix II.

Appendix I: Comments from the U.S. Department of Homeland Security

Appendix II: GAO Contact and Staff Acknowledgments

William Russell (202) 512-8777 or RussellW@gao.gov.

In addition to the contact named above, Christopher Ferencik (Assistant  Director), Mona Nichols Blake (Analyst in Charge), Jason Blake,  Frederick K. Childers, Michele Fejfar, Jonathan G. Felbinger, Eric  Hauswirth, Tracey King, Kristiana D. Moore, Claire Peachey, Jack  Sheehan, Sarah Veale, and Robert Ward made key contributions to this  report.

The Department of Homeland Security’s Science and Technology Directorate and the Transportation Security Administration (TSA) research, develop, and test technologies to address threats to mass transit systems.

In particular, TSA tests the effectiveness of commercially available technologies that could help secure mass transit systems, and produces written assessments of these products. However, TSA does not routinely or comprehensively share its assessments with the mass transit operators.

We recommended that it do so to better inform these operators about the capabilities of technology they may buy to secure their systems.