Public-Safety Broadband Network: Network Deployment Is Progressing, but FirstNet Could Strengthen Its Oversight

Public-safety officials such as police officers and firefighters rely on communications systems to do their jobs. The Department of Commerce's FirstNet must establish a nationwide public-safety broadband network for use by these officials. In March 2017, FirstNet awarded a 25-year, multibillion-dollar contract to AT&T to deploy, operate, and maintain the network. AT&T must meet milestones specified in the contract, such as for providing network coverage and for the network's adoption. FirstNet's oversight of AT&T's progress and performance is critical given the contract's scope and duration.

GAO was asked to review FirstNet's progress and oversight. GAO examined the extent to which (1) AT&T is meeting milestones for the network's coverage and adoption and (2) FirstNet is overseeing AT&T in accordance with key practices. GAO analyzed FirstNet and AT&T documentation; assessed FirstNet's oversight efforts against key contract-oversight practices identified in federal regulations and other government, academic, and industry guidance; and assessed the program's master schedule against GAO best practices. GAO interviewed FirstNet officials, and selected state, local, and tribal officials and first responders representing a variety of viewpoints. Although not generalizable, they provided useful perspectives.

AT&T is meeting—or on track to meet—all nationwide, contractual network coverage and usage (adoption) milestones for the First Responder Network Authority (FirstNet) public-safety broadband network. AT&T has met the first nationwide coverage milestone (20 percent of the final expected coverage by March 2019), but coverage varies across states. Similarly, AT&T is on track to meet the first nationwide adoption milestone (which is to have a certain number of devices connected to the network by March 2020). AT&T has exceeded adoption targets in most states but lags in others. According to FirstNet officials, variances by state are allowable, as the key milestones are nationwide.

FirstNet uses various mechanisms to oversee AT&T; many of which align with key contract-oversight practices. For example, FirstNet uses a quality assurance surveillance plan to evaluate AT&T's performance. However, GAO found that FirstNet lacked (1) a reliable master schedule to review, (2) communication with relevant stakeholders regarding contract oversight, and (3) meaningful information on end-users' satisfaction to gauge performance quality.

Schedule. AT&T is required to provide a current master schedule to FirstNet monthly, but the schedule only partially or minimally meets the characteristics of a reliable schedule per GAO best practices. For example, the schedule only partially captures all activities or the duration or sequence of activities. Key practices call for tracking a contractor's progress toward the expected schedule. Having a more detailed schedule to review could improve FirstNet's insight into AT&T's deployment and strengthen FirstNet's use of the schedule as a management tool.

Stakeholder communication. Numerous public-safety officials GAO interviewed were dissatisfied with the level or quality of information received from FirstNet, noting that FirstNet had communicated little to no information on AT&T's progress or FirstNet's oversight. FirstNet officials said there is no contractual requirement to share such information, but key practices call for communicating appropriate information to relevant stakeholders and reporting on monitoring results. The lack of information has left stakeholders speculating about what, if any, oversight FirstNet conducts; sharing more information about the oversight FirstNet conducts could improve public-safety sentiment for and support of the program.

End-users' satisfaction. FirstNet collects some information that could relate to end-users' satisfaction, but this information provides limited insight into users' experiences. For example, AT&T surveys some users to ask whether they would recommend FirstNet services, but a user might do so due to limited alternatives, not satisfaction. Although end-users' satisfaction is not a performance quality measure in the contract, key practices call for using end-user satisfaction information as a metric to gauge performance quality. By not using this information to inform FirstNet's oversight or related activities, FirstNet could be missing an opportunity to increase assurance of the program's long-term success.

This is a public version of a sensitive report that GAO issued in December 2019. Information that FirstNet deemed proprietary has been omitted.

GAO is making four recommendations, including that FirstNet ensure the schedule aligns with GAO best practices, share additional oversight and other information with appropriate stakeholders, and utilize end-user satisfaction information to gauge performance. FirstNet agreed with GAO's recommendations.

Public-safety officials, especially first responders such as law- enforcement officers, firefighters, and emergency medical-services  personnel, rely on communication systems when conducting daily  operations, overseeing planned events, and responding to emergencies.  To help address long-standing service issues and challenges with the  interoperability of the differing systems used by public-safety entities in  differing jurisdictions, the Middle Class Tax Relief and Job Creation Act of  2012 (the 2012 Act) created the First Responder Network Authority  (FirstNet) as an independent entity within the Department of Commerce’s  (Commerce) National Telecommunications and Information  Administration, and tasked FirstNet with establishing a nationwide,  wireless, public-safety broadband network (hereafter, the network). The  network is intended to meet first responders’ needs for reliable voice and  data communications service and foster greater interoperability among  public-safety entities by establishing a single broadband network  dedicated for their use.

In March 2017, FirstNet entered into a public-private partnership with  AT&T, awarding a 25-year contract for the network’s deployment. As part  of this contract, AT&T will receive $6.5 billion in funding and exclusive  access to 20 megahertz (MHz) of spectrum reserved for the network to  operate on. In exchange, AT&T must deploy, operate, and maintain the  network, including providing the technical solution—the overall design,  development, production, operation, and evolution of the network—and  business functions—the marketing, product management, sales, and  customer service. In deploying the network, AT&T must meet various  milestones established in the contract, such as for providing certain levels  of network coverage at agreed-upon intervals. AT&T plans to invest about  $40 billion in the network over the life of the contract using revenue from  the subscription fees that network users will pay, among other sources.  Use of the network by public-safety entities and officials (i.e., “adoption”)  is voluntary, although the contract establishes goals for adoption as  measured by the number of devices connected to the network. The size  and scope of the project, duration of the contract, and the federal  government’s history of acquisition-management challenges suggests the  need for strong oversight.

You asked us to review FirstNet’s progress in establishing the network  and its efforts to oversee AT&T. This report examines the extent to which  (1) AT&T is meeting the established milestones for deploying the network,  including coverage and adoption goals, via its contract with FirstNet, and  (2) FirstNet is overseeing AT&T’s deployment of the network in  accordance with key practices.

This report is a public version of a sensitive report that we issued in  December 2019. FirstNet deemed some of the information in our  December report to be Controlled Unclassified Information, which must  be protected from public disclosure. Specifically, the December report  contained proprietary business information owned by AT&T related to  network coverage maps, contractual coverage and cell site requirements  and delivery, and device connections that was provided to FirstNet  pursuant to contractual terms preventing further release; this report omits  such sensitive information. Although the information provided in this  report is more limited, the report addresses the same objectives as the  sensitive report and uses the same methodology.

To address these objectives, we reviewed the 2012 Act; the FirstNet- AT&T network contract, corresponding task orders, and relevant  information contained in FirstNet’s contract files; as well as other contract- related documentation. This information and other documentation  included, for example, reports and other records submitted as  “deliverables” by AT&T to FirstNet; contract oversight plans, manuals,  guidance, and other items; and board-meeting materials, reports to  Congress, and fact sheets. In reviewing the deliverables documentation,  we analyzed data as of September 2019 that related to nationwide and  state-level network coverage and adoption. We assessed the reliability of  these data by asking FirstNet officials questions about how they review  the deliverables and about data sources, quality, and timeliness, as well  as by electronically testing the dataset for missing or invalid entries. We  found these data reliable for the purpose of describing progress toward  coverage and adoption milestones. In conducting our analyses, we  focused primarily on the task orders and network-deployment phases  (and associated milestones) that were most relevant to network coverage  and adoption and underway at the time we began our review. We also  conducted case studies of seven states to obtain greater context on  progress. We selected the case studies to include states with very high  density counties or a large number of low-population density counties;  high poverty rates (due to budgetary challenges public-safety entities may  face); different rates of network deployment progress at the time of our  selection; and geographic diversity and tribal lands. The selected states  represent almost a third of the contract dollars allocated for network  coverage deployment, but deployment progress in these states is not  generalizable to the network as a whole.

We assessed FirstNet’s oversight efforts against key acquisition and  contract-oversight practices established in federal acquisition regulations  and other government, academic, and industry guidance on contract  oversight. We selected those practices that were most appropriate given  FirstNet’s contract approach and the stage of the acquisition process  FirstNet was in during the course of our review. We also compared the  network’s integrated master schedule, which AT&T provides to FirstNet,  to scheduling best practices in GAO’s schedule guide. In doing so, we  reviewed the schedule as of January 31, 2019 (the most current at the  time we began our schedule analysis), and the schedule dictionary and  work breakdown structure, among other project documents. We  interviewed FirstNet officials to obtain additional information and  observations on progress and oversight.

We also conducted about 40 interviews with state, local, and tribal  officials and first responders to obtain their perspectives on variations in  state-level coverage and adoption, experiences using the network, and  FirstNet activities. We interviewed state officials in each of our case- study states and received information from other states via a group  discussion and written responses to questions we posed. We also  selected state, local, and tribal public-safety entities from different public- safety disciplines (e.g., law-enforcement, fire, emergency-medical  services) within our case-study states, generally by reviewing AT&T  subscriber management documentation and selecting among the largest  subscribers (at the time of selection) and ensuring representation among  urban, suburban, and rural areas. These stakeholders’ views represent a  wide cross section of geographies and network users but are not  generalizable to those of all FirstNet stakeholders. Appendix I describes  our scope and methodology in greater detail.

We conducted this performance audit from November 2018 to November  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. We subsequently worked  with FirstNet from December 2019 to January 2020 to prepare this non- sensitive report for public release. This public version was also prepared  in accordance with these standards.

The FirstNet-AT&T network contract and its associated task orders define  the requirements AT&T must meet. The contract currently involves five  task orders, four of which relate directly to the network’s deployment.

Task orders 1 and 2 (actions complete). Required AT&T to develop  and deliver individual network deployment plans for each of the 56  states, territories, and the District of Columbia (hereafter, states).  The governor of each state had the opportunity to review the plan and  opt in to allow FirstNet and AT&T to build the network in their state. All  governors opted in by the applicable deadline. The result of this  process was a state deployment plan that included state-specific  commitments made by AT&T.

Task order 3 (actions ongoing). Requires AT&T to deploy, operate,  and maintain the network’s “core” and all of its functions, and provide  for the development of device and application ecosystems for the  network. A network core consists of national and regional data centers  and other elements that store, process, and secure network users’  traffic (activity), and interface with federal, state, and local networks.  AT&T deployed the core in March 2018. The network uses the  spectrum reserved for public-safety use (“Band 14”), as well as the  spectrum that AT&T’s existing, commercial network operates on.  When Band 14 spectrum capacity is not being used by public-safety  users, AT&T can use the excess capacity for its non-public-safety,  commercial-network users. As such, among the functions that task  order 3 provides for are capabilities that allow prioritizing a public- safety user’s network access and traffic over other users and, when  necessary, preempting other users altogether. These functions are  commonly referred to as “priority and preemption.”

Task order 4 (actions ongoing). Requires AT&T to deploy the  network’s Band 14 coverage in the states, including building the  “radio-access network” in each state that connects to the network’s  core and backhaul (which carries network users’ traffic) and fulfilling  the state-specific commitments. Radio-access networks consist of  cell towers, sites, and other elements that connect network-users’  devices to the network core. This task order also requires AT&T to  provide 72 “deployable” cellular assets—meaning, transportable  equipment (typically in a vehicle) that can provide additional network  coverage when needed—dedicated solely for FirstNet network users.  The task order also provides for access to at least 300 additional  deployables in AT&T’s fleet.

The contract and task orders 3 and 4 outline a phased approach for  deploying the network’s capabilities and coverage (in both non-rural and  rural areas), with five “initial” operating-capability phases that build to a  “final” operating capability expected in 2023, as well as ongoing  performance, maintenance, and continuous improvement through 2042.  As described further below, each phase provides for increased  capabilities and coverage—and some outline goals for network user  adoption—and AT&T must meet certain required milestones in each  phase to receive payment for that phase from FirstNet. Figure 1 depicts  the phased timeline for task orders 3 and 4.

The Band 14 spectrum on which AT&T is building the network is a key  component that differentiates it from other commercial networks, as the  network’s full capabilities and functionality are only available via Band 14.  For example, certain high-power user equipment can transmit at stronger  signals; this signal increase can only be done using the Band 14  spectrum. However, at its expected final operating capability, the network  using Band 14 spectrum will not cover the entire country. Public-safety  network users will also have access to the non-Band 14 LTE spectrum  that AT&T uses for its existing, commercial network (with priority and  preemption), though this spectrum does not have all the full capabilities of  Band 14, as in the high-power user equipment example above. According  to AT&T, when including this non-Band 14 spectrum, the network will  cover 76.2 percent of the U.S. geographically and around 99 percent of  the population. Network users are to also have access, by request, to  deployable assets that can provide temporary coverage when needed,  such as in remote and wilderness areas that will not have permanent  coverage.

FirstNet’s Contractor Is Meeting, or Is on Track to Meet, All Nationwide Contractual Coverage and Adoption Milestones, but State- Level Progress Varies

AT&T Has Met the First Nationwide Coverage Milestone but Coverage Is Not Uniform across States

AT&T has met, and exceeded, the first required nationwide network- coverage milestone. According to FirstNet documentation, AT&T is  required to meet certain coverage milestones in both non-rural and rural  areas and by the end of March 2019, AT&T had met the requirement to  provide at least 20 percent of the total expected Band 14 coverage in  both non-rural and rural areas. The Band 14 coverage milestones that  AT&T is contractually required to meet to receive payment increase each  year through March 2023, when AT&T is to have completed 100 percent  of the total expected Band 14 coverage. For example, by March 2021, the  coverage milestones are 80 percent of the total expected Band 14  coverage in both non-rural and rural areas and by March 2022, 95  percent. Per the terms of the contract, prior to meeting the first  milestone, AT&T provided initial coverage via its existing, commercial  wireless network and made 72 deployables (such as mobile cell sites on  trucks) available for network users. AT&T fulfilled the deployables  requirement through a combination of deployables built specifically for  network users and others allocated from AT&T’s existing fleet of  deployables used for disaster relief.

Specifically, to complete the first coverage milestone, AT&T delivered  Band 14 coverage in about 63 percent of the total square miles required  by 2023 in non-rural areas, and about 21 percent of the total square miles  required by 2023 in rural areas, according to FirstNet documentation. For  meeting this milestone, FirstNet paid AT&T approximately $1.2 of the  $6.5 billion. Since completing this milestone, AT&T has continued to  expand coverage and, according to FirstNet officials, is also on track to  meet the next coverage milestone (due March 2020) early, although  FirstNet was in the process of completing final verification and validation  activities as of September 2019.

AT&T constructed or “delivered” (i.e., these sites are all on-air) thousands  of Band 14 cell sites to produce the level of coverage needed to meet the  March 2019 milestone. Since then, according to FirstNet documentation,  AT&T has continued adding Band 14 sites, delivering—as of July 2019— more than one-third of the total Band 14 cell sites planned for the entire  network. AT&T may deliver these cell sites through a combination of  constructing new sites, retrofitting existing AT&T sites, or acquiring or  contracting with local providers, such as rural telecommunications  carriers. Although FirstNet tracks the status of planned cell sites (such as  which sites are undergoing environmental policy review or are currently  operational, or on-air), cell sites are not an explicit part of the contractual  coverage milestones required for AT&T to receive payment. That is,  AT&T’s payment is not contingent upon getting a certain number, type, or  location of cell sites on-air, but rather the amount of coverage (in square  miles) provided on a nationwide level by these sites.

While AT&T met the first coverage milestone and has delivered more  than a third of the planned cell sites nationwide, AT&T also has state- specific commitments. These commitments or targets, like the delivery of  sites, are not explicit contractual payment milestones. AT&T and the  states negotiated the commitments during the state opt-in process, and  AT&T delineated them in the state plans. For example, among states in  our review, AT&T made commitments regarding the number of Band 14  cell sites, including new cell sites, and future coordination with state,  local, or tribal authorities to discuss governance or priority coverage  areas, among other things.

According to our analysis of FirstNet documentation, progress toward  meeting state-specific coverage commitments has varied. For example,  among our case-study states as of July 2019, AT&T’s progress meeting  the total coverage commitment in non-rural areas ranged from  approximately 20 percent complete in one state to nearly 100 percent in  others. In comparison, AT&T’s coverage progress in rural areas ranged  from about 14 percent complete in one state to about 91 percent in  another. Likewise, AT&T’s progress meeting state-specific commitments  for delivery of Band 14 cell sites has varied across states. For example, in  our case-study states, AT&T delivered between 9 and 71 percent of the  total committed Band 14 cell sites as of July 2019.

According to FirstNet documentation and officials, variances in state  progress are allowable, as the contractual payment requirements focus  on outcomes related to nationwide milestones. FirstNet documentation  specifies that if the nationwide payment milestone was met, regardless of  the amount of coverage that was deployed in a specific state, FirstNet  deemed AT&T to have fulfilled that phase for all states. Moreover,  FirstNet officials explained that multiple factors can contribute to delays or  variance in progress across states, including natural and man-made  disasters, subcontractor issues that AT&T must work through with local  partners, and technical challenges common to cellular networks, such as  degraded performance due to mixing of radio-frequency signals.  Furthermore, FirstNet officials explained that AT&T has the first 5 years of  the contract to meet all commitments made to the states.

AT&T Is on Track to Meet the First Nationwide Adoption Milestone, with Adoption among Some Users and States Outpacing Others

AT&T is on track to meet the first adoption milestone, which is to have a  certain number of devices connected or subscribed onto the network  (“device connections”) by the end of March 2020. FirstNet uses device  connections as a proxy for adoption and has set or “forecasted” monthly  targets that build up to the nationwide connections expected by March  2020. Our analysis of FirstNet documentation indicates that AT&T is  making progress in meeting the monthly nationwide targets leading up to  March 2020. Specifically, we found that AT&T was at approximately 165  percent of the July 2019 target. See figure 2 for a comparison of actual  nationwide device connections versus the forecasted targets by month  through July 2019.

Furthermore, while AT&T must meet the nationwide device-connection  milestone to receive payment for the phase ending March 2020, the  targets are to be prorated depending on the month that AT&T meets the  corresponding nationwide coverage requirement. Thus, if AT&T meets  this requirement early (i.e., before March 2020), then the required  adoption milestone is to be reduced accordingly. For example, if AT&T  completes the coverage milestone in September 2019, then it would be  required to meet a corresponding adoption target for that timeframe.

While AT&T is on track to meet the nationwide, forecasted device- connection targets that serve as the payment milestone, our analysis  found that there is variation in who is adopting the network. The targets  are broken out by device connections associated with “primary” versus  “extended-primary” users in different states. FirstNet defines primary  users as those in the law-enforcement, fire, and emergency medical- services disciplines, whereas extended primary encompasses a myriad of  other types of public-safety entities. For example, according to our  analysis of FirstNet documentation, there are extended-primary users  from transit agencies; public-utility and tow-truck companies; school  districts; a state child-protective-services agency; airports; and television- media news outlets. Nationwide, with regard to primary users, AT&T was  at 196 percent of the July 2019 target. For extended-primary users, AT&T  was at approximately 106 percent of the nationwide target. These device  connections are also distributed amongst the different types of public- safety entities. For example, for primary device connections, AT&T was at  more than twice the forecasted nationwide target for law enforcement, as  of July 2019.

Our analysis also shows that there is wide variance in where adoption is  occurring. Specifically, we found that AT&T is exceeding the device  connection targets forecasted in certain states but lagging in others.  Among our case-study states as of July 2019, for example, device  connections for primary users in one state were more than 5 times the  target, whereas in another state, AT&T had met only 33 percent of the  target by July 2019. Adoption by extended-primary users among our  case-study states also varied, with one state at 3 times the target  compared to only 7 percent of the target met in another.

Many types of devices are connected to the network and users’  experiences with network performance can vary based on the specific  device they use. According to FirstNet documentation as of April 2019, 93  device types, 47 of which are Band-14 capable, were vetted and  published on the list of devices certified for use on the network  maintained by Commerce’s National Institute of Standards and  Technology. Our analysis found that a variety of devices and device  models are being used on the network, including smartphones, mobile  hotspots, trunk modems, laptops, and tablets. As of July 2019, the most prevalent type of device was smartphones. FirstNet has acknowledged  that user experiences on the network may vary depending on the type  and model of device. Some public-safety officials we interviewed  described inferior experiences on certain types or models of devices. In at  least one case, AT&T worked with the public-safety entity to address  identified device performance issues.

Aside from device connections, FirstNet also tracks and has reported— via press releases, board presentations, and its most recent annual report  to Congress—on the number of public-safety entities that have started  using the network. For example, in April 2019, FirstNet reported to  Congress that more than 7,000 public-safety agencies were using the  network. This number represents agencies with at least one device  connection, which may indicate piloting of the network. For example, one  agency we interviewed had only about 2 dozen of its approximately 1,300  total devices on the network. Similarly, officials from multiple other public- safety agencies explained they were in the piloting phase (i.e., testing a  small number or types of devices to gauge network performance) and that  they were using or would continue to use another carrier for broadband  services to ensure effective redundancy and emergency planning.  According to FirstNet officials, AT&T provides the count of public-safety  agencies at periodic program-review meetings and documents it in a  required contract deliverable. We analyzed this deliverable and were able  to approximate FirstNet’s reported numbers.

Many FirstNet Oversight Mechanisms Align with Key Practices, but Weaknesses in Some Mechanisms Limit Their Effectiveness

FirstNet’s Approaches to Contract Oversight Generally Align with Key Practices

FirstNet employs a variety of mechanisms to manage and oversee  AT&T’s deployment of the network and monitor contract performance.  We found that many of FirstNet’s approaches to managing and  overseeing AT&T’s network deployment and contract performance  generally align with the key contract-oversight practices identified in  federal acquisition regulations and other government, academic, and  industry guidance on contract oversight that we reviewed, as shown in  table 1.

We analyzed the key performance indicators and other documentation  related to all 46 quality assurance elements that FirstNet monitors as of  April 2019 and found that AT&T’s performance was rated as “excellent” in  over half of these elements but “unsatisfactory” in almost a quarter.  Regarding the number of unsatisfactory ratings, FirstNet officials stated  that these ratings did not raise concerns given where AT&T was in the  deployment lifecycle at the time of our review. That is, the rating may  measure performance on an item that was not yet contractually due. For  example, AT&T cannot achieve an excellent rating for certain elements  that relate to coverage deployment until it is closer to the network’s final  operating capability, expected in March 2023. Relatedly, according to  FirstNet documentation as of April 2019, FirstNet had issued only one  corrective action report since awarding the contract. According to FirstNet  officials at the time of our review, although FirstNet has rejected or  requested corrections to some items submitted by AT&T, no other  concerns have risen to this level because they have been successful in  resolving issues at lower levels first.

FirstNet’s oversight activities leading up to the March 2019 coverage  milestone were the first wherein it had to validate AT&T’s delivery of Band  14 coverage. FirstNet’s methodology for doing so included verifying  AT&T’s prediction of the signal strength at which the necessary  throughput—or, capacity, the amount of data transported successfully in a  given time period—would be achieved, and reviewing AT&T’s lab and  field tests. FirstNet then engaged in a process to verify the validity of  AT&T’s coverage-prediction maps to ensure they were an acceptable  representation of coverage in the field. Finally, FirstNet confirmed that the  on-air coverage as compared to the expected total coverage at the  network’s final operating capability met the contractual requirement.  FirstNet’s methodology did not include conducting its own coverage tests  in the field. According to FirstNet officials, FirstNet does not perform  independent verification of network coverage in the field because FirstNet  officials believe the contract provides an appropriate level of detail within  the contractual deliverables and supporting information that is used to  validate and verify the coverage milestones.

Some FirstNet Oversight Mechanisms Have Weaknesses That Limit Their Effectiveness

While many of FirstNet’s contract-oversight mechanisms generally align  with key practices, we found that some have weaknesses that limit their  effectiveness. Specifically, FirstNet lacks: (1) a reliable master schedule  to review, (2) communication with relevant stakeholders regarding  contract oversight, and (3) meaningful information on end-users’  satisfaction to gauge performance quality.

Key practices for contract oversight call for tracking the contractor’s  performance and progress toward the expected schedule. Furthermore,  GAO’s Schedule Assessment Guide identifies 10 best practices  associated with effective scheduling, and they are grouped into 4  characteristics of a reliable schedule—comprehensive, well-constructed,  credible, and controlled. The contract cites this guide when detailing the  schedule’s requirements.

As described above, AT&T must provide a current master schedule to  FirstNet monthly. However, we found that FirstNet’s use of the schedule  AT&T provides is limited because, based on our assessment, it only  partially or minimally meets the characteristics of a reliable schedule, as  shown in table 2 and described further below.

Comprehensive. We found that the schedule did not reflect all of the  work to be performed, precluding a comprehensive view of the entire  program. For example, although a master schedule should be a  comprehensive plan of all government, contractor, and subcontractor  work that must be performed to complete the project, the schedule did  not capture all government (e.g., FirstNet) activities or cover the entire  contract period. Our schedule guide notes that management should  be aware of how long government activities take because they often  have a clear effect on schedules. An integrated master schedule  should reflect all efforts necessary to successfully complete the  program. Failing to include all work for all deliverables, regardless of  whether they are the government’s responsibility or the contractor’s,  can hamper program members’ understanding of the complete plan.  Further, our analysis showed that there was a 1:1 detail-to-milestone  ratio, meaning there was 1 detail activity for every milestone in the  schedule, which is a low level of planning detail. Activities contained in  the schedule did not always have manageable or reasonable  durations; for example, over 50 percent of remaining activities had  durations greater than 2 standard working months, with 25 percent of  those having durations greater than 1 year. Our schedule guide notes  that, for a schedule to provide a more accurate view of progress,  longer activities should be broken down into smaller efforts where  possible. While some of these activities had long durations because  FirstNet expects AT&T to plan them in the future, some were not  designated as such and had no other noted justification. Moreover,  the schedule did not show any resources (i.e., labor, materials, travel,  facilities, equipment, etc.). Our schedule guide also notes that  resources must be considered in the creation of a schedule because  their availability directly affects an activity’s duration, and a schedule  without resources implies their unlimited supply and availability.

Well-constructed. We found that the schedule had a high number of  date constraints and an unreasonable amount of total float (or slack).  For example, 60 percent of remaining activities and milestones in the  schedule had “start-no-earlier-than” constraints. These date  constraints confine the schedule by preventing tasks from starting  earlier even if predecessor activities are completed ahead of  schedule, which prevent the constrained activities from taking  advantage of possible savings being introduced by predecessor  activities. Our schedule guide recommends minimizing and justifying  (in documentation) date constraints because they override the  schedule’s logic and restrict how planned dates respond to  accomplished effort. Schedules with constrained dates can portray an  artificial view of the program and begin to look more like calendars  than schedules. Moreover, over 50 percent of remaining activities had  total float greater than 2 standard working months, with the average  being over 200 days. In other words, activities in the schedule can slip  an average of 200 working days before delaying the project’s finish  date. Our schedule guide notes that without accurate values of total  float, the schedule cannot be used to identify activities that could be  permitted to slip and thus release and reallocate resources to  activities that require more resources to be completed on time. Finally,  while we found that the schedule had continuous critical paths, there  was not enough detail activities to track the work necessary to  achieve project milestones.

Credible. We found that there was no risk analysis performed for the  schedule. Our schedule guide notes that data about program risks  should be incorporated into a statistical simulation to predict the level  of confidence in meeting a program’s completion date; to determine  the contingency, or reserve of time, needed for a level of confidence;  and to identify high-priority risks. Additionally, our schedule guide  notes that a schedule should be (1) “horizontally traceable,” meaning  that it should link products and outcomes associated with other  sequenced activities; such links are commonly referred to as “hand- offs” and serve to depict the relationships between different program  elements and verify that activities are arranged in the right order, and  (2) “vertically traceable,” meaning data are consistent between  different levels of the schedule. Our analysis found that the schedule  responded when significant delays were introduced into the planned  activities; that is, when we tested the robustness of the schedule by  extending activities’ durations, forecasted dates recalculated  appropriately. However, as described above, we found that the  schedule did not capture all activities or provide sufficient detail,  meaning it cannot be fully traceable horizontally. We also found that,  in general, the schedule provided good vertical traceability—that is,  dates were traceable between status reports and the schedule.  However, when we compared other reported information to the  schedule, there were instances where this traceability was not the  case. For example, one monthly report stated that baseline  information was included for all tasks and milestones of a particular  task order, but we found that the schedule did not in fact include this  information. Vertical traceability provides assurance that the  representation of the schedule to different audiences is consistent and  accurate.

Controlled. We found that the schedule was updated regularly using  actual progress and logic by trained AT&T personnel, with supporting  documentation and review procedures. We also found that not all  activities in the schedule had baseline dates. According to FirstNet  officials, portions of the schedule are baselined on a rolling basis once  the next requirements traceability matrixes are created. However,  some activities with no baseline dates had already begun or been  completed. Further, FirstNet officials stated that no “basis document”  exists for the baselined schedule. Our schedule guide notes that a  corresponding basis document is important because it explains the  overall approach to the program, defines custom fields in the schedule  file, details assumptions used in developing the schedule, and justifies  constraints, lags, long activity durations, and any other unique  features of the schedule. Furthermore, while AT&T was submitting  schedule variance information, it covered only tasks that had been  baselined, when the majority of activities in the schedule were missing  baseline dates. Without formally established baseline-schedule start  and finish dates to measure performance against, FirstNet is limited in  how it can use the schedule to identify or mitigate the effect of  unfavorable performance.

Overall, FirstNet officials said they are not concerned about the gaps in  the AT&T master schedule for a variety of reasons. Namely, officials  stated that FirstNet entered into a contract with AT&T that lays out  specific milestones that AT&T must meet or it does not receive payment.  Accordingly, they said that the summary level of detail is sufficient for  FirstNet’s purposes, as AT&T’s program management office determines  what activities are appropriate to track to meet those milestones and  AT&T maintains its own, more detailed schedule. They further added that  given the firm-fixed price nature of the contract, it is not practical or  helpful for FirstNet to collect information on the resources for AT&T’s  deliverables; if it takes AT&T 50 or 50,000 individuals to complete the  requirement that decision is for AT&T to determine. As such, although the  contract cites GAO’s schedule guide when detailing the schedule’s  requirements, FirstNet excluded requirements related to resources.  Similarly, FirstNet excluded requirements related to schedule risk analysis  primarily, according to FirstNet officials, because risks to the established  schedule milestones were largely considered when evaluating AT&T’s  proposal prior to contract award. Finally, FirstNet officials highlighted that  the schedule is not the only measure for progress and reporting, noting  that it employs many other mechanisms to monitor and oversee AT&T’s  progress and performance, and discusses the schedule during program  management review and other meetings with AT&T.

However, the contract itself states that FirstNet is responsible for ensuring  the overall success of the network and that, to do so, its responsibilities  after contract award include overseeing the program schedule. Regarding  resources in particular, the contract also states that these responsibilities  include managing schedule resources. Thus, while it may not be  necessary for FirstNet to collect information from AT&T on every resource  detail, as FirstNet has stated, it is nevertheless important for FirstNet to  gain an understanding of the overall resources needed to complete the  work. This understanding could include, for example, evidence that  sufficient resources were assigned to activities in the more detailed  schedule that AT&T maintains. Our schedule guide notes that resources  must be considered in the creation of a schedule—and it is important that  FirstNet have sufficient insight into those resources—because their  availability directly affects an activity’s duration. Regarding schedule risk  analyses, consideration of risks to the milestones prior to contract award  may not serve as a substitute for a risk analysis of the current schedule,  which would include detail on activities and risks that could not have been  known or fully understood prior to the award. Finally, while FirstNet  utilizes a variety of other mechanisms to oversee AT&T’s performance,  having a more detailed master schedule from AT&T would strengthen  FirstNet’s use of the schedule as a management and oversight tool. For  example, such a schedule could improve FirstNet’s insight into the  activities driving AT&T’s deployment of the network and completion of  requirements, how each activity relates to others, and any potential risks.  It could also provide FirstNet with additional information that could help it  and AT&T manage tradeoffs and make decisions to maximize the  program’s success across the entire country.

Contract Oversight Communication with Stakeholders

Key practices for contract oversight call for communicating appropriate  information to relevant stakeholders and reporting on monitoring results.  Additionally, the 2012 Act requires FirstNet to consult—via a designated  single point of contact (SPOC) in each state—with regional, state, local,  and tribal jurisdictions regarding a host of activities, such as: ongoing compliance review and monitoring of the management and  operation of the network; practices, procedures, and standards for the management and  operation of the network;  terms of service for use of the network;  radio-access network build out, placement of cell towers, and  coverage areas; and assignment of priority and selection of entities seeking use of the  network.

Furthermore, the contract requires AT&T to report, by state, on the state- specific commitments made as a result of the state opt-in process.  Portions of this report are to be shareable with states, and it is to detail  the deadline by which the commitments will be fulfilled, the status of  fulfilling them, and include evidence of the state’s satisfaction with  progress. Beginning April 2018, AT&T is required to deliver this report  semi-annually. Although two such state-specific commitment reports were  due as of July 2019, only one has been completed by AT&T and  accepted by FirstNet. Additionally, according to FirstNet officials as of  October 2019, the report was not shared with the states.

Numerous state, local, and tribal stakeholders we interviewed described  having had very little contact with FirstNet or being generally dissatisfied  with the level or quality of information they had received from FirstNet and  AT&T. These officials said that FirstNet had communicated little to no  information on AT&T’s progress deploying the network in their area, or if  and how FirstNet was monitoring performance. For example, many  officials said that they had limited interaction with FirstNet beyond public  relations emails or events promoting the network, or noted that their  interactions lacked substantive information and details that would be of  more value. The SPOCs were particularly dissatisfied with the lack of  transparency surrounding the contractual requirements or FirstNet’s  oversight of progress to date. Many of these state officials noted that the  level of communication and information shared by FirstNet post contract  award stood in stark contrast to the level of engagement prior to the  state’s opt-in decision.

Numerous state, local, and tribal stakeholders we interviewed said that  additional information on AT&T’s deployment and FirstNet’s oversight  would be helpful or that greater transparency was needed. Officials  wanted additional information on, among other things: contract  requirements, milestones, and progress; technical details on the network  including operational status and location of cell sites; subscribers within  the official’s agency or agencies across the state that had adopted the  network; and FirstNet’s oversight activities and results, including  assurance from FirstNet that network coverage and performance had  been verified. Even public-safety officials who were pleased with their  experiences on the network to date or their relationship with FirstNet  representatives reported that having more information was important. In  the absence of this type of information, many public-safety entities we  contacted expressed concern that they did not know whether FirstNet  was holding AT&T accountable. For example, several officials indicated  they did not know whether FirstNet or AT&T was “running the show.”

State, local, and tribal stakeholders we interviewed gave a variety of  reasons for wanting greater transparency on contractual requirements  and oversight. Numerous public-safety officials said that they needed to  know this information for tactical response and planning, or state and  local contracting purposes. For example, some local public-safety officials  described wanting to have basic information on the contract coverage  phases in their states so that they could confidently plan out equipment  lifecycles. Additionally, many SPOCs said that there was a duty for  FirstNet as the contracting agency to oversee that state-specific  commitments were met. Many SPOCs also stated that their attempts to  obtain more information from FirstNet or AT&T per the agreed-upon  commitments had been delayed. At times, when they reached out to  FirstNet, they were directed back to AT&T, or vice versa. Numerous  stakeholders agreed that given the nature of the network as a public  resource—involving public investment and funds, with the expressed  purpose of serving public safety—they expected greater transparency  from both FirstNet and AT&T.

FirstNet officials provided several reasons for not communicating the  additional information cited by the stakeholders we spoke to and for not  reporting on monitoring results. In particular, FirstNet officials told us  there is no contractual requirement to communicate or share information  collected, including any performance information or monitoring results,  with any stakeholders or network users. However, its Public Safety  Advocacy team serves as the primary interface to the public-safety  community and conducts considerable outreach to stakeholders, as  described above. Regarding the SPOCs, the officials further said that  they believe the 2012 Act’s consultation requirement applied only to the  initial planning stages (namely, the development of the request for  proposal prior to contract award). As such, they do not believe they are  legally obligated to continue to communicate specifically as identified in  the 2012 Act. Additionally, FirstNet has stated that much of the  information AT&T provides is proprietary and, therefore, cannot be  disclosed to stakeholders. Finally, regarding the state-commitments  report, FirstNet officials have said that FirstNet shares subsets of this  information with states that request it during consultative interactions with  FirstNet and in coordination with AT&T, but does not routinely share the  full report to protect confidential commercial or trade-secret information.

While the 2012 Act does require consultation to occur “in developing  requests for proposals,” it also states “and otherwise carrying out its  responsibilities,” suggesting a broader application than just the initial  planning stages, which is FirstNet’s interpretation. Moreover, while there  are valid concerns about disclosing proprietary information and statutory  prohibitions on doing so, there are opportunities for FirstNet to  communicate additional information in ways it deems appropriate. For  example, communicating how it oversees AT&T, the mechanisms it  employs, and the performance areas it monitors could be done in a  manner that does not disclose proprietary AT&T information, as these are  government activities. Additionally, a state official and some local  government officials we spoke to said that certain AT&T commercial  information (e.g., the location of cell towers) could already be publicly  available through local permitting offices. Further, federal internal-control  standards note that management may select appropriate methods for  external reporting, meaning management can consider what methods are  appropriate for different audiences when communicating and reporting  information. Finally, the contract states that except as specifically  indicated or with explicit written permission from FirstNet, AT&T’s  deliverables documentation shall not contain proprietary information or  have any restriction on reproduction and/or distribution, suggesting that  upon awarding the contract, FirstNet recognized the value of limiting  these instances.

Industry guidance on project management that we reviewed—and which  is cited in the contract—notes that analyses of high-profile project failures  highlight the importance of stakeholder engagement. It also notes that  communicating with stakeholders in an appropriate way can mean the  difference between a project’s success and failure. Stakeholders’ lack of  information on the program and FirstNet’s oversight of AT&T can make it  difficult for stakeholders to assess what benefits have, or have not, been  realized, which may affect their enthusiasm and continued support of the  program. This scarcity of information has also left them speculating about  other matters such as what, if any, oversight FirstNet conducts of AT&T.  By not communicating additional information and reporting on monitoring  results, FirstNet could be unknowingly reinforcing nascent skepticism of  the program overall and of itself as the entity charged with holding AT&T  accountable.

Key practices for contract oversight call for obtaining information on end- users’ satisfaction that can be used as a metric to gauge performance  quality. For example, industry guidance on program management  emphasizes that end-users’ satisfaction is a powerful metric that should  be obtained to gauge program quality, noting that the benefits, product, or  service delivered is best evaluated by those who receive it.

While FirstNet collects some information—via its QASP monitoring, as  described above—that could relate to end-users’ satisfaction, these  metrics provide limited insight into users’ experiences. For example,  although AT&T surveys some customers to ask them whether they would  recommend FirstNet services to a colleague to satisfy a QASP  requirement, a user could recommend the service not because they are  satisfied but because they have limited alternatives.

Indeed, while many state and local public-safety officials we spoke to  were pleased with their experience migrating to or piloting the network,  numerous officials told us about experiences that fell short of their  expectations for a public-safety broadband network backed by the  government. Numerous officials told us that they had concerns about  misleading or disorganized sales tactics from AT&T representatives. For  example, while some officials said that their AT&T representative had  been candid in explaining the limited available coverage in their area,  many officials told us about instances when AT&T representatives had  shown them maps depicting more coverage than actually existed or that  were insufficiently granular for their mission work. Similarly, while many  officials recounted positive experiences with network coverage or  performance or AT&T representatives, many also described instances  when equipment failed to work or perform as expected during piloting  phases or exercises. In some instances, these officials stated that  FirstNet or AT&T representatives explained, after the fact, that differences  in user experience were to be expected depending on the device model  or subscriber identity module (SIM) card being employed. Specifically,  FirstNet or AT&T officials explained that the optimal performance could  only be achieved when Band 14 devices connected to a Band 14 cell site.

According to FirstNet officials, the best experience will be when  subscribers use a Band 14-capable FirstNet-ready device with a FirstNet  SIM card while in a Band 14 coverage area. The officials said any other  combination could result in slightly degraded performance or features  being unavailable. This is notable given that Band 14 coverage is still  limited and generally state and local public-safety officials do not have  insight as to where these sites were located or when, if ever, coverage  will be expanding, as previously discussed. As stated above, at its final  operating capability, the network utilizing Band 14 spectrum will not cover  the entire country.

Many officials also expressed concerns about the network’s quality of  service, priority, and preemption capabilities over the long run or during a  catastrophic event. They speculated about the type or expanding number  of subscribers allowed on the network or whether at some point in the  future, the network would become saturated because non-public safety  organizations or individuals (either extended-primary users or non-verified  public-safety subscribers) were being granted priority and preemption  capabilities. Exacerbating these concerns, many officials noted that they  did not have insight into who had subscribed even within their own  agency or state, or lacked confidence in how FirstNet or AT&T verifies  individuals’ public-safety status, based on anecdotal experiences.  Further, some officials also raised concerns about their inability to test the  network during congested periods or simulate catastrophic power failures  and lack of insight into if or how AT&T had hardened the network. Many  officials discussed or shared after-action reports or their testing results  with us, and several communicated that they had shared or would be  willing to share such information with FirstNet as well to support validation  of the network’s actual performance.

According to FirstNet officials, the key performance indicators identified  via the QASP are the performance quality measures, not end-users’  satisfaction. They also stated that “disincentive” payments embedded in  the contract serve as an incentive for AT&T to ensure end-users’  satisfaction. Specifically, if AT&T does not meet user adoption (i.e.,  device connection) goals specified in the contract, it has to make  payments to FirstNet on a timetable identified in the contract. Additionally,  according to FirstNet officials, they informally hear information on end- users’ satisfaction and the network’s performance through many of the  engagements its Public Safety Advocacy team conducts, which they can  informally share with AT&T.

However, disincentive payments (and the user-adoption goals tied to  them) may be a limited reflection of end-users’ satisfaction for various  reasons. For example, users may continue to subscribe to the service not  because they are satisfied with it but because agency procurement  lifecycles and budgets prevent them from changing providers, or because  they find it difficult to break a sales contract, have already sunk costs into  the transition, or lack alternatives in the market. Additionally, if AT&T  perceives that the value derived from its commercial customers’ use of  the excess Band 14 spectrum capacity is greater than the disincentive  payment it must make to FirstNet, it may view making the payment as an  acceptable tradeoff. Alternatively, aggressively pursuing sales contracts  with potential public-safety users to avoid the payments may not be  welcomed by the public-safety community, which could result in  negatively, not positively, affecting end-users’ satisfaction, as some  public-safety network users we spoke to said it had. Finally, while the  informal collection and sharing of information on satisfaction can be  valuable, it does not serve as a formal performance-quality measure,  which could provide FirstNet with additional recourse should issues arise.

End-user adoption is both a goal of the program and how AT&T plans to  fund the $40 billion of investment in the network. Adoption may be driven  by satisfaction in addition to need. Ultimately, end-users’ dissatisfaction  could affect the success of the program. Thus, FirstNet’s lack of formal  insight into end-users’ satisfaction hampers its ability to take actions that  could increase the program’s chance of succeeding. By not obtaining and  using this information to inform its oversight or related activities, FirstNet  could be missing an opportunity to increase assurance of the program’s  long-term success.

The FirstNet public-safety broadband network has the potential to save  lives every day. Since beginning their 25-year partnership, AT&T has  made progress deploying the network and meeting contractual milestones  and goals, and FirstNet has employed a variety of mechanisms—many of  which align with key practices—to oversee AT&T’s performance.  However, the success of the network depends not only on AT&T’s  contract execution and FirstNet’s oversight but also on the confidence of  the end users, the nation’s first responders. As FirstNet enters the next  phases of its partnership with AT&T, it could reduce the risks to the  network’s long-term success by strengthening its schedule oversight;  increasing transparency, communication, and reporting of additional  information to states and other public-safety stakeholders; and obtaining  and using meaningful information on the satisfaction of the first  responders for whom the network is intended.

We are making the following four recommendations to FirstNet:

FirstNet’s Chief Executive Officer should take steps to ensure that the  integrated master schedule for the program is developed and  maintained in accordance with the best practices provided in GAO’s  Schedule Assessment Guide. (Recommendation 1)

FirstNet’s Chief Executive Officer should identify additional  information about the program, including FirstNet’s oversight and  monitoring activities, that can be shared with public-safety  stakeholders and periodically communicate and report this information  to them. (Recommendation 2)

FirstNet’s Chief Executive Officer should share relevant portions of  the accepted state-specific commitment reports with the states, as  specified in the contract. (Recommendation 3)

FirstNet’s Chief Executive Officer should, in consultation with public- safety stakeholders and its contractor, as appropriate, identify and  obtain periodic information or meaningful indicators on end-users’  satisfaction that would serve as a metric to gauge performance  quality, including the effect of the FirstNet network and products on  public-safety operations. (Recommendation 4)

We provided a draft of the sensitive report to FirstNet for review and  comment. FirstNet’s comments on the sensitive report are reprinted in  appendix II. In these comments, FirstNet stated that it agreed with all of  our recommendations; will take appropriate additional steps to apply  lessons learned and address our concerns; and will continue to find ways  to improve transparency with and feedback from its stakeholders, in  addition to refining the integrated master schedule. Separately, FirstNet  also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees, the Chief Executive Officer of FirstNet, the Secretary of  Commerce, and other interested parties. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-2834 or vonaha@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix III.

This report examines the extent to which (1) AT&T is meeting the  established milestones for deploying the nationwide public-safety  broadband network, including coverage and adoption goals, via its  contract with the First Responder Network Authority (FirstNet), and (2)  FirstNet is overseeing AT&T’s deployment of the network in accordance  with key practices.

To assess progress toward the coverage and adoption milestones, we  reviewed the FirstNet-AT&T network contract, corresponding task orders,  and relevant documentation contained in FirstNet’s contract files,  including information or “deliverables” submitted by AT&T that had been  reviewed by FirstNet for contract compliance. We also reviewed  additional FirstNet documentation, such as board-meeting materials,  annual reports to Congress, press releases, fact sheets, and official blog  postings. We reviewed the Middle Class Tax Relief and Job Creation Act  of 2012 (the 2012 Act), which created FirstNet as an independent  authority charged with establishing a nationwide public-safety broadband  network that would, among other things, be deployed in phases that  included substantial coverage milestones in rural areas. Within the  contract, we identified the various coverage and adoption milestones and  focused our analysis primarily on task order 3, phase 3 (which spanned  March 31, 2018, to March 30, 2019) and task order 4, phase 2 (which  spanned October 1, 2018, to March 30, 2019) milestones. We focused on  these task orders because they are most relevant to the network’s  coverage deployment and adoption, and on these phases because they  were the phases under way at the time we began our review. We did not  review activities or progress as described in AT&T deliverables dated  beyond September 2019 given the timing of our review. We also did not  make any conclusions about progress toward the final phases of these  task orders. However, we did assess the master schedule to determine its  reliability and validity for planning and tracking progress toward the final  phases as described further below and in our report.

The contractual deliverables that we reviewed in some cases included  detailed data broken out by state and public-safety discipline. In  particular, we analyzed data that indicated progress toward nationwide  and state Band 14 network coverage (in square miles); cell site delivery;  monthly adoption targets (i.e., device connections) by discipline; and  types of devices connected. When analyzing these data, in all cases, we  used the most currently available data at the time of our request for the  information, and we report data as of September 2019. Although all data  were the most currently available as of September 2019, because the  deliverables have varying cycles for when AT&T is contractually required  to report the information, we specify throughout the report the “as of”  period these data represent. We assessed the reliability of these data by  asking FirstNet officials questions about how they review the deliverables  and about data sources, quality, and timeliness, as well as by  electronically testing the dataset for missing or invalid entries. We  removed a small number of missing or invalid entries from our analysis of  device types and models and count of public-safety agencies. We did not  assess AT&T’s underlying systems or databases, nor did we interview  AT&T officials about their protocols for producing this data. We found  these data reliable for the purpose of describing FirstNet’s current and  projected progress toward coverage and adoption milestones for the  related task orders and phases.

To further assess deployment progress, we conducted case studies of  seven states to illustrate and obtain greater context on variations in state- level coverage and adoption. We selected our case-study sample to  include states that had very high-density counties; relatively large  numbers of low-population density counties; high poverty rates (due to  budgetary challenges public-safety entities may face); varying levels of  progress in cell site delivery as of January 2019 (the most currently  available data at the time of our selection); and geographic diversity and  tribal lands. In total, the selected states represent almost a third of the  contract dollars allocated for network coverage deployment. Our case- study analyses included reviewing and comparing the deployment plans  and commitment letters for these seven states (detailing the agreed-upon,  state-specific commitments AT&T made to these states) against the  deliverables describing the progress AT&T made on some of these  commitments, as of July 2019. It also included interviewing state, local,  and tribal officials and first responders from these states, as described  further below. The case studies and stakeholders’ views illustrate  experiences with FirstNet’s deployment of the network across a wide  cross section of geographies and network users to date but are not  generalizable to those of all FirstNet stakeholders or the network as a  whole. We also interviewed FirstNet officials to obtain their perspectives  on AT&T’s progress and factors that may explain the variance across  states.

To examine FirstNet’s oversight efforts, we reviewed the FirstNet-AT&T  network contract and documentation contained in FirstNet’s contract files,  as well as additional FirstNet documentation. In addition to the material  described above, this documentation included, for example, the Quality  Assurance Surveillance Plan, requirements traceability matrixes,  verification reports, memos, Contract Administration Plan, FirstNet  Acquisition Manual, guidance documents on contract management and  procedures, and FirstNet officials’ written responses to questions we  posed. For the same reasons described above, we focused primarily on  material related to task order 3, phase 3 and task order 4, phase 2. We  interviewed FirstNet officials to obtain greater context on FirstNet’s  oversight mechanisms and their use, and to observe FirstNet’s  verification activities and the platform it uses to manage its contract files.

Further, we reviewed key acquisition and contract-oversight practices  established in the Federal Acquisition Regulation and the Commerce  Acquisition Regulation, as well as the Commerce Acquisition Manual   and other academic and industry guidance. We also reviewed the 2012  Act and federal standards for internal control. We selected those  practices that were most appropriate given FirstNet’s contract approach  (i.e., Indefinite-Delivery/Indefinite-Quantity, Firm-Fixed-Price contract  vehicle) and the stage of the acquisition process FirstNet was in during  the course of our review. We assessed FirstNet’s oversight efforts against  these practices. We also compared the network’s integrated master  schedule, which AT&T provides to FirstNet, to scheduling best practices  in GAO’s schedule guide. Collectively, these best practices are  organized into four characteristics of a reliable schedule. A schedule is  considered reliable if each of the four characteristics is substantially or  fully met; if any of the characteristics are not met, or minimally or partially  met, the schedule cannot be considered reliable. We reviewed the  schedule as of its status date January 31, 2019, which represented the  latest status update to the schedule at the time we began our schedule  analysis. In reviewing the schedule, we also reviewed the schedule  dictionary, work breakdown structure, and program management review  or monthly progress reports dated October 2018 to January 2019, among  other documents. We provided our criteria and draft schedule analyses to  FirstNet for review.

To inform both of our objectives, we conducted about 40 interviews with  state, local, and tribal officials and first responders. These interviews  represented almost 30 different states’ single point of contact (SPOC) to  FirstNet or their designees, and over 30 different state, local, or tribal  public-safety entities. The public-safety entities we interviewed included  police and fire departments, sheriffs’ offices, emergency medical-services  providers, and emergency-management agencies, among others.

We interviewed the SPOC from each of our case-study states and  received information from other SPOCs (or a designee) via a multi-state  focus-group discussion and written responses to the semi-structured  discussion questions and prompts we posed. A GAO moderator led the  discussion to establish ground rules and keep participants focused on the  specified issues within the discussion time frame. We selected state,  local, and tribal public-safety entities within our case-study states to  interview. To select the state and local public-safety entities to interview,  we reviewed the AT&T subscription management report provided to  FirstNet as of February 2019 (the most current available at the time of our  selection) and asked the SPOCs for recommendations within their state.  Generally, we selected among the largest subscribers (meaning, the most  number of devices on the network) in each of the primary public-safety  disciplines (law enforcement, fire, emergency-medical services) in each  state, and selected others to ensure representation among urban,  suburban, and rural areas. To select the tribal entities to interview we  asked the National Tribal Emergency Management Council for a  recommendation in each state. Not all public-safety entities accepted our  interview requests. Among our case-study states, we conducted a site  visit in one state region. We selected this region for our visit because of  the concentration of subscribers within reasonable geographic proximity  to each other. For additional context, during this visit we also met with the  FirstNet Public Safety Advisors that serve the state and attended a  FirstNet presentation and town hall meeting hosted by the local chapter of  the Association of Public Safety Communications Officials. Because  stakeholders varied in their expertise with various topics, not every  stakeholder provided an opinion on every topic. Throughout this report we  refer to “some” stakeholders if officials from 3–5 entities, “several” if 6–9,  “many” if 10–19, and “numerous” if 20 or more expressed the view.  Finally, for additional perspective we also interviewed the National Public  Safety Telecommunications Council because of its role as a federation of  organizations whose mission is to improve public-safety communications  and interoperability. As noted above, stakeholders’ views are not  generalizable to those of all FirstNet stakeholders.

Appendix III: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Sally Moino (Assistant Director);  Nalylee Padilla (Analyst in Charge); David Aja; Melissa Bodeau; Andrew  Burton; Mark Goldstein; Yvette Gutierrez; David Hooper; Jason Lee;  Andrew Stavisky; Hai Tran; William Woods; and Friendly Vang-Johnson  made key contributions to this report.

The Department of Commerce’s FirstNet must establish a nationwide broadband network for public safety officials (like police officers and firefighters).

In 2017, FirstNet awarded AT&T a contract to deploy, operate, and maintain the network. AT&T is meeting contract milestones.

But since awarding the contract, FirstNet has shared little to no information about its contract oversight with state officials and other stakeholders. This has led to speculation about the extent of FirstNet’s oversight of AT&T, which could affect network users’ and stakeholders’ continued support of the program.

We recommended that FirstNet address this and other issues.