Discretionary Transportation Grants: Actions Needed to Improve Consistency and Transparency in DOT's Application Evaluations

The cost to repair and upgrade the nation's surface transportation system to meet current and future demands is estimated in the hundreds of billions of dollars. In December 2015, Congress established a DOT discretionary grant program to fund nationally significant freight and highway projects. DOT awarded $1.54 billion for such projects for fiscal years 2017 and 2018. GAO was asked to review DOT's process for evaluating and selecting applications for awards.

This report discusses the consistency and transparency of DOT's process for evaluating and awarding INFRA grants for the fiscal-year 2017–2018 round of funding, among other objectives. GAO reviewed DOT's documentation of its evaluation process, and interviewed DOT staff and officials, as well as 11 INFRA applicants selected to ensure diversity in projects' size, type, location, and award status, as well as type of applicant.

The Department of Transportation's (DOT) process for reviewing applications for grants to fund projects under the Infrastructure for Rebuilding America (INFRA) program lacked consistency and transparency in aspects related to following up with applicants and evaluating applications.

Following up with applicants. DOT must determine that an applicant's project meets statutory requirements in order for the project to be eligible for an INFRA award. DOT initially found that 97 applications had insufficient information for an eligibility determination. DOT followed up with 42 of the 97 applicants to request additional information. DOT did not sufficiently document why it followed up with certain applicants over others. If DOT does not clearly communicate and document its process regarding applicant follow-up, the process lacks transparency and the assurance of fairness.

Evaluating applications. In addition to the statutory requirements, DOT established merit criteria (e.g., economic vitality) to evaluate projects against, and stated that competitive projects would substantively address all of the criteria. DOT teams scored the projects on how well they addressed each criterion. However, DOT forwarded the information on all 165 projects that were found to be statutorily eligible to the Secretary for potential award, regardless of how well they scored on the merit criteria. In the end, DOT awarded some projects that did not address all of the criteria. Several applicants told GAO they were uncertain how DOT determines which projects should receive awards. In addition, DOT's documentation does not provide insight into why projects were selected for awards, an issue GAO has previously noted and recommended DOT address.

The above limitations reflect long-standing issues GAO has identified in DOT's discretionary grant programs. Specifically, since 2011, GAO has recommended actions to increase consistency and transparency. In some cases, DOT implemented the recommendations for one program, but GAO later found similar problems in other programs. After finding repeated issues, GAO recommended in 2016 that DOT develop a department-wide directive that would, among other things, require that key decisions be documented. DOT agreed with the recommendation. In a March 2019 memo, DOT directed offices to implement GAO's recommendation by June 2019. However, it is unclear how this action will improve transparency and consistency because, among other things, DOT did not communicate how offices should sufficiently document decisions to ensure that the rationale for decisions is clear. The next reauthorization of surface transportation programs provides Congress the opportunity to build requirements for greater consistency and transparency into DOT's grant programs. This is particularly important as DOT has two additional rounds of INFRA funding to award under the FAST Act, and the President's Budget proposal proposed providing an additional $1 billion to INFRA. Absent effective action by DOT going forward, the recurring and long-standing issues GAO has identified could continue to affect DOT's competitive discretionary grant programs.

GAO is making three recommendations, including that DOT should communicate and document the rationale for asking specific applicants for more information and provide information to applicants on how, if at all, DOT uses merit criteria scores to advance projects through its evaluation and selection process. Also, Congress should consider directing DOT to develop and implement transparency measures in the next surface-transportation reauthorization bill. DOT concurred with GAO's recommendations and provided technical comments that GAO incorporated as appropriate.

The nation’s surface transportation system is under growing strain, and  the cost to repair and upgrade the system to meet current and future  demands is estimated in the hundreds of billions of dollars. Federal  transportation grants provide critical funding to help build and upgrade the  freight and highway networks that support safe, efficient, and reliable  movement of goods and people. In December 2015, the Fixing America’s  Surface Transportation Act (FAST Act) established a nationally significant  freight and highway project program and authorized the appropriation of  $4.5 billion to the Department of Transportation (DOT) to award in  discretionary grants for such projects for fiscal years 2016 through 2020.  In 2016, DOT issued the first round of grants providing approximately  $759 million to 18 projects under this new program, called the Fostering  Advancement in Shipping and Transportation for the Long-term  Achievement of National Efficiencies (FASTLANE) program. We reviewed  DOT’s processes used to evaluate and award FASTLANE grants and  found issues related to consistency and transparency. Specifically, we  found that due to inconsistencies in DOT’s review of applications and  limited documentation of decisions regarding awards, we were unable to  determine the rationale DOT used to award projects. We have repeatedly  found similar issues related to consistency and transparency in our prior  reviews of DOT’s various discretionary grant programs, beginning in  2011.

In July 2017, to reflect the priorities of the new administration, DOT  revised the FASTLANE program by establishing new criteria for  evaluating grant applications and renaming it the Infrastructure for  Rebuilding America (INFRA) program. DOT also announced that up to  $1.6 billion would be awarded for fiscal years 2017 and 2018. After DOT  reviewed the 258 applications it received, it awarded roughly $1.54 billion  to 26 projects.

You asked us to review the INFRA award process. This report: describes DOT’s process for evaluating INFRA grant applications, and assesses the consistency and transparency of DOT’s process for  evaluating INFRA grant applications.

To describe DOT’s processes for evaluating and awarding INFRA grant  applications submitted in response to DOT’s July 2017 call for  applications for the fiscal-year 2017–2018 round of funding, we identified  pertinent statutory requirements in the FAST Act. We then reviewed  DOT’s July 5, 2017, notice of funding opportunity (NOFO) announcing the  availability of INFRA funds, as well as the program’s funding priorities and  the corresponding criteria DOT would use to evaluate the projects  proposed in the grant applications. We also reviewed DOT’s INFRA  evaluation plan that described how DOT staff should evaluate and score  the projects against these requirements and criteria as well as  documentation from an internal DOT spreadsheet showing the results of  the reviews, including project scores and narratives explaining the  rationale for the scores. Finally, we reviewed the documents presented to  the Secretary. To assess the reliability of DOT’s INFRA spreadsheet, we  interviewed DOT officials and conducted checks of the data, such as  identifying blank cells in the spreadsheet and comparing the information  from the spreadsheet against the documents presented to the Secretary  to identify any discrepancies. While we identified instances in which DOT  did not record score changes in its spreadsheet, we were able to identify  final project scores and found the data were reliable for the purposes of  identifying the scores given by DOT to projects and understanding how  DOT evaluated projects.

To assess the consistency and transparency of DOT’s process for  evaluating INFRA grant applications, we compared DOT’s processes for  evaluating applications submitted in response to the July 2017 NOFO to  requirements and best practices related to consistency and transparency  in the administration of discretionary grant programs. We identified  relevant requirements and guidance from the Office of Management and  Budget’s (OMB) Uniform Administrative Requirements, Cost Principles,  and Audit Requirements for Federal Awards (Uniform Guidance), federal  internal control standards related to control activities and communication,  DOT’s Financial Assistance Guidance Manual, and our prior work in  which we identified recommended practices for awarding discretionary  grants. We then conducted document reviews and interviews with DOT  staff and officials to assess DOT’s process against these criteria. For  example, we conducted in-depth reviews of the narratives explaining the  scores and any reviewer’s concerns for all of the awarded projects, and a  non-generalizable sample of 10 non-awarded projects, selected to ensure  diversity in projects’ size, type, urban or rural status, and applicant type.  We followed up with DOT regarding concerns raised by reviewers on  specific projects, as well as potential inconsistencies between the  reviewer’s narrative and the project’s score.

In addition, we interviewed DOT staff with diverse responsibilities,  including: (1) those who conducted technical reviews of INFRA projects  against criteria established by DOT, (2) those responsible for overseeing  the process, and (3) senior officials responsible for deciding which  projects should be forwarded to the Secretary. We asked these staff how  they conducted reviews and documented their decisions. For the  technical review staff, we selected a sample of staff from six of the seven  teams that reviewed projects against criteria. We selected technical  review staff from the Federal Highway Administration (FHWA), Federal  Railroad Administration (FRA), and United States Maritime Administration  (MARAD) to ensure diversity with respect to area of expertise and modal  administration. For the senior officials, we selected two officials who  served on the team that decided which projects should be forwarded to  the Secretary, including one official DOT identified as best able to discuss  how the Secretary reviewed the projects. To obtain applicant perspectives  on the process, we interviewed 11 applicants and 3 consultants that  applicants hired to help them with their INFRA application. Applicants  were selected to ensure diversity in projects’ size; type (highway, rail,  port, multimodal, grade crossing); location; urban or rural setting; award  status; as well as types of applicants (state, city, county government,  among others). To identify consultants to interview, we asked DOT staff  and INFRA applicants we interviewed to provide the names of consultants  that worked on INFRA project applications. The results of our interviews  provide insight into applicants’ experiences with the INFRA process but  are not generalizable to all applicants. When reporting on applicants’ and  consultants’ responses to our questions, we use the following terms to  enumerate responses: several (9 to 13); some (5 to 8); and a few (less  than 5).

We conducted this performance audit from July 2018 to June 2019 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Traditionally, federal surface-transportation funding has been primarily  delivered through formula grant programs based on distributions  prescribed by statute. Discretionary grant programs, such as INFRA,  represent an alternative approach for directing federal funding toward  national priorities. Through a discretionary grant program, Congress or  federal agencies establish desired goals or outcomes—such as improving  the condition of critical infrastructure, enhancing economic  competitiveness, or reducing fatalities. Generally, federal agencies review  grant applications against published selection criteria and statutory and  regulatory requirements before selecting projects to receive awards. This  approach can help assure accountability for federal investment by more  clearly linking program funds to desired outcomes and can support  projects of national or regional significance that cross state lines. In prior  work, we have recommended that a merit-based competitive approach— like INFRA—be used to direct a portion of federal funds to transportation  projects of national and regional significance. The FAST Act authorized  over a dozen discretionary transportation-grant programs, and Congress  may consider additional programs as it considers reauthorizing DOT’s  surface transportation programs in 2020.

State, local, and tribal governments, as well as multistate or  multijurisdictional groups, are among the entities eligible to receive INFRA  funding. Freight or highway projects must meet the statutory requirements  outlined in the FAST Act to receive INFRA funding. Notable statutory  requirements regarding the distribution of awards include:

Ten percent of available funds are reserved for small projects each  fiscal year.

At least 25 percent of available funds are reserved for rural areas  each fiscal year unless DOT does not receive enough qualified rural  project applicants.

No more than $500 million, in aggregate, over fiscal years 2016  through 2020 may be used to fund freight rail, water (including ports),  or other freight intermodal projects.

The Secretary must consider geographic diversity during the selection  process.

Large projects have to meet seven additional statutory requirements to be  eligible for selection by the Secretary. Specifically, the Secretary must  determine that the project: will generate national or regional economic, mobility, or safety will be cost-effective; will contribute to one or more of the national goals for the  transportation system: improved safety, infrastructure maintenance,  congestion reduction, system reliability, freight movement, economic  vitality, environmental sustainability, and reduced project delivery  delays;  is based on the results of preliminary engineering;  for related non-federal financial commitments, has stable and  dependable funding and financing sources to construct, maintain, and  operate the project, and contingency amounts to cover unanticipated  cost increases;  cannot be easily and efficiently completed without other federal  funding or financial assistance; and  is reasonably expected to begin construction no later than 18 months  after the date of obligation of funds for the project.

In the July 2017 NOFO for the INFRA program, DOT established four  new criteria for INFRA outlining how projects would be evaluated (see  table 1). DOT did not require that projects address every criterion. DOT  noted that in addition to these criteria, called merit criteria, it would also  evaluate a project’s readiness, meaning the likelihood of a project’s  successful delivery and that the project will meet statutory deadlines for  certain milestones. In December 2018, DOT issued a NOFO in which it  called for applications for grants of fiscal year 2019 INFRA funds, and  made some changes to the program’s criteria. However, that process is  ongoing and is outside the scope of this review.

DOT Evaluated Projects against Statutory and Merit Criteria Using a Multiphase Review Process

In reviewing applications submitted in response to the July 2017 NOFO,  DOT evaluated proposed projects against the statutory and merit criteria  using a multiphase review process involving technical and senior  management teams. The process had three phases—application intake,  technical evaluation, and senior review—each supported by different  teams. The process also included a Quality Control and Oversight Team  (QCO) that was involved throughout the process and responsible for  ensuring consistent reviews and documentation. QCO consisted of team  leads from each of the seven technical evaluation teams as well as  liaisons from FHWA, MARAD, and FRA (see fig. 1).

The application intake phase consisted of two sequential steps performed  by two different teams. First, the intake review team assessed each of the  projects to:  1.  verify that the applicant type, project type, and cost-sharing met the  statutory requirements;  2.  determine the project’s size as being either small or large;  3.  identify the highway and non-highway cost components;  4.  determine whether the project is in an urban or rural area;  5.  identify which technical evaluation teams should review the  6.  which modal administration should perform the Operating  Administration screen (described below).

DOT received 258 applications for projects in November 2017 and  determined that 24 projects did not qualify for INFRA funding. The  remaining 234 projects then moved to the Operating Administrations’  screen.

As part of the Operating Administrations’ screen, staff from the  appropriate modal agency provided input on:  1.  the applicant’s history with delivering projects on time;  2.  whether the applicant had previously received federal funding from  3.  whether the applicant contacted the agency about their INFRA  project, the nature of the contact, and the level of technical and  financial assistance provided by the agency;  4.  whether the project is on the Transportation Improvement Program or  the Statewide Transportation Improvement Program; and,  5.  any specific issues with the project that evaluators should be aware  of.

The 234 projects then advanced to the Technical Evaluation phase of the  process.

The seven technical evaluation teams, made up of experts from across  the agency, assessed and rated projects against the merit criteria. Each  team was responsible for rating a different merit criterion (as noted in  figure 1, the innovation criterion was split into three factors, so there were  three innovation teams). Since DOT did not require projects to address all  the criteria, teams only reviewed the projects that related to their criterion.  The teams used the factors outlined in DOT’s INFRA evaluation plan to  assess and rate the projects and documented their rating and a narrative  justification for the assigned rating in DOT’s tracking spreadsheet.  Generally, raters assigned scores of high, medium, or low for each  criterion, with some exceptions. For example, the economic vitality team  calculated the project’s benefit-cost ratio and net present value, while also  noting whether the uncertainty associated with the rating was high,  medium, or low. Similarly, the leveraging team assigned a rating score of  high, medium, or low, but also calculated the percentage of non-federal  funding, and noted whether the project included private-sector funding.  Technical teams did not provide an overall rating of projects (such as not  recommended, recommended, or highly recommended), an approach  that differs from prior DOT discretionary grant programs we have  reviewed. For detailed information about the evaluation factors and  possible scores for each criterion, see appendix I. Each technical review  team was assigned a team lead, who was responsible for ensuring that  the projects were evaluated consistently and per the plan that governed  that team’s criterion. All 234 projects received technical evaluation ratings  for their merit criteria and then advanced for further review.

According to DOT’s evaluation plan, the Quality Control and Oversight  Team (QCO) was responsible for ensuring the consistency of reviews and  documentation throughout the INFRA process. QCO consisted of team  leads from each of the seven technical evaluation teams and liaisons  from FHWA, MARAD, and FRA. QCO was also responsible for  performing a “large project determination,” in which QCO assessed  whether projects met each of the seven statutory requirements for large  projects. QCO used information from the technical evaluations and the  information provided in the application to determine whether projects met  the statutory requirements. In cases where QCO could not definitively  determine whether a large project met a statutory requirement, it would  note “additional information is necessary” in DOT’s tracking spreadsheet.  After QCO recorded its assessment, it submitted the projects to the  Senior Review Team for review.

The Senior Review Team was responsible for assembling a list of  projects for consideration by the Secretary, and consisted of senior  officials from the Office of the Secretary, and the Administrators of FHWA,  FRA, FTA, and MARAD. The Senior Review Team, with QCO present  to answer questions, met to review the projects and their technical  evaluation scores for each criterion. The evaluation plan stated the Senior  Review Team could, at its discretion, request that QCO seek additional  information from applicants to help QCO determine if a large project met  the statutory requirements. The final list of projects for consideration  developed by the Senior Review Team contained 165 projects (all of the  small projects and all of the large projects that QCO and the Senior  Review Team determined met the statutory requirements).

At the end of the review process, the Secretary received a series of  spreadsheets ranking each of the 165 projects according to how well they  scored on each merit criteria. According to a member of the Senior  Review Team, the Secretary formally met twice with her chief of staff,  deputy secretary, and other senior advisors to discuss the projects, first to  analyze all of the projects on the list and second to finalize the award  decisions.

In June 2018, DOT announced it had awarded approximately $1.54 billion  in INFRA funding to 26 projects (see fig. 2). For the 26 awarded projects,  44 percent of funds went to rural projects and 5 percent of funds went to  small projects. In addition as shown in figure 2, highway projects  received the largest percentage of funding (85 percent), and rail projects  received the smallest percentage (1 percent).

Despite Progress in Some Areas, the INFRA Evaluation Process Lacked Consistency and Transparency DOT Took Steps to Improve Its Application Evaluations and Better Communicate with Unsuccessful Applicants

In designing its process for evaluating INFRA applications submitted in  response to the July 2017 NOFO, DOT took steps to address issues that  we found led to inconsistencies in DOT’s review of FASTLANE  applications. Specifically, we reported that technical teams were divided  by modal administrations (FHWA, MARAD, and FRA) and lacked clear  guidance on how to score applications. This led to inconsistent scoring  practices among the FASTLANE teams because one team applied a  higher standard than the others. We recommended that DOT develop an  evaluation plan for INFRA that clearly defined how all review teams  should apply criteria, assess applications, and assign ratings to ensure  that all applications are consistently reviewed. In response, DOT  developed an INFRA evaluation plan that provided guidance on how to  evaluate and assign a rating for each criterion, and in some cases,  provided discrete numeric rating categories, allowing for less  interpretation by technical review teams when assigning a score. In  addition, DOT organized technical review teams by merit criteria and  selected staff with the relevant expertise to serve on each team—for  example, economists from the various modal agencies served on the  economic vitality team.

DOT also took steps to improve the transparency of its process by better  communicating with unsuccessful applicants. Specifically, DOT formally  notified unsuccessful INFRA applicants of selection decisions via email,  addressing a concern we raised regarding the FASTLANE process. In our  review of FASTLANE, we recommended that DOT notify unsuccessful  applicants of DOT’s decision and that the notification should include a  brief explanation of the decision. For INFRA, DOT emailed unsuccessful  applicants notifying them of its decision. While the email did not include a  brief explanation of the decision, it did offer applicants the chance to  schedule a debriefing with DOT officials. Some of the selected applicants  and consultants we spoke to said that the debriefing was helpful. For  example, one applicant told us that during the debriefing, DOT shared  how the project was rated by criterion. One applicant we met with said the  debriefing was not helpful because the applicant did not receive a  substantive answer about why they did not receive an award. Another  applicant said he requested a debriefing but did not receive one. A DOT  official told us that prior to issuing the fiscal year 2019 INFRA NOFO,  DOT contacted all previous applicants to notify them of the upcoming  round and again offer debriefs.

The INFRA Evaluation Process Lacked Consistency and Transparency DOT Gave Some Applicants the Opportunity to Provide More Information to Meet Requirements, Potentially Giving These Applicants an Advantage over Others

We found that DOT’s process for following up with applicants lacked  consistency and transparency, due to a lack of guidance and  documentation. Specifically, DOT followed up with some applicants and  not others to request additional information about their projects, and the  rationale behind which applicants were selected for follow-up is not clear.  We identified similar issues in our review of FASTLANE.

As discussed earlier, for large projects to be eligible for an award, DOT  must determine that the project meets several statutory requirements,  such as generating benefits and demonstrating cost-effectiveness, among  others. Our review of DOT documents revealed that DOT staff originally  determined that 97 (of 116) applications for large projects did not include  sufficient information for DOT to assess if the projects met each of the  statutory requirements. At the request of officials on the Senior Review  Team, QCO requested more information from 42 of those 97 applicants to  help DOT determine if their projects met the requirements. Of the 42  applicants that DOT followed up with, 28 provided information that QCO  determined was sufficient to ensure that they met the statutory  requirements, and 13 of the projects received an award. Similarly, at the  request of officials on the Senior Review Team, DOT staff reduced the  scope of a number of projects. QCO staff split 9 projects into  “components,” to scope out pieces of projects that could not meet a  statutory requirement (for example, cost-effectiveness). Four of these  component projects received an award.

OMB guidance states that the intent of a NOFO is to make the application  review process transparent so applicants can make informed decisions  when preparing their applications to maximize fairness of the process.  The guidance also states that federal agencies should make clear  whether an applicant’s failure to meet an eligibility criterion by the time of  an application deadline will result in the awarding agency returning the  application without review or, even though an application may be  reviewed, will preclude the awarding agency from making an award.  Similarly, internal control standards note that federal agencies should  communicate with external entities and enable these entities to provide  quality information to the agency that will help it achieve its objectives.  DOT’s NOFO states that the applications must include sufficient  information for DOT to determine whether projects meet the statutory  requirements, but also notes that DOT may seek additional information  from applicants. The NOFO does not provide information on the basis for  why DOT would follow up with one applicant and not another.

After reviewing DOT’s documentation, we found that the rationales for  following up with specific applicants were insufficient to explain why DOT  followed up with certain applicants over others. The documentation, with  few exceptions, included generally vague statements that additional  information from the applicant could help DOT determine whether the  project met the statutory requirements. We asked two officials from the  Senior Review Team about several specific projects for which those  officials requested additional information. These officials both stated they  could not recall their rationale, given that roughly a year had elapsed and  the large number of projects reviewed. However, they did provide some  reasons why they might have requested additional information, such as  the need for more clarity on a project, a high score on a criterion of  interest to that official, or the desire to ensure that the list provided to the  Secretary included a diverse array of projects (in terms of location, urban  or rural status, and project type). Further, one official noted that there was  insufficient time to follow up with every applicant.

We have previously identified recommended practices for evaluating and  selecting discretionary grant awards and noted that in order to align with  these practices, it is important to document decisions, including decisions  regarding which projects should have the opportunity to advance in the  process. When we identified similar issues related to a lack of  consistent and transparent follow-up with FASTLANE applicants, we  recommended DOT develop an INFRA evaluation plan that clearly  defines how all review teams should apply criteria, assess applications,  and assign ratings to ensure that all applications are consistently  reviewed. DOT’s INFRA evaluation plan states that if QCO has been  unable to make an affirmative determination with respect to whether a  large project meets a statutory requirement, a Senior Review Team  member may direct QCO to seek clarifying information from the applicant  or provide the necessary clarifying information themselves to support a  determination. However, DOT’s evaluation plan does not require  documentation of the reasons why the Senior Review Team asked QCO  to follow up with certain projects over others.

Without clearly outlining in the NOFO and the evaluation plan the  situations in which certain applicants may be asked to provide additional  information, as well as clear documentation for why follow-up does occur  with specific projects over others, the process lacks transparency and the  assurance of fairness. For example, we found examples in which  reviewers noted that additional information could help them determine  whether a project met the statutory requirements (such as whether the  project was cost-effective) but less than half of the projects had the  chance to provide such information. Of the 26 awarded projects, half of  those projects (13 large projects) were afforded the opportunity to provide  additional information to demonstrate that their projects met the statutory  requirements.

DOT Provided Inconsistent Messages and Limited Transparency Regarding Its Award-Making Decisions

We were unable to determine the rationale for the selection of projects for  INFRA awards; an issue we also found with the FASTLANE process. This  is due to:  inconsistency in the NOFO regarding how merit criteria would be used  to select awardees; a large number of applications forwarded for potential award  regardless of merit scoring; and  limited documentation regarding why 26 projects were ultimately  selected out of 165 for award.

Inconsistency Regarding the Use of Merit Criteria for Selection

In the NOFO for INFRA, DOT provided inconsistent and unclear  messages regarding the extent to which the merit criteria should be  addressed to be competitive for an award, which also reduced  transparency and caused confusion for some applicants. OMB guidance  states that the intent of a NOFO is to make the application review process  transparent so applicants can make informed decisions when preparing  their applications to maximize fairness of the process. In the NOFO, DOT  stated it would evaluate applications against four merit criteria, but also  stated, “The Department is neither weighting these criteria nor requiring  that each application address every criterion, but the Department expects  that competitive applications will substantively address all four criteria.” In  some cases, this approach led to confusion among applicants, as several  of the selected applicants and consultants we interviewed noted that it  was difficult to address the innovation merit criterion, with some stating  the criterion was confusing or unclear and others stating that they faced  difficulties adapting their projects to meet the criterion. Compounding  this issue, several applicants and consultants also expressed uncertainty  as to how DOT determined which projects should receive awards and  which factors affected a project’s ability to get an award. For example,  representatives for one applicant noted that they spent a considerable  amount on a consultant for the benefit-cost analysis (which was common  among most of the applicants we interviewed), but it was not clear how  the benefit-cost analysis affected DOT’s decision-making.

Large Number of Applications Moved Forward Regardless of Their Merit Scores

Despite the language in the NOFO, DOT did not use the merit scores— which reflect the extent to which projects addressed all four criteria— when it determined which projects should be provided to the Secretary for  consideration. While DOT reviewers did score applications on all four  merit criteria, all of the 165 projects that QCO found to be statutorily  eligible—47 large projects and 118 small projects—were sent to the  Secretary for potential award, regardless of merit criteria scores or  whether the applicant substantively addressed all four merit criteria. DOT  officials told us that DOT sought a “portfolio” approach in which the  Secretary selected projects that scored highly on at least one criterion.  Thus, the Secretary received a 25-page spreadsheet showing 14 different  lists (7 for small projects and 7 for large projects) sorting all of the projects  against the merit criteria, with each list arranged from highest to lowest  score for that criterion. This method of presenting information on  projects (and the volume of information presented) would make it  challenging for any decision maker to compare projects and readily see  how 165 projects scored across all criteria and whether all criteria were  “substantively addressed.” In addition, projects were provided to the  Secretary for consideration—and in some cases awarded—despite  concerns raised by technical reviewers and regardless of whether  projects addressed all of the merit criteria. For example, we found  instances in which awarded projects had:

Low cost-effectiveness scores. Over 50 percent (14 of 26) of all  awarded projects received a high uncertainty rating related to their  benefit-cost ratio and net present value score, meaning that the  technical team had a low degree of confidence in the assigned score.  Only 38 percent of all projects had this uncertainty rating. Moreover,  of the 14 awarded projects with this rating, 11 had benefit-cost ratios  of 1.0 to 1.5, which, when combined with the high uncertainty rating,  raises the risk that the project would not be cost-effective. For  example, for one large project, a technical reviewer noted, “… we  conclude that the benefits of this project are reasonably likely to  exceed its costs, though the case is very marginal and highly  uncertain, as even a small change in some of the key assumptions  and parameters could result in a negative finding.” Uncertainty  regarding projects’ benefit-cost ratios is particularly important as DOT  used these scores to assess whether large projects met the FAST Act  requirement to be cost-effective. While comments from technical  reviewers were not included in the spreadsheets provided to the  Secretary, an official stated that the Senior Review Team reviewed  each project in-depth with the Secretary, and other DOT officials  noted that the spreadsheet provided to the Secretary included the  uncertainty ratings for each project.

Low scores on multiple criteria, or did not address all criteria.

Two awarded small projects had a benefit-cost ratio of less than one,  and one of those projects did not address the innovation criteria at  all. Three of the 26 awarded projects (11.5 percent) did not address  the innovation criteria at all.

In addition, several of the selected applicants and consultants we  interviewed expressed confusion regarding how DOT reviewed the  applications and moved them forward within DOT. Some of the applicants  and consultants thought that DOT used the project scores to determine  which projects should move forward to the Secretary (similarly to previous  rounds of other DOT grant programs in which projects were sorted into  categories such as “highly recommended,” “recommended,” and “not  recommended”). One applicant noted that it is important to know how  many projects make it to the Secretary in order to understand the extent  to which decisions are based on technical scores versus other  considerations.

Limited Documentation of Rationale for Award Decisions

DOT’s guidance states that grant recipients should be selected based on  technical merit and those projects most likely to achieve the intended  purpose. In addition, we have identified recommended practices for  awarding discretionary grants, one of which includes documenting the  rationale for award decisions. Documenting the rationale for award  decisions becomes even more important in light of DOT’s decision to  provide every eligible INFRA application to the Secretary, rather than  providing the Secretary a list of the projects “most likely to achieve the  intended purpose.” However, DOT’s documentation on the final selection  of projects states the anticipated benefits of the projects but does not  indicate why these projects, according to DOT’s guidance, “best address  program requirements and, therefore, are most worthy of funding.” In our  review of the FASTLANE process, we also noted that due to limited  documentation, we could not determine how DOT selected which projects  should receive awards. We recommended that DOT require program  teams to document their decision-making rationale throughout all levels of  review in the application selection process. DOT agreed with this  recommendation; however, it has not yet been implemented. Therefore, it  remains unclear whether DOT is awarding discretionary grants on the  basis of merit principles or other considerations. An absence of  documentation can give rise to challenges to the integrity of the  evaluation process and thus the decisions made.

Limitations in the INFRA Process Reflect Recurring Issues in DOT’s Discretionary Grant Programs

Since 2011, we have found similar issues with DOT’s management of  other competitive discretionary grant programs, including a lack of  documentation of key award decisions, and have made recommendations  aimed at increasing consistency and transparency. In some cases, DOT  implemented our recommendations for one program, but we  subsequently found similar or recurring problems in other DOT programs.

In 2011, we reviewed DOT’s Transportation Investment Generating  Economic Recovery (TIGER) program and FRA’s High Speed Intercity  Passenger Rail program—two discretionary grant programs funded  through the American Recovery and Reinvestment Act of 2009. For  both programs, we found, among other things, limitations in the agencies’  documentation of the rationale for award decisions. With respect to  TIGER, we noted that a lack of documentation could subject DOT to  criticism that projects were selected for reasons other than merit.  However, we also noted that documenting key decisions could help build  confidence in DOT’s ability to administer competitive discretionary grant  programs. We recommended that DOT and FRA improve their  documentation of key decisions for both programs. DOT implemented  these recommendations by updating its TIGER and FRA guidance to  require additional documentation.

Despite the steps DOT took to address our prior recommendations, in  2014, we found continued issues in the TIGER program and made more  targeted recommendations. Specifically, we found that DOT did not  document key decisions to, among other things, (1) advance projects with  lower technical ratings instead of more highly rated projects, and (2)  change the technical ratings of lower-rated projects that had been  selected for an award. We recommended that the Secretary of  Transportation establish additional accountability measures for  management of the TIGER program, to include using a decision  memorandum or similar mechanism to document a clear rationale for  decisions to:  change the technical evaluation rating of an application, not advance applications rated as highly recommended, and advance for senior review applications other than those rated as  highly recommended.

Subsequently, DOT revised its guidance for the TIGER program to  prohibit changes to the technical ratings, require that all highly rated  projects be advanced, define the conditions through which lower rated  projects may be advanced, and require that all such decisions be fully  documented. DOT did not require that these decisions be documented  through a decision memorandum or similar mechanism, as we had  recommended. However, taken together, we determined DOT’s actions  were sufficient to address our recommendation for the TIGER program.

In December 2016, we found similar problems during our review of the  Hurricane Sandy transit-resilience grant program administered by the  Federal Transit Administration (FTA). For example, we found that FTA  did not document rationales for changes to project ratings nor did it  document how it addressed high-level project concerns raised by  reviewers in their evaluation comments. In addition, we found that DOT  lacked clear department-wide requirements for what should be  documented when evaluating and selecting discretionary grant awards.  We noted that internal control standards state that all transactions and  significant events need to be clearly documented, and that a  recommended practice for evaluating and selecting discretionary grant  awards is documenting the rationale for awards decisions, including  reasons individual projects were selected or not selected. We also found  that FTA did not develop an evaluation plan prior to calling for  applications, despite the fact that this was a requirement in DOT’s  Financial Assistance Guidance Manual and that recommended practices  for administering discretionary grant programs note the importance of  having an evaluation plan that describes a method for overseeing the  technical review panels to ensure a consistent review. Finally, we found  that FTA did not assess projects against the policy priorities it outlined in  its notice of funding availability, despite an OMB directive to provide  sufficient information to help an applicant make an informed decision  about whether to submit a proposal.

At this time, we noted a pattern of problems occurring across DOT and its  modal administrations’ discretionary grant programs and determined that  a department-wide action was needed to address these issues.  Specifically, we recommended that the Secretary issue a department- wide directive that should include requirements to: develop a plan for evaluating project proposals in advance of issuing  a notice of funding availability that defines the stages of the process,  including how the process will be overseen to ensure a consistent  review of applications; document key decisions, including the reason for any rating changes  and the officials responsible for those changes, and how high-level  concerns raised during the process were addressed; and align stated program purpose and policy priorities with the evaluation  and selection process.

DOT concurred with our 2016 recommendation to develop a department- wide directive and initially stated that it would address it by updating its  Financial Assistance Guidance Manual by September 2018 (DOT  recently extended this to December 2019). In response, we noted that in  order to address our recommendation, DOT needed to issue a directive  that incorporates all of the elements identified in our recommendation. In  addition, it remains unclear whether updating the manual would have the  same effect as issuing the department-wide directive that we  recommended. Specifically, we have found that DOT has not always  followed its own guidance despite clear language that certain actions are  required. For example, in our 2017 review of the FASTLANE program, we  noted that the Financial Assistance Guidance Manual required finalization  of the evaluation plan prior to soliciting applications for grants, but this  guidance was not followed. Since 2017, we have sent letters to the Secretary of Transportation noting that this is a high-priority  recommendation that warrants her attention.

In March 2019, DOT issued a one-page memo to all offices and  departments that administer discretionary grants. This memo directed the  offices to update their policies and procedures to implement our 2016  recommendation and to send the updated policies to DOT’s Office of the  Senior Procurement Executive by June 30, 2019. DOT officials told us  that DOT believes this action has addressed the recommendation. Due to  a number of issues, however, it is unclear how this action will address our  recommendation to create clear department-wide requirements aimed at  improving transparency and consistency. Specifically, we found that the  memo was essentially limited to a repetition of our recommendation. That  is, DOT did not take steps to ensure that the various affected offices  consistently interpret and implement the recommendation. For example,  DOT did not define key terms such as “high level concerns,” or “key  decisions.” In addition, DOT did not communicate to offices how they  should sufficiently document their decisions to ensure that the rationale  for those decisions—including the reasons individual projects were  selected or not selected—is clear. DOT officials told us they wanted to  provide the affected offices flexibility to implement the recommendation  and would assess the need for additional guidance based on the  completion of the Financial Assistance Guidance Manual. However, the  lack of information regarding how offices should implement the memo  raises significant questions about whether various offices will interpret  and implement the recommendations differently, and enhances the risk  that DOT will continue to lack a department-wide approach to ensure that  discretionary grant programs are consistently and transparently  administered.

As DOT continues to try to address these long-standing issues with its  discretionary grant programs, Congress has an opportunity through  reauthorization legislation, scheduled for 2020, to build requirements for  enhanced consistency and transparency into these programs. This is  particularly important as DOT has two additional rounds of INFRA funding  to award under the FAST Act, and the President’s Budget proposal  proposed providing an additional $1 billion to INFRA. Moreover, the FAST  Act also authorized over a dozen discretionary transportation grant  programs, and Congress may consider additional programs during the  reauthorization of DOT’s surface transportation programs. Through  legislation, Congress could craft requirements around the administration  of DOT’s discretionary grants to improve the processes for awarding  grants. Absent effective action by DOT going forward, the recurring and  long-standing issues we have identified could continue to affect DOT’s  discretionary grant programs.

While DOT has taken some steps to improve its reviews of INFRA grant  applications since we reviewed the FASTLANE program, issues related to  consistency and transparency remain. Specifically, without clear  communication from DOT regarding (1) the situations in which DOT may  provide certain applicants the opportunity to supplement their applications  with additional information, and (2) how merit scoring is used, if at all, to  determine whether projects advance to the Secretary for selection and  which projects are selected, applicants lack the information needed to  make informed decisions about whether to apply. In addition, without  documentation outlining why DOT decided to request additional  information from certain applicants over others, the process lacks  transparency.

Since the FAST Act was enacted in 2015, we have been unable to  determine the basis for the resulting awards of about $2.3 billion through  the FASTLANE and INFRA program. This lack of clarity is significant and  is the product of long-standing issues that we have identified with DOT’s  discretionary grant programs since 2011. We have previously noted that  competitive discretionary grant programs have promise in better targeting  federal transportation spending to areas of national and regional  significance; however, this promise cannot be fulfilled if DOT’s process  and rationale for making awards remains unclear. In 2019, DOT issued a  department-wide memo aimed at addressing our 2016 recommendation,  but it is unclear how DOT’s approach will improve consistency and  transparency in its management of grant programs. We will continue to  monitor DOT’s efforts to address our recommendation. However, given  the long-standing nature of the issues we identified and the potential that  they could continue to affect DOT’s discretionary grant programs, the  reauthorization of DOT’s surface transportation programs scheduled for  2020 provides Congress the opportunity to require DOT to take additional  action to ensure consistency and transparency in the management of its  discretionary grant programs.

During the next reauthorization for surface transportation programs,  Congress should consider including language in the reauthorization bill  that would require DOT to develop and implement transparency  measures for DOT’s review and selection process for discretionary  grants. Such measures should, at a minimum, help to ensure that the  evaluation process is clearly communicated, that applications are  consistently evaluated, and that the rationale for DOT’s decisions are  clearly documented. Such measures should be developed in line with  OMB guidance, federal internal control standards, and recommended  practices for evaluating and selecting discretionary grant awards (Matter  for Consideration 1)

We are making the following three recommendations to DOT:  The Secretary of Transportation should ensure that DOT, in its notice of  funding opportunity and evaluation plan for each remaining INFRA- funding cycle, clarify the circumstances under which DOT may select  applicants to receive requests for additional information.  (Recommendation 1)

The Secretary of Transportation should develop procedures for each  remaining INFRA-funding cycle to ensure that when additional information  is requested from an applicant, the specific rationale behind the request is  documented (for example, to promote geographic diversity among  projects), as well as to ensure that DOT documents the rationale if similar  projects were not afforded an opportunity to provide additional  information. (Recommendation 2)

The Secretary of Transportation should ensure that DOT provides  information to applicants in its notice of funding opportunity for each  remaining INFRA-funding cycle regarding: (1) how scores on merit criteria  are used, if at all, to determine whether projects advance to the Secretary  for selection, and (2) how, if at all, DOT plans to use merit scores to  determine which projects should receive an award. (Recommendation 3)

We provided a draft of this report to DOT for review and comment. In  comments, reproduced in appendix II, DOT concurred with our  recommendations. DOT noted its efforts to improve the INFRA process  for the 2019 round of funding and stated that it looks forward to assisting  Congress in addressing the matter for congressional consideration in a  manner that is feasible within DOT’s timing and resource constraints.  DOT also provided technical comments, which we incorporated as  appropriate.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of Transportation, and other  interested parties. In addition, the report will be available at no charge on  the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-2834 or flemings@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.

Appendix I: Summary of the Department of Transportation’s Rating Guidelines for the Infrastructure for Rebuilding America Program

Likely NEPA status/type of action required,  based on available information (such as  expecting the project to be found to have no  significant impact on the environment or that  the project would be required to be  reevaluated)

The likelihood the project will be  able to be delivered by its obligation  timeframe

High risk = high likelihood that the project will  not be obligated on time

Medium risk = some possibility the project  will not be obligated on time  Low risk = highly likely the project will be  obligated on time  The Special Experimental Project authorities (SEP 14/15 waiver) is a program that identifies and  tests innovative project-delivery methods (such as non-traditional contracting techniques).

Appendix II: Comments from the Department of Transportation

Appendix III: GAO Contact and Staff Acknowledgments

Susan Fleming, (202) 512-2834 or flemings@gao.gov.

In addition to the contact named above, Steve Cohen (Assistant Director);  Crystal Huggins (Analyst in Charge); Amy Abramowitz; Melissa Bodeau;  Michelle Everett; Geoffrey Hamilton; Joshua Ormond; Oliver Richard;  Kelly Rubin; and Charles Truxillo made key contributions to this report.

The Department of Transportation awarded over $1.5 billion in grants for nationally significant freight and highway projects for 2017-2018. We reviewed its grants process.

We found DOT's application reviews for the Infrastructure for Rebuilding America program lacked consistency and transparency. For example, after DOT found 97 applications were missing information, it followed up with 42. It did not sufficiently document why it only contacted some applicants.

Since 2011, we have recommended DOT improve its grant programs. In this report, we recommended, among other things, that Congress consider requiring DOT to enact transparency measures.