Chapter: Appendix K - Examples of Risk and Uncertainty
« Previous: Appendix J - Multiple Accounts BCA Example
Suggested Citation:"Appendix K - Examples of Risk and Uncertainty." National Academies of Sciences, Engineering, and Medicine. 2017. Guide for Conducting Benefit-Cost Analyses of Multimodal, Multijurisdictional Freight Corridor Investments. Washington, DC: The National Academies Press. doi: 10.17226/24680.
152 K.1 Example 1: National Gateway The National Gateway is a major freight transportation project that has been awarded TIGER funds from the U.S. DOT. Rail links between the Midwest and ports in the Mid-Atlantic region will be upgraded to allow trains to be double-stacked. The project focuses on three corridors: one between Washington, D.C., and Ohio; another between Baltimore and North Carolina going through Washington, D.C.; and a third cutting across the state of North Carolina. Project proponents included a detailed appendix describing the methodology they used to estimate benefits and costs with their TIGER grant application. This document includes short lists of "General Assumptions" and key model input "Variables" along with the nominal values used in the analysis. This is a good example of transparency. It also serves as an identification of potential sources of uncertainty although the authors do not specifically call out the assumptions and variables as sources of uncertainty or categorize the assumptions and variables in terms of types of uncertainty. This document also includes the results of various sensitivity studies. For example, the authors identify seven specific scenarios of interest and provide estimates of project benefits and costs in each of these scenarios reproduced here as Figure K1. This approach allows readers to evaluate the project based on the readers' own interpretations of how likely and/or meaningful different scenarios are. The appendix would be even more helpful if additional scenarios were considered, the impacts of individual assumptions and variables were studied in turn and compared, or if there were one or more pessimistic scenarios studied, following the recommendations of this guidebook. In another table, presented here as Figure K2, the authors list how many jobs will be created in several regions as a function of a key model parameter, the "Average Jobs/1000 Lifts." K.2 Example 2: Downeaster Service Optimization Project The Downeaster Service Optimization Project was a proposed effort to improve rail links in New England to enable passenger trains to run between Brunswick, Maine, and Boston, Massachusetts, five times daily. Project proponents put together an application for a 2014 TIGER program grant. The Northern New England Passenger Rail Authority provided the results of a BCA as part of their application for a 2014 TIGER grant. The BCA included an appendix specifically focused on sensitivity analysis. Examples of Risk and Uncertainty A p p e n d i x K
examples of Risk and Uncertainty 153 Source: National Gateway (1) Figure K1. National Gateway sensitivity analysis, benefits, and costs. Source: National Gateway (1) Figure K2. National Gateway sensitivity analysis, jobs. The document provided by the Northern New England Passenger Rail Authority provides distributions for parameters of interest instead of only point estimates. Figure K3 is an example of the results provided. The reader can examine both a histogram and several summary statistics regarding the distribution for the project's BCR. The publication of confidence intervals is a particularly good idea. On the other hand, much of Figure K3 is not particularly useful. There is a table in the top left that contains only "N/A" values. Data on percentiles duplicates the infor- mation that is better summarized by the histogram. Many summary statistics of questionable usefulness, such as kurtosis or the sum of the 500 estimates of BCR that were apparently gener- ated, are presented alongside actually useful summary statistics, such as median. This guidebook recommends publication of the histogram (minus the empirical cumulative distribution function curve shown in Figure K3), the 95% confidence interval, the mean, and the five number summary of any model result of interest. No other data are required. The document provided by the Northern New England Passenger Rail Authority provides information on the distribution of many model outputs. As an example, Figure K4 describes the values generated by Monte Carlo simulation when monetizing the social costs of SOx emissions. Much of the unnecessary information from Figure K3 is not present in Figure K4. The docu- ment never makes clear what the values at the top of Figure K4 represent. The histogram and 95% confidence interval for the cost of SOx emissions are provided in Figure K4 without the overwhelming number of summary statistics that can be found in Figure K3.
154 Guide for Conducting Benefit-Cost Analyses of Multimodal, Multijurisdictional Freight Corridor investments Source: Northern New England Passenger Rail Authority (2) Figure K3. Distribution of B/C ratio for Downeaster project.
examples of Risk and Uncertainty 155 Source: Northern New England Passenger Rail Authority (2) Figure K4. Distribution of SOx emissions, Downeaster project. References 1. National Gateway. National Gateway TIGER Grant Appendix B: Cost Benefit Methodology in Evaluation of Project Costs and Benefits and Economic Impacts. September 14, 2009. http://www.nationalgateway.org/sites/ default/files/pdfs/EA/Appendix%20B%20-%20Cost%20Benefit%20Methodology.pdf. 2. Northern New England Passenger Rail Authority. Benefit-Cost Analysis: Downeaster Service Optimization Project: FY2014 TIGER Discretionary Grant Program. April 25, 2014. http://www.amtrakdowneaster.com/ sites/default/files/Tiger6_BCA.pdf.
Next: Appendix L - Heartland Corridor Case Study »