<jupyter_start><jupyter_text>Ambulance Demand Forecasting. Generate results summary filesThis notebook is the master notebook that runs all of the results summary notebooks that create the figures and tables used in the paper.> Figures are saved to `paper/figures`;  tables to `paper/tables` and appendicies to `paper/appendix`Results summary:* **Model selection stage 1**: screening 15 methods on a single time series* **Model selection stage 2**: cross validation of FBProphet and Ensemble on 7 time series* **Benchmarking**: multiple simulated forecasts of ensemble on 7 unseen time series from SWAST.* **External validation**: multiple simulated forecasts of ensemble on 13 unseen time series from London, Wales and Yorkshire.  Model Selection Stage 1For the paper this notebook generates* Table 2* Appendix tables S1-3.<jupyter_code>%run results/01_summary_stage_1.ipynb<jupyter_output>Appendix summary tables written to ./paper/appendix/

Table 2: Cross-Validation of Seasonal Naive Point Forecasts
          sMAPE            RMSE         MASE
7   3.48 (1.19)   96.91 (37.11)  0.94 (0.35)
14  3.94 (1.32)  109.71 (41.88)  1.06 (0.39)
21  4.12 (1.38)  115.38 (42.94)  1.12 (0.41)
28  4.19 (1.33)  118.34 (42.92)  1.14 (0.40)
35  4.35 (1.34)  123.26 (42.33)  1.18 (0.40)
42  4.51 (1.33)  128.11 (40.34)  1.23 (0.40)
49  4.61 (1.32)  131.40 (39.79)  1.25 (0.39)
56  4.67 (1.33)  133.95 (40.20)  1.27 (0.39)
63  4.77 (1.36)  136.59 (40.30)  1.30 (0.40)
70  4.82 (1.26)  138.70 (37.25)  1.31 (0.37)
77  4.88 (1.18)  140.73 (34.49)  1.33 (0.34)
84  4.94 (1.21)  143.04 (34.07)  1.34 (0.35)<jupyter_text>Model Selection Stage 2.* Figure 3: Cross-validation variation in MASE across regions by horizon* Figure 4: Cross-validation variation in Prediction Interval Coverage across regions by horizon<jupyter_code>%run results/02_summary_stage_2.ipynb<jupyter_output>Figure 4: Cross-validation variation in Prediction Interval Coverage across regions by horizon<jupyter_text>Benchmark For the paper this notebook generates* Table 3: ./paper/tables/table3.tex  (horizon summary)* Table 4: ./paper/tables/table4.tex  (forecast distribution by region)* Statistics used in results section 3.4<jupyter_code>%run results/03_benchmark_summary.ipynb<jupyter_output>******************Summary of benchmark results*****************



Table 4: Prediction interval coverage by sub-region.
                                   60%                    70%  \
BNSSG            0.639 (0.634 - 0.644)  0.746 (0.739 - 0.753)   
Cornwall         0.637 (0.634 - 0.639)  0.762 (0.760 - 0.764)   
Devon            0.691 (0.684 - 0.699)  0.793 (0.789 - 0.797)   
Dorset           0.712 (0.708 - 0.715)  0.803 (0.800 - 0.805)   
Gloucestershire  0.626 (0.621 - 0.632)  0.730 (0.728 - 0.733)   
Somerset         0.629 (0.625 - 0.633)  0.723 (0.718 - 0.728)   
Wiltshire        0.665 (0.662 - 0.668)  0.762 (0.760 - 0.765)   

                                   80%                    90%  \
BNSSG            0.844 (0.839 - 0.850)  0.922 (0.918 - 0.926)   
Cornwall         0.856 (0.855 - 0.857)  0.921 (0.921 - 0.922)   
Devon            0.857 (0.856 - 0.859)  0.939 (0.935 - 0.942)   
Dorset           0.873 (0.871 - 0.876)  0.940 (0.938 - 0.941)   
Gloucestershire  0.835 (0.833 - 0.[...]<jupyter_text>External validation* Statistics used in section 3.5* Table 5: External validation: Point forecast and coverage performance by region<jupyter_code>%run results/04_summary_external_validation.ipynb<jupyter_output>******************Summary of External validation results*****************



Overall External Evaluation Statistics for MASE
mean: 0.7275 95% CI (0.7193-0.7357)
median: 0.734, IQR 0.084
middle 90%: 0.641 - 0.834
{'fraction_non_alphanumeric': 0.13674519360953155, 'fraction_numerical': 0.16978066612510154, 'mean_word_length': 2.746450304259635, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '79403', 'license': 'MIT', 'n_tokens_mistral': 1672, 'n_tokens_neox': 1332, 'n_words': 412, 'path': 'results_summary_main.ipynb', 'repo_name': 'TomMonks/swast-benchmarking', 'size': '3693'}