extfieldsofstudy:
Computer Science
Engineering
Biology

text:
StationPlot: A New Non-stationarity Quantification Tool for Detection of Epileptic Seizures

A novel non-stationarity visualization tool known as StationPlot is developed for deciphering the chaotic behavior of a dynamical time series. A family of analytic measures enumerating geometrical aspects of the non-stationarity&degree of variability is formulated by convex hull geometry (CHG) on StationPlot. In the Euclidean space, both trend-stationary (TS)&difference-stationary (DS) perturbations are comprehended by the asymmetric structure of StationPlot's region of interest (ROI). The proposed method is experimentally validated using EEG signals, where it comprehend the relative temporal evolution of neural dynamics&its non-stationary morphology, thereby exemplifying its diagnostic competence for seizure activity (SA) detection. Experimental results&analysis-of-Variance (ANOVA) on the extracted CHG features demonstrates better classification performances as compared to the existing shallow feature based state-of-the-art&validates its efficacy as geometry-rich discriminative descriptors for signal processing applications.

I. INTRODUCTION
Robust biomarkers are essential for early stage epileptic seizure detection from electroencephalogram (EEG) signals. EEG signal contains information related to neurological activities of the human brain [1] & neural impediments in the transmission flow paths manifests in chaos within the brain segments, resulting in physio-pathological brain-related disorders [2]. Several time-domain techniques such as waveletchaos in EEG sub-bands [1], empirical mode decomposition (EMD), EMD modulation bandwidths & least-square support vector machine (LS-SVM) [2], EMD-variation features [3], Hilbert marginal spectrum analysis [4] & Hilbert Huang Transform (HHT) aided field programmable gate array (FPGA) [5] implementation have been reported for SA detection. Further, joint time-frequency techniques like wavelet transform & machine learning classifiers [6,7], intra-cranial spatiotemporal correlation structures [8], time-frequency image descriptors [9] & multi-level wavelet decomposition [10] have shown potency in epileptic seizure diagnosis. Non-linear methods like permutation entropy & support vector machines (SVM) [11], multi-scale sample entropy [12], rational discrete short-time Fourier transform [13], sparse coding using Slantlet transform [14] for multi-resolution quantification & sparse representation of seizure free (interictal) & seizure (ictal) EEG signals is studied in [15]. Application of local pattern transformation *This paper is accepted for presentation at IEEE Global Conference on Signal and Information Processing (IEEE GlobalSIP), California, USA, 2018 features [16], ARIMA-GARCH models [17] & L l regularized EEG attributes [18] have shown adequacy as potential SA biomarkers. Mode decomposition based EEG modeling [2][3][4] & multi-fractal detrended fluctuation analysis (MFDFA) [23,24] have been studied to exploit the underlying nonlinearity & non-stationary nature of EEG fluctuations. Highdimensional EEG data embedding via t-SNE based projection by maximizing the inter-class & minimizing the intra-class variances of seizure & healthy class with high probability of visualization is discussed in [25]. A comprehensive feature engineering for EEG signal analysis can be traced from [1,2,21], where the significance of an exhaustive & robust geometry-rich model to abstract the non-stationary EEG time series analytics cannot be over-emphasized. As such, a new non-stationarity quantification tool, a.k.a, StationPlot is proposed. Stationplot is a novel geometry rich non-stationarity quantification tool for generic time series analysis. It gives succinct feature engineering & analytical insights about the temporal evolution of stationarity in a time series. Ongoing, StationPlot theory & its CHG features are given in Section II. Experimental results on EEG dataset is given in Section III. Section IV concludes the paper.

A. Trend-stationary (TS) & Difference stationary (DS) Signals
Non-stationary signals have an unpredictable joint probability distribution due to combined presence of trends, random walks & cycles, which poses significant difficulty in statistical parameter estimation such as mean & co-variances. De-trending or successive differencing of a non-stationary signal yields a TS or DS time-series respectively [19]. Deterministic mean trend removal of a signal results in persistent forecasting interval widths, which exhibits recurrent long-run trending behavior, whereas successive differencing is required for stochastic mean trend removal. If a timeseries becomes stationary after differencing 'D' times, it is called D th order DS time-series. Such signals never recuperate from the random fluctuations & forecasting intervals grows over time [19]. Mathematically, Box-Jenkins non-stationary time series analysis by successive differencing to attain stationarity can be traced as ∆ Dyt = µ + Ψ(L)ε t . Where, ∆ D = (1 − L) D is the D th order differencing operator & Ψ(L) = (1 + Ψ 1 L + Ψ 2 L 2 + ...) is the infinite-degree operator polynomial with coefficients being absolutely summable & all roots lying outside the unit circle.

B. StationPlot
The trend present in the given time series, x t is de-trended by subtracting the mean value or linear trend of the feature vector & a least-squares fit of the time series is envisaged. For the original time series, we have, x t = µ t + ε t . Where, µ t signifies the deterministic mean trend in the time series & ε t denotes the stochastic stationary process with zero means. De-trending is effectuated by parametric regression techniques or by non-parametric decomposition using filters such that: In the Euclidean space, we define the 2D & 3D StationPlot & its subsequent feature extraction thereof. For, the 2D-planar case, n th order StationPlot is defined as the plot of X 1 (n) versus X 2 (n), where, While, for 3D, n th order StationPlot is defined as the surface generated by plotting X 1 (n), 2D-StationPlot depicts the analytical variability of ∆ n X(t) versus ∆ (n+1)X(t) . It indicates the successive differencing effects on data sampling with system dynamics approaching continuum in the limiting scenario of sampling. In order to quantify the variability & asymmetric behavior of the StationPlot, convex hull analytics (CHA) of the sampled datapoints is done for variability characterization. Fig. 1 shows typical representative 3D-StationPlot of EEG signals.

C. Convex hull geometry (CHG) on StationPlot 1) Convex hull (CH) of a set X:
In the affine space, CH defines the convex envelope which completely encloses all the data points in X, such that no concavities are present Algorithm 1 : 2-D StationPlot visualization Input: (Degree of Trend-removal) / (Differencing) order = m,n f (n) ∼ = EEG time series, ∀nǫ EEG data points Output: 2D-stationPlot representation of EEG signal.
1: function 2D-StationPlot (f (n), m, n) 2: Local variables: X 1 , X 2 , n max = size (Data points set) [21]. The convex combination of k number of datapoints in X, i.e., CH spans the set of all possible convex combinations of datapoints in X by taking all the permutation of the coefficients, θ k . In the closed form, the convex hull can be expressed as: For our case, the set X denotes all the time series data-points projected in the n-Dimensional StationPlot. In the present study, n is restricted to 2 & 3. Quickhull algorithm [20] has been used to compute the CH & CHG thereof. Following CHG of the StationPlot, various features pertaining to the non-stationarity space geometry analytical significance are comprehended. Fig. 2 shows a representative 2-D StationPlot. x 1 x 2 y 1 y 2 + x 2 x 3 y 2 y 3 + ... + x n x 1 y n y 1 CHP is mathematically written as:

3) Circularity (C):
The perturbed elliptical shape of the 2D-StationPlot's ROI exhibits asymmetric geometry, which is apprehended by C measuring the degree of roundness of the convex hull & its deviation from its circular nature. C is independent of linear transformations. It is defined as: 4) Aspect ratio: The aspect ratio measures the ratio of ROI's main inertia axis length, I main to ROI's minor inertia axis length, I minor .
Aspect ratio = I main I minor

A. EEG dataset & its acquisition
EEG signals being highly non-linear & non-stationary in nature has been used for experimental validation of the proposed method was validated using. We have used publicly available EEG dataset provided by Bonn University [26].

B. Statistical analysis
Prior to data sampling, the obtained EEG data records are subjected to band-pass filtering based pre-processing. For each of these 5 classes, CHG features are extracted from StationPlot's ROI & are initially subjected to Analysis-ofvariance (ANOVA) test for testing p-values. ANOVA, a statistical test used to analyze the group means differences in a sample & characterize the discriminating significance in terms of p-values. A smaller p-value is associated with a stronger evidence in favor of the alternative hypothesis that the feature is effective for segregation. Fig. 4 shows box-plot of the extracted convex hull geometric feature variations. It clearly exhibits that the non-overlapping distribution band of the feature values. Table I gives the corresponding p-values of the feature descriptors for the two-class classification problem as contemplated & demonstrates the class discriminating capability of the proposed CHG feature attributes. It can be noted that p-values of ≤ 0.01 from Table 1 signifies the efficacy of CHG features in SA detection using EEG signals.

C. Classification Methodology & Performance Measures
By employing Kruskal-Wallis based feature ranking, most dominant features are selected & fed to support vector machine (SVM) classifier for classification. In the present study, we confine ourselves to the following kernels: linear, quadratic, polynomial & RBF kernel, where the optimal kernel parameters selection are governed by Mercer's theorem. For each of these feature vector classes, 70% samples are randomly selected for training & the rest 30% is used for testing. These data partitioning, training & testing, is repeated 100 times with training & testing samples randomly selected for each run & the mean performance measures are reported.    it requires heuristic hyper-parameter tuning for optimal performance, whereas StationPlot is an analytical closed form nonstationarity quantification framework subjugate these drawbacks & give state-of-the-art performances.

IV. CONCLUSION
A novel non-stationarity quantification tool for early stage SA detection using EEG signals is presented. The proposed method can be used as an effective chaos modeling tool for any non-stationary time series visualization & analysis of temporal evolutionary behavior & its quantification thereof, in terms of trend-stationary & difference-stationary components via CHG. The D th order differencing statistics exhibits significant knowledge about the underlying system progression over time & adequately captures the underlying non-stationarity structure analytics, which is otherwise inaccessible. Moreover, classification accuracy of 99.31% for (A vs E) & 98.79% for (ABCD vs E) problem, shows its adequacy in EEG signal classification. Ongoing, we are escalating our method for noise robustness via inclusion of area moments to study the distribution of non-stationary points w.r.t. to any arbitrary axis.


extfieldsofstudy:
Computer Science
Medicine

text:
Wrangling distributed computing for high-throughput environmental science: An introduction to HTCondor

Biologists and environmental scientists now routinely solve computational problems that were unimaginable a generation ago. Examples include processing geospatial data, analyzing -omics data, and running large-scale simulations. Conventional desktop computing cannot handle these tasks when they are large, and high-performance computing is not always available nor the most appropriate solution for all computationally intense problems. High-throughput computing (HTC) is one method for handling computationally intense research. In contrast to high-performance computing, which uses a single "supercomputer," HTC can distribute tasks over many computers (e.g., idle desktop computers, dedicated servers, or cloud-based resources). HTC facilities exist at many academic and government institutes and are relatively easy to create from commodity hardware. Additionally, consortia such as Open Science Grid facilitate HTC, and commercial entities sell cloud-based solutions for researchers who lack HTC at their institution. We provide an introduction to HTC for biologists and environmental scientists. Our examples from biology and the environmental sciences use HTCondor, an open source HTC system.

Scientists have traditionally processed "big" data sets and run large-scale simulations using either desktop computing (and long wait times) or high-performance computing (HPC; with higher overhead costs in a central location). These "supercomputers" use parallel computing to locally run a simulation or process data on a single, large computer. Historically, only researchers at large institutes, national laboratories, and large universities had access to HPC systems and the funding to restructure code to run on such a computer.

High-throughput computing
Many problems can be broken down into smaller, loosely coupled jobs (defined in Box 1) and distributed across computer pools. Examples from the environmental sciences can include Box 1. HTCondor terminology adapted from [19] and the HTCondor Manual [27] • Job: The task a user wants HTCondor to run. Sometimes the official HTCondor Documentation refers to a "job" as the large task being submitted, and other times, it refers to the jobs as the specific files being submitted.
• Batch/shell script: A file with instructions for a computer to execute.
• Submit machine: One or more computers used by HTCondor to send jobs out at the start of a run and collect jobs at the end of a run. The submit machine can also run jobs but is not required to.
• ClassAds: A language used by HTCondor to advertise resources on machines running HTCondor. The submit files request resources based upon the ClassAds.
• Submit file: A file submitted by the user to the central manager. This file tells HTCondor the job requirements and what files to run as part of the job.
simulating study designs to compare model performance and study performance [14], model calibrations [16], and processing telemetry data [17]. This alternative paradigm is called highthroughput computing (HTC) [18][19][20]. Originally designed as a method to "scavenge" unused computational resources (e.g., idle desktop computers) in the 1980s, HTC has emerged as a method for distributed computing that can be used with dedicated servers. HTC-based approaches require users to break a large problem into many small, independent jobs that are distributed across a pool by the HTC software. Computers then process the task in many small parts. Furthermore, this allows both the task from a specific user as well as the tasks from multiple users to be distributed and processed in parallel. HTC software automatically manages the individual jobs and is designed to be robust. Jobs may be stopped (e.g., a desktop computer is no longer idle because the owner has returned to work) or lost (e.g., a connection is lost and a job needs to be restarted). This distributed computing paradigm has led to HTC being used to run other large tasks that would be too big even for one supercomputer. Currently, many different industries and scientific fields use HTC. One specific implementation of HTC is HTCondor. HTCondor, originally called Condor, coordinates computing resources on either dedicated machines or idle machines such as desktop computers [18]. When first developed in the 1980s, HTCondor could submit and manage thousands of jobs. Currently, HTCondor can handle more than 200,000 jobs [21]. Many different organizations use HTCondor: the Open Science Grid uses HTCondor to pool computational resources across institutions [22]; the European Organization for Nuclear Research (commonly known by its French acrynomn CERN) uses HTCondor to process high-energy physics data [23], such as finding the "God particle" [24]; Disney and DreamWorks use HTCondor to create movies (http://research.cs.wisc.edu/htcondor/images/WildLetter.pdf); the National Aeronautics and Space Administration (NASA) uses HTCondor to process telescope, atmospheric, and space data as well as run simulations [25]; the National Oceanic and Atmospheric Administration (NOAA) uses HTCondor to process weather data [26]; the Search for Extraterrestrial Intelligence (SETI) uses HTCondor to search for extraterrestrial life (http:// research.cs.wisc.edu/htcondor/manual/v7.8/3_12Setting_Up.html); and the United States Geological Survey (USGS) uses HTCondor for groundwater modeling [16].
Although HTC is currently used by some environmental sciences (e.g., genomics [20]), we see the opportunity for expanded use of HTC in the environmental sciences based upon our own experiences working with researchers and resource managers. HTC could help these scientists with their current computationally intensive computing tasks. Hence, we provide an introduction to HTC for these scientists.

Overview of HTCondor
HTCondor "has enabled ordinary users to do extraordinary computing" [19]. Broadly, HTCondor is a distributed-batch computing system, and our description of HTCondor applies to other HTC systems as well. Users submit a job to HTCondor. HTCondor chooses when and where to run the job based upon the job requirement and available worker machines. HTCondor monitors the jobs' progress, and HTCondor notifies the user upon completion. Practically, this means users submit their jobs from a submit machine (see Box 1 for definitions). A job is typically a batch script on Windows or a shell script on Linux/Unix/ macOS. The batch/shell script calls other programs in the workflow. A key aspect of HTCondor is that it requires no custom programming or linking to run in the cluster. This eases debugging and increases confidence of correctness, for the same job can usually be run interactively on a local machine. The submit machine submits jobs to different node machines. These node machines are computers with HTCondor installed. Node machines can be either "normal" computers (e.g., the administrative assistant's computer) or dedicated computers/servers that only HTCondor uses. Nodes have ClassAds that are like "classified advertisements" in a newspaper and communicate a machine's resources (e.g., operating system, available memory, or central processing unit [CPUs]). ClassAds are a language developed by the HTCondor project. End users will not need to change ClassAds unless they are configuring machines in the HTCondor pool. Users send submit files to the submit machine, which matches jobs to machines. Once jobs are done, HTCondor gathers up and returns output files to the user.
Ideally, a user will not need to setup their own HTCondor pool. Many universities, government agencies, and national laboratories offer HTC facilities. Additionally, many countries have science grids available that use HTCondor (e.g., Open Science Grid in the United States [22]; GridPP in the United Kingdom [https://www.gridpp.ac.uk/]). Commercial vendors also offer HTCondor (e.g., Amazon Web Service). Last, HTCondor is free and open source, and anybody may setup their own local pool. We have included documentation outlining our installation process (AR1), but otherwise, assume users have access to their own pool.

HTCondor Quick Start example
The HTCondor Manual provides a Quick Start Example (https://research.cs.wisc.edu/ htcondor/manual/quickstart.html), also in AR1. This demonstrates HTCondor by submitting a job (sleep.bat for Windows; sleep.sh for Linux) that tells the computer to "sleep" for 6 s. The submit file (sleep.sub) instructs HTCondor where the job is located, what HTCondor should do with the executable, and what to do with the results from the executable. In this example, the "results" include a log file, an errors file, and a message file. The job is submitted using the command condor_submit sleep.sub. The job may be monitored with the condor_q command. When the job is finished running, HTCondor will update the log file and no longer show the job when condor_q is entered.

Conceptual HTCondor example
This section describes our five steps for using HTCondor, which also apply to other HTC systems.
Step 1: Define the computing problem. Where are the bottlenecks and what can be broken down into small tasks? The answer may not be HTC. HPC or even code optimization or language changes may be an appropriate solution (e.g., using C++ rather than R or Python). An important consideration is if a problem is processor limited or memory limited [28]. Memory limited means that a program runs out of random access memory (RAM) memory, whereas processor limited means that calculations done by a processor constrains the program. A tradeoff exists between these two constraints. Additionally, large computing jobs can be network-intensive tasks and limited by data transfers. For these tasks, HTC may not be an ideal solution or may require special consideration to avoid network limitations. Also, some computing tasks may be able to run in parallel but cannot be tightly coupled. These jobs are ill suited for HTC and require HPC.
Step 2: Discretize the problem into smaller jobs that may be run on HTCondor. These small discrete problems should be "pleasantly" parallel (also known as "embarrassingly" parallel) and run independently of one another. This involves defining all input files for each job and ensuring output files do not overwrite output files from other jobs. For simulations, this involves deciding how to run simulations (e.g., should one simulation be a job or multiple simulations that share the same inputs?). Simulations are often processor limited. For data processing, breaking a task down may involve splitting data into smaller files. Data processing can often be memory limited but may be processor limited if it involves many calculations.
Step 3: Preprocess the inputs. For large data, this may mean breaking the data up into smaller files and having a way to identify files by some naming convention. For large simulations, for instance, this may require creating an input table of parameters in which each row corresponds to a job.
Step 4: Run the jobs on HTCondor. This step requires placement of a submit file and corresponding files on the central manager. Depending upon the size of the job being run, the processing might take a few minutes to run or possibly take weeks or even months.
Step 5: Postprocess the data. This step consists of combining the output from the previously run HTCondor jobs. Often, this step is most easily done using a scripting language such as Python or R. Pre-and postprocessing can often be done on personal computers before or after the data have been moved to the HTCondor submit machine. In some cases, both the pre-and postprocessing steps might be done using their own HTCondor submit files for large projects (i.e., breaking files or jobs down into smaller jobs to run on many machines). DAG-Man is a program that is included with HTCondor capable of submitting multiple HTCondor batches (conceptually, it is a program to run and manage HTCondor submit files) and is documented in the HTCondor Manual [27].

Sandbox code
An important and sometimes difficult step with HTCondor is "sandboxing" code. Sandboxing is necessary so that all programs will run independently and predictably across platforms. The easiest method for using HTCondor avoids sandboxing by having the same version of programs installed on local machines. For example, simple scripts that only use base R or Python might not require sandboxing. There are two downsides to this approach. First, the software must be installed on the local machines. Second, machines might not have the same version installed (e.g., Python 2.x verses 3.x).
On Windows, sandboxing can sometimes be easier because Windows executables are often independent. Thus, sandboxing the code only requires using HTCondor to send all required programs (e.g., Python.exe) with the script. All required dependencies are then locally installed with the program and deleted when HTCondor is finished running. We include examples of Sandboxing R script (AR1).
On Linux/Unix systems (including macOS), multiple options exist. In theory, programs could be sandboxed like Windows. However, Unix software architecture is built upon many small building blocks that are combined, creating intricate dependencies [29]. This can be side stepped with container programs such as Docker. Docker has the added bonus of having an HTCondor "Universe" setting that allows it to integrate well with HTCondor.
Docker works by building images (i.e., a collection of software that makes programs able to run) similar to a virtual box but with a smaller footprint (i.e., size, memory, and processor use) and takes advantage of the shared kernel across Linux platforms. An image is based upon a specified version of Linux (e.g,. Ubuntu 16.04) and has specific software installed in it (e.g., R 3.0.1 with ggplot2 2.0.1). Users may build their own image either from a Dockerfile (a script that tells Docker how to build an image) or using the command line and an existing Docker image. Many images have already been developed for commonly used programs such as Python and R, and these can often meet the needs of HTCondor users or, at the very least, serve as starting places for building your own image. We prepared a brief tutorial on Docker Images (AR1) and additional documentation may be found on the Docker homepage (https:// docs.docker.com/engine/tutorials/dockerimages/). Additionally, we regularly use (and document) HTCondor. For example, we have simulated data sets to see how statistical trends can be recovered (AR3). This code demonstrates how to use HTCondor on Windows, including sandboxing code. We have also simulated study designs and examined how well parameters could be estimated (AR2). This example demonstrates how to use HTCondor in Linux, including Docker for sandboxing code.

Examples
Briefly, we provide a walkthrough of the statistical trends example (AR3). First, we defined our computer problem as the amount of time necessary to fit models across a wide parameter space. Second, we discretized our problem by parameter combinations (i.e., each HTCondor "job" corresponded to a set of parameter combinations). Third, we presimulated all data sets for each parameter combination and also recorded these combinations in a comma-seperated values (CSV) file. Fourth, we used a submit file to run our jobs. Fifth, we used R to combine, summarize, and plot our results.

When not to use HTC
The previous examples all worked well with HTC because they could be discretized into smaller jobs. When jobs cannot be readily broken down, HTC may not always be the solution. However, HTCondor allows for jobs to request multiple CPUs and HTCondor can be used to manage multiple HPC jobs across HPC resources. As a concrete example, running a computationally intense individual-based model might require HPC resources because the model is closely connected and is not pleasantly parallel. But running multiple instances of these models could be managed through HTCondor. Last, we have observed cases in which simply optimizing code removed our need for HTC or HPC.

Additional resources
Learning how to use HTCondor can be overwhelming because of the amount of documentation. The manual for the stable version (8.6.9) is 1,125 pages [27]! However, the project homepage (http://research.cs.wisc.edu/htcondor/) contains several different resources that are more accessible than the manual. Talks from previous HTCondor Weeks are one resource for learning about HTCondor that include many different introductions and getting-started tutorials. Fienen and Hunt [15] also provided an introduction to HTC and HTCondor for groundwater modeling, and this may be helpful for environmental scientists. Researchers at major universities may not be aware of access they already have to HPC and HTC computing facilities. Finally, computer scientists may find an environmental or ecological computational problem an interesting computer science problem if life scientists seek out collaborations with them (personal communication, R.A. Erickson). Furthermore, working with software engineers can help to improve environmental modeling [30].

AR2: Git repository of simulated study design
This Git repository contains an example of how to use HTCondor to simulate and recover a study design using R, RStan, and Docker: https://my.usgs.gov/bitbucket/users/rerickson_usgs. gov/repos/edna-sampling-design/browse.

Conclusion
Biology and the environmental sciences have become computationally intense. HTC is an important framework for dealing with computationally intense problems. We provided an introduction to HTC and an overview of using HTCondor. Our specific five steps were 1) define the computational problem and confirm that HTC is the solution, 2) determine how a problem can be broken down into small discrete jobs and can be identified by a unique number, 3) preprocess inputs into small jobs that can be distributed, 4) run the job using HTC (e.g., HTCondor), and 5) postprocess the data and combine the results from the many distributed jobs.


Using agent-based modeling to determine collision risk in complex TMA environments: the turn-onto-ILS-final safety case

The continuously growing traffic demand1,2,3 in civil aviation forces the ATM system to utilize existing airport infrastructure and surrounding airspace at its best. Increasing traffic volumes impose additional approach and departure capacities concentrating at the runway or runway system. These may induce higher workloads for both air traffic controllers (ATCO) and pilots as the highly efficient use of pre-set sequencing and spacing constraints becomes crucial while they are also considering environmental issues such as noise abatement. While introducing important measures to minimize aircraft emissions and maximize throughput, the current high safety standards in aviation must not be compromised, but instead, pushed further in order to compensate numerically the projected traffic growth. The performance-based navigation concept,4 which is based on the RNP/RNAV philosophy, is a promising innovation. The European SESAR2020 project PJ01 03 focuses particularly on dynamic and enhanced routes and airspace by, for example, investigating PBN for parallel approaches. PBN allows for a more flexible route design, e.g., circumnavigating densely populated or other noise-sensitive areas during intermediate and even final approach by introducing procedure legs such as “turn to fix” or “radius to fix.” Since those new procedures will have to demonstrate equivalent safety standards compared to conventional procedures, in the course of this paper, we first present these reference values through an analysis of conventional procedures and published safety cases. Depending on the airport runway operation mode (segregated or mixed), individual requirements have been established by ICAO to assure safe operations. However, these safety measures are somehow “static” and do not foresee specific adjustments to local conditions because no explicit cause-to-risk correlations exist in current guidance material except for the final approach segment.5 To overcome this lack of cause-and-effect reasoning, the present project investigates pair wise aircraft collision risk modeling (CRM), also explicitly taking controllers and flight crews into account, modelled mathematically as so-called agents. The case study assumes radar vectoring towards the final ILS approach, laterally intercepting localizer (LOC) and subsequently the glide slope (G/S) until the aircraft has crossed the runway threshold. The following four sections summarize and present (in Section 4) the results of a literature review regarding current regulations and the scientific state-of-the-art knowledge in the field of assessing operational safety during independent parallel approach procedures. We make conclusions about input parameters of the model and summarize on CRM Acceptance Criteria as the output of the model. In Section 5, we specify how the fast-time model is integrated into the overall architecture and explain how the agents are modeled. In Section 6, we transfer the specification into software in order to calculate safety and capacity metrics (i.e., probability of separation infringements and the nuisance breakout rates as detailed in Section 4.D) for various scenarios depicting traffic, runway/airspace layout and agent behavior (i.e., procedures and hazards as detailed in Section 7.A). In Section 7, we discuss the model verification concept and the results gained.

Introduction
The continuously growing traffic demand 1,2,3 in civil aviation forces the ATM system to utilize existing airport infrastructure and surrounding airspace at its best. Increasing traffic volumes impose additional approach and departure capacities concentrating at the runway or runway system. These may induce higher workloads for both air traffic controllers (ATCO) and pilots as the highly efficient use of pre-set sequencing and spacing constraints becomes crucial while they are also considering environmental issues such as noise abatement. While introducing important measures to minimize aircraft emissions and maximize throughput, the current high safety standards in aviation must not be compromised, but instead, pushed further in order to compensate numerically the projected traffic growth. The performance-based navigation concept, 4 which is based on the RNP/RNAV philosophy, is a promising innovation. The European SESAR2020 project PJ01 03 focuses particularly on dynamic and enhanced routes and airspace by, for example, investigating PBN for parallel approaches. PBN allows for a more flexible route design, e.g., circumnavigating densely populated or other noise-sensitive areas during intermediate and even final approach by introducing procedure legs such as "turn to fix" or "radius to fix." Since those new procedures will have to demonstrate equivalent safety standards compared to conventional procedures, in the course of this paper, we first present these reference values through an analysis of conventional procedures and published safety cases.
Depending on the airport runway operation mode (segregated or mixed), individual requirements have been established by ICAO to assure safe operations. However, these safety measures are somehow "static" and do not foresee specific adjustments to local conditions because no explicit cause-to-risk correlations exist in current guidance material except for the final approach segment. 5 To overcome this lack of cause-and-effect reasoning, the present project investigates pair wise aircraft collision risk modeling (CRM), also explicitly taking controllers and flight crews into account, modelled mathematically as so-called agents. The case study assumes radar vectoring towards the final ILS approach, laterally intercepting localizer (LOC) and subsequently the glide slope (G/S) until the aircraft has crossed the runway threshold. The following four sections summarize and present (in Section 4) the results of a literature review regarding current regulations and the scientific state-of-the-art knowledge in the field of assessing operational safety during independent parallel approach procedures. We make conclusions about input parameters of the model and summarize on CRM Acceptance Criteria as the output of the model. In Section 5, we specify how the fast-time model is integrated into the overall architecture and explain how the agents are modeled. In Section 6, we transfer the specification into software in order to calculate safety and capacity metrics (i.e., probability of separation infringements and the nuisance breakout rates as detailed in Section 4.D) for various scenarios depicting traffic, runway/airspace layout and agent behavior (i.e., procedures and hazards as detailed in Section 7.A). In Section 7, we discuss the model verification concept and the results gained.

State-of-the-art in safety assessment focusing on the approach segment A. Independent parallel approach, safety
The FAA began research in the field of independent approaches to parallel runways as early as the 1960s. Since then, the MITRE Corporation in particular has progressed in developing and conducting simulations aiming at reducing runway separation requirements. 6,7 Besides developing technology like high update rate radars (down to 1 Hz), the FAA also started three major research programs: The Multiple Parallel Approach Program, 8 the Precision Runway Monitoring (PRM) Demonstration Program 9 and the Converging Approach Standards Technical Working Group (CASTWG). For the purposes of the MPAP and CASTWG, the FAA created a fast-time model called Airspace Simulation and Analysis for TERPS (ASAT). An ASAT study is split into the two stages of real-time, human-in-the-loop and fast-time computer-only simulation. Otherwise hard-to-reach human factors (reaction times, workload, etc.) and aircraft behavior data (e.g., roll rate, climb rate, airspeed, etc.) is acquired in the first stage. This empirical data is statistically fitted and transferred into continuous probability density functions (PDF). By means of Monte-Carlo simulation, this data is recombined with additional data from technical specifications (e.g., radar errors) while a vast amount of simulation runs helps to increase statistical significance and sharpen the confidence interval. 10 Along MPAP, which was started in the late 1980s, 8 ASAT has been used to investigate simultaneous parallel approach scenarios with dual, triple and even quadruple runway configurations within the US. Rare blunder situations (see Section 4.C) were the main interest due to the great safety impact. Therefore, the blunder occurrence probability (or blunder rate) was the central hazard for collision risk with aircraft operating on the adjacent approach track and not taking any evasive actions. To efficiently collect the most relevant data focus was merely put on such blunder-evader constellations solely along the project's

B. Intercept of the final approach
Published studies regarding the final approach segment always assume that separation requirements will be met during the intercept maneuver even though workload is considerably high and accepted as relevant during this flight segment and important changes in the aircraft guidance mode are necessary for safe operations. As the aircraft's headings converge, violations of separation standards could (in theory) occur suddenly and without meaningful precursor events, e.g., in case of a late or incorrect turn anticipation by the flight crew or degraded/misconfigured aircraft guidance systems. The literature does, however, not yield any safety or human factor considerations concerning these potential issues. [6][7][8][9][10][11][12] Therefore, we can only take note of the fact that conservative procedure design usually foresees a vertical buffer, e.g., by displaced runway thresholds, thereby safeguarding the intercept against these and any other connected issues.

C. The blunder hazard
In the scope of independent parallel approaches, so-called "blunder" events require special attention. 6 A blunder describes an unexpected deviation of an aircraft from its intended flight path (e.g., the final approach track) that does not result from ubiquitous navigational errors or could otherwise be attributed to the total system error in general, but from human error or improper functionality of technical equipment. The extent of the initial deviation from the desired track forms the blunder angle. Immediately after detection of such abnormal behavior, ATC will issue a correction advisory to the crew. In case of a non-responsive evasive aircraft, this one is called a worst-case blunder. Any potentially threatened aircraft on the adjacent approach track will receive a breakout command to relax the situation and avoid a collision (the so-called evader). By searching the FAA's Aviation System Performance Metrics database and reviewing nearly 1,000,000 approach reports from 2008 to 2010, the authors Massimini et al., 13 determined a blunder rate of 1/24,000 with a blunder angle not exceeding 29° and no occurrence of a single worst-case-blunder diverting by an angle of 30° or more.
As a blunder risk mitigation measure, ICAO calls for a 2,000 ft wide No Transgression Zone (NTZ, Figure 2) placed equidistantly between the two final approach tracks, starting laterally at the distance outbound where vertical separation is first lost and extending towards the first (if staggered) runway threshold. The remaining lateral space to each runway is called Normal Operating Zone (NOZ), starting longitudinally at the localizer intercept, extending towards the individual runway threshold. Consequently, the NOZ has no pre-defined width but depends entirely on the runway spacing while considering the pre-set NTZ width. This is not at all compliant to modern concepts (e.g., the PBN philosophy 4 ), indicating a necessity for further research. The NTZ dimensions are the result of a geometric construction of the flight paths of a worst-case blunder and an adjacently operating aircraft executing an evasive maneuver, which effectively dictate the minimum distance at the closest point of approach (CPA) for an aircraft pair. The case of dual blunders lies outside of this rationale and is therefore not part of this construction but rather a very rare though potentially risk-intensive corner-case.

D. Acceptance criteria
Obviously, safety acceptance especially of TMA procedures competes with both acceptable system capacity and also with emission abatement. Consequently, the relationship between a given CR and its implication on runway throughput is crucial for both the regulator and the operator. Therefore, we define different measures that allow an assessment of safety versus capacity.

Safety
In case of a blunder event, the risk of a collision is deter-mined by calculating the probability of two aircraft coming into physical contact. In general, this is the case if the slant range between the aircraft falls below a threshold defined by the aircraft's physical dimensions. This "conflict risk" approach was generalized and adopted by FAA/ MITRE and subsequently ICAO, and is defined as a threshold of 500ft as the 'hit' distance regardless of the orientation in 3D space. In ICAO's and-built on it-SESAR's safety regulation, the Target Level of Safety (TLS) concept, aims at defining, by means of mathematical proof, a Maximum Tolerable Frequency of Occurrence (MTFoO) for hazardous events concentrating on, but not being limited to, the 'fatal accident' category induced by a physical contact according ICAO Annex 13. 14 For independent parallel approaches, regulation requires less than 1 violation of the 500 ft. slant-range criterion in 56 million approaches, equaling a Near Mid-Air Collision (NMAC) rate of 1.8•10 -8 per approach. 15 Assuming a final approach segment length of 10 NM and an average final approach speed of 150 KIAS, the time per final approach is approx. 4 minutes, resulting in a "conflict" frequency of 2.7•10 -7 per flight hour. That NMAC may remain an incident but it might recur in different conditions and become an accident, depending on the moderating events and conditions. The ICAO TLS for fatal obstacle collision accidents during ILS final approaches is 1•10 -7 per approach. 5 A study from MITRE however proposes a much stricter target for NMAC rate in close-to-terrain operations, with a value as low as 1•10 -9 per approach even though a non-catastrophic consequence is associated to that hazard, argued with the fact that escape procedures are physically constrained downwards. 16 It is therefore important to stress that this value does not equal an accident-type TLS; historical incident data proves that only a small portion of slant-range violations indeed led to an accident, whereas the majority fell into the severe incident category according ICAO Annex 13. 14 Therefore, it is necessary to consult recent safety publications such as SESAR 17 for comparison and interpretation.

Capacity
During parallel approaches, a blunder should induce a breakout maneuver of an aircraft operating on the adjacent approach track, this one then following the published missed approach procedure. This scheme enforces safety at a minimal cost of capacity. Since aircraft are subjected to navigational tolerances, NTZ alerts may be triggered by aircraft deviating from but autonomously returning to their approach track. Nevertheless, the (automatically triggered) NTZ alert requires ATC to 'break' the adjacent aircraft 'out' of the approach. This is called a nuisance breakout in the sense of a false positive alert (see scenario A in Figure 2). The frequency of such events is called nuisance breakout rate (NBR) and forms a (lost) capacity measure. Fixing a general NBR target is however unrealistic since it depends on many factors, e.g., on the traffic mix and the given runway spacing.
To give indications about NBR estimates, we find in Massimini 7 that the NBR estimated is considerably below 5% of all approaches. In Mc Cartor & Ladecky, 12 aircraft performing an RNP 0.3 (non-precision) approach are observed to cause similar nuisance breakouts 2-4% of the time.

CRM model architecture A. Overview
The configuration of our integrated agent-based traffic simulation and collision risk estimation model, which studies though is not limited to the safety case of independent parallel approach operations ( Figure  3), comprises multiple interacting modules that are triggered repeatedly following a Monte-Carlo parameter variation scheme. Previous work in development and application of the model is described in. [18][19][20][21][22] For the safety case focused on in this paper, the following CRM parts were customized: the independent parallel approach configuration including all associated parameters (green colors in Figure 3), the definition and calculation of relevant metrics (blue) and customizations to the visualization. After a short introduction to the core model (traffic simulation, collision risk quantification, parameter variation, orange), these customizations are covered in detail, followed by geometric construction of the infrastructure and scenario definition descriptions.

B. CRM-agent based traffic simulation
The main idea of an agent-based model is creating software entities, which closely resemble real-world entities in terms of knowledge (information base), behavior (rules of deciding and acting) and communication (sensing the environment, interacting with other agents) without 'cheating' for the sake of efficient modeling or computing. The main benefit is emergent behavior, where a multiplicity of well-modeled software agents exhibit the same features as their realworld counterparts without the need for explicit modeling of complex traits. 1 Although getting the behavior 'right' is often troublesome, the major benefit is that the model is generative, meaning it is able to predict patterns outside the evident behavior specification which can be (and often are) 'wrong' but which may provide valuable insight into unforeseeable hazards; this is particularly relevant in the aviation domain. Except for the aircraft and flight guidance agents, all other work is purely event-driven (e.g., radar pattern detected, reaction/ reading time passed). The emergent behavior is the timed sequence of events shaping the synchronous aircraft movements, whereby the concept explicitly addresses the missing cause-to-effect relations as elaborated in Section 3 ( Table 1).

C. Agent action sequence for the blunder event
The agent logic is best understood following an assumed blunderevader constellation as depicted in Figure 2. Agent entities will be underlined in this sub-section. Initially, both aircraft are following their specific approach track. Upon blunder initiation, the blundering aircraft's flight guidance reverts to vectoring mode with a new heading given by the blunder angle and the initial approach track. It is reached by modeling a constant bank rate of 10 / s Φ =° by the autopilot either until reaching the bank limitation max 25 Φ =° 5 (for clean configuration) or the standard procedure turn rate of 3 / s Ψ =° . The final turn radius r and the required bank Φ for a procedure turn observing the bank limitation are calculated as follows: During the period of 'rolling up,' the aircraft will follow an EULER spiral (clothoid). The on-following curve with constant bank and speed is circular. It is correctly terminated by the autopilot with the appropriate ramp-down of the roll, resulting in a clothoid transition towards the anticipated blunder heading. While performing the described maneuver, the blunder position is radar-detected and passed on to the controller, upholding all performance assumptions (e.g., delay, resolution). Upon such 'evident' NTZ penetration, the according software module inside the controller agent will trigger the 'alert.' The behavior is pre-defined: break out of the (previously paired) aircraft on the adjacent approach track again by means of vectoring, optionally adding a vertical guidance and requesting the blunder aircraft to immediately return to its initial approach track. For the 'classic' ICAO SOIR blunder scenario, all reaction and communication times are subsumed by ICAO's original assumption of statically 8s. 15 This incorporates the controller's reaction time to the NTZ alert, voice-com and the pilot's reaction time until aircraft compliance with the advisory. Following ICAO's assumptions, we only assume 8s total reaction time from NTZ alert to aircraft reaction and procedure turns (i.e., no manual aircraft handling assumed). It is obvious, that this static assumption does not correctly reflect a humanmachine-system, which is always stochastic. The fixed value was chosen as an initial baseline for validation reasons.
To minimize time-based discretization effects induced by the Monte Carlo simulation engine, blunder events are also synchronized with the chosen simulation update period (e.g. 0.5s). When initiating the evasive maneuver, small corrections (<1 update cycle) are applied to compensate time-discretization errors (radar, controller reaction). By the more sophisticated design of our simulation system mostly overridden for this study, the controller reaction time would be built from a blocking resource model resembling the top-level task items of scanning the radar screen, conflict inspection, decision-making 23 and advisory transmission (including interruptible read-back and compliance monitoring). The advisory transmission would require a free voice-com channel and the utterance would be accounted for according to ACT-R 24 with syllable and word pause times, etc. Finally, the pilot reaction time will be modeled with a PDF evaluated in a previous human-in-the-loop study. 22 Using knowledge about human factors, the total reaction time could be estimated as: Let (1) be almost immediate due to uniqueness of the alert and the pre-defined actions, except for the necessity to look up the evader call sign at the radar screen (GOMS complex information look-up, 1200 ms). 25 (2) can be calculated by the syllable/word count of the advisory according to ICAO phraseology and phonetics and metrics from ACT-R, 24 e.g. air-line_1_2_3_turn_left_hea-ding_4_5_6_imme-diate.
15 syllables+10 word gaps+initiation+comprehension 15*50ms+10*100ms+150ms+180ms=2,080ms With 1,540ms and 0 to 2,000ms on average (depending on the input strategy), (3) and (4) can be adopted from Vogel et al., 22 where a pure human factors estimate is presented as well. In total, we can account for 5 to 7 s (4,820 to 6,820 ms on average) of the 8 s assumption without taking the radar update rate into account; this then appears roughly valid considering that much time can be saved by observing the controller's acute voicing in hazardous situations and early cue words in ICAO phraseology. Once configured by the pilot, the evader autopilot will initiate the same aircraft turn maneuver as described in the beginning of this sub-section. In parallel, a second controller who simultaneously monitors the blunder's approach track will initiate a comparable advisory to the blundering aircraft, whose pilot would subsequently start implementing the corrective maneuver ('normal' blunder) or fail to react ('worst case' blunder).

D. Aircraft navigation performance
With respect to the defined navigation aids, all trajectories produced by the traffic simulation are ideal and incorporate all of the features of the agents' decision-making including timing, which we call microscopic because of the fine-grained dependencies between agents (e.g., milliseconds of reaction time deciding upon which aircraft is being served first by ATC). In addition, all flight movements are known to exhibit a navigational error consisting of path definition, navigation system and flight technical errors. We handle these errors on a macroscopic level. By analyzing large radar data sets and filtering flights navigating autonomously only on flight guidance systems (no manual control, no active ATC advisories), and statistically fitting (often heavily tailed) PDF, all micro-effects stemming from weather (e.g., turbulent wind), control system dynamics, etc. can be summed up into one distribution (independently per axis of the trajectoryfixed coordinate system; i.e., cross-track, along-track, vertical-track tolerance, XTT, ATT, VTT). We refer to these PDF of navigational accuracy as actual navigation performance (ANP) equivalent, but not limited to, the term used in PBN. 26,27

E. Monte-Carlo evaluation and numeric complexity
We utilize pseudo-random variables in order to efficiently and reproducibly assess the influence of arbitrarily distributed stochastic parameters present in the form of aircraft pairing, navigation tolerances (ANP), radar accuracy and reaction times. Normally, to determine a risk of less than 10 -8 collisions per approach (due to blundering) with an accuracy of 5%, the simulation needs to span far more than 10 quadrillion approaches according to the Bernoulli Law of Large Numbers, as quantified by Chebyshev 's inequality in (3). Here, the simulation is considered a Bernoulli experiment (collision yes/no) which is repeated n times, in which a collision is recorded 1 n times. The type-I error probability is set to 1-P=0.05 and the probability estimation tolerance is set to The computational effort can, however, be reduced by focusing on approach "points of interest:" Assuming a blunder rate of 1/24,000 leads to 500 billion approaches to simulate. In contrast to this theoretic approximation, FAA conducted studies where a similar simulation was run 100,000 times. 9

A. Definition of metrics
First, we defined the assessment metrics, which we call acceptance criteria. For each such criterion, tool users have to define acceptance thresholds for the given ATM procedure design with regard to safety (collision probability) and capacity (aircraft throughput). According to historical models and studies, we adopt the concept of the number of Test Criteria Violations (TCV) to define a safety metric as outlined in Section 4.D, targeting severe incidents instead of a pure collision risk (CR). The latter can, however, be estimated by reducing the 'hit' distance to the aircraft's physical dimensions. The slant range between blunder-evader aircraft pairs is therefore continuously traced. If it falls below a given threshold, a TCV is recorded. Finally, the probability of a TCV based on its number of occurrence The widely accepted assumption of a fixed blunder probability Blunder P helps to increase computational efficiency. The blunderevader results are collated with inherently safe normal operations as a post-processing step. The NBR is also calculated analytically during post-processing using the NTZ penetration probability and the physical runway layout. Both approach tracks' NBR are calculated separately by integrating geometrically the cross-track ANP distribution function along the extended runway centerline, as shown in (5). Figure 4 illustrates this process. Yellow-grey areas under the Normal Distribution function depict NTZ penetration probabilities at single points as the integral of the cross-track deviation. The length of the final approach track along the NTZ boundary is denoted as l . The overall NBR is the counter-probability of the event that "on neither approach track, a nuisance breakout occurs." All NBR values depend on the current navigation accuracy.

B. Geometric design, stochastic variables
For the targeted safety case, airport, final approach configuration and runways are assumed to operate in single mode (approaches only) with guidance and surveillance equipment compliant to ICAO SOIR Mode 1 specifications. 15 A standard ILS final approach begins with intercepting the LOC. Then, a straight horizontal flight segment is followed for at least 2NM, allowing the aircraft to establish and stabilize laterally before vertically intercepting the G/S and start to further descent until touchdown. Aircraft approaching the same or the adjacent runway must maintain radar separation (3NM laterally or 1,000 ft vertically) at all times even though they are considered 'independent.' The angle between intercept vector and final approach course must not exceed 30°. 15 The model, however, allows for certain violations, thus reflecting navigational, control and human error. The actual LOC intercept point is subject to uncertainties (aircraft/pilot behavior) and variability's resulting from sequencing in a vectoring onto ILS environment. These, however, are not reported in the paper and will be modeled by an appropriate PDF.
Surveillance system requirements depend on the actual runway spacing: Between 3,400ft and 4,300ft (centerline to centerline), ICAO requires a radar update rate of every 2.5 seconds. The azimuthal accuracy of the recorded aircraft positions shall match a Normal Distribution with 0.06 σ =°. Beyond 4,300ft runway spacing, an update rate of only 5 seconds as well as an azimuthal accuracy of 0.3 σ =°is required. Aircraft speeds are also stochastically modeled.
While historical investigations by ICAO assume an unrealistic constant speed of 150 KIAS, our model is able to handle various distributions. Besides a static value, aircraft speeds may also be subjected to a Normal Distribution 2 ( , ) N µ σ  or following the ICAO approach category scheme. 5 Finally, blunder parameters are stochastic variables as well. Those are the distribution of blunder occurrence along the approach track, the blunder angle and the worst-case blunder rate.

C. Implementation of the turn-onto-final safety case
For each simulation run, blunder-evader pairs are positioned on the parallel approach tracks. First, the distance from threshold for the blundering aircraft is chosen randomly. Subsequently, based on the provided ANP function, cross-and vertical-track deviations are determined, assuring the necessary initial adherence to the NOZ before the actual blunder event. Additional stochastic parameters are speed, blunder angle and vertical blunder behavior (leveling-off due to lost G/S signal versus upholding the current rate of descent). Now, a deterministic blunder trajectory is generated. The crossing point with the adjacent approach track is calculated and forms the mid-point of the evader placing area, which extends, plus/minus, half of the required in-trail separation, thereby covering precisely the location of the most threatened aircraft for the independent parallel approach configuration. The assigned evader placement follows a uniform distribution. Once placed, the evader is shifted back in time in order to let blunder and evader enter the simulation at the same time. After having assigned the initial positions and all aircraft parameters, the simulation continues as described in Section III.C and stops as soon as the closest point of approach (CPA) has been passed. For the minimum distance, the TCV is evaluated.

D. Model limitations
We currently assume that human actions are purely event driven. The controller will initiate corrective commands only after receiving a NTZ alert. The pilot agent only consecutively intervenes after receiving a command. However, in reality, the controller will most likely detect deviations during the approach well prior NTZ alert and issue appropriate commands. Additionally, pilots may also be able to detect deviations and undertake corrective maneuvers without ATCO recall. The model therefore is rather conservative, probably too deterministic at this point. Furthermore, the distribution functions used to represent navigation tolerances presume a Brownian motion, flight mechanics are widely neglected. This theoretically allows an aircraft to suddenly change attitude in space between two simulation steps. At the same time, the employed ANP functions are considered to be independent while deviations from the flight path may not. ANP functions were deduced from recorded real flight paths and are therefore environment specific. Data contains environmental effects like winds, which will often be specific for the location, and the time the data was recorded. Therefore, reference data should contain a very large range of all external effects. Alternatively, locally fitted ANP functions should be developed and human reaction times modelled stochastically. The model is technically prepared to do so. We will consider these issues in consecutive research.

A. Model verification using the 'classic' ICAO Scenario
The fast-time model shows that results are comparable to those of previous simulation studies by ICAO. This is shown in the following, by running two exemplary use cases. While for intercept maneuvers there is no comparable study available, we configured the mode (final approach blunder, nuisance breakout and aircraft behavior modules) to be comparable with historical studies. Hence, we expect the model to remain valid and to deliver results that are directly comparable with the 'classic' baseline. Using these settings, multiple simulations, initialized with individual random 'seed' values, were conducted. Figure 5 visualizes the development of resulting probability over an increasing number of simulated blunder-evader pairings. The results stabilize at a probability of around (one TCV per 56 million approaches). 15 The remaining differences do result from implementation details (e.g. clothoid) and model parameterization (e.g. ILS deviations). Especially, we found high sensitivity to TCV probability towards the final approach length, which was set to 10 resp. 13.5NM for both runways to allow for immediate blundering towards NTZ. When applying 10NM for both runways, the probability drops significantly down to around TCV per approach. This ILS specific effect is subject to further research.

B. The turn-onto-final safety case
The 'novel' safety case now covers additionally the LOC intercept including the additional hazard of weak interception due to human error such as e.g., flight guidance mode-confusion. The model handles such events identically to those on final approach, with the difference that the blunder initiation is linked to the nominal intercept point and the 'blunder' heading is assumed to be already reached. In contrast to the 'classic' scenario, the NTZ, extending outwards until vertical separation is no longer granted, 15 does not cover that airspace entirely. Instead, vertical separation set to 1000ft. between left and right approach safeguards the intercept procedure. Following the stated assumptions, proactive controller interventions (without NTZ alert) are not considered. This is a rather weak assumption from reality. Therefore, the current agent behavior represents a conservative (static & worst-case) safety consideration. The resulting points of closest approach (slant distance color-coded) are shown in Figure 6.  shown below depicts the measured aircraft slant ranges and TCV violation threshold. Although the worst-case assumptions and visual images laid out above suggest a more critical behavior than in the 'classic' scenario, the opposite is true: due to the 1000ft. vertical separation set out until G/S intercept, no TCV occurred over the 10 million simulated pairings. Therefore, we cannot show any stabilization of TCV rate results as part of Figure 7. To investigate further, a more critical case lowering virtually vertical separation down to 800ft was selected. Figure 8 depicts the corresponding effects: The slant range distribution is shifting towards lower values (rightwards), TCVs do occur: Its rate now stabilizes similarly to Figure 5 at a value of 9 3.04 10 − ⋅ per approach. These kinds of procedure design studies, along with re-shaped (elongated) NTZ geometries, form potential future application of the presented model.

C. Discussion of runtime complexity and outlook
First, the quality of results does depend on the time discretization performed during all computer-numerical simulations. By applying corrections below one time increment (update period), we were able to compensate and stabilize the results ( Figure 9). Second, the runtime requirement certainly depends on the expected probability of 'positive' observations (e.g., NMACs). Running multiple simulations over a wide range of blunder events shows that results stabilize at around 3 million events, as depicted in Figure 5 & Figure 8. 100,000 simulation runs as published in 15 are, consequently, far too little for reliable results. Third, it is important to recall that the probability value for a TCV, as given by ICAO, does not represent an accident-type of TLS since historical incident data proves that only a small minority of slant-range violations indeed led to accidents, whereas the majority fall into the severe incident category (NMAC) of ICAO Annex 13. 14 Therefore, it is necessary to consult recent safety publications such as 17 for comparison and interpretation. The document offers a translation of a singular TLS value into seven incident/accident severity categories and correctly connects safety with reliability by declaring a "Maximum Tolerable Frequency of Occurrence" (MTFoO). (Bottom) Figure 9 Reduced influence of time stepping (simulation update period) on resulting TCV rate, with comparison to published value in ICAO SOIR. 15