Proposed methods for prediction interval estimation so far focus on cases where input variables are numerical. In datasets with solely nominal input variables, we observe records with the exact same input @math , but different real valued outputs due to the inherent noise in the system. Existing prediction interval estimation methods do not use representations that can accurately model such inherent noise in the case of nominal inputs. We propose a new prediction interval estimation method tailored for this type of data, which is prevalent in biology and medicine. We call this method Distribution Adaptive Prediction Interval Estimation given Nominal inputs (DAPIEN) and has four main phases. First, we select a distribution function that can best represent the inherent noise of the system for all unique inputs. Then we infer the parameters @math (e.g. @math ) of the selected distribution function for all unique input vectors @math and generate a new corresponding training set using pairs of @math . III). Then, we train a model to predict @math given a new @math . Finally, we calculate the prediction interval for a new sample using the inverse of the cumulative distribution function once the parameters @math is predicted by the trained model. We compared DAPIEN to the commonly used Bootstrap method on three synthetic datasets. Our results show that DAPIEN provides tighter prediction intervals while preserving the requested coverage when compared to Bootstrap. This work can facilitate broader usage of regression methods in medicine and biology where it is necessary to provide tight prediction intervals while preserving coverage when input variables are nominal.
This paper evaluates the four leading techniques proposed in the literature for construction of prediction intervals (PIs) for neural network point forecasts. The delta, Bayesian, bootstrap, and mean-variance estimation (MVE) methods are reviewed and their performance for generating high-quality PIs is compared. PI-based measures are proposed and applied for the objective and quantitative assessment of each method's performance. A selection of 12 synthetic and real-world case studies is used to examine each method's performance for PI construction. The comparison is performed on the basis of the quality of generated PIs, the repeatability of the results, the computational requirements and the PIs variability with regard to the data uncertainty. The obtained results in this paper indicate that: 1) the delta and Bayesian methods are the best in terms of quality and repeatability, and 2) the MVE and bootstrap methods are the best in terms of low computational load and the width variability of PIs. This paper also introduces the concept of combinations of PIs, and proposes a new method for generating combined PIs using the traditional PIs. Genetic algorithm is applied for adjusting the combiner parameters through minimization of a PI-based cost function subject to two sets of restrictions. It is shown that the quality of PIs produced by the combiners is dramatically better than the quality of PIs obtained from each individual method.