Uncertainty Quantification: Previous Studies
UQ has been critical for robust learning under different contexts (known or unknown) in mobile activity recognition [1]}, healthcare [2]}, [3]}, signal processing [4]}, and manufacturing [5]}. Existing models with Gaussian process [1]} or Bayesian approximation methods [7]} either rely on the assumption that variables in a system can be characterized by explicit probabilistic relationships (e.g., Bayesian models) or rely on generating one best model in the learning algorithm. However, measuring the predictive uncertainty for deep neural networks remains a challenging problem, closely related to the problem of detecting test samples that are drawn sufficiently far away from the distribution of training samples.
For example, Malinin & Gales proposed a framework called Prior Networks (PNs) for modeling predictive uncertainty that explicitly modeled distributional uncertainty [8]}, Hendrycks & Gimpel proposed a framework that utilized probabilities from softmax distributions and detected out-of-distribution examples, by introducing confidence scores based on density estimators [9]}, later improved by processing the input and output of DNNs in [10]}. Lee et al. proposed a method for detecting abnormal test samples by including both out-of-distribution and adversarial samples, to obtain the class conditional Gaussian distributions by introducing confidence scores based on the Mahalanobis distance [11]}.
Our method is different from these works as we aim to learn a distribution of deep models rather than one single best one.
