Quantitative performance evaluation of Bayesian neural networks
Due to the growing adoption of deep neural networks in many fields of science
and engineering, modeling and estimating their uncertainties has become of
primary importance. Various approaches have been investigated including
Bayesian neural networks, ensembles, deterministic approximations, amongst
others. Despite the growing litterature about uncertainty quantification in
deep learning, the quality of the uncertainty estimates remains an open
question. In this work, we attempt to assess the performance of several
algorithms on sampling and regression tasks by evaluating the quality of the
confidence regions and how well the generated samples are representative of the
unknown target distribution. Towards this end, several sampling and regression
tasks are considered, and the selected algorithms are compared in terms of
coverage probabilities, kernelized Stein discrepancies, and maximum mean
discrepancies.