Deep neural networks (NNs) have become ubiquitous
and achieved state-of-the-art results in a wide variety of research problems [1]}. To prevent over-parameterized NNs from overfitting, we often need to appropriately regularize their training.
One way to do so is to use Bayesian NNs that treat the NN weights as random variables and regularize them with appropriate prior distributions [2]}, [3]}.
More importantly, we can obtain the model's confidence on its predictions
by evaluating the consistency between the predictions that are conditioned on different posterior samples of the NN weights.
However, despite significant recent efforts in developing various types of approximate inference for Bayesian NNs [4]}, [5]}, [6]}, [7]}, [8]}, [9]},
the large number of NN weights makes it difficult to
scale to real-world applications.
