Dropout has been demonstrated as another effective regularization strategy, which can be viewed as imposing a distribution over the NN weights [1]}.
Relating dropout to Bayesian inference provides a much simpler and more efficient way than using vanilla Bayesian NNs to provide uncertainty estimation [1]}, as there is no more need to explicitly instantiate multiple sets of NN weights.
For example, Bernoulli dropout randomly shuts down neurons during training [3]}, [4]}. Gaussian dropout
multiplies the neurons with independent, and identically distributed (\(iid\) ) Gaussian random variables drawn from \(\mathcal {N}(1,\alpha )\) , where the variance \(\alpha \)  is a tuning parameter [4]}. Variational dropout generalizes Gaussian dropout by reformulating it under a Bayesian setting and allowing \(\alpha \)  to be learned under a variational objective [6]}, [7]}.
