The estimation of weight and bias in neural networks is currently dominated by approaches employing point estimates when learning model parameters using gradient backpropagation [1]}. Although these approaches allow for the state-of-the-art performance in many domains of applications, it is recognized that they fall short in situations where, for instance, datasets are small, when the task requires quantifying the uncertainty about the prediction made, or for continual learning [2]}, [3]}, [4]}. In such situations, the Bayesian approach for infering the parameters' posterior probability, is known to be theoretically better suited than a point estimate. This is in theory because applying exact Bayesian inference on large neural networks has been considered to be intractable [5]}.
