Many researchers have already proposed approximate inference methods for Bayesian Neural Networks (BNN). Early on, it was proposed to employ the Extended and Unscented Kalman Filter to estimate the Gaussian posterior for feedforward neural networks [1]}, [2]}, [3]}. More recently, this Gaussian framework was extended with the Cubature filter [4]}, which, like the extended and unscented methods, is limited by the computational complexity of \(\mathcal {O}(n^3)\) , where \(n\)  is the number of parameters. In another approach, [5]} ([5]}) employed the Laplace approximation to describe the posterior covariance of parameters. Later, [7]} ([7]}) explored the potential of BNN using sampling methods. In parallel, several researchers have applied variational inference for estimating the posterior distribution of neural networks' parameters [9]}, [10]}. The development of moment matching and variational approaches for BNN is still nowadays an active research area [11]}, [12]}, [13]}, [14]}. Recently, the technique of using Dropout as a Bayesian approximation has received a lot of attention in the community [15]}. For all the recent methods that are either based on variational approaches or dropout, the inference of parameters is still treated as an optimization problem relying on gradient backpropagation.
