Why not just...?   The proposed training procedure may at first appear non-trivial, and it is reasonable to wonder if existing methods could be applied to similar results. Variational inference often achieves similar results to Laplace approximations, so could we use `Bayes by Backprop' [1]} to get an alternative Gaussian approximate posterior over \({\theta }\) ? Similar to the supervised experiences of [2]}, we unfortunately found this approach too brittle to allow for practical model fitting. But then perhaps a post-hoc la as proposed by [3]} for supervised learning? Empirically, we found it to be important to center the approximate posterior around a point where the Hessian provides useful uncertainty estimates. Our online training moves in this direction as the Hessian is part of the procedure, but this is not true for the post-hoc la.
