The FMRIB Variational Bayes Tutorial: Variational Bayesian Inference for a Non Linear Forward Model
Bayesian methods have proved powerful in many applications, including MRI, for the inference of model, e.g. physiological, parameters from data. These methods are based on Bayes' theorem, which itself is deceptively simple. However, in practice the computations required are intractable even for simple cases. Hence methods for Bayesian inference are either significantly approximate, e.g. Laplace approximation, or achieve samples from the exact solution at significant computational expense, e.g. Markov Chain Monte Carlo methods. However, more recently the Variational Bayes (VB) method has been proposed (Attias 2000) that facilitates analytical calculations of the posterior distributions over a model. The method makes use of the mean field approximation, making a factorised approximation to the true posterior, although unlike the Laplace approximation does not need to restrict these factorised posteriors to a Gaussian form. Practical implementations of VB typically make use of factorised approximate posteriors and priors that belong to the conjugate exponential family, making the required integral tractable. The procedure takes an iterative approach resembling an Expectation Maximisation method and whose convergence is guaranteed. Since the method is approximate the computational expense is significantly less than MCMC approaches and is also less than a Laplace approximation since no Hessian need be evaluated. Attias (2000) provides the original derivation of the 'Variational Bayes Framework for Graphical Models' (although is not the first person to take such an approach) He introduces the concept of 'free form' selection of the posterior given the chosen model and priors, although this is ultimately limited by the need for the priors and factorised posteriors to belong to the conjugate exponential family (Beal 2003) A comprehensive example of the application of VB to a one dimensional Gaussian mixture model has been presented by Penny et al. (2000) Beal (2003) has provided a thorough description of variation Bayes and its relationship to MAP and MLE, as well as its application to a number of standard inference problems. He has shown that Expectation Maximisation algorithm is a special case of VB. Friston et al. (2007) additionally has considered the VB approach and variational free energy in the context of the Laplace approximation and ReML. In this context they use a fixed 3 multi variate Gaussian form for the approximate posterior, in contrast to the 'free form' approach. To ensure tractablility of the VB approach the models to which it can be applied are limited (Beal 2003) However, Woolrich Behrens (2006) have avoided this problem, in