Figure REF  shows the posterior mean and the \(95\%\)  predictive interval of the mis-specified regression line \(\theta ^{\top }\phi \)  from (A) the Bayesian posterior; (B) the mean field variational approximation restricted to isotropic Gaussians; and (C)–(F) variational approximations with re-scaled marginal variances.
In each plot, we overlaid the observed data \(\mathcal {D}_{100}\) , with the true data generating function in dashed black.
We also report the 2-Wasserstein distance between the true posterior and each approximation and the \(\textrm {lpd}\)  averaged over \(N^* = 10^4\)  test data points drawn from Equation (REF ); note that the 2-Wasserstein distance can be used to bound differences in means and variances [1]}.
The variational approximation (panel (B) of Figure REF ) is quite accurate: the 2-Wasserstein distance between the approximation and the exact posterior is \(\sim 10^{-4}.\) 
As we scale up the variance of this approximation, we move away from the exact posterior over the parameters but the posterior predictive distribution covers more of the data, yielding higher \(\textrm {lpd}\) .
Figure REF  in Appendix  shows the contours of these approximate posterior distributions.
