Note that \(\mathcal {I}_\mathcal {F}[\mathbf {x};\theta ]\)  is not strictly a bound on \(\mathcal {I}[\mathbf {x};\theta ]\) , but may be seen as an approximation to it [1]}. Briefly, this is because the original Stam's inequality, as stated in (REF ), assumes \(\theta \)  is a scalar location parameter, and assumes the high-precision limit where \({\rm q}(\hat{\theta }|\mathbf {x})\)  is well-approximated by a Gaussian. Despite this, \(\mathcal {I}_\mathcal {F}[\mathbf {x};\theta ]\)  is well-suited for our purposes, since (i) it leads to remarkably simple expressions for \(\psi (\theta )\)  below; (ii) it is concave in \(\psi \) ; (iii) it is exact in the “sampling” limit using narrow Gaussian-like components (where \(\mathcal {F}(\theta )\)  is the precision matrix of \({\rm q}(\hat{\theta }|\mathbf {x})\) ); (iv) in the large-\(\lambda \)  limit Expected KL still dominates \(\mathcal {I}_\mathcal {F}[\mathbf {x};\theta ]\) , and (v) finally, the inequality in (REF ) is likely strict in cases where the entropy of \(\psi (\theta )\)  itself is small (i.e. large \(\lambda \) ), since it neglects this additional prior information contained in \(\psi (\theta )\)  when estimating \(\hat{\theta }\)  and therefore over-estimates the entropy of \({\rm q}(\hat{\theta }|\mathbf {x})\) .By analogy to the Bayesian Cramér-Rao bound [2]}, [3]}, a tighter variant of (REF ) might be applied here, though possibly at the expense of added complexity; we leave this to future work. Importantly, points (ii)–(iv) in this list are enough to ensure that Theorems REF  and REF  still hold, i.e. that substituting \(\mathcal {I}_\mathcal {F}[\mathbf {x};\theta ]\)  for \(\mathcal {I}[\mathbf {x};\theta ]\)  in our original optimization problem (REF ) results in families of mixture-approximations that effectively interpolate sampling and VI. However, (REF ) is most applicable when using Gaussian or Gaussian-like components, and that the monotonicity arguments in Theorem REF  now hold with respect to \(\mathcal {I}_\mathcal {F}[\mathbf {x};\theta ]\)  rather than \(\mathcal {I}[\mathbf {x};\theta ]\) .
