There are many methods that use mixture models to reduce the bias of variational inference. In past work, mixtures are often directly optimized, reducing bias and maintaining low variance but at a \(\mathcal {O}(T^2)\)  computational cost using a mixture of \(T\)  components (Section REF  above). Simplifications include using greedy optimization [1]} or more efficient bounds on the Mutual Information term [2]}.
Our method reduces the cost per component at the expense of increased variance, since components may be independently sampled. Further, with some notable exceptions [3]}, [4]}, most mixture VI methods make strong assumptions about the family of components [5]}, [2]}, [7]}, [1]}. While our general framework is agnostic to the family of \({\rm q}\) , our approximation in Section  is most applicable to Gaussian components.
