There are rapidly diminishing returns to increasing \(T\) , since Mutual Information is upper-bounded by \(\mathcal {I}[\mathbf {x};\theta ] \le \log T\) , with equality only when all components are mutually non-overlapping [1]}. In past work using mixture approximations, these diminishing returns were compounded by \(\mathcal {O}(T^2)\)  time and \(\mathcal {O}(T)\)  space complexity, since solving the joint optimization problem over \(T\)  components means selecting the parameters of each component depending on the other \(T-1\)  components, all of which must be in memory at once [1]}, [3]}, [4]}, [5]}, [6]} (but the \(\mathcal {O}(T^2)\)  complexity may be hardware-accelerated). We circumvent this by bounding Mutual Information using only local geometric information in (REF ), allowing each component to be selected independently of the others. This means we can select and evaluate \(T\)  components in \(\mathcal {O}(T)\)  time and either \(\mathcal {O}(T)\)  space (if all are stored) or \(\mathcal {O}(1)\)  space (if components are evaluated and accumulated online). Further, drawing samples of \(\theta \sim \psi (\theta )\)  is embarrassingly parallelizable into multiple sampling chains for a constant factor speedup.
