Regarding the decomposition of sample moment tensors REF , streaming and implicit techniques can be applied to address the computational burden in the data-rich and high-dimensional regime.
For covariance matrices which are centered second-order moments, the streaming PCA scheme a.k.a. Oja's method [1]}, and its adaptations [2]}, [3]}, [4]}, have been thoroughly studied.
Recently, a similar strategy is applied to optimization with higher-order moment tensors for the task of learning Gaussian mixture models, see the
works [5]}, [6]}.
The common ideas with the present work are that we pass over only a subset of samples
at a time and drive the optimization without forming gigantic moment tensors.
