We consider the Principal Component Analysis problem for large tensors of arbitrary order @math under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component can be estimated using unbounded computational resources. It turns out that this is possible as soon as the signal-to-noise ratio @math becomes larger than @math (and in particular @math can remain bounded as the problem dimensions increase). On the other hand, we analyze several polynomial-time estimation algorithms, based on tensor unfolding, power iteration and message passing ideas from graphical models. We show that, unless the signal-to-noise ratio diverges in the system dimensions, none of these approaches succeeds. This is possibly related to a fundamental limitation of computationally tractable estimators for this problem. We discuss various initializations for tensor power iteration, and show that a tractable initialization based on the spectrum of the matricized tensor outperforms significantly baseline methods, statistically and computationally. Finally, we consider the case in which additional side information is available about the unknown signal. We characterize the amount of side information that allows the iterative algorithms to converge to a good estimate.
Our Theorem is analogous to the main result of @cite_22 although incomparable: In @cite_22 the signal' part of the tensor @math is assumed to have an orthogonal decomposition @math with @math bounded away from zero. Here, the signal part has rank one (equivalently, all the @math 's but one vanish). In @cite_22 only the case of third order tensors ( @math ) is considered. We characterize power iteration for general @math . We establish convergence in a number of iterations @math that is @math . In @cite_22 the number of iterations is bounded by a polynomial in @math . We evaluate our bounds in the case of Gaussian noise. This allows a comparison with other methods, such as tensor unfolding.
@cite_22
2950741027
We consider the Principal Component Analysis problem for large tensors of arbitrary order @math under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component can be estimated using unbounded computational resources. It turns out that this is possible as soon as the signal-to-noise ratio @math becomes larger than @math (and in particular @math can remain bounded as the problem dimensions increase). On the other hand, we analyze several polynomial-time estimation algorithms, based on tensor unfolding, power iteration and message passing ideas from graphical models. We show that, unless the signal-to-noise ratio diverges in the system dimensions, none of these approaches succeeds. This is possibly related to a fundamental limitation of computationally tractable estimators for this problem. We discuss various initializations for tensor power iteration, and show that a tractable initialization based on the spectrum of the matricized tensor outperforms significantly baseline methods, statistically and computationally. Finally, we consider the case in which additional side information is available about the unknown signal. We characterize the amount of side information that allows the iterative algorithms to converge to a good estimate.