We study the problem of finding orthogonal low-rank approximations of symmetric tensors. In the case of matrices, the approximation is a truncated singular value decomposition which is then symmetric. Moreover, for rank-one approximations of tensors of any dimension, a classical result proven by Banach in 1938 shows that the optimal approximation can always be chosen to be symmetric. In contrast to these results, this article shows that the corresponding statement is no longer true for orthogonal approximations of higher rank. Specifically, for any of the four common notions of tensor orthogonality used in the literature, we show that optimal orthogonal approximations of rank greater than one cannot always be chosen to be symmetric.
The convergence guarantees of the orthogonal approximation problem have been improved by imposing structural assumptions on the tensors under consideration. An important special case that guarantees also global convergence is when @math is completely orthogonally decomposable, i.e., when @math has an exact orthogonal (but unknown) decomposition @math with @math for all @math . Optimal rank- @math orthogonal decompositions for these tensors can be computed by successively computing the optimal rank-one approximations, subtracting these from @math and iterating, in a deflation procedure @cite_10 . An optimal rank-one approximation of orthogonally decomposable tensors can be computed efficiently by the tensor power iteration method @cite_24 and each term corresponds to a singular vector of the original tensor. This method has also been extended to nearly completely orthogonally decomposable tensors @cite_24 @cite_4 @cite_6 .
@cite_24 @cite_10 @cite_6 @cite_4
2242384245 2553316781 1965727018 2963282659
Many idealized problems in signal processing, machine learning, and statistics can be reduced to the problem of finding the symmetric canonical decomposition of an underlying symmetric and orthogonally decomposable (SOD) tensor. Drawing inspiration from the matrix case, the successive rank-one approximation (SROA) scheme has been proposed and shown to yield this tensor decomposition exactly, and a plethora of numerical methods have thus been developed for the tensor rank-one approximation problem. In practice, however, the inevitable errors (say) from estimation, computation, and modeling necessitate that the input tensor can only be assumed to be a nearly SOD tensor---i.e., a symmetric tensor slightly perturbed from the underlying SOD tensor. This paper shows that even in the presence of perturbation, SROA can still robustly recover the symmetric canonical decomposition of the underlying tensor. It is shown that when the perturbation error is small enough, the approximation errors do not accumulate with ... A recent work (Wang et. al., NIPS 2015) gives the fastest known algorithms for orthogonal tensor decomposition with provable guarantees. Their algorithm is based on computing sketches of the input tensor, which requires reading the entire input. We show in a number of cases one can achieve the same theoretical guarantees in sublinear time, i.e., even without reading most of the input tensor. Instead of using sketches to estimate inner products in tensor decomposition algorithms, we use importance sampling. To achieve sublinear time, we need to know the norms of tensor slices, and we show how to do this in a number of important cases. For symmetric tensors T = ∑ki=1 λiui⊗p with λi > 0 for all i, we estimate such norms in sublinear time whenever p is even. For the important case of p = 3 and small values of k, we can also estimate such norms. For asymmetric tensors sublinear time is not possible in general, but we show if the tensor slice norms are just slightly below || T ||F then sublinear time is again possible. One of the main strengths of our work is empirical - in a number of cases our algorithm is orders of magnitude faster than existing methods with the same accuracy. In this paper, we propose a method for the approximation of the solution of high- dimensional weakly coercive problems formulated in tensor spaces using low-rank approximation for- mats. The method can be seen as a perturbation of a minimal residual method with a measure of the residual corresponding to the error in a specified solution norm. The residual norm can be de- signed such that the resulting low-rank approximations are optimal with respect to particular norms of interest, thus allowing to take into account a particular objective in the definition of reduced order approximations of high-dimensional problems. We introduce and analyze an iterative algorithm that is able to provide an approximation of the optimal approximation of the solution in a given low-rank subset, without any ap rioriinformation on this solution. We also introduce a weak greedy algorithm which uses this perturbed minimal residual method for the computation of successive greedy correc- tions in small tensor subsets. We prove its convergence under some conditions on the parameters of the algorithm. The proposed numerical method is applied to the solution of a stochastic partial differential equation which is discretized using standard Galerkin methods in tensor product spaces. Mathematics Subject Classification. 15A69, 35J50, 41A63, 65D15, 65N12. A new algorithm is presented for computing a canonical rank- @math tensor approximation that has minimal distance to a given tensor in the Frobenius norm, where the canonical rank- @math tensor consists of the sum of @math rank-one tensors. Each iteration of the method consists of three steps. In the first step, a tentative new iterate is generated by a stand-alone one-step process, for which we use alternating least squares (ALS). In the second step, an accelerated iterate is generated by a nonlinear generalized minimal residual (GMRES) approach, recombining previous iterates in an optimal way, and essentially using the stand-alone one-step process as a preconditioner. In particular, the nonlinear extension of GMRES we use that was proposed by Washio and Oosterlee in [Electron. Trans. Numer. Anal., 15 (2003), pp. 165--185] for nonlinear partial differential equation problems (which is itself related to other existing acceleration methods for nonlinear equation systems). In the third step, a line search is perfor...