In this work we generalize the Jacobi-Davidson method to the case when eigenvector can be reshaped into a low-rank matrix. In this setting the proposed method inherits advantages of the original Jacobi-Davidson method, has lower complexity and requires less storage. We also introduce low-rank version of the Rayleigh quotient iteration which naturally arises in the Jacobi-Davidson method.
There are two standard ways to solve eigenvalue problems in low-rank format: optimization of Rayleigh quotient based on alternating minimization, which accounts for multilinear structure of the decomposition, and iterative methods with rank truncation. The first approach has been developed for a long time in the matrix product state community @cite_21 @cite_22 @cite_7 . We also should mention altenating minimization algorithms that were recently proposed in the mathematical community. They are based either on the alternating linear scheme (ALS) procedure @cite_4 @cite_2 or on basis enrichment using alternating minimal energy method (AMEn) @cite_14 @cite_1 . Rank truncated iterative methods include power method @cite_26 @cite_18 , inverse iteration @cite_16 , locally optimal block preconditioned conjugate gradient method @cite_0 @cite_3 @cite_24 . For more information about eigensolvers in low-rank formats see @cite_17 . To our knowledge no generalization of the Jacobi-Davidson method was considered.
@cite_18 @cite_14 @cite_4 @cite_22 @cite_7 @cite_26 @cite_21 @cite_1 @cite_3 @cite_0 @cite_24 @cite_2 @cite_16 @cite_17
2126407450 2089088255 2189538311 2949254743
We address the numerical problem of recovering large matrices of low rank when most of the entries are unknown. We exploit the geometry of the low-rank constraint to recast the problem as an unconstrained optimization problem on a single Grassmann manifold. We then apply second-order Riemannian trust-region methods (RTRMC 2) and Riemannian conjugate gradient methods (RCGMC) to solve it. A preconditioner for the Hessian is introduced that helps control the conditioning of the problem and we detail preconditioned versions of Riemannian optimization algorithms. The cost of each iteration is linear in the number of known entries. The proposed methods are competitive with state-of-the-art algorithms on a wide range of problem instances. In particular, they perform well on rectangular matrices. We further note that second-order and preconditioned methods are well suited to solve badly conditioned matrix completion tasks. In recent years, matrix rank minimization problems have received a significant amount of attention in machine learning, data mining and computer vision communities. And these problems can be solved by a convex relaxation of the rank minimization problem which minimizes the nuclear norm instead of the rank of the matrix, and has to be solved iteratively and involves singular value decomposition (SVD) at each iteration. Therefore, those algorithms for nuclear norm minimization problems suffer from high computation cost of multiple SVDs. In this paper, we propose a Fast Tri-Factorization (FTF) method to approximate the nuclear norm minimization problem and mitigate the computation cost of performing SVDs. The proposed FTF method can be used to reliably solve a wide range of low-rank matrix recovery and completion problems such as robust principal component analysis (RPCA), low-rank representation (LRR) and low-rank matrix completion (MC). We also present three specific models for RPCA, LRR and MC problems, respectively. Moreover, we develop two alternating direction method (ADM) based iterative algorithms for solving the above three problems. Experimental results on a variety of synthetic and real-world data sets validate the efficiency, robustness and effectiveness of our FTF method comparing with the state-of-the-art nuclear norm minimization algorithms. In this paper we propose a global optimization-based approach to jointly matching a set of images. The estimated correspondences simultaneously maximize pairwise feature affinities and cycle consistency across multiple images. Unlike previous convex methods relying on semidefinite programming, we formulate the problem as a low-rank matrix recovery problem and show that the desired semidefiniteness of a solution can be spontaneously fulfilled. The low-rank formulation enables us to derive a fast alternating minimization algorithm in order to handle practical problems with thousands of features. Both simulation and real experiments demonstrate that the proposed algorithm can achieve a competitive performance with an order of magnitude speedup compared to the state-of-the-art algorithm. In the end, we demonstrate the applicability of the proposed method to match the images of different object instances and as a result the potential to reconstruct category-specific object models from those images. In this paper we propose a global optimization-based approach to jointly matching a set of images. The estimated correspondences simultaneously maximize pairwise feature affinities and cycle consistency across multiple images. Unlike previous convex methods relying on semidefinite programming, we formulate the problem as a low-rank matrix recovery problem and show that the desired semidefiniteness of a solution can be spontaneously fulfilled. The low-rank formulation enables us to derive a fast alternating minimization algorithm in order to handle practical problems with thousands of features. Both simulation and real experiments demonstrate that the proposed algorithm can achieve a competitive performance with an order of magnitude speedup compared to the state-of-the-art algorithm. In the end, we demonstrate the applicability of the proposed method to match the images of different object instances and as a result the potential to reconstruct category-specific object models from those images.