Low-rank approximation of a matrix by means of random sampling has been consistently efficient in its empirical studies by many scientists who applied it with various sparse and structured multipliers, but adequate formal support for this empirical phenomenon has been missing so far. Our novel insight into the subject leads to such an elusive formal support and promises significant acceleration of the known algorithms for some fundamental problems of matrix computations and data mining and analysis. Our formal results and our numerical tests are in good accordance with each other. We also outline extensions of low-rank approximation algorithms and of our progress to the Least Squares Regression, the Fast Multipole Method, and the Conjugate Gradient algorithms.
Part (ii) of our Theorem is implied by [Theorem 10.8] HMT11 , but our specific supporting estimates are more compact, cover the case of any @math (whereas @cite_36 assumes that @math ), and we deduce them by using a shorter proof (see Remark ). Our approach, our results listed in Section , some of our techniques, e.g., expan -sion compres -sion in Section , and even the concept of factor-Gaussian matrices are new, and so are our families of multipliers and policies of their generation, combination, and application in Sections and as well. Moreover, our progress can be extended to a variety of important matrix computations. In Section (Conclusions) we outline such novel extensions to the highly popular and much studied computations for Least Squares Regression, the Fast Multipole Method and the Conjugate Gradient Algorithms. Hereafter we use the acronyms LSR" for Least Squares Regression", FMM" for Fast Multipole Method", and CG " for Conjugate Gradient". The extensions provide new insights and new opportunities and should motivate further effort and further progress.
@cite_36
2051142108
We describe new algorithms of the locally optimal block preconditioned conjugate gradient (LOBPCG) method for symmetric eigenvalue problems, based on a local optimization of a three-term recurrence, and suggest several other new methods. To be able to compare numerically different methods in the class, with different preconditioners, we propose a common system of model tests, using random preconditioners and initial guesses. As the "ideal" control algorithm, we advocate the standard preconditioned conjugate gradient method for finding an eigenvector as an element of the null-space of the corresponding homogeneous system of linear equations under the assumption that the eigenvalue is known. We recommend that every new preconditioned eigensolver be compared with this "ideal" algorithm on our model test problems in terms of the speed of convergence, costs of every iteration, and memory requirements. We provide such comparison for our LOBPCG method. Numerical results establish that our algorithm is practically as efficient as the ideal'' algorithm when the same preconditioner is used in both methods. We also show numerically that the LOBPCG method provides approximations to first eigenpairs of about the same quality as those by the much more expensive global optimization method on the same generalized block Krylov subspace. We propose a new version of block Davidson's method as a generalization of the LOBPCG method. Finally, direct numerical comparisons with the Jacobi--Davidson method show that our method is more robust and converges almost two times faster.