The Nystr " o m method is routinely used for out-of-sample extension of kernel matrices. We describe how this method can be applied to find the singular value decomposition (SVD) of general matrices and the eigenvalue decomposition (EVD) of square matrices. We take as an input a matrix @math , a user defined integer @math and @math , a matrix sampled from the columns and rows of @math . These are used to construct an approximate rank- @math SVD of @math in @math operations. If @math is square, the rank- @math EVD can be similarly constructed in @math operations. Thus, the matrix @math is a compressed version of @math . We discuss the choice of @math and propose an algorithm that selects a good initial sample for a pivoted version of @math . The proposed algorithm performs well for general matrices and kernel matrices whose spectra exhibit fast decay.
SVM training is a convex optimization problem which scales with the training set size rather than the feature space dimension. While this is usually considered to be a desired quality, in large scale problems it may cause training to be impractical. The common techniques to handle this difficulty basically build a solution by solving a sequence of small scale subproblems. Our current effort is concentrated on the rank of the kernel matrix as a source for further enhancement of the training procedure. We first show that for a low rank kernel matrix it is possible to design a better interior point method (IPM) in terms of storage requirements as well as computational complexity. We then suggest an efficient use of a known factorization technique to approximate a given kernel matrix by a low rank matrix, which in turn will be used to feed the optimizer. Finally, we derive an upper bound on the change in the objective function value based on the approximation error and the number of active constraints (support vectors). This bound is general in the sense that it holds regardless of the approximation method.