The columnwise Khatri–Rao product of two matrices is an important matrix type, reprising its role as a structured sensing matrix in many fundamental linear inverse problems. Robust signal recovery in such inverse problems is often contingent on proving the restricted isometry property (RIP) of a certain system matrix expressible as a Khatri–Rao product of two matrices. In this paper, we analyze the RIP of a generic columnwise Khatri–Rao product matrix by deriving two upper bounds for its @math th order restricted isometry constant ( @math -RIC) for different values of @math . The first RIC bound is computed in terms of the individual RICs of the real-valued input matrices participating in the Khatri–Rao product. The second RIC bound is probabilistic and is specified in terms of the input matrix dimensions. We show that the Khatri–Rao product of a pair of @math sized random matrices comprising independent and identically distributed sub-Gaussian entries satisfies @math -RIP with arbitrarily high probability, provided @math exceeds @math . This is a substantially milder condition compared to @math rows needed to guarantee @math -RIP of the input sub-Gaussian random matrices participating in the Khatri–Rao product. Our RIC bounds confirm that the Khatri–Rao product exhibits stronger restricted isometry compared to its constituent matrices for the same RIP order. The proposed RIC bounds are potentially useful in obtaining improved performance guarantees in several sparse signal recovery and tensor decomposition problems.
Perhaps the most direct approach for analyzing the RICs of the KR product matrix is to use the eigenvalue interlacing theorem @cite_1 , which relates the singular values of any @math -column submatrix of the KR product between two matrices to the singular values of their Kronecker product. This is possible because any @math columns of the KR product can together be interpreted as a submatrix of the Kronecker product. However, barring the maximum and minimum singular values of the Kronecker product, there is no available explicit characterization of its non-extremal singular values that can be used to obtain tight bounds for the @math -RIC of the KR product. Bounding the RIC using the extreme singular values of the Kronecker product matrix turns out to be too loose to be useful. In this context, it is noteworthy to mention that an upper bound for the @math -RIC of the Kronecker product is derived in terms of the @math -RICs of the input matrices in @cite_6 @cite_28 . However, the @math -RIC of the Khatri-Rao product is yet to be analyzed.
@cite_28 @cite_1 @cite_6
2899347127 2963373272 1554201860 2101043704
Given an m ×n matrix A and an integer k less than the rank of A, the “best” rank k approximation to A that minimizes the error with respect to the Frobenius norm is Ak, which is obtained by projecting A on the top k left singular vectors of A. While Ak is routinely used in data analysis, it is difficult to interpret and understand it in terms of the original data, namely the columns and rows of A. For example, these columns and rows often come from some application domain, whereas the singular vectors are linear combinations of (up to all) the columns or rows of A. We address the problem of obtaining low-rank approximations that are directly interpretable in terms of the original columns or rows of A. Our main results are two polynomial time randomized algorithms that take as input a matrix A and return as output a matrix C, consisting of a “small” (i.e., a low-degree polynomial in k, 1 e, and log(1 δ)) number of actual columns of A such that ||A–CC+A||F ≤(1+e) ||A–Ak||F with probability at least 1–δ. Our algorithms are simple, and they take time of the order of the time needed to compute the top k right singular vectors of A. In addition, they sample the columns of A via the method of “subspace sampling,” so-named since the sampling probabilities depend on the lengths of the rows of the top singular vectors and since they ensure that we capture entirely a certain subspace of interest. Abstract We derive bounds on the extremal singular values and the condition number of N × K , with N ⩾ K , Vandermonde matrices with nodes in the unit disk. The mathematical techniques we develop to prove our main results are inspired by a link—first established by Selberg [1] and later extended by Moitra [2] —between the extremal singular values of Vandermonde matrices with nodes on the unit circle and large sieve inequalities. Our main conceptual contribution lies in establishing a connection between the extremal singular values of Vandermonde matrices with nodes in the unit disk and a novel large sieve inequality involving polynomials in z ∈ C with | z | ⩽ 1 . Compared to Bazan's upper bound on the condition number [3] , which, to the best of our knowledge, constitutes the only analytical result—available in the literature—on the condition number of Vandermonde matrices with nodes in the unit disk, our bound not only takes a much simpler form, but is also sharper for certain node configurations. Moreover, the bound we obtain can be evaluated consistently in a numerically stable fashion, whereas the evaluation of Bazan's bound requires the solution of a linear system of equations which has the same condition number as the Vandermonde matrix under consideration and can therefore lead to numerical instability in practice. As a byproduct, our result—when particularized to the case of nodes on the unit circle—slightly improves upon the Selberg–Moitra bound. Given a matrix A e ℝ m ×n of rank r, and an integer k < r, the top k singular vectors provide the best rank-k approximation to A. When the columns of A have specific meaning, it is desirable to find (provably) "good" approximations to A k which use only a small number of columns in A. Proposed solutions to this problem have thus far focused on randomized algorithms. Our main result is a simple greedy deterministic algorithm with guarantees on the performance and the number of columns chosen. Specifically, our greedy algorithm chooses c columns from A with @math such that @math where C gr is the matrix composed of the c columns, @math is the pseudo-inverse of C gr ( @math is the best reconstruction of A from C gr), and ¼(A) is a measure of the coherence in the normalized columns of A. The running time of the algorithm is O(SVD(A k) + mnc) where SVD(A k) is the running time complexity of computing the first k singular vectors of A. To the best of our knowledge, this is the first deterministic algorithm with performance guarantees on the number of columns and a (1 + e) approximation ratio in Frobenius norm. The algorithm is quite simple and intuitive and is obtained by combining a generalization of the well known sparse approximation problem from information theory with an existence result on the possibility of sparse approximation. Tightening the analysis along either of these two dimensions would yield improved results. We design a new distribution over poly(r e-1) x n matrices S so that for any fixed n x d matrix A of rank r, with probability at least 9 10, SAx2 = (1 pm e)Ax2 simultaneously for all x ∈ Rd. Such a matrix S is called a subspace embedding. Furthermore, SA can be computed in O(nnz(A)) + O(r2e-2) time, where nnz(A) is the number of non-zero entries of A. This improves over all previous subspace embeddings, which required at least Ω(nd log d) time to achieve this property. We call our matrices S sparse embedding matrices. Using our sparse embedding matrices, we obtain the fastest known algorithms for overconstrained least-squares regression, low-rank approximation, approximating all leverage scores, and lp-regression: to output an x' for which Ax'-b2 ≤ (1+e)minx Ax-b2 for an n x d matrix A and an n x 1 column vector b, we obtain an algorithm running in O(nnz(A)) + O(d3e-2) time, and another in O(nnz(A)log(1 e)) + O(d3log(1 e)) time. (Here O(f) = f ⋅ logO(1)(f).) to obtain a decomposition of an n x n matrix A into a product of an n x k matrix L, a k x k diagonal matrix D, and a n x k matrix W, for which F A - L D W ≤ (1+e)F A-Ak , where Ak is the best rank-k approximation, our algorithm runs in O(nnz(A)) + O(nk2 e-4log n + k3e-5log2n) time. to output an approximation to all leverage scores of an n x d input matrix A simultaneously, with constant relative error, our algorithms run in O(nnz(A) log n) + O(r3) time. to output an x' for which Ax'-bp ≤ (1+e)minx Ax-bp for an n x d matrix A and an n x 1 column vector b, we obtain an algorithm running in O(nnz(A) log n) + poly(r e-1) time, for any constant 1 ≤ p