This paper considers the problem of recovering an unknown sparse p⇥ p matrix X from an m ⇥ m matrix Y = AXB T ,w hereA and B are known m ⇥ p matrices with m ⌧ p. The main result shows that there exist constructions of the “sketching” matrices A and B so that even if X has O(p) non-zeros, it can be recovered exactly and eciently using a convex program as long as these non-zeros are not concentrated in any single row column of X. Furthermore, it suces for the size of Y (the sketch dimension) to scale as m = O p # nonzeros in X ⇥ logp . The results also show that the recovery is robust and stable in the sense that if X is equal to a sparse matrix plus a perturbation, then the convex program we propose produces an approximation with accuracy proportional to the size of the perturbation. Unlike traditional results on sparse recovery, where the sensing matrix produces independent measurements, our sensing operator is highly constrained (it assumes a tensor product structure). Therefore, proving recovery guarantees require non-standard techniques. Indeed our approach relies on a novel result concerning tensor products of bipartite graphs, which may be of independent interest. This problem is motivated by the following application, among others. Consider a p ⇥ n data matrix D, consisting of n observations of p variables. Assume that the correlation matrix X := DD T is (approximately) sparse in the sense that each of the p variables is significantly correlated with only a few others. Our results show that these significant correlations can be detected even if we have access to only a sketch of the data S = AD with A 2 R m⇥p .
Our work is also motivated by the work on sketching in the computer science community; this literature deals with the idea of compressing high-dimensional data vectors via projection to low-dimensions while preserving pertinent geometric properties. The celebrated Johnson-Lindenstrauss Lemma @cite_2 is one such result, and the idea of sketching has been explored in various contexts @cite_20 @cite_27 . The idea of using random bipartite graphs and their related expansion properties, which motivated our approach to the problem, have also been studied in past work @cite_33 @cite_26 @cite_14 .
@cite_26 @cite_14 @cite_33 @cite_27 @cite_2 @cite_20
1972939134 2963274201 2149463264 2036959865
The Johnson-Lindenstrauss lemma provides a way to map a number of points in high-dimensional space into a low-dimensional space, with only a small distortion of the distances between the points. The proofs of the lemma are non-constructive: they show that a random mapping induces small distortions with high probability, but they do not construct the actual mapping. In this paper, we provide a procedure that constructs such a mapping deterministically in time almost linear in the number of distances to preserve times the dimension of the original space. We then use that result (together with Nisan's pseudorandom generator) to obtain an efficient derandomization of several approximation algorithms based on semidefinite programming. This paper resolves one of the longest standing basic problems in the streaming computational model. Namely, optimal construction of quantile sketches. An e approximate quantile sketch receives a stream of items x1,·,xn and allows one to approximate the rank of any query item up to additive error e n with probability at least 1-δ.The rank of a query x is the number of stream items such that xi ≤ x. The minimal sketch size required for this task is trivially at least 1 e.Felber and Ostrovsky obtain a O((1 e)log(1 e)) space sketch for a fixed δ.Without restrictions on the nature of the stream or the ratio between e and n, no better upper or lower bounds were known to date. This paper obtains an O((1 e)log log (1 δ)) space sketch and a matching lower bound. This resolves the open problem and proves a qualitative gap between randomized and deterministic quantile sketching for which an Ω((1 e)log(1 e)) lower bound is known. One of our contributions is a novel representation and modification of the widely used merge-and-reduce construction. This modification allows for an analysis which is both tight and extremely simple. The same technique was reported, in private communications, to be useful for improving other sketching objectives and geometric coreset constructions. The Johnson--Lindenstrauss lemma shows that any n points in Euclidean space (i.e., ℝn with distances measured under the e 2 norm) may be mapped down to O((log n) e2) dimensions such that no pairwise distance is distorted by more than a (1 p e) factor. Determining whether such dimension reduction is possible in e 1 has been an intriguing open question. We show strong lower bounds for general dimension reduction in e 1 . We give an explicit family of n points in e 1 such that any embedding with constant distortion D requires nΩ(1 D2) dimensions. This proves that there is no analog of the Johnson--Lindenstrauss lemma for e 1 ; in fact, embedding with any constant distortion requires nΩ(1) dimensions. Further, embedding the points into e 1 with (1pe) distortion requires nh−O(e log(1 e)) dimensions. Our proof establishes this lower bound for shortest path metrics of series-parallel graphs. We make extensive use of linear programming and duality in devising our bounds. We expect that the tools and techniques we develop will be useful for future investigations of embeddings into e 1 . We study the metric properties of finite subsets of L1. The analysis of such metrics is central to a number of important algorithmic problems involving the cut structure of weighted graphs, including the Sparsest Cut Problem, one of the most compelling open problems in the field of approximation algorithms. Additionally, many open questions in geometric non-linear functional analysis involve the properties of finite subsets of L1.We present some new observations concerning the relation of L1 to dimension, topology, and Euclidean distortion. We show that every n-point subset of L1 embeds into L2 with average distortion O(√log n), yielding the first evidence that the conjectured worst-case bound of O(√log n) is valid. We also address the issue of dimension reduction in Lp for p ∈ (1, 2). We resolve a question left open by M. Charikar and A. Sahai [Dimension reduction in the l1 norm, in: Proceedings of the 43rd Annual IEEE Conference on Foundations of Computer Science, ACM, 2002, pp. 251-260] concerning the impossibility of dimension reduction with a linear map in the above cases, and we show that a natural variant of the recent example of Brinkman and Charikar [On the impossibility of dimension reduction in l1, in: Proceedings of the 44th Annual IEEE Conference on Foundations of Computer Science, ACM, 2003, pp. 514-523], cannot be used to prove a lower bound for the non-linear case. This is acomplished by exhibiting constant-distortion embeddings of snowflaked planar metrics into Euclidean space.