Consider an Erd "os-Renyi random graph in which each edge is present independently with probability 1 2, except for a subset @math of the vertices that form a clique (a completely connected subgraph). We consider the problem of identifying the clique, given a realization of such a random graph. The best known algorithm provably finds the clique in linear time with high probability, provided @math dekel2011finding . Spectral methods can be shown to fail on cliques smaller than @math . In this paper we describe a nearly linear time algorithm that succeeds with high probability for @math for any @math . This is the first algorithm that provably improves over spectral methods. We further generalize the hidden clique problem to other background graphs (the standard case corresponding to the complete graph on @math vertices). For large girth regular graphs of degree @math we prove that local' algorithms succeed if @math and fail if @math .
A rich line of research in statistics addresses the problem of identifying the non-zero entries in a sparse vector (or matrix) @math from observations @math where @math has typically i.i.d. standard Gaussian entries. In particular @cite_38 @cite_2 @cite_31 @cite_0 study cases in which the sparsity pattern of @math is structured'. For instance, we can take @math a matrix with @math if @math and @math otherwise. This fits the framework studied in this paper, for @math the complete graph and @math , @math . This literature however disregards computational considerations. Greedy search methods were developed in several papers, see e.g. @cite_5 @cite_22 .
@cite_38 @cite_22 @cite_0 @cite_2 @cite_5 @cite_31
2949962875 2052602889 2127300249 2764111657
In sparse principal component analysis we are given noisy observations of a low-rank matrix of dimension @math and seek to reconstruct it under additional sparsity assumptions. In particular, we assume here each of the principal components @math has at most @math non-zero entries. We are particularly interested in the high dimensional regime wherein @math is comparable to, or even much larger than @math . In an influential paper, johnstone2004sparse introduced a simple algorithm that estimates the support of the principal vectors @math by the largest entries in the diagonal of the empirical covariance. This method can be shown to identify the correct support with high probability if @math , and to fail with high probability if @math for two constants @math . Despite a considerable amount of work over the last ten years, no practical algorithm exists with provably better support recovery guarantees. Here we analyze a covariance thresholding algorithm that was recently proposed by KrauthgamerSPCA . On the basis of numerical simulations (for the rank-one case), these authors conjectured that covariance thresholding correctly recover the support with high probability for @math (assuming @math of the same order as @math ). We prove this conjecture, and in fact establish a more general guarantee including higher-rank as well as @math much smaller than @math . Recent lower bounds berthet2013computational, ma2015sum suggest that no polynomial time algorithm can do significantly better. The key technical component of our analysis develops new bounds on the norm of kernel random matrices, in regimes that were not considered before. The subject of sparse matrices has its roots in such diverse fields as management science, power systems analysis, surveying, circuit theory, and structural analysis. Mathematical models in all these areas give rise to very large systems of linear equations which could not be solved were it not for the fact that the matrices contain relatively few non-zero entries. Only comparatively recently, in the last fifteen years or so, has it become apparent that the equations can be solved even when the pattern is irregular, and it is primarily the solution of such problems that is considered in this book. The subject is intensely practical and this book is written with practicalities ever in mind. Whenever two methods are applicable, their rival merits are considered, and conclusions are based on practical experience where theoretical comparison is not possible. Non-numeric computing techniques have been included, as well as frequent illustrations, in an attempt to bridge the usually wide gap between the printed page and the working computer code. Despite this practical bias, it is recognized that many aspects of the subject are of interest in their own right, and the book aims to be suitable also for a student course, probably at MSc level. Exercises have been included to illustrate and strengthen understanding of the material, as well as to extend it. Efficient use of sparsity is a key to solving large problems in many fields. This book will supply both insight and answers for those attempting to solve these problems. The problem of consistently estimating the sparsity pattern of a vector β* ∈ RP based on observations contaminated by noise arises in various contexts, including signal denoising, sparse approximation, compressed sensing, and model selection. We analyze the behavior of l1-constrained quadratic programming (QP), also referred to as the Lasso, for recovering the sparsity pattern. Our main result is to establish precise conditions on the problern dimension p, the number k of nonzero elements in β*, and the number of observations n that are necessary and sufficient for sparsity pattern recovery using the Lasso. We first analyze the case of observations made using deterministic design matrices and sub-Gaussian additive noise, and provide sufficient conditions for support recovery and l∞-error bounds, as well as results showing the necessity of incoherence and bounds on the minimum value. We then turn to the case of random designs, in which each row of the design is drawn from a N(0, Σ) ensemble. For a broad class of Gaussian ensembles satisfying mutual incoherence conditions, we compute explicit values of thresholds 0 0, if n > 2(θu + δ)k log(p - k), then the Lasso succeeds in recovering the sparsity pattern with probability converging to one for large problems, whereas for n < 2(θl - δ)k log(p - k), then the probability of successful recovery converges to zero. For the special case of the uniform Gaussian ensemble (Σ = I p×p), we show that θl = θu = 1, so that the precise threshold n = 2k log (p - k) is exactly determined. In this paper we consider memoryless one-bit compressed sensing with randomly subsampled Gaussian circulant matrices. We show that in a small sparsity regime and for small enough accuracy @math , @math measurements suffice to reconstruct the direction of any @math -sparse vector up to accuracy @math via an efficient program. We derive this result by proving that partial Gaussian circulant matrices satisfy an @math RIP-property. Under a slightly worse dependence on @math , we establish stability with respect to approximate sparsity, as well as full vector recovery results.