Consider an Erd "os-Renyi random graph in which each edge is present independently with probability 1 2, except for a subset @math of the vertices that form a clique (a completely connected subgraph). We consider the problem of identifying the clique, given a realization of such a random graph. The best known algorithm provably finds the clique in linear time with high probability, provided @math dekel2011finding . Spectral methods can be shown to fail on cliques smaller than @math . In this paper we describe a nearly linear time algorithm that succeeds with high probability for @math for any @math . This is the first algorithm that provably improves over spectral methods. We further generalize the hidden clique problem to other background graphs (the standard case corresponding to the complete graph on @math vertices). For large girth regular graphs of degree @math we prove that local' algorithms succeed if @math and fail if @math .
A rich line of research in statistics addresses the problem of identifying the non-zero entries in a sparse vector (or matrix) @math from observations @math where @math has typically i.i.d. standard Gaussian entries. In particular @cite_38 @cite_2 @cite_31 @cite_0 study cases in which the sparsity pattern of @math is structured'. For instance, we can take @math a matrix with @math if @math and @math otherwise. This fits the framework studied in this paper, for @math the complete graph and @math , @math . This literature however disregards computational considerations. Greedy search methods were developed in several papers, see e.g. @cite_5 @cite_22 .
@cite_38 @cite_22 @cite_0 @cite_2 @cite_5 @cite_31
1984238247 2952259914 2107307520 2072157164
We investigate the minimization of Newton's functional for the problem of the body of minimal resistance with maximal height @math cite butt in the class of convex developable functions defined in a disc. This class is a natural candidate to find a (non-radial) minimizer in accordance with the results of cite lrp2. We prove that the minimizer in this class has a minimal set in the form of a regular polygon with @math sides centered in the disc, and numerical experiments indicate that the natural number @math is a non-decreasing function of @math . The corresponding functions all achieve a lower value of the functional than the optimal radially symmetric function with the same height @math . The following problem is considered: given a matrix @math in @math , ( @math rows and @math columns), a vector @math in @math , and @math , compute a vector @math satisfying @math if such exists, such that @math has the fewest number of non-zero entries over all such vectors. It is shown that the problem is NP-hard, but that the well-known greedy heuristic is good in that it computes a solution with at most @math non-zero entries, where @math is the optimum number of nonzero entries at error @math , @math is the matrix obtained by normalizing each column of @math with respect to the @math norm, and @math is its pseudo-inverse. In sparse principal component analysis we are given noisy observations of a low-rank matrix of dimension @math and seek to reconstruct it under additional sparsity assumptions. In particular, we assume here each of the principal components @math has at most @math non-zero entries. We are particularly interested in the high dimensional regime wherein @math is comparable to, or even much larger than @math . In an influential paper, johnstone2004sparse introduced a simple algorithm that estimates the support of the principal vectors @math by the largest entries in the diagonal of the empirical covariance. This method can be shown to identify the correct support with high probability if @math , and to fail with high probability if @math for two constants @math . Despite a considerable amount of work over the last ten years, no practical algorithm exists with provably better support recovery guarantees. Here we analyze a covariance thresholding algorithm that was recently proposed by KrauthgamerSPCA . On the basis of numerical simulations (for the rank-one case), these authors conjectured that covariance thresholding correctly recover the support with high probability for @math (assuming @math of the same order as @math ). We prove this conjecture, and in fact establish a more general guarantee including higher-rank as well as @math much smaller than @math . Recent lower bounds berthet2013computational, ma2015sum suggest that no polynomial time algorithm can do significantly better. The key technical component of our analysis develops new bounds on the norm of kernel random matrices, in regimes that were not considered before. We consider the problem of estimating an unknown signal @math from noisy linear observations @math . In many practical instances, @math has a certain structure that can be captured by a structure inducing convex function @math . For example, @math norm can be used to encourage a sparse solution. To estimate @math with the aid of @math , we consider the well-known LASSO method and provide sharp characterization of its performance. We assume the entries of the measurement matrix @math and the noise vector @math have zero-mean normal distributions with variances @math and @math respectively. For the LASSO estimator @math , we attempt to calculate the Normalized Square Error (NSE) defined as @math as a function of the noise level @math , the number of observations @math and the structure of the signal. We show that, the structure of the signal @math and choice of the function @math enter the error formulae through the summary parameters @math and @math , which are defined as the Gaussian squared-distances to the subdifferential cone and to the @math -scaled subdifferential, respectively. The first LASSO estimator assumes a-priori knowledge of @math and is given by @math . We prove that its worst case NSE is achieved when @math and concentrates around @math . Secondly, we consider @math , for some @math . This time the NSE formula depends on the choice of @math and is given by @math . We then establish a mapping between this and the third estimator @math . Finally, for a number of important structured signal classes, we translate our abstract formulae to closed-form upper bounds on the NSE. We introduce the framework of blind regression motivated by matrix completion for recommendation systems: given @math users, @math movies, and a subset of user-movie ratings, the goal is to predict the unobserved user-movie ratings given the data, i.e., to complete the partially observed matrix. Following the framework of non-parametric statistics, we posit that user @math and movie @math have features @math and @math respectively, and their corresponding rating @math is a noisy measurement of @math for some unknown function @math . In contrast with classical regression, the features @math are not observed, making it challenging to apply standard regression methods to predict the unobserved ratings. Inspired by the classical Taylor's expansion for differentiable functions, we provide a prediction algorithm that is consistent for all Lipschitz functions. In fact, the analysis through our framework naturally leads to a variant of collaborative filtering, shedding insight into the widespread success of collaborative filtering in practice. Assuming each entry is sampled independently with probability at least @math with @math , we prove that the expected fraction of our estimates with error greater than @math is less than @math plus a polynomially decaying term, where @math is the variance of the additive entry-wise noise term. Experiments with the MovieLens and Netflix datasets suggest that our algorithm provides principled improvements over basic collaborative filtering and is competitive with matrix factorization methods. In sparse recovery we are given a matrix @math (the dictionary) and a vector of the form @math where @math is sparse, and the goal is to recover @math . This is a central notion in signal processing, statistics and machine learning. But in applications such as sparse coding, edge detection, compression and super resolution, the dictionary @math is unknown and has to be learned from random examples of the form @math where @math is drawn from an appropriate distribution --- this is the dictionary learning problem. In most settings, @math is overcomplete: it has more columns than rows. This paper presents a polynomial-time algorithm for learning overcomplete dictionaries; the only previously known algorithm with provable guarantees is the recent work of Spielman, Wang and Wright who gave an algorithm for the full-rank case, which is rarely the case in applications. Our algorithm applies to incoherent dictionaries which have been a central object of study since they were introduced in seminal work of Donoho and Huo. In particular, a dictionary is @math -incoherent if each pair of columns has inner product at most @math . The algorithm makes natural stochastic assumptions about the unknown sparse vector @math , which can contain @math non-zero entries (for any @math ). This is close to the best @math allowable by the best sparse recovery algorithms even if one knows the dictionary @math exactly. Moreover, both the running time and sample complexity depend on @math , where @math is the target accuracy, and so our algorithms converge very quickly to the true dictionary. Our algorithm can also tolerate substantial amounts of noise provided it is incoherent with respect to the dictionary (e.g.,