Performing statistical inference in high-dimension is an outstanding challenge. A major source of difficulty is the absence of precise information on the distribution of high-dimensional estimators. Here, we consider linear regression in the high-dimensional regime @math . In this context, we would like to perform inference on a high-dimensional parameters vector @math . Important progress has been achieved in computing confidence intervals for single coordinates @math . A key role in these new methods is played by a certain debiased estimator @math that is constructed from the Lasso. Earlier work establishes that, under suitable assumptions on the design matrix, the coordinates of @math are asymptotically Gaussian provided @math is @math -sparse with @math . The condition @math is stronger than the one for consistent estimation, namely @math . We study Gaussian designs with known or unknown population covariance. When the covariance is known, we prove that the debiased estimator is asymptotically Gaussian under the nearly optimal condition @math . Note that earlier work was limited to @math even for perfectly known covariance. The same conclusion holds if the population covariance is unknown but can be estimated sufficiently well, e.g. under the same sparsity conditions on the inverse covariance as assumed by earlier work. For intermediate regimes, we describe the trade-off between sparsity in the coefficients and in the inverse covariance of the design. We further discuss several applications of our results to high-dimensional inference. In particular, we propose a new estimator that is minimax optimal up to a factor @math for i.i.d. Gaussian designs.
Most of the work on statistical inference in high-dimensional models has been focused so far on linear regression. The debiasing method admits a natural extension to generalized linear models that was analyzed in @cite_29 . Robustness to model misspecification was studied in @cite_36 . An R-package for inference in high-dimension that uses the node-wise Lasso is available @cite_46 . An R implementation of the method @cite_9 (which does not make sparsity assumptions on @math ) is also available See http: web.stanford.edu montanar sslasso . .
@cite_36 @cite_9 @cite_29 @cite_46
1531316514 2962726112 2037228647 2950190315
We consider high-dimensional inference when the assumed linear model is misspecified. We describe some correct interpretations and corresponding sufficient assumptions for valid asymptotic inference of the model parameters, which still have a useful meaning when the model is misspecified. We largely focus on the de-sparsified Lasso procedure but we also indicate some implications for (multiple) sample splitting techniques. In view of available methods and software, our results contribute to robustness considerations with respect to model misspecification. We consider the problem of fitting the parameters of a high-dimensional linear regression model. In the regime where the number of parameters p is comparable to or exceeds the sample size n, a successful approach uses an l1-penalized least squares estimator, known as Lasso. Unfortunately, unlike for linear estimators (e.g. ordinary least squares), no well-established method exists to compute confidence intervals or p-values on the basis of the Lasso estimator. Very recently, a line of work [8], [7], [13] has addressed this problem by constructing a debiased version of the Lasso estimator. We propose a special debiasing method that is well suited for random designs with sparse inverse covariance. Our approach improves over the state of the art in that it yields nearly optimal average testing power if sample size n asymptotically dominates s0(logp)2, with s0 being the sparsity level (number of non-zero coefficients). Earlier work achieved similar performances only for much larger sample size, namely it requires n to asymptotically dominates (s0 log p)2. We evaluate our method on synthetic data, and compare it with earlier proposals. We consider linear regression in the high-dimensional regime where the number of observations @math is smaller than the number of parameters @math . A very successful approach in this setting uses @math -penalized least squares (a.k.a. the Lasso) to search for a subset of @math parameters that best explain the data, while setting the other parameters to zero. Considerable amount of work has been devoted to characterizing the estimation and model selection problems within this approach. In this paper we consider instead the fundamental, but far less understood, question of . More precisely, we address the problem of computing p-values for single regression coefficients. On one hand, we develop a general upper bound on the minimax power of tests with a given significance level. On the other, we prove that this upper bound is (nearly) achievable through a practical procedure in the case of random design matrices with independent entries. Our approach is based on a debiasing of the Lasso estimator. The analysis builds on a rigorous characterization of the asymptotic distribution of the Lasso estimator and its debiased version. Our result holds for optimal sample size, i.e., when @math is at least on the order of @math . We generalize our approach to random design matrices with i.i.d. Gaussian rows @math . In this case we prove that a similar distributional characterization (termed standard distributional limit') holds for @math much larger than @math . Finally, we show that for optimal sample size, @math being at least of order @math , the standard distributional limit for general Gaussian designs can be derived from the replica heuristics in statistical physics. High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless @math , a line of recent work has studied models with various types of low-dimensional structure, including sparse vectors, sparse and structured matrices, low-rank matrices and combinations thereof. In such settings, a general approach to estimation is to solve a regularized optimization problem, which combines a loss function measuring how well the model fits the data with some regularization function that encourages the assumed structure. This paper provides a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive some existing results, and also to obtain a number of new results on consistency and convergence rates, in both @math -error and related norms. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure corresponding regularized M-estimators have fast convergence rates and which are optimal in many well-studied cases.