Performing statistical inference in high-dimension is an outstanding challenge. A major source of difficulty is the absence of precise information on the distribution of high-dimensional estimators. Here, we consider linear regression in the high-dimensional regime @math . In this context, we would like to perform inference on a high-dimensional parameters vector @math . Important progress has been achieved in computing confidence intervals for single coordinates @math . A key role in these new methods is played by a certain debiased estimator @math that is constructed from the Lasso. Earlier work establishes that, under suitable assumptions on the design matrix, the coordinates of @math are asymptotically Gaussian provided @math is @math -sparse with @math . The condition @math is stronger than the one for consistent estimation, namely @math . We study Gaussian designs with known or unknown population covariance. When the covariance is known, we prove that the debiased estimator is asymptotically Gaussian under the nearly optimal condition @math . Note that earlier work was limited to @math even for perfectly known covariance. The same conclusion holds if the population covariance is unknown but can be estimated sufficiently well, e.g. under the same sparsity conditions on the inverse covariance as assumed by earlier work. For intermediate regimes, we describe the trade-off between sparsity in the coefficients and in the inverse covariance of the design. We further discuss several applications of our results to high-dimensional inference. In particular, we propose a new estimator that is minimax optimal up to a factor @math for i.i.d. Gaussian designs.
A parallel line of research develops methods for performing valid inference after a low-dimensional model is selected for fitting high-dimensional data @cite_34 @cite_2 @cite_50 @cite_60 . The resulting significance statements are typically conditional on the selected model. In contrast, here we are interested in classical (unconditional) significance statements: the two approaches are broadly complementary.
@cite_60 @cite_34 @cite_50 @cite_2
2009462809 2164275109 1599875939 1974403130
It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid post-selection inference'' by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all submodels. By purchasing simultaneity insurance'' for all possible submodels, the resulting post-selection inference is rendered universally valid under all possible model selection procedures. This inference is therefore generally conservative for particular selection procedures, but it is always less conservative than full Scheffe protection. Importantly it does not depend on the truth of the selected submodel, and hence it produces valid inference even in wrong models. We describe the structure of the simultaneous inference problem and give some asymptotic results. We develop a framework for post model selection inference, via marginal screening, in linear regression. At the core of this framework is a result that characterizes the exact distribution of linear functions of the response @math , conditional on the model being selected ( condition on selection" framework). This allows us to construct valid confidence intervals and hypothesis tests for regression coefficients that account for the selection procedure. In contrast to recent work in high-dimensional statistics, our results are exact (non-asymptotic) and require no eigenvalue-like assumptions on the design matrix @math . Furthermore, the computational cost of marginal regression, constructing confidence intervals and hypothesis testing is negligible compared to the cost of linear regression, thus making our methods particularly suitable for extremely large datasets. Although we focus on marginal screening to illustrate the applicability of the condition on selection framework, this framework is much more broadly applicable. We show how to apply the proposed framework to several other selection procedures including orthogonal matching pursuit, non-negative least squares, and marginal screening+Lasso. This paper proposes a unified framework to quantify local and global inferential uncertainty for high dimensional nonparanormal graphical models. In particular, we consider the problems of testing the presence of a single edge and constructing a uniform confidence subgraph. Due to the presence of unknown marginal transformations, we propose a pseudo likelihood based inferential approach. In sharp contrast to the existing high dimensional score test method, our method is free of tuning parameters given an initial estimator, and extends the scope of the existing likelihood based inferential framework. Furthermore, we propose a U-statistic multiplier bootstrap method to construct the confidence subgraph. We show that the constructed subgraph is contained in the true graph with probability greater than a given nominal level. Compared with existing methods for constructing confidence subgraphs, our method does not rely on Gaussian or sub-Gaussian assumptions. The theoretical properties of the proposed inferential methods are verified by thorough numerical experiments and real data analysis. We review the method of Parallel Factor Analysis, which simultaneously fits multiple two-way arrays or ‘slices’ of a three-way array in terms of a common set of factors with differing relative weights in each ‘slice’. Mathematically, it is a straightforward generalization of the bilinear model of factor (or component) analysis (xij = ΣRr = 1airbjr) to a trilinear model (xijk = ΣRr = 1airbjrckr). Despite this simplicity, it has an important property not possessed by the two-way model: if the latent factors show adequately distinct patterns of three-way variation, the model is fully identified; the orientation of factors is uniquely determined by minimizing residual error, eliminating the need for a separate ‘rotation’ phase of analysis. The model can be used several ways. It can be directly fit to a three-way array of observations with (possibly incomplete) factorial structure, or it can be indirectly fit to the original observations by fitting a set of covariance matrices computed from the observations, with each matrix corresponding to a two-way subset of the data. Even more generally, one can simultaneously analyze covariance matrices computed from different samples, perhaps corresponding to different treatment groups, different kinds of cases, data from different studies, etc. To demonstrate the method we analyze data from an experiment on right vs. left cerebral hemispheric control of the hands during various tasks. The factors found appear to correspond to the causal influences manipulated in the experiment, revealing their patterns of influence in all three ways of the data. Several generalizations of the parallel factor analysis model are currently under development, including ones that combine parallel factors with Tucker-like factor ‘interactions’. Of key importance is the need to increase the method's robustness against nonstationary factor structures and qualitative (nonproportional) factor change.