We study the problem of robust subspace recovery (RSR) in the presence of adversarial outliers. That is, we seek a subspace that contains a large portion of a dataset when some fraction of the data points are arbitrarily corrupted. We first examine a theoretical estimator that is intractable to calculate and use it to derive information-theoretic bounds of exact recovery. We then propose two tractable estimators: a variant of RANSAC and a simple relaxation of the theoretical estimator. The two estimators are fast to compute and achieve state-of-the-art theoretical performance in a noiseless RSR setting with adversarial outliers. The former estimator achieves better theoretical guarantees in the noiseless case, while the latter estimator is robust to small noise, and its guarantees significantly improve with non-adversarial models of outliers. We give a complete comparison of guarantees for the adversarial RSR problem, as well as a short discussion on the estimation of affine subspaces.
Other RSR methods compute the top eigenspaces of robust covariance estimators. Many examples of robust covariance estimators are built on Maronna's original M-estimator of covariance @cite_33 , which was extended by @cite_18 @cite_45 @cite_34 @cite_26 @cite_10 @cite_44 , among many others. While some robust covariance estimators are not affected by scale differences between inliers and outliers @cite_34 @cite_10 @cite_44 , we are not aware of any tractable RSR estimator based on a robust covariance that is guaranteed to be robust to adversarial outliers. In this work we use for initialization purposes the spherical PCA @cite_10 @cite_44 @cite_17 , which is the simplest subspace estimator that is based on a robust covariance. It computes the top eigenspace of the spherized sample covariance matrix @math , which is also known as the spatial sign matrix'' @cite_44 . Specifically, the spherized sample covariance is given by
@cite_18 @cite_26 @cite_33 @cite_44 @cite_45 @cite_34 @cite_10 @cite_17
2039892753 2019740336 1716364890 1981469185 20683584 2008418230 2063016434 149129625
Let y1, y2,..., yn∈ Rq be independent, identically distributed random vectors with nonsingular covariance matrix Σ, and let S = S(y1,..., yn) be an estimator for Σ. A quantity of particular interest is the condition number of Σ-1 S. If the yi are Gaussian and S is the sample covariance matrix, the condition number of Σ-1 S, i.e. the ratio of its extreme eigenvalues, equals 1 + Op((q n)1 2) as q →∞ and q n → 0. The present paper shows that the same result can be achieved with two estimators based on Tyler's (1987, Ann. Statist., 15, 234-251) M-functional of scatter, assuming only elliptical symmetry of ℒ(yi) or less. The main tool is a linear expansion for this M-functional which holds uniformly in the dimension q. As a by-product we obtain continuous Frechet-differentiability with respect to weak convergence. Classical methods in multivariate analysis require the estimation of means and covariance matrices. Although the sample mean and covariance matrix are optimal estimates of multivariate location and scatter when the data are multivariate normal, a small fraction of atypical points in the data (outliers) suffices to drastically alter them. This article describes different approaches to the development of substitutes to sample means and covariances that are resistant to outliers. Keywords: M-estimates; S-estimates; MM-estimates; τ-estimates; Stahel–Donoho estimates; P-estimates; regularization; missing data; independent contamination; equivariance The robust estimation of multivariate location and shape is one of the most challenging problems in statistics and crucial in many application areas. The objective is to find highly efficient, robust, computable and affine equivariant location and covariance matrix estimates. In this paper, three different concepts of multivariate sign and rank are considered and their ability to carry information about the geometry of the underlying distribution (or data cloud) are discussed. New techniques for robust covariance matrix estimation based on different sign and rank concepts are proposed and algorithms for computing them outlined. In addition, new tools for evaluating the qualitative and quantitative robustness of a covariance estimator are proposed. The use of these tools is demonstrated on two rank-based covariance matrix estimates. Finally, to illustrate the practical importance of the problem, a signal processing example where robust covariance matrix estimates are needed is given. On etablit l'existence et l'unicite d'une forme limite d'un estimateur M du type Huber d'un eparpillement multivariable A method for exploring the structure of populations of complex objects, such as images, is considered. The objects are summarized by feature vectors. The statistical backbone is Principal Component Analysis in the space of feature vectors. Visual insights come from representing the results in the original data space. In an ophthalmological example, endemic outliers motivate the development of a bounded influence approach to PCA. Classical statistical techniques fail to cope well with deviations from a standard distribution. Robust statistical methods take into account these deviations while estimating the parameters of parametric models, thus increasing the accuracy of the inference. Research into robust methods is flourishing, with new methods being developed and different applications considered. Robust Statistics sets out to explain the use of robust methods and their theoretical justification. It provides an up-to-date overview of the theory and practical application of the robust statistical methods in regression, multivariate analysis, generalized linear models and time series. This unique book: Enables the reader to select and use the most appropriate robust method for their particular statistical model. Features computational algorithms for the core methods. Covers regression methods for data mining applications. Includes examples with real data and applications using the S-Plus robust statistics library. Describes the theoretical and operational aspects of robust methods separately, so the reader can choose to focus on one or the other. Supported by a supplementary website featuring time-limited S-Plus download, along with datasets and S-Plus code to allow the reader to reproduce the examples given in the book. Robust Statistics aims to stimulate the use of robust methods as a powerful tool to increase the reliability and accuracy of statistical modelling and data analysis. It is ideal for researchers, practitioners and graduate students of statistics, electrical, chemical and biochemical engineering, and computer vision. There is also much to benefit researchers from other sciences, such as biotechnology, who need to use robust statistical methods in their work.