Many applications, such as photon-limited imaging and genomics, involve large datasets with noisy entries from exponential family distributions. It is of interest to estimate the covariance structure and principal components of the noiseless distribution. Principal Component Analysis (PCA), the standard method for this setting, can be inefficient when the noise is non-Gaussian. We develop @math PCA (exponential family PCA), a new methodology for PCA on exponential family distributions. @math PCA can be used for dimensionality reduction and denoising of large data matrices. @math PCA involves the eigendecomposition of a new covariance matrix estimator, constructed in a simple and deterministic way using moment calculations, shrinkage, and random matrix theory. We provide several theoretical justifications for our estimator, including the finite-sample convergence rate, and the Marchenko-Pastur law in high dimensions. @math PCA compares favorably to PCA and various PCA alternatives for exponential families, in simulations as well as in XFEL and SNP data analysis. An open-source implementation is available.
There have been several approaches suggested for extending PCA to non-Gaussian distributions, see. e.g., @cite_5 , Sec. 14.4. One possibility is to use robust estimates of the covariance matrix [see][Sec. 14.4, for references] jolliffe2002principal . Another approach assumes that the natural parameter lies in a low dimensional space , and then attempts to maximize the log-likelihood. This leads to a non-convex optimization problem for which an alternating maximization method is proposed, without global convergence guarantees. More recently, described a similar generalization of PCA, while proposed another likelihood-based method, both without global convergence guarantees. Scalable methods include @cite_10 , albeit without precise performance guarantees in high dimensions.
@cite_5 @cite_10
2148694408 2963481531
Introduction * Properties of Population Principal Components * Properties of Sample Principal Components * Interpreting Principal Components: Examples * Graphical Representation of Data Using Principal Components * Choosing a Subset of Principal Components or Variables * Principal Component Analysis and Factor Analysis * Principal Components in Regression Analysis * Principal Components Used with Other Multivariate Techniques * Outlier Detection, Influential Observations and Robust Estimation * Rotation and Interpretation of Principal Components * Principal Component Analysis for Time Series and Other Non-Independent Data * Principal Component Analysis for Special Types of Data * Generalizations and Adaptations of Principal Component Analysis We develop a flexible framework for low-rank matrix estimation that allows us to transform noise models into regularization schemes via a simple bootstrap algorithm. Effectively, our procedure seeks an autoencoding basis for the observed matrix that is stable with respect to the specified noise model; we call the resulting procedure a stable autoencoder. In the simplest case, with an isotropic noise model, our method is equivalent to a classical singular value shrinkage estimator. For non-isotropic noise models--e.g., Poisson noise-- the method does not reduce to singular value shrinkage, and instead yields new estimators that perform well in experiments. Moreover, by iterating our stable autoencoding scheme, we can automatically generate low-rank estimates without specifying the target rank as a tuning parameter.