Q:

Eigenvalues and Eigenvectors of a singular Covariance matrix

I am working on a research in which my data matrix $\bf X$ has dimension of $N\times P$ where $P>>>>N$.ie. its a small sample size problem. I need to compute the covariance of $\bf X$, which is denoted by $\bf C$ with dimension $P\times P$. Is it mathematically possible to do the Eigen-value decomposition of $\bf C$ because I know that $\bf C$ is  not stable matrix as  its Eigen value have very high variance? Also, I need to reduce the dimension of $\bf X$ using the Eigen-vectors of $\bf C$, would it be correct if I did it using Eigen vectors of $\bf C$ without regularizing it?  To add to the point, based on some preliminary calculation the covariance matrix $\bf C$ has only one signifiant eigen values , rest are all very small (almost close to zero).

A:

The following paper should have answers to your question:
"Piecewise polynomial solutions to linear inverse problems"
 Authors: Per Christian Hansen, Klaus Mosegaard

The implementation of the algorithm is available at:
http://www.imm.dtu.dk/~pcha/Regutools/pptsvd.m

