In order to achieve a 2D representation $X\in\mathbb{R}^{n\times 2}$ of some high-dimensional data residing in $Y\in\mathbb{R}^{n\times k}$, I use PCA:$$X=Y\cdot U,$$where $U\in\mathbb{R}^{k\times 2}$ contains eigenvectors of $Y^TY$ corresponding to its dominant eigenvalues.However, in case there are multiple occurrences of, e.g., first dominant eigenvalue, my PCA solution, as defined above, will be inconsistent: it would depend on the actual eigenvalue that is declared as 'the first dominant' by themethod that I use for the eigendecomposition. What is the recipe to allow for a consistent solution?Perhaps more important is the following. Namely, PCA provides guarantees on maximal variance along axes; what impact does the above problem have on the solution with maximal variance along axes? Will maximal variance be retained with each solution, regardless on which eigenvector corresponding to dominant first eigenvalue is used?
Handling inconsistent solutions obtained by PCA
linear algebra;matrices;high dimensional;machine learning
PCA is called this way since it picks the principal components. If you happen to have several components with the same or almost the same eigenvalue and you pick one but not the other, then you can't claim that you picked the principal components. You picked a subset of the principal components. In other words, if your second eigenvalue is doubled, then it's not a useful strategy to choose only two principal components, but you need to consider three.