Principal Component Analysis (PCA) is a well-known data dimensionality reduction technique [1]}. It works by projecting a dataset of \(m\)  vectors \(\mathbf {x}_j \in \mathbb {R}^n\) , \(j=1,\ldots ,m\)  (represented by a data matrix \(\mathbf {X} \in \mathbb {R}^{m \times n}\) , whose rows are such vectors) onto a reduced \(d\) -dimensional subspace of \(\mathbb {R}^n\) , which is generated by the first \(d<n\)  so-called principal directions. These are orthonormal eigenvectors of the symmetric matrix \(\mathbf {C} \doteq \frac{1}{m}\mathbf {X}^{\prime } \mathbf {X} \in \mathbb {R}^{n \times n}\) , associated with its \(d\)  largest positive eigenvalues. The latter are proportional (via the multiplicative factor \(\frac{1}{m}\) ) to the \(d\)  largest positive eigenvalues of the related Gram matrix \(\mathbf {G} \doteq \mathbf {X} \mathbf {X}^{\prime } \in \mathbb {R}^{m \times m}\) , whose element in position \((i,j)\)  is the inner product between the vectors \(\mathbf {x}_i\)  and \(\mathbf {x}_j\) . Focusing on such eigenvalues is important because the eigenvalues corresponding to discarded principal directions (the ones associated with the successive eigenvalues, not selected by PCA) provide information about the mean squared error of approximation of the dataset when only the first \(d\)  principal directions are kept to construct that approximation (as a consequence, knowing these eigenvalues is useful also to select a suitable value for \(d\) ). Moreover, when the dataset has zero mean, each such eigenvalue represents the empirical variance of the projection of the dataset onto the corresponding principal direction.
