Subspace clustering in high-dimensions: Phase transitions \& Statistical-to-Computational gap
A simple model to study subspace clustering is the high-dimensional
$k$-Gaussian mixture model where the cluster means are sparse vectors. Here we
provide an exact asymptotic characterization of the statistically optimal
reconstruction error in this model in the high-dimensional regime with
extensive sparsity, i.e. when the fraction of non-zero components of the
cluster means $\rho$, as well as the ratio $\alpha$ between the number of
samples and the dimension are fixed, while the dimension diverges. We identify
the information-theoretic threshold below which obtaining a positive
correlation with the true cluster means is statistically impossible.
Additionally, we investigate the performance of the approximate message passing
(AMP) algorithm analyzed via its state evolution, which is conjectured to be
optimal among polynomial algorithm for this task. We identify in particular the
existence of a statistical-to-computational gap between the algorithm that
require a signal-to-noise ratio $\lambda_{\text{alg}} \ge k / \sqrt{\alpha} $
to perform better than random, and the information theoretic threshold at
$\lambda_{\text{it}} \approx \sqrt{-k \rho \log{\rho}} / \sqrt{\alpha}$.
Finally, we discuss the case of sub-extensive sparsity $\rho$ by comparing the
performance of the AMP with other sparsity-enhancing algorithms, such as
sparse-PCA and diagonal thresholding.