On Finite-Sample Identifiability of Contrastive Learning-Based Nonlinear Independent Component Analysis
Nonlinear independent component analysis (nICA) aims at recovering
statistically independent latent components that are mixed by unknown nonlinear
functions. Central to nICA is the identifiability of the latent components,
which had been elusive until very recently. Specifically, Hyv\"arinen et al.
have shown that the nonlinearly mixed latent components are identifiable (up to
often inconsequential ambiguities) under a generalized contrastive learning
(GCL) formulation, given that the latent components are independent conditioned
on a certain auxiliary variable. The GCL-based identifiability of nICA is
elegant, and establishes interesting connections between nICA and popular
unsupervised/self-supervised learning paradigms in representation learning,
causal learning, and factor disentanglement. However, existing identifiability
analyses of nICA all build upon an unlimited sample assumption and the use of
ideal universal function learners -- which creates a non-negligible gap between
theory and practice.
  Closing the gap is a nontrivial challenge, as there is a lack of established
``textbook'' routine for finite sample analysis of such unsupervised problems.
This work puts forth a finite-sample identifiability analysis of GCL-based
nICA. Our analytical framework judiciously combines the properties of the GCL
loss function, statistical generalization analysis, and numerical
differentiation. Our framework also takes the learning function's approximation
error into consideration, and reveals an intuitive trade-off between the
complexity and expressiveness of the employed function learner. Numerical
experiments are used to validate the theorems.