On the Identifiability of Nonlinear ICA: Sparsity and Beyond
Nonlinear independent component analysis (ICA) aims to recover the underlying
independent latent sources from their observable nonlinear mixtures. How to
make the nonlinear ICA model identifiable up to certain trivial indeterminacies
is a long-standing problem in unsupervised learning. Recent breakthroughs
reformulate the standard independence assumption of sources as conditional
independence given some auxiliary variables (e.g., class labels and/or
domain/time indexes) as weak supervision or inductive bias. However, nonlinear
ICA with unconditional priors cannot benefit from such developments. We explore
an alternative path and consider only assumptions on the mixing process, such
as Structural Sparsity or Independent Influences. We show that under specific
instantiations of such constraints, the independent latent sources can be
identified from their nonlinear mixtures up to a permutation and a
component-wise transformation, thus achieving nontrivial identifiability of
nonlinear ICA without auxiliary variables. We provide estimation methods and
validate the theoretical results experimentally. The results on image data
suggest that our conditions may hold in a number of practical data generating
processes.