How do Variational Autoencoders Learn? Insights from Representational Similarity
The ability of Variational Autoencoders (VAEs) to learn disentangled
representations has made them popular for practical applications. However,
their behaviour is not yet fully understood. For example, the questions of when
they can provide disentangled representations, or suffer from posterior
collapse are still areas of active research. Despite this, there are no
layerwise comparisons of the representations learned by VAEs, which would
further our understanding of these models. In this paper, we thus look into the
internal behaviour of VAEs using representational similarity techniques.
Specifically, using the CKA and Procrustes similarities, we found that the
encoders' representations are learned long before the decoders', and this
behaviour is independent of hyperparameters, learning objectives, and datasets.
Moreover, the encoders' representations up to the mean and variance layers are
similar across hyperparameters and learning objectives.