Reconstruction has the potential downside that the model has no prior knowledge of what parts of the representation are more important than others, so all errors are weighted equally.
When dealing with high-dimensional signals (e.g. speech or images), modeling the underlying relationships between the dimensions in the original feature space is difficult and costly.
Reconstructing every detail is likely unnecessary as there are many nuisance dimensions that are irrelevant for the eventual task.
We know the number of relevant classes/categories are orders of magnitude smaller than the number of unique signals.
We also know the objective function cannot distinguish between irrelevant differences in the reconstructed signal.
Approaches like CPC [1]}, wav2vec 2.0 [2]}, DeCoAR 2.0 [3]} use quantized or latent representations representations to allow the model to focus on the more relevant dimensions during reconstruction.
