Another line of research concerns learning good representations through self-supervised learning,
with an unsupervised auxiliary loss to encourage the discovery of better representations.
These better representations could lead to improved performance on supervised tasks,
especially when labeled data is scarce.
In [1]}, for instance, an auxiliary infill loss allows the BERT model to benefit from pretraining on
a large corpus of unlabeled data.
Our approach to object-centric self-supervised learning is heavily inspired by the BERT infilling loss.
Other studies have shown similar benefits to auxiliary learning in vision as well [2]}, [3]}, [4]}. These works apply various forms of contrastive losses to predict scene dynamics,
and
the better representations that result carry
downstream benefits to supervised and reinforcement learning tasks.
