Among the three ideas described above, learning a generalisable representation has benefited from the recent advance of self-supervised representation learning [1]}, [2]}.
In particular, contrastive learning, e.g. SimCLR [3]} and SimSiam [4]}, has shown its effectiveness in learning generalisable image representation.
For instance, SimCLR leverages the composition of multiple carefully chosen data augmentations, such as cropping and colour distortion, to generate random views, and the model learns to align these random views in the representation space with the ones of the same labelIn the unsupervised setting for the contrastive learning, the image index is commonly treated as the label..
