To reduce the reliance on output class labels during training, and also to learn better representations, self-supervised learning [1]}, [2]}, [3]} has emerged as an effective method to pre-train deep neural networks. Different approaches for self-supervised learning such as contrastive learning techniques like SimCLR [4]} make use of the input data and create pseudo-labels using augmentations to train the network. Experiments have shown that such approaches can achieve competitive results even without the full datasets.
<FIGURE>