As Yann LeCun said, “if intelligence is a cake, the bulk of the cake is unsupervised learning”. This sentence reflects that Un-/Self-supervised Learning played a central role in the resurgence of deep learning. Common approaches focuses on designing different pretext tasks [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]} and aim to learn useful representations of the input data without relying on human annotations. It then uses those representations in downstream tasks, such as image classification, objection detection, and semantic segmentation.
