[1]} includes two pretext tasks with the contrastive loss after passing input data through a vision transformer. These tasks include predicting the rotation degree of \((0°, 90°, 180°, 270°)\)  and the image reconstruction. The image is first augmented and partitioned into different patches. The rotation task is to match the predicting degree and the actual degree of the input image. The reconstruction task is to reconstruct the augmented image back to the original image. Finally, the contrastive loss maximizes the similarity between the same inputs.
