In computer vision, previous methods focus on designing different pretext tasks. One of the most promising directions among them is contrastive learning/instance discrimination [1]}, [2]}, which regards each instance in the training dataset as a single category. Based on instance discrimination [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, some methods show the effectiveness in the image classification task. They successfully bridge the performance gap between self-supervised and full-supervised methods. However, almost all of self-supervised learning methods, which formulate the learning as image-level prediction using global features, are suboptimal in the pixel-level predictions [3]}, [8]}, [7]}, such as object detection and semantic segmentation. Also, InfoMin [12]} finds that high-level features do not truly matter in transferring to dense prediction tasks. Here, current self-supervised learning may overfit to image classification while not being well tamed for downstream tasks requiring dense prediction.
