Based on the existing instance discrimination, some researchers propose self-supervised dense prediction methods.
Self-EMD [1]} adopts Earth Mover's Distance (EMD) to compute the similarity between two embedding. Insloc [2]} pastes image instances at various locations and scales onto background images. The pretext task is to predict the instance category given the composited images as well as the foreground bounding boxes. PixPro [3]} directly applies contrastive learning at the pixel level. DenseCL [4]} presents dense contrastive learning by optimizing a pairwise contrastive loss at the pixel level between two views of input images.
These methods also show the effectiveness in detection and segmentation tasks but get poor performance on image classification tasks. In a word, these methods overfit a single task and cannot train a general pre-training model.
