Self-supervised learning (SSL) methods [1]}, [2]}, [3]}, [4]} have been explored to learn feature representations from unlabeled images. As a representative SSL method, contrastive learning (CL) [5]}, [6]}, [7]} has been very successful in the natural image field, which defines a contrastive prediction task that trying maximize the similarity between features from differently augmented views of the same image and simultaneously maximize the distance between features from different images, via a contrastive loss. However, the application of CL in the fundus image field [8]}, [9]} is relatively limited, mainly due to fundus images' high resolution and low proportion of diagnostic features. First, fundus images are of high resolution so as to clearly reveal structural and pathological details. And it is challenging for CL to train high-resolution images with large batch sizes which are nevertheless generally required by CL so as to provide more negative samples. Secondly, data augmentation is typically applied in CL to generate different views of the same image. However, some strong data augmentation operations applied to fundus images may destroy important domain-specific information, such as cropping and Cutout [10]}. In a natural image, salient objects generally occupy a very large proportion and each part of them may contribute to the recognition of the object of interest, whereas the diagnostic features in a fundus image, such as lesions, may only occupy a small part of the whole image. This may result in that most cropped patches inputted to the CL framework have few or no diagnostic features. In this way, the network is prone to learn feature representations that are distinguishable for different views of the image but not discriminative for downstream tasks. CL in fundus image based DR grading is even rarer.
