Contrastive learning: The task of learning unsupervised representations by contrasting the representations of an image with other images is called contrastive learning [1]}. Contrastive learning is essentially positive/negative classification where the positive and negative pairs of embeddings can be defined in various ways. In [2]}, [3]}, [4]}, the positive pairs are augmented views of the same image while the negative pairs are those of different images. In [5]}, the positive pairs are a patch and context embeddings from the same image while the negatives pairs are a patch and context embedding from different images. In [6]}, [7]}, the positives pairs are global and local features from the same image while the negative pairs are global and local features from different images. In [8]}, [9]}, [10]}, the positive pairs are members of the same cluster while the negative pairs are member of different clusters. We are different from these contrastive methods in that we do not consider all negatives equally: we calculate a soft labeling for the negatives using similarity of the data points. Also, we do not consider positive pairs directly: the comparison for the positive pairs is done through the similarity distillation.
