Contrastive Learning.
As the fundamental method in metric learning, contrastive learning aims to learn an
embedding space where similar instances can be distinguished from dissimilar data, and are also at the core of several recent works on self-supervised representation learning [1]}, [2]}, [3]}, [4]}, [5]}, [6]}.
Then according to computing similarity metric on real samples or virtual proxies, these methods could be divided into pair-based [7]}, [8]}, [9]} or proxy-based [10]}, [11]}, [12]}, [13]} branches, respectively.
In the contrastive learning setting, positive and negative pairs are generated in mini-batch of samples for each anchor image, often using hard mining. Different contrastive learning methods usually vary in their specific sampling and weighting strategy to design loss function, such as distance-weighted sampling in [14]}, sampling informative instances from memory bank in [12]}, uniformly weighting in vanilla contrastive loss [9]}, log-sum-exp weighting in N-pair loss [17]} and InfoNCE loss[18]}.
