Typical contrastive learning methods are instance-specific, in the sense that positive examples are constructed such that they are different augmented “views” of the same sample. Mathematically speaking, given an image \(\mathbf {x}\)  and a set of possible transformations \(\mathcal {T}\) , two transformations are sampled as \(t_1, t_2 \sim \mathcal {T}\)  and \(\mathbf {x}\)  is augmented as \(t_1(\mathbf {x}), t_2(\mathbf {x})\) . The resulting \(t_1(\mathbf {x})\)  and \(t_2(\mathbf {x})\)  are considered as positive examples to each other because they originated from the same image. All other images are considered as negative examples, regardless of their ground truth labels for downstream tasks (which we don't have access to during training).
We will later mention that this instance-specific method is proven to be effective by several works [1]}, [2]}, [3]}, [4]}, while is also criticized by several works [5]}.
