\(\dagger \)  Leave-one-out contrastive learning (LooC) [1]} LooC is the main artifact in this section. proposed a model that learns to capture both varying and invariant factors for representations by constructing multiple separate projector spaces, each of which is invariant to all but one augmentation (as shown in Fig. REF ). By creating separate embedding spaces and concatenating the resulting representations, the final feature is aware of the augmentation-related information because each augmentation will be recognized by one feature space.
Denote \(E_i^{+}\)  as a positive logit and \(E_i^{-}\)  as a negative logit for the \(i\) -th embedding space \(\mathcal {Z}_i\) , where \(i=0\)  represents the original contrastive learning embedding space (all augmentation invariant) and \(i=1, \ldots , k\)  represents additional \(i\) -th augmentation covariant embedding space, where \(k\)  is the total amount of augmentations. The loss function of LooC is simply the summation of \(k+1\)  contrastive loss, as shown below:
\(\mathcal {L}_{\text{LooC}}=-\frac{1}{n+1}\left(\log \frac{E_{0}^{+}}{E_{0}^{+}+\sum E_{0}^{-}}+\sum _{i=1}^{k} \log \frac{E_{i}^{+}}{E_{i}^{+}+ \sum E_{i}^{-}}\right)\) 
