where \(\tau \)  is a temperature hyper-parameter and \(\mathcal {N}\)  is a set of negatives, corresponding to the representations of the other images in the batch.
Following previous work [1]}, we symmetrize the contrastive loss, and apply it similarly on the representation \(\mathbf {x}_b\)  from the subset \(\mathcal {B}\) .
The motivation for adding this contrastive loss is to encourage the model to produce globally coherent features that are consistent across different choices of observed subsets without relying on any hand-designed transformations. Using our design of SplitMask, we attain such signal with almost no overhead.
