In addition to the mim loss, which is computed at the patch level, our approach also uses a contrastive loss at the image level.
To this end, we apply an average pooling operation over all the output representations of the decoder (including representations of the masked patches).
For each image, we obtain two representations \(\mathbf {x}_{a}\)  and \(\mathbf {x}_b\) , corresponding to the subsets \(\mathcal {A}\)  and \(\mathcal {B}\)  of observed patches.
We then apply the InfoNCE loss [1]} over these representations:
\(\ell (\mathbf {x}_a) = \frac{\exp (\mathbf {x}_a^\top \mathbf {x}_b / \tau )}{\sum _{\mathbf {y} \in \lbrace \mathbf {x}_b\rbrace  \cup \mathcal {N} } \exp (\mathbf {x}_a^\top \mathbf {y} / \tau )},\) 
