paragraph40.75ex plus1ex minus.2ex-1emForeground-focused clustering. To focus the clustering on the foreground tokens, we further leverage the ViT's CLS token attention maps \(A_{i} \in [0,1]^N\)  of each of its attention heads.
To create a foreground clustering mask that can be used during training, we first average the attention heads to one map and apply a Gaussian filter for smoothing.
We then obtain a binary mask \(A_b\)  by thresholding the map to keep 60% of the mass following [1]}.
We use \(\alpha _{B_{ji}}\)  to align the global crop's attention to the intersection with crop \(j\) .
The resulting mask is then applied as 0-1 weighting to the 2D cross entropy loss, \(l \odot A_b\) .
Note that we extract the attention maps and spatial tokens with the same forward pass, thus not impacting training speed.
