On the other hand, group-level objectives explicitly allow learning shared concepts between images by leveraging clustering losses [1]}, [2]}, [3]}, [4]}.
[1]} proposes k-means clustering in feature space to produce pseudo labels for training a neural network.
[2]} casts the problem of finding pseudo labels as an optimal transport problem unifying the clustering and representation learning objectives.
This formulation was adapted to an online setting in SwAV [4]} together with a new multi-crop augmentation strategy, a random cropping method that distinguishes between global and local crops of an image.
The IIC method [8]}, also conducts clustering, however using a mutual information objective.
While it can also be used densely, it has been found to focus on lower-level features specific to each dataset [9]}.
Another recent line of works completely refrains from group level clustering or instance-based discrimination by predicting targets from a slowly moving teacher network [10]}, [11]}.
