Self-supervised Learning:
Self-supervised learning is used to learn representations in an unsupervised way by defining some pretext tasks. There are many different flavors of self-supervised learning such as instance discrimination tasks [1]}, [2]} and contrastive techniques [3]}, [4]}. In instance discrimination tasks, each image is considered as its own category so that the learned embeddings are well separated [1]}. Building on the Instance Discrimination (ID) task, Zhuang et al., [2]} propose a Local Aggregation (LA) method based on a robust clustering objective (using multiple runs of k-means) to move statistically similar data points closer in the representation space and dissimilar data points further away. In contrastive techniques such as SimCLR [4]}, MoCo [8]}, representations are learned by maximizing agreement between different augmented views of the same data example (known as positive pairs) and minimizing agreement between augmented views of different examples (known as negative pairs). Recent works BYOL [9]}, SwAV [10]} achieve state-of-the-art results without requiring negative pairs. Although self-supervised learning methods show impressive performance on a variety of problems, it is not clear whether learned representations are good for clustering.
