Clustering with Representation Learning:
DEC [1]} is one of the first algorithms to show deep learning can be used effectively to cluster images in an unsupervised way, where they use features learned from an autoencoder to finetune the cluster assignments. DeepCluster [2]} shows that it is possible to train deep convolutional neural networks end-to-end with pseudo labels that are generated by a clustering algorithm. Subsequently, there have been several works [3]}, [4]}, [5]}, [6]}, [7]} that introduce end-to-end clustering based objectives and achieve state-of-the-art clustering results.
For example, in Gaussian ATtention network for image clustering (GATCluster) [5]}, training is performed in two distinct steps, similar to [2]} where the first step is to compute pseudo targets for a large batch of data and the second step is to train the model in a supervised way using the pseudo targets. Both DeepCluster and GATCluster use k-means to generate pseudo labels which may not scale well. [6]} propose Deep Comprehensive Correlation Mining (DCCM), where discriminative features are learned by taking advantage of the correlations of the data using pseudo-label supervision and triplet mutual information among features. However, DCCM may be susceptible to trivial solutions [5]}. Invariant Information Clustering (IIC) [4]} maximizes mutual information between the class assignments of two different views of the same image (paired samples) in order to learn representations that preserve what is common between the views while discarding instance specific details. It is argued that the presence of an entropy term in mutual information plays an important role in avoiding the degenerate solutions. However a large batch size is needed for the computation of mutual information in IIC, which may not be scalable for larger image sizes which are common in popular datasets [4]}, [5]}. [7]} extend the celebrated maximal margin clustering idea to the deep learning paradigm, by learning the most semantically plausible clustering through minimizing a proposed partition uncertainty index. Their algorithm PICA uses a stochastic version of the index, thereby facilitating mini-batch training. PICA fails to assign a sample-correct cluster when that sample either has high foreground or background similarity to samples in other clusters.
In a more recent approach contrastive clustering [16]}, contrastive learning loss like in SimCLR [17]} was adopted along with entropy term to avoid the degenerate solution. Similarly, [18]} combines instance discrimination [19]} with a novel softmax-formulated decorrelation constraints for representation learning and clustering. Their approach outperforms state of the art methods and improves on instance discrimination method. Our method also improves on instance discrimination and outperforms [18]} on all datasets considered. There are other non end-to-end approaches such as SCAN [21]} which use the learned representations from a pretext task to find semantically closest images to the given image using nearest neighbors.
