In the representation learning for clustering literature, e.g. [1]}, [2]}, [3]}, for evaluating performance of different algorithms the following methodology has been used: as set of models with some hyper-parameters are trained and these models are sorted by the observed clustering performance. Finally the best model's results are reported. This methodology will be named as max-performance in the remaining of the paper. We assess the quality of learned embeddings by using five challenging image data sets for clustering and report their performance with max-performance strategy. Although max-performance procedure gives some insights about the performance of the method under consideration, we give additional insights by significantly extending the evaluations. In practice, it is desirable that learned models can be utilized for different dataset other than the training dataset. To assess the performance of ConCURL, we have designed two additional experiments that focuses on performance of cross-model features and performance of ConCURL under distribution shift. Furthermore, we also assess the quality of the learned embeddings in image retrieval tasks. Finally, we present detailed ablation study to assess the impact of data augmentations, hyper-parameters and architecture choice in order to have a more complete picture.
