Clustering is a fundamental machine learning method. The quality of its results is dependent on the data distribution. For this reason, deep neural networks can be used for learning better representations of the data. In this paper, we propose a systematic taxonomy for clustering with deep learning, in addition to a review of methods from the field. Based on our taxonomy, creating new methods is more straightforward. We also propose a new approach which is built on the taxonomy and surpasses some of the limitations of some previous work. Our experimental evaluation on image datasets shows that the method approaches state-of-the-art clustering quality, and performs better in some cases. Clustering is one of the most fundamental unsupervised machine learning problems. Its main goal is to separate data into clusters of similar data points. Besides having its own applications, it is beneficial for multiple other fundamental tasks. For instance, it can serve for automatic data labeling for supervised learning and as a pre-processing step for data visualization and analysis.However, the performance of clustering algorithms is dependent on the type of the input data, such that different problems and datasets could require different similarity measures and different separation techniques. As a result, dimensionality reduction and representation learning have been extensively used alongside clustering, in order to map the input data into a feature space where separation is easier with respect to the problem's context. Using deep neural networks (DNNs), it is possible to learn non-linear mappings allowing to transform the data into more clustering-friendly representations.In the past, dimensionality reduction (or representation learning) and clustering have been treated separately, and sequentially applied on the data BID3 BID22 BID23 . However, recent research has shown that jointly optimizing for both problems can achieve decent results BID20 BID28 BID29 BID13 .One of our main contributions is the formulation of a taxonomy of methods that use deep learning for clustering. Our taxonomy facilitates the overview of existing methods and the creation of new ones by using the best properties of the existing ones in a modular manner.Based on the taxonomy, we propose a new method that combines advantageous properties of some existing methods. We use an autoencoder-based method for learning better representations of the data which are clustering-friendly, with a state-of-the-art training procedure. The training has two phases, the first one being standard autoencoder training with the mean squared error reconstruction loss, and the second one is based on a loss function combining the reconstruction loss and a clustering-specific loss. Moreover , in the second phase, we alternate between optimizing the network model, and updating the clustering assignments.The rest of the paper is organized as follows: the taxonomy of clustering with deep learning and the corresponding building blocks is described in Section 2. In Section 3, several related methods are briefly described and compared based on the taxonomy. Subsequently , in Section 4, a new method is proposed and discussed based on the building blocks of the taxonomy. Results of the proposed method are shown in Section 5, followed by conclusions in Section 6. In this work, we present a taxonomy for clustering with deep learning, identifying the general framework, and discussing different building blocks and possible options. In addition, a summary of methods in the field and their specific use of the taxonomy is presented alongside a general comparison of many of these methods. Using this taxonomy and the summary of previous methods, generating new methods is clearer and easier and can be done by creating new combinations of the taxonomy's building blocks. Moreover, we present a new method to the field, which is based on such a new combination. Our method overcomes the limitations of several previous ones, approaches state-ofthe-art performance and performs better in some cases.
Unifying framework to perform clustering using deep neural networks
{"src": "scitldr-AIC_train", "paper_id": "B1eT9VMgOX"}