For the clustering part, the MLP projection head \(g\)  consists of a hidden layer of size 2048, followed by batch normalization and ReLU layers, and an output layer of size 256. The prototypes are thus chosen to be of dimension 256. Note that we fixed the number of prototypes to be equal to the number of ground truth classes of the dataset. It was shown however that over-clustering leads to better representations [1]}, [2]}, [3]} and we can extend our model to include an over-clustering block with a larger set of prototypes [2]} and alternate the training procedure between the blocks.
