with \(A\)  being the outer product of the normalized class-averaged encoding vectors and \(I\)  the identity matrix. Li et al. [1]} suggest that all methods for which encoding (pre-classification) features hold the property \(\text{OH} \ll 1\) , as the method in [2]}, are flawed. It is unclear why, according to [1]}, having features which can be linearly mapped to one-hot vectors at the model encoding layer should be problematic.
We argue that the one-hotness behavior described in [1]}, where the outer product of class-averaged features is shown to be close to the identity matrix, is just the expected behavior of any well-trained neural network.
To demonstrate this, we have prepared an examplePlease find it at https://colab.research.google.com/drive/1Y6HyToZv6HkRKK48D663Fthv8_-4n-nI, where a ResNet-18 model is trained on MNIST and the same analysis of pre-output features is carried out. Of course, using ResNet on MNIST is overkill: the choice of the model is meant to show that the phenomenon described by [1]} is not an anomaly and is not model-related or embedding-related, while the dataset was chosen to make the example quickly reproducible. Unsurprisingly, the behavior observed with ResNet-18 is the same of the [2]}'s EEG encoder as observed by [1]}. This is not an issue related to the embeddings, but, again, simply the normal behavior that any machine learning researcher would reasonably expect.
Beside ResNet-18, we tested several other state of the art models, including InceptionV3, DenseNet-121 and GoogleNet on the MNIST dataset. We measure the one-hotness behaviour claimed by [1]} in Table REF .
Results show that when a classifier works properly (i.e., it is able to discriminate the input data), \(|\text{det} (A-I)| \ll 1\) , i.e., features (at the encoding layer) show a “one-hot representation”. On the contrary when classifiers have not learned any linear separation among classes (“Untrained models” in Table REF ), the one-hotness measure is significantly higher than 0, as expected.
<TABLE>