The invalid reasoning in [1]} can also be shown with an example. Let us suppose that we replace the EEG encoder, in the regression schema in [2]}, with a ResNet. Then, we regress visual features extracted from GoogleNet to those learned by a ResNet.
Of course, within the limits of intra-class variability, such a regression succeedsWe implemented this experiment and it can be found here: https://colab.research.google.com/drive/1et3Pnlv9Iivtlcku8ck9-KEe2cKHijmv. To evaluate the regressor, we compared the classification accuracy on a random subset of the ImageNet validation set when using the original features and when using the regressed features. Those two values were respectively about 78% and 65%, showing a performance drop that can be expected, given the approximation introduced by regression and the original difference in classification performance between GoogleNet and ResNet (about 9 percent points)., since a class mapping exists between the two groups of features.
Then, if we replace ResNet with the codebook proposed in [1]}, the regression would still work, as implied by the fact that the authors can successfully carry out that regression from VGG-16, that has similar performance as GoogleNetSee https://pytorch.org/docs/stable/torchvision/models.html.. This, according to the reasoning in [1]}, should imply that ResNet does not derive any image-related features, which is of course absurd.
What the authors of [1]} overlook is the fact that, if ResNet had not learned image-related features in the first place, then the resulting representations would not be class-separable and the regression would fail. Similarly, if the EEG encoder in [2]} had not learned EEG-related features, the regression would also simply fail.
