As an ablation, we also studied the effect of changing the backbone architecture on the results of this experiment. In particular, as a backbone, we also tried using a ViT-base model [1]} with \(16 \times 16\)  patches pre-trained on ImageNet-1k images of size \(224 \times 224\)  (\(s=196\) , \(d=768\) ). Then, we followed the same average pooling approach as for the gMLP backbone experiments, and kept all other network parameters the same. The results of this experiment are summarized in Table REF .
