To validate our statement, we pre-train different ResNet50 models on the Cifar-100 dataset by using different levels of data augmentations and different levels of adversarial training strength. Once a model is pre-trained, we test its transfer accuracy on 4 downstream datasets: Cifar-10 [1]}, FGVS-Aircraft [2]}, DTD [3]} and Oxford-IIIT-Pet [4]} datasets. In fig:cediffdownstreamtransfer, we observe that the \(\mathcal {NC}_1\)  on Cifar-10 dataset has a negative (almost linear) correlation with the transfer accuracy on these downstream tasks. The smaller \(\mathcal {NC}\)\(_1\)  on the Cifar-10 dataset, the higher the transfer accuracy.When evaluating the relationship between \(\mathcal {NC}_1\)  and transfer accuracy measured on the same downstream dataset, the correlation is not as strong as we find on Cifar-10 dataset, we leave the results and discussion in Appendix . As such, the \(\mathcal {NC}_1\)  metric on Cifar-10 evaluated on pre-trained models can serve as a good performance indicator for the transfer accuracy on downstream tasks. To further strengthen our argument, we conduct experiments on the same set of downstream tasks using publicly available pre-trained models on ImageNet-1k [5]}, such as ResNet [6]}, DenseNet [7]} and MobileNetV2 [8]}. We observe the same negative correlation between \(\mathcal {NC}_1\)  and transfer accuracy in fig:archscediffdownstreamtransfer. This implies that such a relationship not only exists across different training scenarios of the same network but also exists more universally across different network architectures.
<FIGURE><FIGURE><FIGURE>