Second, given the pre-trained models, on downstream data we experimentally investigate the relationship between \(\mathcal {NC}\)\(_1\)  metric of their representations and transfer accuracy. Transferring pre-trained large models to smaller downstream tasks has become the dominant approach in both vision [1]} and language [2]} domains. Moreover, because of the huge size and the limited access nature of the source datasets,e.g., JFT dataset [3]}, which is used in the pretraining of Vision Transformer, is not publicly available. it is often intractable to get the statistics of the \(\mathcal {NC}\)\(_1\)  metric on the source datasets. They both motivate us to study the \(\mathcal {NC}\)\(_1\)  metric on the downstream data.
