We derive the second ranking method also from the evaluation procedure \(\mathbf {G}\) , which replaces a checkpoint's output layer by a linear classifier tailored for the downstream task. We train the linear classifier while freezing the other layers. The ranking score equals the classifier's test accuracy. It is worth mentioning that self-supervised learning [1]}, [2]}, [3]} often adopts this practice as well to evaluate the learned feature representations. We shall see that the linear separability of the features extracted from a checkpoint is a strong indicator of the performance of fine-tuning the full checkpoint.
