A task usually refers to a joint distribution over input and label. Task transferability aims to predict how well a deep neural network pre-trained on a source task transfers to the target task. One may estimate the task transferability by data similarities regardless of models being used. Some work in this line includes conditional entropy [1]}, data set distance as optimal transport [2]}, \(F\) -relatedness [3]}, \(A\) -distance [4]}, and discrepancy distance [5]}. Besides, Poole et al. [6]} derived information theoretic bounds. These methods are generally hard to compute in practice and rely on the availability of the source data. Some recent task transferability estimators involve both data and the models. Taskonomy [7]} is a fully computation method, where task similarity scores are obtained by transfer learning experiments. Dwivedi et al. [8]} analyzed the representation similarities to construct a task taxonomy. More recently, Puigcerver et al. [9]} proposed a novel approach for transfer learning by firstly training experts on large-scale datasets and then selecting the relevant expert by simple heuristics for downstream tasks. Besides the models trained on source tasks, all these methods also require a fine-tuned or independently trained model from the target task. In contrast, our work aims to find checkpoint ranking measures that are lightweight in computing and requires no access to the source tasks.
