We study the pre-training and finetuning of computer vision models on a variety of datasets, see Table REF  for details.
For image classification, we consider the iNaturalist 2018 and 2019 [1]}, Stanford Cars [2]} and Food101 [3]} datasets, which all contain fine-grained categories.
We also consider three subsets from the DomainNet dataset [4]}, clipart, painting and sketch, which are not natural images and hence from different domains than ImageNet.
For object detection and instance segmentation, we use the COCO dataset [5]}.
Finally, we also use the ADE20k dataset [6]} for semantic segmentation.
The training set sizes of these different datasets vary from 8k to 437k images, thus all being significantly smaller than ImageNet, some more than two order of magnitude smaller.
This allows to investigate under different data regimes how feasible it is to pre-train directly on the target task data, alleviating the need for a large scale curated dataset as ImageNet.
