For most “off-the-shelf” models in computer vision, it is hard to anticipate the exact application of models and impossible to train a model on precisely the data distribution that the model will be applied to. Inevitably, the model will encounter out-of-domain data on which the model performance can vary widely. For instance, even though deep learning models have surpassed human performance on ImageNet dataset [1]}, recent works [2]}, [3]} have demonstrated that these models still make simple mistakes and have lower accuracy (than ImageNet and human) on new benchmarks [4]}, [5]}. Therefore, understanding the out-of-domain generalization of models is important. Motivated by this, we probe the performance of our models on out-of-domain datasets. To measure the generalization capabilities of our model, we report the performance of the finetuned model on several alternative test sets.
