Not every OOD sample is equally likely in a real-world scenario.
The CheXnet model can encounter OOD X-ray images, as the distinction between ID and OOD X-ray images is based on manual, error-prone, tagging.
Photographs on the other hand are not part of the image processing pipeline in a radiology department and can thus be assumed not to be found in a real-world scenario.
For our experiments, we selected three publicly available radiographic data sets, IRMA [1]}, MURA [2]}, and BoneAge [3]}, containing images of various body parts as realistic OOD test data sets to test cross-data set generalization [4]}.
While the CheXnet model has been pre-trained on predicting the ImageNet classes, they are OOD regarding the target task of chest X-ray classification, as the data set does not include chest X-rays.
Therefore, we use it as additional non chest X-ray OOD data set, allowing us to investigate the performance of our propoosed method “In-Distribution Voting” (IDV) when trained with a large-scale unrealistic OOD data set.
This is relevant for use cases where no or only few realistic OOD images are available.
The OOD data sets are illustrated in Figure REF :
