In this work, we investigated the effect of OOD images on a “radiologist-level” chest X-ray classifier.
We showed that the model, reportedly performing as good as radiologists [1]}, [2]}, was not able to filter OOD images, leading to obvious false positives to the human observer.
We assume its predictions are conditioned on chest X-rays, because the model was only trained on chest X-rays, leading to overconfident predictions given OOD images.
As hypothesises by Lee et al. [3]}, this leads to an ID-overfitted output space.
This interpretation explains why established output-based OOD detection methods failed in our experiments, when compared to detecting OOD samples in the feature space.
Our solution, ID voting and training with OOD images, regularizes the output space and expands the model's knowledge horizon, leading to a 100 % ID sensitivity and 98 % specificity.
