Assessing whether the tested model performance in a benchmark translates to an intended production setting, including potential OOD data is a necessary step before deploying a machine learning model.
This is particularly important in safety critical applications, e.g. when classifying chest X-rays to assist radiologists in diagnosing patients.
Our results show that the so-called “radiologist-level” CheXnet model [1]} cannot handle OOD samples.
A model that cannot handle OOD images, making confident predictions based on wrong evidence, will lead to worse quality of care, eroding the trust of physicians into the model's predictions when facing ID images, and impede the potential benefits of computer assisted diagnosis.
