Transformer-based out-of-distribution detection for clinically safe segmentation
In a clinical setting it is essential that deployed image processing systems
are robust to the full range of inputs they might encounter and, in particular,
do not make confidently wrong predictions. The most popular approach to safe
processing is to train networks that can provide a measure of their
uncertainty, but these tend to fail for inputs that are far outside the
training data distribution. Recently, generative modelling approaches have been
proposed as an alternative; these can quantify the likelihood of a data sample
explicitly, filtering out any out-of-distribution (OOD) samples before further
processing is performed. In this work, we focus on image segmentation and
evaluate several approaches to network uncertainty in the far-OOD and near-OOD
cases for the task of segmenting haemorrhages in head CTs. We find all of these
approaches are unsuitable for safe segmentation as they provide confidently
wrong predictions when operating OOD. We propose performing full 3D OOD
detection using a VQ-GAN to provide a compressed latent representation of the
image and a transformer to estimate the data likelihood. Our approach
successfully identifies images in both the far- and near-OOD cases. We find a
strong relationship between image likelihood and the quality of a model's
segmentation, making this approach viable for filtering images unsuitable for
segmentation. To our knowledge, this is the first time transformers have been
applied to perform OOD detection on 3D image data.