A comparative study between vision transformers and CNNs in digital pathology
Recently, vision transformers were shown to be capable of outperforming
convolutional neural networks when pretrained on sufficient amounts of data. In
comparison to convolutional neural networks, vision transformers have a weaker
inductive bias and therefore allow a more flexible feature detection. Due to
their promising feature detection, this work explores vision transformers for
tumor detection in digital pathology whole slide images in four tissue types,
and for tissue type identification. We compared the patch-wise classification
performance of the vision transformer DeiT-Tiny to the state-of-the-art
convolutional neural network ResNet18. Due to the sparse availability of
annotated whole slide images, we further compared both models pretrained on
large amounts of unlabeled whole-slide images using state-of-the-art
self-supervised approaches. The results show that the vision transformer
performed slightly better than the ResNet18 for three of four tissue types for
tumor detection while the ResNet18 performed slightly better for the remaining
tasks. The aggregated predictions of both models on slide level were
correlated, indicating that the models captured similar imaging features. All
together, the vision transformer models performed on par with the ResNet18
while requiring more effort to train. In order to surpass the performance of
convolutional neural networks, vision transformers might require more
challenging tasks to benefit from their weak inductive bias.