Vision Transformers (ViTs) [1]} are an adaptation of the popular Transformer model [2]}, conventionally applied to natural language processing (NLP), to computer vision domains. ViTs have been shown to achieve state-of-the-art performance across a wide range of tasks, such as image classification [1]}, [4]}, object detection [5]}, and text-to-image generation [6]}. However, while these models generally outperform other approaches, such as Convolutional Neural Networks (CNNs), they have significantly higher training and inference computational requirements due to quadratic processing time between all pairs of image patches (tokens).
