Transformer. Transformer were first proposed in [1]} for the sequence-to-sequence machine translation task, which has revolutionized machine translation and natural language processing.
The Transformer models are then extended to some popular computer-vision tasks including image processing [2]}, object detection [3]}, semantic segmentation [4]}, object tracking [5]}, video instance segmentation [6]}, etc. DETR [3]} builds an object detection system based on Transformers, which largely simplifies the traditional detection pipeline, and achieves on par performances compared with highly-optimized CNN based detectors [8]}. ViT [9]} introduces the Transformer to image recognition and models an image as a sequence of patches, which attains excellent results compared to state-of-the-art convolutional networks. The above works show the effectiveness of Transformers in image understanding tasks. More detailed information of the application of the Transformer in the field of computer vision can be found in survey [10]}, [11]}.
