Transformers [1]} have originally achieved a great success in natural language processing [2]}, [3]} and can be adopted in various applications, including sentiment classification, machine translation, word prediction, and summarization. The key feature of transformers is the self-attention mechanism, which helps a model learn the global contexts and enables the model to acquire the long-range dependencies.
Motivated by the great success in natural language processing, transformers have been adopted by computer vision tasks, leading to the development of vision transformers. Vision transformers have become prevalent in these years and have reached considerable success in many fields such as image classification [4]}, [5]}, video classification [6]}, object detection [7]}, [8]}, semantic segmentation [9]}, [10]}, and pose estimation [11]}.
