Transformers [1]} has achieved great success in natural language processing.
Transformer architectures are based on a self-attention mechanism that summarizes content from the source sequence and is capable of modeling complex and arbitrary dependencies within a limited number of layers.
Recently, many works [2]}, [3]}, [4]}, [5]}, [6]} have revealed the great potential of Transformers in the computer vision task.
Inspired by the successful application of Transformers in various fields, we intuitively take advantage of Transformers in modeling long-range contextual information. In this paper, we utilize the Transformer-alike structure to devise three attention units.
