The Attention mechanism is inspired by the human vision which does not treat all data equally but enhances or weakens them. Recent work have been proposed to improve the localization performance of the model. For example, Hu et. al proposed a squeeze-and-excitation network (SENet)[1]} to model the inter-channel dependencies, which generates a weight of \(1 \times 1 \times C\)  via a global average pooling layer and two FC layers and multiply it into the input feature map to get an enhanced feature map. Based on SENet, SKNet[2]} built multiple branches of different receptive fields and used the information summarized by the multiple scale feature to channel-wise guide how to allocate the representation of which kernel to focus on. Wang et. al proposed Non-local Neural networks to fuse the global information and bring richer semantic information to the following layers. Convolutional Block Attention Module (CBAM)[3]} are also proposed to enhance features in channel and spatial dimensions in a cascading manner.
