Attention mechanism is inspired by human's perception and visual cognition, and has been commonly applied in computer vision tasks [1]} [2]} [3]} [4]}. Attention mechanism in computer vision tasks is to generate the spatial or channel weight maps on feature representations extracted by the neural networks. For instance, Woo et al. [2]} developed a convolutional block attention module (CBAM) to introduce a fused attention which includes both channel attention and spatial attention. Such attention module can be inserted to commonly-used classification or segmentation networks. Oktay el al. [4]} proposed a novel attention gate in the Attention U-Net, which is utilized to refine the feature representations extracted by the network encoder to facilitate the network to focus on the ROIs. In summary, attention mechanism in neural networks is utilized to highlight the ROIs and suppress irrelevant information, which improves the network capacity and the robustness against noise.
<FIGURE>