The earliest application of attention mechanisms is in the field of computer vision. In the literature [1]}, researchers adopt the attention mechanism on RNN models to implement image classification. Then, Bahdanau et al [2]}conducted the attention mechanism to machine translation tasks, which means that the attention mechanism has attracted a lot of attention in the natural language processing. In 2017, the Google machine translation team [3]} built the whole model framework with the Attention mechanism to replace the traditional RNN method. It contains a fully connected feed-forward network between each layer in the encoder and decoder structure.Feifan Fan [4]} proposed a fine-grained attention mechanism to capture word-level interactions between contexts and aspects in Twitter, and experiments demonstrate that the approach can effectively improve performance.
