Image Captioning  
In recent years, a large number of neural systems have been proposed for image captioning [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]}, [12]}, [13]}, [14]}, [15]}. The state-of-the-art approaches [3]}, [7]}, [6]}, [8]}, [9]} depend on the encoder-decoder framework to translate the image to a descriptive sentence.
Specifically, the encoder network computes visual representations for the image and the decoder network generates a target sentence based on the visual representations.
To allow a more efficient use of the visual representations, a series of attention models have been proposed and achieve great success in multiple sequence-to-sequence learning tasks [21]}, [22]}, [2]}, [24]}, [25]}.
For image captioning, [2]} proposes the visual attention to help the model to focus on the most relevant image regions instead of the whole image; in [27]}, an adaptive attention model is designed to decide when to employ the visual attention.
There are other attention mechanisms such as: spatial and channel-wise attention [28]}, semantic attention [29]} and attention on attention [7]}.
However, these methods calculate attentional weights without explicitly knowing the word it should ground.
