The task of image captioning [1]} aims to generate a textual description for an input image and has received extensive research interests.
Recently, the attention-enhanced encoder-decoder framework [2]}, [3]}, [4]}, [5]}, [6]}, [7]} have achieved great success in advancing the state-of-the-arts.
Specifically, they use a Faster-RCNN [2]}, [9]} to acquire region-based visual representations and an RNN [10]}, [11]} to generate the coherent captions, where the attention model [12]}, [13]}, [14]}, [15]} guides the decoding process by attending the hidden state to the image regions at each time step.
Many sequence-to-sequence learning systems, including machine translation [12]}, [15]} and text summarization [18]}, have proven the importance of the attention mechanism in generating meaningful sentences. Especially for image captioning, the attention model can ground the salient image regions to generate the next word in the sentence [14]}, [2]}, [13]}, [22]}.
<FIGURE>