Graves \(et\ al\) . [1]} first proposed an attention model in the context of handwriting synthesis. Another attention model was proposed by [2]} for machine translation. It also uses a weighted summation of local annotations as a context feature. But they computed the weights by softmax function. Following [2]}, [4]} proposed a soft attention model and a hard attention model for image captioning. The soft attention model uses a weighted summation of local region features as a context vector. The hard attention model uses a single local feature as a context feature. The context feature captures visual information relevant to words in the caption. Our attention model aims to find image regions relevant to the entire caption. Different from [5]}, our model does not directly use attention to generate images. We only apply attention in the discriminator to facilitate the comparison between images and captions.
