Works related to ours also include the attention model proposed by Xu et al. [1]} and [2]}. [1]} first applied both stochastic hard attention and deterministic soft attention mechanisms for spatial locations of images for image captioning. [2]} instead used weighting on image patches to implement a region-level attention. In this paper, similar to [1]}, both of stochastic hard attention and deterministic soft attention are studied. However, when implementing hard attention, [1]} borrowed the idea of REINFORCE whilst we also propose to apply a more recent Gumbel-softmax to estimate discrete neurons in the attention mechanism.
