In hard attention mechanism, the model selects one important region instead of taking the expectation. Hence, it is a stochastic unit which can not be trained using back propagation. [1]} applied the REINFORCE to estimate the gradient of the stochastic neuron. Although the REINFORCE is an unbiased estimator, the variance of gradient is large and the algorithm is complex to implement. To solve these problems, we propose to apply Gumbel-softmax to estimate the gradient of the discrete units in our model. Gumbel-softmax is better in practice [2]} and much easier to implement.
