
For Bernoulli contextual dropout, backpropagating the gradient efficiently is not straightforward, as the Bernoulli distribution is not reparameterizable, restricting the use of the reparameterization trick. In this case, a commonly used gradient estimator is the REINFORCE estimator [1]} (see details in Appendix ). This estimator, however, is known to have high Monte Carlo estimation variance.
To this end, we estimate \(\nabla _{\varphi } \mathcal {L}\)  with the augment-REINFORCE-merge (ARM) estimator
[2]}, which provides unbiased and low-variance gradients for the parameters of Bernoulli distributions. We defer the details of this estimator to Appendix . We note there exists an improved ARM estimator [3]}, [4]}, applying which could further improve the performance.
