Now, we consider the gradient of \(\mathcal {L}\)  with respect to \(\varphi \) , the components of \(=\lbrace \theta , \varphi \rbrace \)  not copied from the decoder. For Gaussian contextual dropout, we estimate the gradients via the reparameterization trick [1]}. For \(z^l\sim N(\mathbf {1},\sigma (\alpha ^{l})/(1-\sigma (\alpha ^{l})))\) , we rewrite it as \(z^l = 1+\sqrt{\sigma (\alpha ^{l})/(1-\sigma (\alpha ^{l}))}\epsilon ^l\) , where \(\epsilon ^l\sim \mathcal {N}(\mathbf {0},\mathbf {I})\) . Similarly, sampling a sequence of \(z=\lbrace z^l\rbrace _{l=1}^L\)  from \(q_{}(z\,|\,x)\)  can be rewritten as \(f_{}(\epsilon ,x)\) , where \(f_{}\)  is a deterministic differentiable mapping and \(\epsilon \)  are \(iid\)  standard Gaussian. The gradient \(\nabla _{\varphi }\mathcal {L}(x,y)\)  can now be expressed as (see pseudo code of Algorithm  in Appendix)
\(\textstyle \nabla _{\varphi }\mathcal {L}(x,y)=\mathbb {E}_{\epsilon \sim \mathcal {N}(\mathbf {0},\mathbf {1})}[\nabla _{\varphi } (\log p _{\theta }(y\,|\,x,f_{}(\epsilon ,x))-\frac{\log q_{}(f_{}(\epsilon ,x)\,|\,x)}{\log p_{\eta }(f_{}(\epsilon ,x))})]. \) 
