Denote the output of layer \(l\)  by a multidimensional array (tensor) \(\mathbf {U}^l=g_{\theta }^{l}(x^{l-1})\in \mathbb {R}^{C_1^l \times \ldots \times C_{D^l}^l}\) , where \(D^l\)  denotes the number of the dimensions of \(\mathbf {U}^l\)  and \(C_d^l\)  denotes the number of elements along dimension \(d\in \lbrace 1,\ldots ,D^l\rbrace \) .
For efficiency, the output shape of \(h_{\varphi }^l\)  is not matched to the shape of \(\mathbf {U}^l\) . Instead, we make it smaller and broadcast the contextual dropout masks \(z^{l}\)  across the dimensions of \(\mathbf {U}^l\)  [1]}. Specifically, we parameterize dropout logits \(\alpha ^l\)  of the variational distribution to have \(C_{d}^l\)  elements, where \(d\in \lbrace 1, ...., D^l\rbrace \)  is a specified dimension of \(\mathbf {U}^l\) . We sample \(z^{l}\)  from the encoder and
broadcast them across all but dimension \(d\)  of \(\mathbf {U}^l\) . We sample \(z^{l}\sim \text{Ber} (\sigma (\alpha ^l))\)  under contextual Bernoulli dropout, and follow [2]} to use \(z^{l}\sim N(1, \sigma (\alpha ^{l})/(1-\sigma (\alpha ^{l})))\)  for contextual Gaussian dropout.
To obtain \(\alpha ^l\in \mathbb {R}^{C_{d}^l}\) , we first take the average pooling of \(\mathbf {U}^l\)  across all but dimension \(d\) , with the output denoted as \(F_{\text{avepool}, d}(\mathbf {U}^l)\) , and then apply two fully-connected layers \(\Phi _1^l\)  and \(\Phi _2^l\)  connected by \(F_\text{NL}\) , a
(Leaky) ReLU based nonlinear activation function, as
\(\alpha ^l =h_{\varphi }^l (\mathbf {U}^l)= \Phi _2^l( F_\text{NL}(\Phi _1^l(F_{\text{avepool}, d}(\mathbf {U}^l)))),\) 
