Contextual dropout module for convolutional layers:
Assume layer \(l\)  is a convolutional layer with \(C_3^l\)  as convolutional channels and \(\mathbf {U}^l \in \mathbb {R}^{C_1^l\times C_2^l\times C_3^l}\) .
Similar to Spatial Dropout [1]}, we set \(\alpha ^l\in \mathbb {R}^{C_3^l}\)  and broadcast its corresponding \(z^l\)  spatially as illustrated in Figure REF .
Such parameterization is similar to the squeeze-and-excitation unit for convolutional layers, which has been shown to be effective in image classification tasks [2]}. However, in squeeze-and-excitation, \(\sigma (\alpha ^l)\)  is used as channel-wise soft attention weights instead of dropout probabilities, therefore it serves as a deterministic mapping in the model instead of a stochastic unit used in the inference network.
