where the \(\circledast _s\)  indicates fractionally strided convolution, which can be viewed as the reverse convolution operation via adding stride \(s\)  into the input. The stride \(s\)  is a upsampling factor. Activation functions can also be attached for learning a nonlinear upsampling [1]}.
For restoring downsampled feature map \(\textbf {X}^{l}\)  of \(l\) -th layer to a fine feature map with the same size as input, a decoder with multiple deconvolution layers could be added on the top of \(\textbf {X}^{l}\) :
\(\begin{aligned}&\textbf {Y}^{l} = D(\textbf {X}^{l};\textbf {W}^{l}_{decon}),\end{aligned}\) 
