
We apply a traditional convolution operation to transform the feature map form \((C, H, W)\)  into \((C \times r \times r, H, W)\)  where \(r\)  represents the size of the convolution kernel, \(H\)  and \(W\)  represents respectively the height and width of the image.
Hence, we increase the number of feature map channels. Each feature map represents a neighborhood map that will be used in the step. For example, in Figure REF , we have four feature maps. We denote this step the Projection convolution step. Let us denote \(f\)  the input image and \(\omega \)  the convolution kernel of spatial size \(1 \times 1\)  and \(b\)  the bias of the projection convolution. Let us consider that the convolutional layer has \(C_{\mbox{in}}\)  input channels and \(C_{\mbox{out}}\)  output channels. Hence, the resulting feature map at pixel \(n_1,n_2\)  is equal to:
\(g[n_1,n_2,c]=\sum _{k=0}^{C_{\mbox{in}} } f[n_1,n_2,k]\omega [0,0,k] + b[c] \mbox{ , with } k_2 \in [1,C_{\mbox{out}}]\) 

We then apply the Pixel Shuffle transformation to the feature maps representing coordinates as illustrated in Figure REF . Pixel Shuffle transformation, also known as sub-pixel convolution, was introduced [1]} for the super-resolution task. The Pixel Shuffle transformation reorganizes the low-resolution image channels to obtain a bigger image with fewer channels. Specifically, it increases the spatial size of the feature map by reducing the number of channels. Hence it rearranges the input tensor elements expressed as \((C \times r \times r, H, W)\)  to form a scaled \((C, H \times r , W \times r )\) . This operation is interesting since it is stable, compatible with deep learning back propagation, and does not add any artifacts. We denote this step the Pixel Shuffle step.
More formally, each channel \(c\)  of the previous step represents a neighborhood as illustrated in Figure REF . Let us decompose equation REF  in two terms such that for all \(c\)  we have \( g[n_1,n_2,c]=\tilde{f}[n_1,n_2,c] + b[c]\)  where \(\tilde{f}\)  is the results of the cross correlation of the convolution layer. Note that \(C_{\mbox{out}} = r^2 C_{\mbox{in}}\) . The output of pixel shuffle is :
\(h[n_1,n_2,c]= g[ \lfloor n_1/ r \rfloor ,\lfloor n_2/ r \rfloor ,c^{\prime }]\) 
where \(\lfloor n_1/ r \rfloor \)  is the floor fraction: it takes as input a real number \(n_1/ r\)  and outputs the greatest integer value. The channel \(c^{\prime }\)  is equal to \(c^{\prime }=c+n_1\mbox{.mod}(r)+n_2\mbox{.mod}(r)+r\mathbf {1}_{n_2\mbox{.mod}(r)>0}\) , where :\( n_1{\mbox{.mod}(r)}=n_1-(\lfloor n_1/r\rfloor \times r)\) .

On the output of the Pixel Shuffle step we apply a max-pooling of
stride \(r\) . We denote this step the Max-pooling step.
We apply the max pooling into \(h\)  with kernel of spatial size of \(r \times r \)  and a stride \(r\) , let us denote \(h^{\prime }\)  the result. Then \(h^{\prime }\)  is equal to :

\(h^{\prime }[ n_1 ,n_2 ,c]= \max _{(i,j) \in \mathcal {N}} \tilde{f}[\lfloor (n_1 +i)/ r\rfloor ,\lfloor (n_2 +j)/ r\rfloor ,c^{\prime }[i,j]] + b[c^{\prime }[i,j]]\) 
where \(c^{\prime }[i,j] = c+(n_1+i)\mbox{.mod}(r)+(n_2+j)\mbox{.mod}(r)+r\mathbf {1}_{(n_2+j)\mbox{.mod}(r)>0}\) . So we ended having a dilation on \(\tilde{f}\)  where the bias \(b\)  is the structuring element, with a connectivity depending on the channel.

