For the given restored blackpriors \(\mathbf {\tilde{P}}\)  in arbitrary scales, a full convolutional network is employed to process them into a feature space. Our SFE is also an autoencoder model with 3 layers downsampling convolutions (encoder), 3 layers residual blocks with dilated convolutions [1]} (middle), and 3 layers upsampling convolutions (decoder). For the encoder and the decoder in SFE, we use Gated Convolutions (GCs) [2]} to transfer useful features selectively. GC learns another sigmoid activation with the same channels. Then the sigmoid features are multiplied to the convoluted ones as outputs. Although GCs are widely used in image inpainting for the better generalization to irregular masks, we use GCs to filter useful features to FTR. Because the grayscale sketch space is sparse, and not all features are necessary for the inpainting. Then, 4 coarse-to-fine feature maps \(\lbrace \mathbf {S}_k\rbrace _{k=0}^3\)  from the last middle layer and 3 decoder layers are selected to transfer structural features to FTR as
black
\(\mathbf {S}_0,\mathbf {S}_1,\mathbf {S}_2,\mathbf {S}_3=\mathrm {SFE}(\mathbf {\tilde{P}},\mathbf {M}),\) 
