We used the same architecture as [1]} for the SFN but used a new padding approach in every convolutional layer. We note that padding is necessary to keep the size of the stylized image the same as that of the original image due to the convolution operations in the model that shrink the size of the input image. This network does not have pooling layers but has convolutional layers with stride larger than 1. Even though upsampling layers are present, without appropriate padding, the network still outputs smaller images than the input images. In experiments, we found that the ordinary padding approaches were prone to produce bounding box artifacts (horizontal or vertical bars appearing around the boundary of the stylized video), even if no temporal loss was applied. We found that this artifact could be largely suppressed by adopting an interpolation padding approach. First, the feature maps with spatial size \(H\times W\)  are bilinearly upsampled to the desired size \(H^{\prime }\times W^{\prime }\) . Then, the interior parts with the spatial sizes of \(H\times W\)  are substituted by original feature maps. In other words, the padded area is obtained by bilinear interpolation. This approach was adopted in both the SFN and the RNN in all experiments. Its effect is demonstrated in Section  REF .
