We highlight several contributions in our ZITS++:
(1) The TSR module is proposed to learn grayscale structural priors. The attention-based TSR can efficiently learn superior holistic structures with long-range dependency.
black(2) We study various priors including canny edges [1]}, wireframes [2]}, L-Edges [3]}, gradients, semantic segmentation, RTV [4]}, LR-RGB and Histograms of Oriented Gradients (HOG) [5]}.
Generally, L-Edges, wireframe lines and gradients work better than others, as empirically discussed and compared in Sec. REF  and Sec. REF  respectively. Thus they are introduced as structural priors to ZITS++.
As compared in Fig. REF , canny edges used in our conference version [6]} extract many trivial edges from textures, while edges from L-Edges (CATS) are much cleaner on the texture regions.
The L-Edges give meaningful holistic structures, and can be well recovered by TSR to improve the inpainting as in Fig. REF (a)-(e).
(3) We introduce the E-NMS technique and SSU module to flexibly upsample L-Edges and lines
to arbitrary resolutions. This provides critical structural clues for HR inpainting.
(4) We present a novel ZeroRA strategy to
incrementally incorporate the structural priors into a pre-trained FTR. This can largely save computation compared with training from the scratch. Critically, we also propose a novel masking positional encoding is proposed to improve the generalization of the HR inpainting.
black(5) Our FTR is further improved with the maxpool mask resizing strategy and LKA for a more stable training and better performance.
black(6) Additionally, we contribute to the community a high-quality HR image dataset –HR-Flickr, as the testset for image inpainting.
