In this paper, we propose an incrementally structure enhanced inpainting model called ZITS. We use a transformer-based structure restorer to get much better holistic structures compared with previous methods. Then, a novel ZeroRA strategy is leveraged to incorporate auxiliary structures into a pre-trained inpainting model with a few finetuning steps. The proposed masking positional encoding can further improve the inpainting performance.
blackMoreover, we further study different image priors for inpainting, and select to leverage CATS edges [1]} instead of canny [2]} used in the conference version [3]}, which provides more informative structures to the inpainting model.
Besides, we comprehensively upgraded LaMa [4]} that used as the pre-trained FTR of ZITS with LKA and several useful techniques.
The newly proposed ZITS++ can achieve significant improvements based on the state-of-the-art model in experiments of various resolutions, even outperform our previous ZITS.
