Some pioneering works can partially solve these problems.
For the limited receptive fields, the attention mechanism is leveraged in [1]}, [2]}, [3]}. Fast Fourier Convolution (FFC) of global receptive fields is utilized in [4]} to
encode features in the frequency domain for resolution-robust inpainting, but the holistic structure is not explicitly modeled in [4]}.
Some transformer-based methods [6]}, [7]} with long-range dependency firstly fill Low-Resolution (LR) tokens, and then upsample them with CNNs to address the huge memory footprint of transformers on large images.
blackOn the other hand, various prior information of the images has been explored for inpainting, such as
canny edges [8]}, [9]}, segmentation [10]}, [11]}, Relative Total Variation (RTV) [12]}, [13]}, learning based edges [14]}, gradients [15]}, and even semantic prior features [16]}. Critically, our previous work [17]} proposes the concept of sketch tensor space, and exploits the structural priors of edges and wireframes [18]} in the space to facilitate holistic structure recovery for image inpainting.
To sum up, these methods typically have the multi-stage or multi-model designs, and are very expensive to be trained from the scratch.
Essentially, it is also non-trivial to directly enable amenable structural priors to enhance a well pre-trained inpainting model. Additionally, there is relative less study and discussion of using
position information in image inpainting, while such position information is quite critical to the inpainting model [19]}, [20]}.
