Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing
Existing GAN inversion and editing methods work well for aligned objects with
a clean background, such as portraits and animal faces, but often struggle for
more difficult categories with complex scene layouts and object occlusions,
such as cars, animals, and outdoor images. We propose a new method to invert
and edit such complex images in the latent space of GANs, such as StyleGAN2.
Our key idea is to explore inversion with a collection of layers, spatially
adapting the inversion process to the difficulty of the image. We learn to
predict the "invertibility" of different image segments and project each
segment into a latent layer. Easier regions can be inverted into an earlier
layer in the generator's latent space, while more challenging regions can be
inverted into a later feature space. Experiments show that our method obtains
better inversion results compared to the recent approaches on complex
categories, while maintaining downstream editability. Please refer to our
project page at https://www.cs.cmu.edu/~SAMInversion.