Which layers to edit.   To prevent model overfitting and accelerate fine-tuning speed, we only modify the weights of the mapping network of StyleGAN2 [1]}, which essentially remaps \(z \sim \mathcal {N}(0, I)\)  to a different intermediate latent space (\(\mathcal {W}\)  space). We observe this to be effective, since modifying the mapping network is sufficient to obtain our target distribution, which is a subset of the original distribution. This choice has also been shown effective in previous few-shot GAN works [2]}. We have also experimented with optimizing the entire generator, and observe that generated output contains severe artifacts.
