In terms of the loss function, optimization and training configuration, we adopt the settings and techniques used in StyleGAN2 and GANformer [1]}, [2]}, including in particular style mixing, Xavier Initialization, stochastic variation, exponential moving average for weights, and a non-saturating logistic loss with lazy R1 regularization. We use Adam optimizer with batch size of 32 (4 times 8 using gradient accumulation), equalized learning rate of \(0.001\) , \(\beta _1 = 0.9\)  and \(\beta _1 = 0.999\)  as well as leaky ReLU activations with \(\alpha = 0.2\) , bilinear filtering in all up/downsampling layers and minibatch standard deviation layer at the end of the discriminator. The mapping layer of the generator consists of 8 layers, and ResNet connections are used throughout the model, for the mapping network, synthesis network and discriminator. All models have been trained for the same number of training steps, roughly spanning 10 days on 1 NVIDIA V100 GPU per model.
