
We trained our model using the Adam optimizer, with momentum values set at \(\beta _1 \)  = 0.5 and \(\beta _2\)  = 0.99, we used a batch size of 8 for most experiments on CelebA [1]}, Celeba-HQ [2]} and FairFAce [3]}, respectively. A learning rate of 0.0001 for the first 10 epochs which linearly decayed to 0 over the corresponding epochs. We trained the entire model on three NVIDIA Titan X GPUs.
