Figure REF  plots the training losses of the generator and the discriminator for TTUR based on Adam with constant learning rates \(\alpha ^G = 1\)  and \(\alpha ^D = 1\) . The results show that the convergent point of TTUR with \(\alpha ^G = 1\)  and \(\alpha ^D = 1\)  is different from the one of TTUR with \(\alpha ^G = 5 \times 10^{-4}\)  and \(\alpha ^D = 10^{-5}\)  (Figure REF ). To verify whether TTUR with \(\alpha ^G = 1\)  and \(\alpha ^D = 1\)  converged to an appropriate point, we checked the Fréchet inception distance (FID) [1]} of which a small value is desirable for training GANs. FID for TTUR with \(\alpha ^G = 1\)  and \(\alpha ^D = 1\)  was between 300 and 330, while FID for TTUR with \(\alpha ^G = 5 \times 10^{-4}\)  and \(\alpha ^D = 10^{-5}\)  was 120. Hence, we can see that TTUR with \(\alpha ^G = 1\)  and \(\alpha ^D = 1\)  is not good for training the DCGAN.
