Figure REF  plots the training losses of the generator and the discriminator for TTUR based on Adam with decaying learning rates \(\alpha ^G = 1/\sqrt{n}\)  and \(\alpha ^D = 1/\sqrt{n}\) . The results show that TTUR using decaying learning rates is not good from a practical viewpoint, while in theory it can be used to train GANs [1]}. Overall, the above results indicate that it is desirable to use small constant learning rates such as \(\alpha ^G = 5 \times 10^{-4}\)  and \(\alpha ^D = 10^{-5}\)  [1]} to train DCGANs.
