First, we evaluated TTUR based on Adam with different learning rates and a fixed batch size \(b = 2^6\) . Figure REF  plots the training losses of the generator and the discriminator for TTUR based on Adam with constant learning rates \(\alpha ^G = 5 \times 10^{-4}\)  and \(\alpha ^D = 10^{-5}\)  [1]}. The results show that there are subsequences of the whole sequences \((L_G(\mathbf {\theta }_n, \mathbf {w}_n))_{n\in \mathbb {N}}\)  and \((L_D(\mathbf {\theta }_n, \mathbf {w}_n))_{n\in \mathbb {N}}\)  which converge to some point. This result supports Theorem REF  indicating that using small constant learning rates \(\alpha ^G\)  and \(\alpha ^D\)  can approximate a stationary local Nash equilibrium.
