The previous study in [1]} used a decaying learning rate \(\mathcal {O}(n^{-\tau })\) , where \(\tau \in (0,1]\) , for TTUR based on Adam to train GANs. This paper investigates the performance of TTUR based on adaptive methods using constant learning rates defined by (C1). Condition (C2) is used here to provide upper bounds on the performance measures defined by the expectations of the gradients of \(L_G\)  and \(L_D\)  (see Theorem REF  for details). (C2) was also used to analyze adaptive methods for training deep neural networks (see, e.g., [2]}, [3]}). Condition (C3) was used to provide upper bounds on the performance measures and for analyzing both convex and nonconvex optimization in deep neural networks (see, e.g., [4]}, [5]}, [3]})
