Next, we evaluated the TTUR with different batch sizes and fixed constant learning rates \(\alpha ^G = 5 \times 10^{-4}\)  and \(\alpha ^D = 10^{-5}\)  [1]}. The performance measures were the number of steps \(N\)  and the SFO complexity \(Nb\)  satisfying that FID is less than or equal to 200. Figure REF  plots the number of steps \(N\)  versus the batch size \(b\) . The figure indicates that \(N\)  for TTUR based on Adam was monotone decreasing for \(b \ge 2^1\) , as in Theorem REF . TTUR based on Momentum also decreased \(N\)  except for \(b = 2^7\) . Meanwhile, \(N\)  for TTUR based on SGD was monotone decreasing for \(b \ge 2^4\) .
