Different from original SGDA, we have to make two queries to SFO in each iteration. One for \(z_t=(x_t,y_t)\) , and the other for the intermediate step \((x_{t+1},y_t)\) . Since original SGDA is not going to work even in the convex-concave setting (such as \(\min _x\max _y f(x,y)=xy\) ), so researchers propose the following “theoretical correct modification”. Prior theoretical works [1]} that shows the global convergence of GANs training over real-world distributions also focus on SGDA due to its simplicity.
