From Table REF , we can see that many related convergence proofs rely on assuming the MVI condition of \(-V(z)\) , which means \(\langle V(z), z-z^*\rangle \leqslant 0~~\forall z\in \mathcal {Z}\) . Although MVI condition is theoretically known to be true in many standard supervised deep learning settings [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}. However, this is a rather unrealistic assumption for GANs: In some practical scenarios such as DCGAN [9]}, it is unclear whether the training objective can satisfy the MVI condition: While the generator might have a consistent gradient direction towards the optimal generator (which is the one that generates the target distribution), it is very unlikely that there is a “optimal discriminator” where the discriminator's gradient is pointing to through the course of the training. Indeed, different generator should in principle requires different discriminator to discriminate it from the target distribution, which precludes the MVI condition to hold on \(y\) .
