Neural architecture search (NAS) that aims at automatic architecture design has achieved remarkable results in various fields [1]}. There are mainly four approaches in NAS: reinforcement learning (RL) [2]}, [3]}, [4]}, gradient descent (GD) [5]}, [6]}, [7]}, surrogate model-based optimization (SMBO) [8]}, and evolutionary algorithm (EA) based methods [9]}, [10]}, [11]}, [12]}. Many works have applied NAS to search GANs, as summarized in Tab. REF . AGAN [13]} and AutoGAN  [14]} are among the first RL-based NAS methods to search GANs, but they only use IS as the reward to guide the search. E2GAN [15]} is rewarded by a linear combination of IS and FID. However, these RL-based NAS-GAN studies only search generators. The second group is gradient descent (GD) based methods that apply softmax function to relax the discrete search space to allow differential optimization of architectures. E.g., DEGAS [16]} uses the generative latent optimization (GLO) [17]} framework to search generators. Although AdversarialNAS [18]} proposes a reconstruction adversarial loss function to search the architectures of generator and discriminator simultaneously, the architectures of the generators and discriminators are deeply coupled, which further aggravates the instability of GAN training. AlphaGAN [19]} targets at finding pure Nash equilibrium of generator and discriminator and exploits a fully differentiable search framework by formalizing a bi-level min-max optimization problem. The third category is the evolutionary algorithm (EA) based methods, which are the most relevant to our proposed method. EGAN  [20]} only searches generator and doesn't perform well on the CIFAR-10 dataset. COEGAN [21]} searches both G and D, but its experiment lacks generality as it only searches on a very small search space (up to 4 layers of convolution or linear layers) using the MNIST dataset [22]}.
