For two search stages, we use hinge loss [1]} and Adam optimizer [2]} with an initial learning rate of 0.0002. The number of searching rounds is 15, each containing 10 epochs. The noise data is sampled from the Gaussian distribution. During each round, a population of 32 individuals is trained and evolved. The batch sizes for generator and discriminator are 40 and 80, respectively. Besides, we adopt a low-fidelity evaluation strategy to improve search efficiency. Specifically, the number of images used to calculate FID and IS is reduced to 5,000, which greatly reduce the evaluation time and keep the performance of the searched architectures. Stage-1 and stage-2 take 0.8 and 0.4 GPU days, respectively.
