Multi objective Neural Architecture Search via Predictive Network Performance Optimization
Neural Architecture Search (NAS) has shown great potentials in finding a better neural network design than human design. Sample based NAS is the most fundamental method aiming at exploring the search space and evaluating the most promising architecture. However, few works have focused on improving the sampling efficiency for a multi objective NAS. Inspired by the nature of the graph structure of a neural network, we propose BOGCN NAS, a NAS algorithm using Bayesian Optimization with Graph Convolutional Network (GCN) predictor. Specifically, we apply GCN as a surrogate model to adaptively discover and incorporate nodes structure to approximate the performance of the architecture. For NAS oriented tasks, we also design a weighted loss focusing on architectures with high performance. Our method further considers an efficient multi objective search which can be flexibly injected into any sample based NAS pipelines to efficiently find the best speed/accuracy trade off. Extensive experiments are conducted to verify the effectiveness of our method over many competing methods, e.g. 128.4xmore efficient than Random Search and 7.8xmore efficient than previous SOTA LaNAS for finding the best architecture on the largest NAS dataset NASBench 101.
ArXiv