NAS is an optimization problem with specific targets (e.g., high classification accuracy) over a set of possible candidate architectures. The set of candidate architectures defines the (typically vast) search space, while the optimizer defines the search algorithm. Recent breakthroughs in NAS can simplify the tricky (and error-prone) ad-hoc architecture design process [1]}, [2]}. Moreover, the networks obtained via NAS have higher test accuracy and significantly fewer parameters than the hand-designed networks [3]}, [4]}. These advantages of NAS have attracted significant attention from researchers and engineers alike [5]}. However, most of the existing NAS approaches do not explicitly consider the hardware constraints (e.g., latency and energy consumption). Consequently, the resulting neural networks still cannot be deployed on real devices.
