Time-efficient NAS: To reduce the time cost of training candidate networks, authors in [1]}, [2]} introduced the weight sharing mechanism (WS-NAS). Specifically, candidate networks are generated by randomly sampling part of a large network (supernet). Hence, candidate networks share the weights of the supernet and update these weights during training. By reusing these trained weights instead of training from scratch, WS-NAS significantly improves the time efficiency of NAS. However, the accuracy of these models obtained via WS-NAS is typically far below those obtained from training from scratch. Several optimization techniques have been proposed to fill the accuracy gap between the shared weights and stand-alone training [3]}, [4]}. For example, authors in [4]} propose a progressive shrinking algorithm to train the supernet. However, in many cases, the resulting networks still need some fine-tuning epochs to get the final architecture. To further accelerate the NAS process, some works propose the differentiable NAS to accelerate the NAS process [6]}, [7]}. The differentiable NAS approaches search for the optimal architecture by learning the optimal architecture parameters during the training process. Hence, differentiable NAS only needs to train the supernet once, thus reducing the training time significantly. Nevertheless, due to the significantly large number of parameters of the supernet, differentiable NAS requires a high volume of GPU memory. In order to further improve the time-efficiency of NAS, several approaches have been proposed to do training-free NAS [8]}, [9]}. These approaches leverage some training-free proxy that indicates the test performance of some given architectures; hence, the training time is eliminated from the entire NAS process. However, these methods usually use gradient-based information to build the proxy [8]}, [9]}. Therefore, in order to calculate the gradients, GPUs are still necessary for the backward propagation process. To totally decouple the NAS process from using GPU platforms, our work proposes a GPU-free proxy to do training-free NAS. We provide more details in Section REF .
<FIGURE>