
We propose a new topology-based analytical metric (NN-Degree) to quantify the topological characteristics of DNNs with skip connections. We demonstrate that the NN-Degree enables a training-free NAS within seconds. Moreover, we use the NN-Degree metric to build a new lightweight (three-parameter) accuracy predictor by training as few as 25 samples out of a vast search space with more than 63 billion configurations.
Without any significant loss in accuracy, our proposed accuracy predictor requires 6.88\(\times \)  fewer samples and provides a \(65.79\times \)  reduction of the fine-tuning time cost compared to existing GNN/GCN based approaches [1]}.

We construct analytical models to estimate the latency, area, and energy consumption of various DNN architectures. We show that our proposed analytical models are applicable to multiple hardware architectures and achieve a high accuracy with less than one second fine-tuning time cost.

We design a hierarchical simplicial homology global optimization (SHGO)-based algorithm, to search for the optimal architecture. Our proposed hierarchical SHGO-based algorithm enables 27729\(\times \)  faster (less than 0.1 seconds) NAS compared to RL-based baseline approach.

We demonstrate that our methodology enables NAS on a Raspberry Pi 3B with less than 3 seconds computational time. To our best knowledge, this is the first work showing NAS running directly on edge devices with such low computational requirements.

