Based on the above accuracy predictor and analytical hardware performance models, we perform the second stage of our NAS methodology, i.e., searching for the optimal neural architecture by considering both test accuracy and hardware performance on the target hardware. To this end, we use a modified version of the Simplicial Homology Global Optimization (SHGO [1]}) algorithm to search for the optimum architecture. SHGO has mathematically rigorous convergence properties on non-linear objective functions and constraints and can solve derivative-free optimization problemsThe detailed discussion of SHGO is beyond the scope of this paper. More details are available in [1]}. Moreover, the convergence of SHGO requires much fewer samples and less time than reinforcement learning approaches [3]}.
Hence, we use SHGO for our new hierarchical searching algorithm.
