Even with state-of-the-art NAS and our enhanced search space as described in Section , it is too costly to search an entire family of models. For example, directly searching for the entire EfficientNet family (B0\(\sim \) B7) is \(\sim \) 100X more expensive than searching for EfficientNet-B0, the base model in the EfficentNet family [1]}. Therefore, model scaling is commonly used together with NAS. Model scaling has changed from simple scaling [2]}, [3]}, [4]} to more sophisticated compound scaling [1]}. Compound scaling [1]} is essentially a search algorithm as it searches for the best scaling factors for depth, width, and resolution, under a given objective and constraint. However, although the SOTA compound scaling has demonstrated better results than simple scaling, by systematically scaling depth, width, and resolution of CNNs, there is still a major hurdle preventing it from harvesting the full potential of hardware and working optimally with NAS. Concretely, by using accuracy as the sole objectiveAlthough compound model scaling also uses FLOPs as the constraints of the scaling factors, the model accuracy is the only objective when searching for the compound scaling factors. during searching for scaling factors, the existing SOTA compound scaling method cannot consider the performance/speed impact for the resulted model families.
