As shown in Figure REF , all 5 blocks \(B_n\)  in the DONNA space can be replaced by a choice out of \(M=384\)  options: k \(\in \)  {3,5,7}; expand \(\in \)  {2,3,4,6}; depth \(\in \)  {1,2,3,4}; activation/attention \(\in \)  {ReLU/None, Swish[1]}/SE[2]}}; layer-type \(\in \)  {grouped, depthwise inverted residual bottleneck}; and channel-scaling \(\in \)  {\(0.5\times \) , \(1.0\times \) }. The search-space can be expanded or arbitrarily constrained to known efficient architectures for a device. Each of these \(5\times 384=1920\)  alternative blocks is trained using BKD to complete the Block Library. Once the Block Library is trained, we use the BKD-based ranking metric from DNA[3]} to sample a set of architectures uniformly spread over the ranking space. For the DONNA search space, we finally finetune the sampled networks for 50 epochs starting from the BKD initialization, building an Architecture Library with accuracy targets used to fit the linear accuracy predictor. Typically, 20-30 target networks need to be finetuned to yield good results, see Appendix REF .
