Figures REF , REF , REF , REF  and REF  visualize some of the diverse network architectures found through DONNA in the DONNA search space. Results are shown for a simulator, the Nvidia V100 GPU, the number of operations, the number of parameters, and the Samsung S20 GPU. Note that all of these networks have different patterns of Squeeze-and-Excite (SE [1]}) and activation functions (whenever SE is used, Swish is also used), channel scaling, expansion rates, and kernel factors, as well as varying network depths. In Figure REF , grouped convolutions are also used as parts of optimal networks as a replacement of depthwise separable kernels.
