This prototyping capability is also showcased for the DONNA search space on a V100 GPU in Figure REF (right). Here we interpolate, using the original accuracy predictor for exploration. In doing this, Fig. REF  shows search-space diversity is crucial to achieve good performance. Especially the impact of optimally adding SE-attention [1]} is very large, predicting a 25\(\%\)  speedup at 76\(\%\)  accuracy (line C vs D), or a 1\(\%\)  accuracy boost at 26ms (line E vs D).
Every plotted line in Figure REF  (right) is a predicted Pareto-optimal. A baseline (A) considers SE/Swish in every block and k \(\in \)  {7}, expand \(\in \)  {3,4,6} and depth \(\in \)  {2,3,4}. Other lines show results for search spaces built starting from (A), e.g. (B) considers k \(\in \)  {5,7}, (C) k \(\in \)  {3,5,7}, (D) removes SE/Swish, (E) allows choosing optimal placement of SE/Swish, (F) adds a channel-width multiplier.
