Figure REF  illustrates the block-level architecture of our search spaces and some parameters that can be varied within it. This search space is comprised of a stem, head, and \(N\)  variable blocks, each with a fixed stride. The choice of stem, head and the stride pattern depends on the choice of the reference model. The blocks used here are comprised of repeated layers, linked together by feedforward and residual connections.
The blocks in the search space are denoted \(B_{n,m}\) , where \(B_{n,m}\)  is the \(m^{th}\)  potential replacement out of \(M\)  choices for block \(B_n\)  in the reference model. These blocks can be of any style of neural architecture (See Appendix  for Vision Transformers [1]}), with very few structural limitations; only the spatial dimensions of the input and output tensors of \(B_{n,m}\)  need to match those of the reference model, which allows for diverse search. Throughout the text and in Appendix , other reference models based on MobileNetV3 [2]} and EfficientNet [3]} are discussed.
