The search strategy defines how the search space is explored to select candidate architectures, usually leveraging performance estimations of previous candidates to select more promising candidates. Common strategies include evolutionary algorithms [1]} and reinforcement learning methods [2]}. Gradient-based techniques can also be used: DARTS leverages weight sharing, optimizing both the meta-parameters and the model weights at the same time using bilevel optimization, thus evaluating multiple architectures simultaneously. While this is very efficient, the downside is that many performance estimation strategies are not applicable to DARTS.
