A solution to this problem is offered by Neural Architecture Optimization (NAO), by [1]}. It maps a neural network architecture \(x\)  to an embedding space using an encoder-decoder model, as shown in Figure REF . The output of the encoder is a continuous feature vector \(e_x\)  (i.e., an embedding), that captures the semantics of the architecture that served as input for the encoder. A performance predictor (NAO uses an MLP) then uses the embedding \(e_x\)  to predict the architecture's accuracy. The output of the performance predictor is then maximized using gradient ascent to find \(e_x^{\prime }\) , the embedding of an improved architecture \(x^{\prime }\) . To get architecture \(x^{\prime }\) , the decoder maps \(e_x^{\prime }\)  back to the discrete space. This framework, using a performance predictor in a continuous search space, allows NAO to perform gradient-based search in the embedding space quickly. An important limitation of NAO is that the performance predictor (MLP) first needs to be trained, which is a practical bottleneck. This study, as described in detail in section , proposes a modification of NAO that uses a performance estimation strategy that makes the MLP redundant.
<FIGURE>