Although many algorithms successfully shortened the search time of NAS from months to hours [1]}, [2]}, some research has shown their reliance on specific search spaces and datasets [3]}. There are also questions on these algorithms' efficiency when dealing with a large number of tasks [4]}. A rising direction of NAS research is thus looking for solutions that acquire transferable knowledge and generalize well across multiple tasks and search spaces [5]}, [6]}, [7]}, [8]}, [9]}. [4]} explores meta-learning to transfer network design knowledge from small tasks to larger tasks, surpassing many efficient solutions based on parameter sharing. [11]} proposes a highly memory-efficient and effective transfer solution that does not require back-propagation for adaptation. Central to these works' investigations are two key considerations: the transferability of an algorithm, namely how much information an algorithm can effectively reuse, and generalizability, which evaluates whether a solution can be applied to different settings (e.g., search spaces) and still performs well.
<FIGURE>