Unlike most NAS benchmarks that focus on classification tasks only, TransNAS-Bench-101 encourages the evaluation of algorithms across different tasks. Considered that Transferable NAS research is still in its infancy, we hope to provide as many common grounds for transfer as possible. This makes selecting proper datasets challenging since, ideally, the datasets should share some commonalities while covering a diversity of tasks.
Thanks to the great previous work Taskonomy [1]}, which provides sufficient images with labels on different tasks (see Figure REF ), we can study algorithms without worrying that the datasets are by nature too distinct for transfer. The original dataset consists of 4.5M images of indoor scenes from about 600 buildings. To control the computational budget, we randomly select 24 buildings containing a total of 120K images from the original dataset and split the subset into 80K train / 20K val / 20K test set.
<TABLE>