Proxyless Neural Architecture Adaptation for Supervised Learning and Self-Supervised Learning
Recently, Neural Architecture Search (NAS) methods have been introduced and
show impressive performance on many benchmarks. Among those NAS studies, Neural
Architecture Transformer (NAT) aims to adapt the given neural architecture to
improve performance while maintaining computational costs. However, NAT lacks
reproducibility and it requires an additional architecture adaptation process
before network weight training. In this paper, we propose proxyless neural
architecture adaptation that is reproducible and efficient. Our method can be
applied to both supervised learning and self-supervised learning. The proposed
method shows stable performance on various architectures. Extensive
reproducibility experiments on two datasets, i.e., CIFAR-10 and Tiny Imagenet,
present that the proposed method definitely outperforms NAT and is applicable
to other models and datasets.