The field of neural architecture search (NAS) has seen advances in recent years. In the early stage, researchers focus on the reinforcement learning-based approaches [1]}, [2]}, [3]}, [4]} and evolution-based approaches [5]}, [6]}, [7]}, [8]}, [9]}. These methods can produce architectures that outperform human-designed ones [10]}, [11]}. However, the computational cost is almost unbearable since it needs to fully train and evaluate every candidate network found in the search process. Weight sharing [12]}, [13]} is a practical solution where a super network is trained, and its sub-graphs form the search space. [14]} proposed DARTS to use search parameters together with a super network, which allows searching with gradient descent. Gradient-based methods [15]}, [16]}, [17]}, [18]}, [19]} attracts researchers' attention since it is computationally efficient and easy to implement. We base our method on DARTS and take one step further to reduce the memory consumption of training the super network. Another recent trend is the one-stage NAS [20]}, [21]}, [22]}, [23]}. Many NAS algorithms are in two stages. In the first stage, one searches for a good candidate network. In the second stage, the resulting network is re-initialized and re-trained. One-stage NAS tries to search and optimize the network weights simultaneously. After searching, one can have a ready-to-run network. We use a simple one-stage NAS algorithm [24]} as a baseline in Section REF .
