In interleaving learning, we perform two tasks: image classification on CIFAR-100 and image classification on CIFAR-10, using two classification models \(A\)  and \(B\) .
CIFAR-10 contains 10 classes and CIFAR-100 contains 100 classes.
For CIFAR-10 and CIFAR-100, each of them is split into a 25K training set, a 25K validation set, and a 10K test set. The training and validation set of CIFAR-100 is used as \(D_A^{(\textrm {tr})}\)  and \(D_A^{(\textrm {val})}\)  respectively; the training and validation set of CIFAR-10 is used as \(D_B^{(\textrm {tr})}\)  and \(D_B^{(\textrm {val})}\)  respectively. For the architecture search space of the feature extractors, we experimented with the search spaces of DARTS [1]}, P-DARTS [2]}, and PC-DARTS [3]}. These search spaces are composed of \(3\times 3\)  and \(5\times 5\)  (dilated) separable convolutions, \(3\times 3\)  max pooling, \(3\times 3\)  average pooling, zero, and identity.
For the CIFAR-100 classification head, we set it to a 100-way linear classifier. For the CIFAR-10 classification head, we set it to a 10-way linear classifier.
