In this section we complete the details of our training and regularization setup. We used a weight decay of \(2e^{-5}\)  and using a cosine annealing scheme [1]} with learning rate \(0.1\) . The largest models consistently overfit at the very end of training, which we attribute to the learning rate going to 0 at the end of training [2]}. To combat this, we set the end of the cosine annealing to be \(\frac{1.0}{128}\)  of the original learning rate instead of 0. For RandAugment [3]}, we grow our RangAugment magnitudes for the smallest \(H0\)  to the the largest \(H7\)  models as \(6, 8, 10, 14, 17, 19, 24\)  and 31. Note that we have not extensively tuned the RandAugment magnitudes.
