We use the standard data augmentations and normalizations for all the methods. ASkewSGD is implemented in Pytorch, and the experiments are run on a NVIDIA Tesla-P100 GPU. Standard multiclass cross-entropy loss is used for all experiments unless otherwise stated. We perform cross-validation of the hyperparameters, such as the learning rate, the tradeoff between constraints \(\alpha \) , the rate of increase of the annealing hyperparameter, and their respective schedules. The search space for tuning the hyperparameters and the final hyperparameters can be found in sec:app:numerical-results. All models are fine-tuned for 100 epochs using the Adam [1]} optimizer with dynamics of \(0.9\)  and \(0.999\) , and batch of size 100.
