We use a training setting similar to sec:experiments. We have adapted the code of [1]}, [2]} to Resnet-18 for ImageNet. The hyperparameters for AdaSTE and MD are those prescribed for TinyImageNet. We use the same default data normalizations as the methods we compare to: we resize the input images to \(256 \times 256\)  and then randomly crop them to \(224 \times 224\)  while centering them to the appropriate sizes during training. Standard multiclass cross entropy loss is used. All models are fine-tuned for 100 epochs using the Adam [3]} optimizer with dynamics of \(0.9\)  and \(0.999\)  and a batch of size 512. The full precision NN is trained with an initial learning rate of \(0.08\) . The projected full precision NN uses a projected gradient algorithm. The same hyperparameters are used as in the "simple" algorithm, except that a deterministic projection onto the hypersphere is performed at each iteration. The ASkewSGD method is described in alg:detailed, and we have set \(\alpha \)  to \(0.5\) . The precision threshold \(\epsilon \)  is decreased from epoch to epoch: it is set to 1 at the beginning and then exponentially annealed to \(.88^t\)  in the last 50 epochs, where \(t\)  is the epoch.
