In this section, we benchmark ASkewSGD with BC [1]}, [2]}, Mirror Descent [3]}, and ProxQuant [4]} on classical computer vision datasets. To avoid overloading the figures, the AdaSTE results are reported separately in sec:app:numerical-results. We also report performance with a standard full precision NN and a full precision NN projected onto the hypersphere. We compare the different methods using the same NN architecture. We do not add bias on any neuron. We introduce batch normalisation (without learning scale and bias parameters) after each layer. We emphasise that our method is generic and not specific to the classical ConvNet architecture. We have also obtained SOTA results for large ResNet architectures (see tab:accuracytinyimgimgnet).
