ResNet on CIFAR In Fig. REF  we train a ResNet18 [1]} on the CIFAR-10 dataset [2]}. Once again, we impose an orthogonality constraint on each convolution kernel. We take a batch size of 128, and use SGD with momentum to train each algorithm.
The landing algorithm is here once again \(50\%\)  faster than retraction-based methods. Also, we notice in this case that trivialization methods fail to reach a high accuracy.
<FIGURE>