The ImageNet dataset consists of 1.28 million training and 50,000 test images in 1000 classes.
We use \(128\times 128\)  resultion, and the standard cross-entropy loss.
We train on a 24 GPUs (GeForce RTX 2080 Ti) distributed training system [1]} for 120 epochs. Adam optimizer was used with effective batch size 384 and learning rate \(0.024\)  (using warm-up as in [2]}), which is divided by 10 when the error plateaus. Similar to CIFAR, data augmentation is implemented with \(20^\circ \)  rotation range, up to \(20\%\)  shift of height and width, and horizontal flip on \(50\%\)  of the data.
All convolutions have kernel size \(k=3\) .
In the cubic models, the first convolution outputs 128 features, followed by cycles of the following tensor widths: 106, 106, 106, 106, 12.
The non-cubic CycleNet starts with a convolution with 260 features (\(128\times 128\times 260\) ) followed by five cycles (\(106\times 106\times 260\) , \(106\times 106\times 190\) , \(106\times 106\times 159\) , \(106\times 106\times 159\) , \(10\times 10\times 11\) ).
The non-cubic 2:3 CNN:CycleNet model has a convolution of 200 filters, followed by two CNN cycles (equivalent to 6 convolutions of output features \(200, 200, 200, 190, 170, 159\) ) and three CycleNet cycles (\(106\times 106\times 159\) , \(106\times 106\times 159\) , \(10\times 10\times 11\) ).
