The CIFAR [1]} is one of the most popular databases for image classification consists of a total of 60k \(32\times 32\)  RGB images and is divided into 50k training and 10k test images. CIFAR has two different datasets- CIFAR10 and CIFAR100 with a total of 10 and 100 classes, respectively. We report the top-1 accuracy on Table REF  and Table REF  on CIFAR10 dataset CIFAR100 datasets respectively. We consider MobileNet V2 (MN V2) [2]}, Shufflenet V2 (SF V2) [3]}, and EfficientNet B0 (EN-B0) [4]}. For all the experiments to train a model on these two datasets, we use a batch size of 128, stochastic gradient descent ([5]}, [6]}) optimizer with 0.9 momentum & \(5e^{-4}\)  weight decay, and trained all networks up-to 200 epochs. We begin with 0.01 learning rate and decay the learning rate with cosine annealing [7]} learning rate scheduler. We have applied Standard data augmentation methods like width shift, height shift, horizontal flip, and rotation on both datasets. It is noticeable from these two tables that replacing ReLU by SAU, there is an increment in top-1 accuracy from 1% to more than 5%.
<TABLE><TABLE><FIGURE>