We trained models using AdamW optimizer (\(\alpha = 0.001, \beta _{1} = 0.9, \beta _{2}=0.999, \epsilon = 10^{-8}\) ) with a weight decay coefficient of 0.01 during initial epochs and then used SGD (stochastic gradient descent) with learning rate of \(0.001\)  and momentum \(= 0.9\)  during the final 20 epochs [1]}. We used automatic mixed precision in PyTorch during training to optimize speed and memory consumption. Almost all experiments were done with a 16 GB Tesla V100-SXM2 GPU available in Google Colab Pro+®. No image augmentations were used while training the models. Maximum number of epochs in all experiments was set to 150. GPU usage for a batch size of 64 was reported for image classification along with top-1% accuracy from best of three runs with random initialization based on prevailing protocols [2]}. We report the semantic segmentation performance using mean intersection over union (mIoU) metric. Cross-entropy loss was used for image classification and pixel-wise focal loss was used for semantic segmentation. Few segmentation experiments were run on A100-SXM4 GPU in Google Colab Pro+®. Due to resource constraints, for segmentation experiments in 16 GB V100 GPU, the original \(1024\times 2048\)  image was resized to \(256\times 512\)  and for 40 GB A100 GPU, it was resized to \(512\times 1024\) . We also adjusted the stride of the initial convolutional layers in all WaveMix-Lite models that handled high-resolution images to ensure that smaller side of input was always 64 before it reached WaveMix-Lite blocks. The classification head of ConvMixer was replaced with a segmentation head similar to WaveMix-Lite for segmentation. No pre-training was performed on any of the WaveMix-Lite models.
