We focus on 4 different components based on the design of EfficientNet [1]}, 2 architecture modules and 2 regularizations:
Squeeze-and-Excitation (SE) [2]}, a channel attention module used after the spatial convolution;
SiLU/Swish-1 [3]}, [4]}, [5]}, an activation function with the form \(x \cdot \text{sigmoid}(x)\) ;
RandAugment (RA) [6]}, a data augmentation scheme that simplifies AutoAugment [7]};
and Label Smoothing (LS) [8]}, a smoothing of the label distribution.
