Regularization in deep neural networks such as dropout, weight decay, early stopping etc. are popular and effective training strategies to improve the training accuracy, robustness, model performance, while also avoiding overfitting to some extent on limited training data. Among these, dropout is a widely-recognized technique in training neural network which is mainly used in fully connected layers of CNN [1]} due to its spatial correlation and dependence property. In recent years, an interesting range of regional dropout or replacement strategies have been proposed, such as Cutout [2]}, DropBlock [3]}, CutMix [4]}, etc. Specifically, Cutout proposed to randomly mask out square regions of input during training in order to improve the robustness and overall performance of CNNs. DropBlock developed a structured form of dropout that is particularly effective in regularizing CNNs. During training, a contiguous region of a feature map is dropped instead of individual elements in the feature map. CutMix is motivated by Mixup [5]}, where the regions in an image are randomly cut and pasted among training images and the ground truth labels are also mixed proportional to the area of the regions. Although regional dropout or replacement operation methods have shown great effectiveness of recognition and localization performance in some benchmarks, the dropping or replacing operation is usually randomly conducted on the input. We argue that this strategy may reduce the efficiency of training and also limit the improvement if the networks are unable to capture the most discriminative regions. A representative comparison of our method with other strategies is shown in Fig. REF .
<FIGURE>