Global Mixup: Eliminating Ambiguity with Clustering
Data augmentation with \textbf{Mixup} has been proven an effective method to
regularize the current deep neural networks. Mixup generates virtual samples
and corresponding labels at once through linear interpolation. However, this
one-stage generation paradigm and the use of linear interpolation have the
following two defects: (1) The label of the generated sample is directly
combined from the labels of the original sample pairs without reasonable
judgment, which makes the labels likely to be ambiguous. (2) linear combination
significantly limits the sampling space for generating samples. To tackle these
problems, we propose a novel and effective augmentation method based on global
clustering relationships named \textbf{Global Mixup}. Specifically, we
transform the previous one-stage augmentation process into two-stage,
decoupling the process of generating virtual samples from the labeling. And for
the labels of the generated samples, relabeling is performed based on
clustering by calculating the global relationships of the generated samples. In
addition, we are no longer limited to linear relationships but generate more
reliable virtual samples in a larger sampling space. Extensive experiments for
\textbf{CNN}, \textbf{LSTM}, and \textbf{BERT} on five tasks show that Global
Mixup significantly outperforms previous state-of-the-art baselines. Further
experiments also demonstrate the advantage of Global Mixup in low-resource
scenarios.