Context-based Virtual Adversarial Training for Text Classification with Noisy Labels
Deep neural networks (DNNs) have a high capacity to completely memorize noisy
labels given sufficient training time, and its memorization, unfortunately,
leads to performance degradation. Recently, virtual adversarial training (VAT)
attracts attention as it could further improve the generalization of DNNs in
semi-supervised learning. The driving force behind VAT is to prevent the models
from overfitting data points by enforcing consistency between the inputs and
the perturbed inputs. This strategy could be helpful in learning from noisy
labels if it prevents neural models from learning noisy samples while
encouraging the models to generalize clean samples. In this paper, we propose
context-based virtual adversarial training (ConVAT) to prevent a text
classifier from overfitting to noisy labels. Unlike the previous works, the
proposed method performs the adversarial training at the context level rather
than the inputs. It makes the classifier not only learn its label but also its
contextual neighbors, which alleviates the learning from noisy labels by
preserving contextual semantics on each data point. We conduct extensive
experiments on four text classification datasets with two types of label
noises. Comprehensive experimental results clearly show that the proposed
method works quite well even with extremely noisy settings.