Data Augmentation (DA) is an effective technique for improving the accuracy of text classification models [1]} and has also been shown to boost the performance of KD for NLP algorithms [2]} as well as KD in models for tabular data [3]}. [1]} proposed the Easy Data Augmentation (EDA) technique that randomly replaces synonyms, inserts, swaps and deletes characters in the sentence. [2]} proposed to utilize the pretrained BERT model and GloVe word embeddings [6]} to augment the input sentence via random word-level replacement. MixKD [7]} adopts mixup [8]} and backtranslation [9]} in augmenting the text data to boost the performance of sentence classification models. Unlike these papers, we propose a novel search space for DA policies that supports stacking elementary augmentation operations such as EDA, mixup, and backtranslation. Thus, our considered DA module is similar to AutoAugment [10]}, except it is used for KD in NLP with different elementary operators.
