For contextual augmentation, we use the pretrained BERT model to do word level replacement by filling in randomly masked tokens. As in EDA [1]}, our random augmentation randomly swaps words in the sentence or replaces words with their synonyms. For backtranslation, we translate the sentence from one language (in this paper, English) to another language (in this paper, German) and then translate it back.
Additionally, mixup can be used to synthesize augmented training samples. First proposed for image classification [2]}, mixup constructs a synthetic training example via the weighted average of two samples (including the labels) drawn at random from the training data. To use it in NLP, [3]}, [4]} applied mixup on the word embeddings at each sentence position \(x_{i,t}\)  with \(\lambda \in [0,1]\)  as the mixing-ratio for a particular pair of examples \(x_i, x_j\) :
\(\hat{x}_{i,t}=\lambda x_{i,t} + (1-\lambda ) x_{j,t},~ \hat{y}_i=\lambda y_i + (1-\lambda ) y_j,\) 
