
Basic editing operations (BE)
We randomly edit tokens from the descriptions except few tokens that convey high semantic importance with respect to the programming languages. For example, the token concatenation conveys special meaning to the sentence and should not be edited. We reserve \(\sim 10\%\)  of the vocabulary tokens as non-editable. The non-editable list includes tokens such as times, sum, digits, maximum, prime, last, etc. We define a parameter \(\alpha \)  to regulate the number editable tokens in a sentence. The number of editable token is given by \(\lfloor \alpha L \rfloor \) , where \(L\)  is the length of the sentence. In our experiments, we assign \(\alpha = 0.1\) . Next, we define three basic token-level edit operations:
Random Deletion (RD): Randomly removing one or more words from the sentences.
Random Insertion (RI):  Randomly inserting one or more words in the sentences.
Random Substitution (RS): Randomly substituting one or more words in the sentences.
For RI and RS, we use BERT [1]} uncased language model trained on monolingual English language. In RI, we randomly add \(\lfloor \alpha L \rfloor \)  masked tokens to the input sentence and predict tokens corresponding to these masked positions. In the case of RS, we randomly select \(\lfloor \alpha L \rfloor \)  editable tokens in a problem description and mask them. Further, the masked sentence is fed to the pre-trained BERT model to predict the mask tokens. RD is reasonably straightforward as we randomly pick \(\lfloor \alpha L \rfloor \)  tokens from a sentence and delete them.

Back-Translation (BT)
In Back Translation (BT), a sentence is, first, translated to an intermediate language and again translated back to the original language. BT leads to the paraphrasing of the original sentence [2]}. In our case, the original language is English, and the intermediate language is GermanWe use German as one of the representative language due to the availability of good quality translations.. Table REF  presents an illustrative example of a BT operation. We leverage native Google Translate API for English to German translation and vice-versa.

Attention-based replace operation (AR)
Inspired by the quality of augmented sentences in  [3]}, we propose an attention-based augmentation operation that extracts the attention vector from the first encoder layer of the transformer and randomly replaces the maximally attended word with a random word in the vocabulary except the non-editable words to preserve the meaning of the sentence [4]}.

<TABLE><TABLE>