We use a library that contains a pre-trained version of XLNet, an implementation of the transformer-XL architecture [14] . It was previously trained on BooksCorpus [15] and English Wikipedia which have 13GB of plain text combined <cite>[11]</cite> .