Most of the top systems build upon either BERT<cite> (Devlin et al., 2019)</cite> or XLNet , both of which rely on additional external training data. In contrast, our submission does not use any additional data.