BERT is based on the encoder of the transformer model (Vaswani et al., 2017) , which has been proven to obtain state-of-the-art accuracy across a broad range of NLP applications <cite>(Devlin et al., 2018)</cite> .