Model Architecture In the same way as the original BERT architecture proposed by<cite> Devlin et al. (2019)</cite> our model is composed by stacked layers of Transformer encoders (Vaswani et al., 2017) .