We fine-tune BERT models pre-trained for each language to ensure compatibility with each language. For English, we use Vanilla BERT\(_{BASE}\)  [1]}, for French CamemBERT [2]}, and for Spanish BETO\(_{BASE}\)  [3]}. In addition, we also fine-tune a multilingual BERT (mBERT) [1]} model with the concatenation of all languages.
