Cross-lingual Language Models (XLMs) [1]} aim to build a universal cross-lingual language encoder that maps sentences of different languages into a shared embedding space.
The pre-training of an XLM model involves both monolingual and cross-lingual objectives, including masked language modeling and translation language modeling.
