The second predictor we utilize is a pre-trained cross-lingual language model (XLM) [1]} to further improve the feature extraction ability.
Unlike monolingual language models such as BERT [2]}, XLM models are pre-trained on parallel corpora and construct cross-lingual representations on text, which suit better to the translation quality estimation task.
In contrast to the single predictor design in most prior work, e.g., [3]}, [4]}, [5]}, [6]}, our two predictors are capable of modeling the sentence pair from different perspectives, leading to a more compact representation, hence improving the estimation results.
