Findings of the WMT 2020 Shared Task on Automatic Post Editing
We present the results of the 6th round of the WMT task on MT Automatic Post Editing. The task consists in automatically correcting the output of a "black box" machine translation system by learning from existing human corrections of different sentences. This year, the challenge consisted of fixing the errors present in English Wikipedia pages translated into German and Chinese by state ofthe art, not domain adapted neural MT (NMT) systems unknown to participants. Six teams participated in the English German task, submitting a total of 11 runs. Two teams participated in the English Chinese task submitting 2 runs each. Due to i) the different source/domain of data compared to the past (Wikipedia vs Information Technology) ii) the different quality of the initial translations to be corrected and iii) the introduction of a new language pair (English Chinese) this year's results are not directly comparable with last year's round. However, on both language directions, participants' submissions show considerable improvements over the baseline results. On English German, the top ranked system improves over the baseline by 11.35 TER and +16.68 BLEU points, while on EnglishChinese the improvements are respectively up to 12.13 TER and +14.57 BLEU points. Overall, coherent gains are also highlighted by the outcomes of human evaluation, which confirms the effectiveness of APE to improve MT quality, especially in the new generic domain selected for this year's round.
WMT