PreQuEL: Quality Estimation of Machine Translation Outputs in Advance
We present the task of PreQuEL, Pre-(Quality-Estimation) Learning. A PreQuEL
system predicts how well a given sentence will be translated, without recourse
to the actual translation, thus eschewing unnecessary resource allocation when
translation quality is bound to be low. PreQuEL can be defined relative to a
given MT system (e.g., some industry service) or generally relative to the
state-of-the-art. From a theoretical perspective, PreQuEL places the focus on
the source text, tracing properties, possibly linguistic features, that make a
sentence harder to machine translate.
  We develop a baseline model for the task and analyze its performance. We also
develop a data augmentation method (from parallel corpora), that improves
results substantially. We show that this augmentation method can improve the
performance of the Quality-Estimation task as well. We investigate the
properties of the input text that our model is sensitive to, by testing it on
challenge sets and different languages. We conclude that it is aware of
syntactic and semantic distinctions, and correlates and even over-emphasizes
the importance of standard NLP features.