Long distance word reordering is a major challenge in statistical machine translation research . Previous work has shown using source syntactic trees is an effective way to tackle this problem between two languages with substantial word order difference . In this work , we further extend this line of exploration and propose a novel but simple approach , which utilizes a ranking model based on word order precedence in the target language to reposition nodes in the syntactic parse tree of a source sentence . The ranking model is automatically derived from word aligned parallel data with a syntactic parser for source language based on both lexical and syntactical features . We evaluated our approach on largescale Japanese-English and English-Japanese machine translation tasks , and show that it can significantly outperform the baseline phrasebased SMT system . * This work has been done while the first author was visiting Modeling word reordering between source and target sentences has been a research focus since the emerging of statistical machine translation . In phrase-based models ( Och , 2002 ; Koehn et al . , 2003 ) , phrase is introduced to serve as the fundamental translation element and deal with local reordering , while a distance based distortion model is used to coarsely depict the exponentially decayed word movement probabilities in language translation . Further work in this direction employed lexi-calized distortion models , including both generative ( Koehn et al . , 2005 ) and discriminative ( Zens and Ney , 2006 ; Xiong et al . , 2006 ) variants , to achieve finer-grained estimations , while other work took into account the hierarchical language structures in translation ( Chiang , 2005 ; Galley and Manning , 2008 ) . Long-distance word reordering between language pairs with substantial word order difference , such as Japanese with Subject-Object-Verb ( SOV ) structure and English with Subject-Verb-Object ( SVO ) structure , is generally viewed beyond the scope of the phrase-based systems discussed above , because of either distortion limits or lack of discriminative features for modeling . The most notable solution to this problem is adopting syntax-based SMT models , especially methods making use of source side syntactic parse trees . There are two major categories in this line of research . One is tree-to-string model ( Quirk et al . , 2005 ; Liu et al . , 2006 ) which directly uses source parse trees to derive a large set of translation rules and associated model parameters . The other is called syntax pre-reordering -an approach that re-positions source words to approximate target language word order as much as possible based on the features from source syntactic parse trees . This is usually done in a preprocessing step , and then followed by a standard phrase-based SMT system that takes the re-ordered source sentence as input to finish the translation . In this paper , we continue this line of work and address the problem of word reordering based on source syntactic parse trees for SMT . Similar to most previous work , our approach tries to rearrange the source tree nodes sharing a common parent to mimic the word order in target language . To this end , we propose a simple but effective ranking-based approach to word reordering . The ranking model is automatically derived from the word aligned parallel data , viewing the source tree nodes to be reordered as list items to be ranked . The ranks of tree nodes are determined by their relative positions in the target language -the node in the most front gets the highest rank , while the ending word in the target sentence gets the lowest rank . The ranking model is trained to directly minimize the mis-ordering of tree nodes , which differs from the prior work based on maximum likelihood estimations of reordering patterns ( Li et al . , 2007 ; Genzel , 2010 ) , and does not require any special tweaking in model training . The ranking model can not only be used in a pre-reordering based SMT system , but also be integrated into a phrasebased decoder serving as additional distortion features . We evaluated our approach on large-scale Japanese-English and English-Japanese machine translation tasks , and experimental results show that our approach can bring significant improvements to the baseline phrase-based SMT system in both preordering and integrated decoding settings . In the rest of the paper , we will first formally present our ranking-based word reordering model , then followed by detailed steps of modeling training and integration into a phrase-based SMT system . Experimental results are shown in Section 5 . Section 6 consists of more discussions on related work , and Section 7 concludes the paper . In this paper we present a ranking based reordering method to reorder source language to match the word order of target language given the source side parse tree . Reordering is formulated as a task to rank different nodes in the source side syntax tree according to their relative position in the target language . The ranking model is automatically trained to minimize the mis-ordering of tree nodes in the training data . Large scale experiment shows improvement on both reordering metric and SMT performance , with up to 1.73 point BLEU gain in our evaluation test . In future work , we plan to extend the ranking model to handle reordering between multiple levels of source trees . We also expect to explore better way to integrate ranking reorder model into SMT system instead of a simple penalty scheme . Along the research direction of preprocessing the source language to facilitate translation , we consider to not only change the order of the source language , but also inject syntactic structure of the target language into source language by adding pseudo words into source sentences .
Long distance word reordering remains a challenge for statistical machine translation and existing approaches do it during the preprocessing.
They propose a ranking-based reordering approach where the ranking model is automatically derived from the word aligned parallel data using a syntax parser.
Large scale evaluation of Japanese-English and English-Japanese shows that the proposed approach significantly outperforms the baseline phrase-based statistical machine translation system.