Yes
BOOLEAN
Yes
BOOLEAN
We built a phrase-based Chinese-to-English SMT system by using Moses BIBREF18 . Our parallel training text is a collection of parallel corpora from LDC, which we divide into older corpora and newer corpora. Due to the dominant older data, we duplicate the newer corpora of various domains by 10 times to achieve better domain balance. To reduce the possibility of alignment errors, parallel sentences in the corpora that are longer than 85 words in either Chinese (after word segmentation) or English are discarded. In the end, the final parallel text consists of around 8.8M sentence pairs, 228M Chinese tokens, and 254M English tokens (a token can be a word or punctuation symbol). We also added two dictionaries by concatenating them to our training parallel text. The total number of words in these two corpora is 1.81M for Chinese and 2.03M for English.
We built a phrase-based Chinese-to-English SMT system by using Moses BIBREF18 . Our parallel training text is a collection of parallel corpora from LDC, which we divide into older corpora and newer corpora. Due to the dominant older data, we duplicate the newer corpora of various domains by 10 times to achieve better domain balance. To reduce the possibility of alignment errors, parallel sentences in the corpora that are longer than 85 words in either Chinese (after word segmentation) or English are discarded. In the end, the final parallel text consists of around 8.8M sentence pairs, 228M Chinese tokens, and 254M English tokens (a token can be a word or punctuation symbol). We also added two dictionaries by concatenating them to our training parallel text. The total number of words in these two corpora is 1.81M for Chinese and 2.03M for English.
1608.01084
Introduction </s> Re ordering Ġin Ġmachine Ġtranslation Ġ( MT ) Ġis Ġa Ġcrucial Ġprocess Ġto Ġget Ġthe Ġcorrect Ġtranslation Ġoutput Ġword Ġorder Ġgiven Ġan Ġinput Ġsource Ġsentence , Ġas Ġword Ġorder Ġreflects Ġmeaning . ĠIt Ġremains Ġa Ġmajor Ġchallenge , Ġespecially Ġfor Ġlanguage Ġpairs Ġwith Ġa Ġsignificant Ġword Ġorder Ġdifference . ĠPh rase - based ĠMT Ġsystems ĠB IB REF 0 Ġgenerally Ġadopt Ġa Ġre ordering Ġmodel Ġthat Ġpredicts Ġre ordering Ġbased Ġon Ġthe Ġspan Ġof Ġa Ġphrase Ġand Ġthat Ġof Ġthe Ġadjacent Ġphrase ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ. </s> The Ġabove Ġmethods Ġdo Ġnot Ġexplicitly Ġpreserve Ġthe Ġrelationship Ġbetween Ġwords Ġin Ġthe Ġsource Ġsentence , Ġwhich Ġreflects Ġthe Ġsentence Ġmeaning . ĠWord Ġrelationship Ġin Ġa Ġsentence Ġcan Ġbe Ġcaptured Ġby Ġits Ġdependency Ġparse Ġtree , Ġin Ġwhich Ġeach Ġword Ġ$ w $ Ġis Ġa Ġtree Ġnode Ġconnected Ġto Ġits Ġhead Ġnode Ġ$ h _ w $ Ġ, Ġanother Ġword , Ġindicating Ġthat Ġthe Ġformer Ġis Ġa Ġdependent Ġ( child ) Ġof Ġthe Ġlatter . </s> D epend ency Ġparsing Ġhas Ġbeen Ġused Ġfor Ġre ordering Ġin Ġstatistical Ġmachine Ġtranslation Ġ( SM T ). ĠIts Ġusage Ġis Ġwell - known Ġin Ġthe Ġpre - ordering Ġapproach , Ġwhere Ġa Ġsource Ġsentence Ġis Ġre ordered Ġbefore Ġthe Ġactual Ġtranslation . ĠDepend ency - based Ġpre - ordering Ġcan Ġbe Ġperformed Ġeither Ġby Ġa Ġrule - based Ġapproach Ġbased Ġon Ġmanually Ġspecified Ġhuman Ġlinguistic Ġknowledge ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, Ġor Ġby Ġa Ġlearning Ġapproach ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġ, ĠB IB REF 12 Ġ. ĠDepend ency Ġparsing Ġhas Ġalso Ġbeen Ġused Ġin Ġre ordering Ġapproaches Ġintegrated Ġwith Ġdecoding Ġto Ġdetermine Ġthe Ġnext Ġsource Ġphrase Ġto Ġtranslate Ġafter Ġtranslating Ġthe Ġcurrent Ġsource Ġphrase ĠB IB REF 13 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ. </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġnovel Ġre ordering Ġapproach Ġintegrated Ġwith Ġtranslation . ĠWe Ġpropose Ġsparse Ġfeature Ġfunctions Ġbased Ġon Ġthe Ġpre - ordering Ġrules Ġof ĠB IB REF 6 Ġ. ĠHowever , Ġin Ġcontrast Ġto Ġthe Ġmanual Ġrule - based Ġpre - ordering Ġapproach Ġof ĠB IB REF 6 Ġ, Ġthe Ġweights Ġof Ġour Ġsparse Ġfeature Ġfunctions Ġare Ġautomatically Ġlearned Ġand Ġused Ġduring Ġthe Ġactual Ġtranslation Ġprocess , Ġwithout Ġan Ġexplicit Ġpre - ordering Ġstep . ĠOur Ġapproach Ġdetects Ġand Ġexploits Ġthe Ġre ordering Ġof Ġeach Ġdependency Ġword Ġpair Ġin Ġthe Ġsource Ġsentence Ġduring Ġphrase - based Ġdecoding . </s> D epend ency ĠWord ĠPair ĠFeatures </s> We Ġdefine Ġa Ġset Ġof Ġsparse Ġfeatures Ġbased Ġon Ġdependency Ġtree Ġword Ġpairs Ġto Ġbe Ġlearned Ġand Ġused Ġin Ġa Ġphrase - based ĠSM T Ġbeam Ġsearch Ġdecoding Ġalgorithm . </s> Capt uring ĠWord ĠPair ĠOrder ing Ġin ĠPh rase - Based ĠBeam ĠSearch </s> The Ġphrase - based Ġapproach Ġis Ġa Ġstate - of - the - art Ġapproach Ġfor ĠSM T , Ġtaking Ġphrases , Ġdefined Ġas Ġa Ġsequence Ġof Ġone Ġor Ġmore Ġwords , Ġas Ġits Ġtranslation Ġunits . ĠIt Ġis Ġperformed Ġby Ġa Ġbeam Ġsearch Ġalgorithm ĠB IB REF 16 Ġ, Ġin Ġwhich Ġthe Ġsearch Ġprocess Ġproduces Ġtranslation Ġfrom Ġleft Ġto Ġright Ġin Ġthe Ġtranslation Ġoutput . ĠThe Ġsearch Ġis Ġorganized Ġinto Ġhypotheses , Ġeach Ġof Ġwhich Ġrepresents Ġan Ġinput Ġsentence Ġphrase Ġcovered Ġand Ġits Ġpossible Ġtranslation . </s> As Ġthe Ġbeam Ġsearch Ġcan Ġchoose Ġinput Ġphrases Ġin Ġany Ġorder , Ġthe Ġtarget - language Ġphrase Ġsequence Ġin Ġthe Ġtranslation Ġoutput Ġmay Ġnot Ġfollow Ġthe Ġoriginal Ġsource Ġsentence Ġorder . ĠThe Ġsequence Ġdetermines Ġthe Ġtranslation Ġoutput Ġorder Ġand Ġenables Ġtranslation Ġoutput Ġre ordering Ġfor Ġlanguage Ġpairs Ġwith Ġdifferences Ġin Ġword Ġorder , Ġsuch Ġas ĠChinese Ġand ĠEnglish . </s> When Ġa Ġword Ġ$ f _ i $ Ġin Ġa Ġsource Ġsentence Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ Ġis Ġcovered Ġby Ġa Ġhypothesis , Ġit Ġis Ġknown Ġthat Ġthe Ġwords Ġ$\ l brace Ġf _{ l } | f _{ l } Ġ\ in Ġf _{ 1 } ^{ i - 1 } Ġ\ w edge Ġ\ l not Ġ{ trans lated ( f _ l ) }\ r brace Ġ$ Ġon Ġthe Ġleft Ġof Ġ$ f _ i $ Ġand Ġthe Ġwords Ġ$\ l brace Ġf _{ r } | f _{ r } Ġ\ in Ġf _{ i + 1 } ^{ N } Ġ\ w edge Ġ\ l not Ġ{ trans lated ( f _ r ) }\ r brace Ġ$ Ġon Ġthe Ġright Ġof Ġ$ f _ i $ Ġthat Ġhave Ġnot Ġbeen Ġtranslated Ġwill Ġbe Ġtranslated Ġafter Ġ( app earing Ġon Ġthe Ġright Ġof ) Ġthe Ġtranslation Ġof Ġ$ f _ i $ Ġ. ĠAs Ġ$ f _ l $ Ġis Ġbefore Ġ$ f _ i $ Ġin Ġthe Ġsource Ġsentence , Ġbut Ġthe Ġtranslation Ġof Ġ$ f _ l $ Ġis Ġafter Ġthe Ġtranslation Ġof Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 0 Ġ, Ġthe Ġtranslations Ġof Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 1 Ġand Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 2 Ġare Ġswapped . ĠMeanwhile , Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 3 Ġis Ġafter Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 4 Ġboth Ġin Ġthe Ġsource Ġsentence Ġand Ġin Ġthe Ġtranslation , Ġtherefore Ġthe Ġtranslations Ġof Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 5 Ġand Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 6 Ġare Ġin - order . ĠIntern ally Ġwithin Ġa Ġphrase , Ġthe Ġordering Ġof Ġeach Ġof Ġits Ġwords Ġin Ġthe Ġtranslation Ġdepends Ġon Ġthe Ġphr as al Ġword Ġalignment , Ġwhich Ġis Ġstored Ġin Ġthe Ġphrase Ġtable . </s> As Ġeach Ġword Ġin Ġthe Ġsource Ġsentence Ġis Ġa Ġnode Ġof Ġthe Ġsource Ġdependency Ġparse Ġtree , Ġthe Ġabove Ġnotion Ġcan Ġbe Ġused Ġfor Ġre ordering Ġbased Ġon Ġthe Ġsource Ġdependency Ġparse Ġtree . ĠInstead Ġof Ġcapturing Ġall Ġpair wise Ġrelations , Ġwe Ġare Ġonly Ġinterested Ġin Ġthe Ġrelations Ġbetween Ġa Ġword Ġand Ġits Ġrelated Ġwords , Ġdefined Ġcollectively Ġas Ġits Ġhead , Ġsibling , Ġand Ġchild Ġwords Ġin Ġthe Ġdependency Ġparse Ġtree . </s> D epend ency ĠSwap ĠFeatures </s> We Ġdefine Ġour Ġdependency Ġswap Ġfeatures , Ġfollowing Ġthe Ġrule Ġtemplate Ġdefinition Ġfor Ġdependency Ġswap Ġrules ĠB IB REF 6 Ġ, Ġwhich Ġdefines Ġrule Ġinstances Ġbased Ġon Ġthe Ġword Ġpairs Ġwith Ġhead - child Ġor Ġsibling Ġrelationship . ĠHowever , Ġthe Ġdifference Ġis Ġthat Ġour Ġapproach Ġdoes Ġnot Ġrequire Ġmanually Ġspecifying Ġwhich Ġdependency Ġlabels Ġare Ġthe Ġconditions Ġto Ġswap Ġwords , Ġbut Ġlearns Ġthem Ġautomatically . </s> In Ġour Ġapproach , Ġeach Ġrule Ġinstance Ġbased Ġon Ġthe Ġabove Ġtemplate Ġbecomes Ġa ĠBoolean Ġsparse Ġfeature Ġfunction ĠB IB REF 17 Ġ. ĠThe Ġfunction Ġparameters Ġare Ġthe Ġword Ġpair Ġspecification Ġand Ġoutput Ġorder . ĠWhile ĠB IB REF 6 Ġca ide pend ency - based 2014 Ġdefined Ġthe Ġrules Ġonly Ġby Ġthe Ġdependency Ġlabels , Ġwe Ġdefine Ġour Ġfeature Ġfunctions Ġfor Ġeach Ġword Ġpair Ġby Ġthe Ġdependency Ġlabels , Ġthe ĠPOS Ġtags , Ġand Ġthe Ġcombination Ġof Ġboth , Ġresulting Ġin Ġa Ġgroup Ġof Ġfour Ġfeature Ġfunctions Ġfor Ġevery Ġword Ġpair Ġordering . ĠThe Ġdependency Ġlink Ġlabel Ġof Ġa Ġword Ġ$ x $ Ġis Ġdefined Ġas Ġthe Ġlabel Ġof Ġthe Ġlink Ġconnecting Ġ$ x $ Ġto Ġits Ġhead Ġword . ĠHence forth Ġfor Ġeach Ġword Ġ$ x $ Ġ, Ġ$ L ( x ) $ Ġand Ġ$ T ( x ) $ Ġdenote Ġthe Ġdependency Ġlink Ġlabel Ġand ĠPOS Ġtag Ġof Ġ$ x $ Ġrespectively . </s> Following Ġthe Ġdependency Ġswap Ġrule Ġtemplate , Ġwe Ġdefine Ġtwo Ġtypes Ġof Ġfeature Ġfunction Ġtemplates , Ġnamely Ġhead - child Ġand Ġsibling . ĠThe Ġhead - child Ġfeature Ġfunctions Ġare Ġequal Ġto Ġ1 Ġif Ġa Ġhead Ġword Ġ$ x _ h $ Ġand Ġits Ġchild Ġword Ġ$ x _ c $ Ġ( where Ġ$ x _ h $ Ġis Ġon Ġthe Ġ$ p \ in Ġ\ l brace Ġleft , right \ r brace Ġ$ Ġof Ġ$ x _ c $ Ġin Ġthe Ġsource Ġsentence ) Ġtake Ġa Ġcertain Ġordering Ġ$ o $ Ġ( which Ġcan Ġbe Ġin - order Ġ( Ġ$ io $ Ġ) Ġor Ġswapped Ġ( Ġ$ sw $ Ġ)) Ġin Ġthe Ġtranslation Ġoutput , Ġand Ġ0 Ġotherwise . ĠThis Ġgroup Ġof Ġfeature Ġfunctions Ġis Ġdefined Ġas </s> $$ H _{ h c } ( x _ h , x _ c , p , o ) Ġ= Ġ\ left [ \ begin { array }{ c } Ġh _{ h c } ( L ( x _ h ), L ( x _ c ), p , o ) Ġ\\ Ġh _{ h c } ( T ( x _ h ), T ( x _ c ), p , o ) Ġ\\ Ġh _{ h c } ( L ( x _ h ), T ( x _ c ), p , o ) Ġ\\ Ġh _{ h c } ( T ( x _ h ), L ( x _ c ), p , o ) Ġ\ end { array }\ right ] $$ Ġ Ġ Ġ( E q . Ġ6 ) </s> Similarly , Ġthe Ġsibling Ġfeature Ġfunctions Ġare Ġequal Ġto Ġ1 Ġif Ġsiblings Ġ$ x _ l $ Ġand Ġ$ x _ r $ Ġ( Ġ$ x _ l $ Ġon Ġthe Ġleft Ġof Ġ$ x _ r $ Ġin Ġthe Ġsource Ġsentence ) Ġtake Ġa Ġcertain Ġordering Ġ$ o $ Ġin Ġthe Ġtranslation Ġoutput , Ġand Ġ0 Ġotherwise . ĠThis Ġgroup Ġof Ġfeature Ġfunctions Ġis Ġdefined Ġas </s> $$ H _{ s ib } ( x _ l , x _ r , o ) Ġ= Ġ\ left [ \ begin { array }{ c } Ġh _{ s ib } ( L ( x _ l ), L ( x _ r ), o ) Ġ\\ Ġh _{ s ib } ( T ( x _ l ), T ( x _ r ), o ) Ġ\\ Ġh _{ s ib } ( L ( x _ l ), T ( x _ r ), o ) Ġ\\ Ġh _{ s ib } ( T ( x _ l ), L ( x _ r ), o ) Ġ\ end { array }\ right ] $$ Ġ Ġ Ġ( E q . Ġ7 ) </s> UTF 8 g bs n ĠEach Ġdependency Ġswap Ġfeature Ġthat Ġis Ġset Ġto Ġ1 Ġwhen Ġa Ġhypothesis Ġis Ġgenerated Ġcaptures Ġtwo Ġsource Ġwords , Ġone Ġis Ġcovered Ġby Ġthe Ġcurrent Ġhypothesis , Ġwhile Ġthe Ġother Ġhas Ġnot Ġyet Ġbeen Ġtranslated . ĠThese Ġword Ġpairs Ġhave Ġeither Ġhead - child Ġor Ġsibling Ġrelationship Ġas Ġdefined Ġabove . ĠFigure Ġ1 Ġis Ġan Ġillustration Ġof Ġhow Ġword Ġorder Ġis Ġdetected Ġduring Ġbeam Ġsearch Ġdecoding Ġand Ġthe Ġswap Ġfeatures Ġinvolved . ĠWhen ĠâĢ ľ åı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ Ġ$\ right arrow Ġ$ Ġmade Ġa Ġspeech âĢ Ŀ Ġis Ġtranslated Ġafter ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ Ġ$\ right arrow Ġ$ ĠJ ok ow i âĢ Ŀ , Ġit Ġis Ġknown Ġthat Ġthe Ġhead ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ Ġis Ġbefore Ġthe Ġchild ĠâĢ ľ è ® ² è¯ Ŀ âĢ Ŀ Ġ( Ġ$ p = left $ Ġ) Ġand Ġtheir Ġtranslations Ġalso Ġfollow Ġthe Ġsame Ġword Ġorder , Ġsetting Ġthe Ġvalue Ġof Ġthe Ġfollowing Ġfour Ġfeatures Ġto Ġ1 : Ġ$ H _{ h c } (\ text { åı ĳ è¡ ¨ }, \ text { è ® ² è¯ Ŀ }, left , io ) Ġ= Ġ\ left [ \ begin { array }{ c } Ġh _{ h c } ( root , d obj , left , io ) Ġ\\ Ġh _{ h c } ( V V , NN , left , io ) Ġ\\ Ġh _{ h c } ( root , NN , left , io ) Ġ\\ Ġh _{ h c } ( V V , d obj , left , io ) Ġ\ end { array }\ right ] $ </s> On Ġthe Ġother Ġhand , Ġthe Ġhead ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ Ġis Ġafter Ġthe Ġchild ĠâĢ ľ åľ ¨ âĢ Ŀ Ġin Ġthe Ġsource Ġsentence Ġ( Ġ$ p = right $ Ġ), Ġbut ĠâĢ ľ åľ ¨ âĢ Ŀ Ġhas Ġnot Ġbeen Ġtranslated . ĠTherefore , Ġthe Ġtranslation Ġof ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ Ġwill Ġbe Ġswapped Ġwith Ġthat Ġof ĠâĢ ľ åľ ¨ âĢ Ŀ , Ġsetting Ġthe Ġfour Ġfeatures Ġin Ġ$ H _{ h c } (\ text { åı ĳ è¡ ¨ }, \ text { åľ ¨ }, right , sw ) $ Ġto Ġ1 . ĠSimilarly , ĠâĢ ľ è ® ² è¯ Ŀ âĢ Ŀ Ġis Ġafter Ġits Ġsibling ĠâĢ ľ åľ ¨ âĢ Ŀ Ġin Ġthe Ġsource Ġsentence , Ġbut ĠâĢ ľ åľ ¨ âĢ Ŀ Ġhas Ġnot Ġbeen Ġtranslated , Ġresulting Ġin Ġthe Ġtranslation Ġof Ġthe Ġtwo Ġwords Ġbeing Ġswapped Ġand Ġsetting Ġthe Ġfour Ġfeatures Ġin Ġ$ H _{ s ib } (\ text { åľ ¨ }, \ text { è ® ² è¯ Ŀ }, sw ) $ Ġto Ġ1 . ĠThe Ġfeatures Ġfor ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ âĢ Ŀ Ġsuch Ġas Ġ$ H _{ h c } (\ text { åı ĳ è¡ ¨ }, \ text { ä½ Ĳ ç § ĳ å ¨ ģ }, right , io ) $ Ġand Ġ$ H _{ s ib } (\ text { ä½ Ĳ ç § ĳ å ¨ ģ }, \ text { è ® ² è¯ Ŀ }, io ) $ Ġare Ġnot Ġset Ġto Ġ1 Ġwhen Ġthe Ġhypothesis ĠâĢ ľ åı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ Ġ$\ right arrow Ġ$ Ġmade Ġa Ġspeech âĢ Ŀ Ġis Ġgenerated . ĠThose Ġfeatures Ġwere Ġset Ġto Ġ1 Ġwith Ġthe Ġprevious Ġhypothesis ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ Ġ$\ right arrow Ġ$ ĠJ ok ow i âĢ Ŀ . </s> D epend ency ĠDist ortion ĠPenalty </s> To Ġencourage Ġthe Ġtranslation Ġoutput Ġthat Ġconform s Ġto Ġthe Ġdependency Ġparse Ġstructure , Ġwe Ġimpose Ġa Ġpenalty Ġfeature Ġthat Ġdiscour ages Ġtranslation Ġcandidates Ġnot Ġcon forming Ġto Ġthe Ġdependency Ġparse Ġsubt ree ĠB IB REF 13 Ġ. ĠThis Ġassigns Ġa Ġpenalty Ġif Ġthe Ġtranslation Ġof Ġthe Ġcurrent Ġphrase Ġresults Ġin Ġa Ġsource Ġdependency Ġparse Ġsubt ree Ġto Ġbe Ġsplit Ġin Ġthe Ġoutput Ġtranslation . </s> UTF 8 g bs n ĠThe Ġtranslation Ġoutput Ġshown Ġin ĠFigure Ġ1 Ġhas Ġeach Ġdependency Ġparse Ġsubt ree Ġgrouped Ġtogether , Ġtherefore Ġinc urring Ġno Ġpenalty . ĠHowever , Ġthere Ġcan Ġbe Ġa Ġcase Ġwhen Ġthe Ġtranslation Ġhas Ġproduced ĠâĢ ľ J ok ow i Ġmade Ġa Ġspeech Ġin âĢ Ŀ , Ġthen Ġtranslate ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ Ġto ĠâĢ ľ yes terday âĢ Ŀ . ĠThis Ġtranslation Ġwill Ġbreak Ġthe Ġcohesion Ġof Ġthe Ġsource Ġdependency Ġparse Ġsubt ree ĠâĢ ľ åľ ¨ Ġå Į Ĺ äº ¬ âĢ Ŀ Ġand Ġis Ġbad . ĠThis Ġis Ġthe Ġcase Ġwhen Ġa Ġdependency Ġdistortion Ġpenalty Ġis Ġincurred . </s> Mod ified ĠFuture ĠCost </s> Ph rase - based ĠSM T Ġbeam Ġsearch Ġdecoding Ġinvolves Ġfuture Ġcost , Ġnot Ġaccumulated Ġover Ġthe Ġtranslation Ġhypotheses Ġbut Ġused Ġto Ġreduce Ġsearch Ġspace ĠB IB REF 16 Ġ. ĠThis Ġpredicts Ġthe Ġcost Ġto Ġtranslate Ġthe Ġremaining Ġunt rans lated Ġphrases . </s> Based Ġon Ġour Ġdependency Ġswap Ġfeatures , Ġwe Ġincorporate Ġthe Ġfuture Ġcost Ġfor Ġeach Ġunt rans lated Ġdependency Ġword Ġpairs . ĠThe Ġfuture Ġcost Ġassumes Ġthat Ġthe Ġunt rans lated Ġwords Ġwill Ġbe Ġordered Ġin Ġthe Ġmost Ġlikely Ġordering . ĠHowever , Ġif Ġan Ġunt rans lated Ġword Ġ$ x $ Ġis Ġan Ġancestor Ġ( in Ġthe Ġdependency Ġparse Ġtree ) Ġof Ġa Ġword Ġcovered Ġby Ġthe Ġcurrent Ġhypothesis , Ġthe Ġfuture Ġcost Ġassumes Ġthat Ġ$ x $ Ġpreced es Ġall Ġof Ġits Ġrelated Ġwords . ĠThis Ġfollows Ġthe Ġassumption Ġthat Ġeach Ġsubt ree Ġwill Ġhave Ġa Ġcontiguous Ġtranslation ĠB IB REF 13 Ġ. </s> Other ĠSp arse ĠFeatures </s> This Ġsection Ġdescribes Ġother Ġsparse Ġfeatures Ġfrom Ġprevious Ġwork Ġto Ġcompare Ġto Ġour Ġmethod . </s> S parse ĠRe ordering ĠOrient ation ĠFeatures </s> We Ġincorporate Ġsparse Ġre ordering Ġorientation Ġfeatures ĠB IB REF 4 Ġ. ĠThe Ġfeatures Ġare Ġderived Ġfrom Ġthe Ġre ordering Ġorientation Ġmodel , Ġcapturing Ġthe Ġsource Ġposition Ġof Ġthe Ġcurrent Ġphrase Ġbeing Ġtranslated Ġwith Ġrespect Ġto Ġthe Ġpreviously Ġtranslated Ġphrase ( s ), Ġfor Ġwhich Ġthree Ġorientation Ġtypes Ġare Ġdefined : </s> Mon ot one Ġ( M ), Ġif Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġpreviously Ġtranslated Ġunit Ġare Ġadjacent Ġin Ġthe Ġinput Ġsentence Ġand Ġthe Ġformer Ġfollows Ġthe Ġlatter Ġin Ġit , Ġe . g ., ĠâĢ ľ Be ijing âĢ Ŀ Ġwith Ġrespect Ġto Ġthe Ġprevious Ġphrase ĠâĢ ľ in âĢ Ŀ Ġin ĠFigure Ġ1 Ġ. </s> Sw apped Ġ( S ), Ġif Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġpreviously Ġtranslated Ġunit Ġare Ġadjacent Ġin Ġthe Ġinput Ġsentence Ġbut Ġthe Ġformer Ġpreced es Ġthe Ġlatter Ġin Ġit Ġ( example Ġbelow ). </s> Disc ont in uous Ġ( D ), Ġif Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġpreviously Ġtranslated Ġunit Ġare Ġnot Ġadjacent Ġin Ġthe Ġinput Ġsentence , Ġe . g ., ĠâĢ ľ made Ġa Ġspeech âĢ Ŀ Ġwith Ġrespect Ġto Ġthe Ġprevious Ġphrase ĠâĢ ľ J ok ow i âĢ Ŀ Ġin ĠFigure Ġ1 Ġ. </s> UTF 8 g bs n ĠB IB REF 4 Ġcherry impro ved 2013 Ġdesigned Ġhis Ġsparse Ġre ordering Ġorientation Ġmodel Ġfollowing Ġthe Ġhierarchical Ġre ordering Ġ( HR ) Ġmodel ĠB IB REF 3 Ġ, Ġcapturing Ġthe Ġrelative Ġposition Ġof Ġthe Ġcurrent Ġphrase Ġ( covered Ġby Ġthe Ġcurrent Ġhypothesis ) Ġwith Ġrespect Ġto Ġthe Ġlargest Ġchunk Ġof Ġcontiguous Ġsource Ġphrases Ġthat Ġform Ġa Ġcontiguous Ġtranslation Ġbefore Ġthis Ġphrase . ĠTherefore , Ġin ĠFigure Ġ1 Ġ, Ġwhen Ġthe Ġdecoding Ġproduces Ġa Ġphrase ĠâĢ ľ yes terday âĢ Ŀ Ġafter ĠâĢ ľ Be ijing âĢ Ŀ , Ġthe Ġorientation Ġof ĠâĢ ľ yes terday âĢ Ŀ Ġis Ġswapped Ġinstead Ġof Ġdiscontin uous , Ġas ĠâĢ ľ made Ġa Ġspeech Ġin ĠBeijing âĢ Ŀ Ġis Ġformed Ġby Ġcontiguous Ġphrases ĠâĢ ľ åľ ¨ Ġå Į Ĺ äº ¬ Ġå ı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ âĢ Ŀ , Ġwhich Ġis Ġadjacent Ġto ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ . </s> While Ġthe Ġoriginal Ġ( non - s parse ) Ġre ordering Ġorientation Ġmodel Ġis Ġbased Ġon Ġthe Ġphrase Ġorientation Ġprobability Ġin Ġthe Ġparallel Ġtraining Ġdata Ġand Ġdefines Ġa Ġsingle Ġfeature Ġfunction Ġon Ġit , Ġthe Ġsparse Ġmodel Ġdefines Ġone Ġfeature Ġfunction Ġfor Ġeach Ġre ordering Ġphenomenon Ġduring Ġdecoding . ĠWe Ġdefine Ġthe Ġsparse Ġfeature Ġfunctions Ġtaking Ġinto Ġaccount Ġthe Ġphrase Ġorientation Ġ$ o \ in Ġ\ l brace ĠM , S , D \ r brace Ġ$ Ġduring Ġdecoding Ġand Ġimportant Ġlocations Ġ$ loc $ Ġpertaining Ġto Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġprevious Ġphrase Ġwith Ġthe Ġfollowing Ġtemplate : </s> $$ h _{ s \ _ hr } ( loc : = rep ( loc ), o ) $$ Ġ Ġ Ġ( E q . Ġ14 ) </s> where Ġlocations Ġ$ loc $ Ġare Ġthe Ġfirst Ġand Ġlast Ġwords Ġof Ġthe Ġcurrent Ġsource Ġphrase Ġ( Ġ$ s _{ first } $ Ġ, Ġ$ s _{ last } $ Ġ), Ġthe Ġpreviously Ġtranslated Ġunit Ġ( Ġ$ p _{ first } $ Ġ, Ġ$ p _{ last } $ Ġ), Ġi . e ., Ġthe Ġlargest Ġcontiguous Ġchunk Ġof Ġphrases Ġforming Ġa Ġcontiguous Ġtranslation , Ġand Ġthe Ġspan Ġbetween Ġthe Ġcurrent Ġand Ġthe Ġprevious Ġsource Ġphrase , Ġor Ġgap Ġ( Ġ$ g _{ first } $ Ġ, Ġ$ g _{ last } $ Ġ) Ġonly Ġfor Ġdiscontin uous Ġorientation . ĠEach Ġword Ġin Ġ$ loc $ Ġis Ġrepresented , Ġ$ rep ( loc ) $ Ġ, Ġby Ġits ĠPOS Ġtag , Ġand Ġthe Ġsurface Ġlex ical Ġform Ġif Ġit Ġbelongs Ġto Ġthe Ġ80 - most Ġfrequent Ġwords Ġin Ġthe Ġtraining Ġdata . </s> UTF 8 g bs n ĠAssuming Ġthat Ġonly ĠâĢ ľ åľ ¨ âĢ Ŀ Ġbelongs Ġto Ġthe Ġtop - 80 Ġwords , Ġwhen Ġthe Ġphrase ĠâĢ ľ yes terday âĢ Ŀ Ġis Ġgenerated Ġafter ĠâĢ ľ Be ijing âĢ Ŀ , Ġthe Ġsparse Ġre ordering Ġorientation Ġfeatures Ġthat Ġare Ġequal Ġto Ġ1 Ġare Ġshown Ġin ĠFigure Ġ2 Ġ. </s> While Ġthe Ġapproach Ġlever ages Ġthe Ġre ordering Ġorientation Ġmodel Ġby Ġdefining Ġfeatures Ġon Ġthe Ġcontext Ġinformation , Ġnot Ġjust Ġthe Ġcurrent Ġphrase , Ġit Ġdoes Ġnot Ġcapture Ġdependency Ġrelation , Ġby Ġwhich Ġthe Ġimportant Ġrelation Ġbetween Ġwords Ġin Ġthe Ġsource Ġsentence Ġis Ġcaptured . ĠTherefore , Ġwe Ġintroduce Ġour Ġsparse Ġdependency Ġswap Ġfeatures Ġto Ġenable Ġthe Ġtranslation Ġsystem Ġto Ġarrive Ġat Ġre ordering Ġdecisions Ġbased Ġon Ġthe Ġsource Ġdependency Ġrelations . </s> D epend ency ĠPath ĠFeatures </s> We Ġalso Ġutilize Ġdependency Ġpath Ġfeatures ĠB IB REF 15 Ġfor Ġphrase - based ĠSM T , Ġdefined Ġover Ġthe Ġshortest Ġpath Ġof Ġdependency Ġparse Ġtree Ġlinks Ġbrid ging Ġthe Ġcurrent Ġand Ġthe Ġprevious Ġsource Ġphrase . ĠB IB REF 15 Ġchang disc rim in ative 2009 Ġdefined Ġthe Ġdependency Ġpath Ġfeatures Ġon Ġa Ġmaximum Ġentropy Ġphrase Ġorientation Ġclass ifier , Ġtrained Ġon Ġtheir Ġword - aligned Ġparallel Ġtext Ġand Ġlabeled Ġby Ġthe Ġtwo Ġpossible Ġphrase Ġorder ings Ġin Ġthe Ġtranslation Ġoutput : Ġin - order Ġand Ġswapped . ĠMeanwhile , Ġwe Ġuse Ġthe Ġfeatures Ġas Ġsparse Ġdecoding Ġfeatures , Ġwith Ġthe Ġfollowing Ġtemplate : </s> $$ h _{ path } ( short est \ _ path ( p _{ last }, s _{ first }); o ) $$ Ġ Ġ Ġ( E q . Ġ18 ) </s> where Ġ$ o Ġ\ in Ġ\ l brace Ġin \ _ order , sw apped \ r brace Ġ$ Ġdenotes Ġthe Ġorientation Ġof Ġthe Ġtwo Ġphrases Ġin Ġthe Ġtranslation . </s> Given Ġa Ġsource Ġsentence Ġand Ġits Ġtranslation Ġoutput , Ġa Ġpath Ġis Ġdefined Ġbetween Ġthe Ġlast Ġword Ġof Ġthe Ġprevious Ġsource Ġphrase Ġ$ p _{ last } $ Ġand Ġthe Ġfirst Ġword Ġof Ġthe Ġcurrent Ġsource Ġphrase Ġ$ s _{ first } $ Ġ. ĠPath Ġtravers al Ġis Ġalways Ġfrom Ġleft Ġto Ġright Ġbased Ġon Ġthe Ġsource Ġsentence Ġword Ġposition . ĠTherefore , Ġif Ġthe Ġcurrent Ġsource Ġphrase Ġbeing Ġtranslated Ġis Ġto Ġthe Ġright Ġof Ġthe Ġprevious Ġsource Ġphrase , Ġthe Ġtravers al Ġis Ġfrom Ġ$ p _{ last } $ Ġto Ġ$ s _{ first } $ Ġ. ĠOtherwise , Ġif Ġthe Ġcurrent Ġsource Ġphrase Ġis Ġto Ġthe Ġleft Ġof Ġthe Ġprevious Ġsource Ġphrase , Ġit Ġis Ġfrom Ġ$ s _{ first } $ Ġto Ġ$ p _{ last } $ Ġ. ĠIn Ġaddition , Ġpath Ġedges Ġgoing Ġagainst Ġthe Ġdependency Ġlabel Ġarrow Ġare Ġdistinguished Ġfrom Ġthose Ġfollowing Ġthe Ġarrow . </s> UTF 8 g bs n ĠAs Ġan Ġexample , Ġin ĠFigure Ġ1 Ġ, Ġwhen Ġthe Ġtranslation Ġgenerates ĠâĢ ľ åı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ Ġ$\ right arrow Ġ$ Ġmade Ġa Ġspeech âĢ Ŀ Ġafter ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ Ġ$\ right arrow Ġ$ ĠJ ok ow i âĢ Ŀ , Ġthe Ġpath Ġis Ġfrom ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ âĢ Ŀ Ġto ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ , Ġconsisting Ġof Ġa Ġdirect Ġlink Ġfollowing Ġthe Ġarrow Ġof Ġ$ ns ub j $ Ġ, Ġresulting Ġin Ġthe Ġfeature Ġ$ h _{ path } ( ns ub j , in \ _ order )= 1 $ Ġ. ĠHowever , Ġwhen Ġthe Ġtranslation Ġgenerates ĠâĢ ľ æĺ ¨ å¤© Ġ$\ right arrow Ġ$ Ġyesterday âĢ Ŀ Ġafter ĠâĢ ľ åĮ Ĺ äº ¬ Ġ$\ right arrow Ġ$ ĠBeijing âĢ Ŀ , Ġas ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ Ġis Ġbefore ĠâĢ ľ åĮ Ĺ äº ¬ âĢ Ŀ Ġin Ġthe Ġsource Ġsentence , Ġthe Ġtravers al Ġis Ġfrom ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ Ġto ĠâĢ ľ åĮ Ĺ äº ¬ âĢ Ŀ , Ġconsisting Ġof Ġthe Ġlink Ġsequence Ġ$ t mod , prep R , p obj R $ Ġ. ĠAs Ġit Ġgoes Ġagainst Ġthe Ġarrows Ġof Ġ$ prep $ Ġand Ġ$ p obj $ Ġ, Ġthe Ġsuffix Ġ$ R $ Ġis Ġadded Ġto Ġdistinguish Ġit . ĠThis Ġresults Ġin Ġthe Ġfeature Ġ$\ right arrow Ġ$ 0 Ġ. </s> The Ġapproach Ġlever ages Ġthe Ġphrase - based Ġre ordering Ġby Ġguiding Ġthe Ġordering Ġof Ġtwo Ġadjacent Ġphrases Ġusing Ġdependency Ġparse . ĠHowever , Ġthe Ġfeatures Ġdo Ġnot Ġcapture Ġthe Ġpair wise Ġordering Ġof Ġevery Ġword Ġwith Ġits Ġrelated Ġword , Ġas Ġthe Ġfeatures Ġare Ġinduced Ġonly Ġwhen Ġthe Ġwords Ġare Ġused Ġin Ġthe Ġtwo Ġadjacent Ġtranslation Ġphrases . </s> Data ĠSet Ġand ĠTool k its </s> We Ġbuilt Ġa Ġphrase - based ĠChinese - to - English ĠSM T Ġsystem Ġby Ġusing ĠMoses ĠB IB REF 18 Ġ. ĠOur Ġparallel Ġtraining Ġtext Ġis Ġa Ġcollection Ġof Ġparallel Ġcorpor a Ġfrom ĠL DC , Ġwhich Ġwe Ġdivide Ġinto Ġolder Ġcorpor a Ġand Ġnewer Ġcorpor a . ĠDue Ġto Ġthe Ġdominant Ġolder Ġdata , Ġwe Ġduplicate Ġthe Ġnewer Ġcorpor a Ġof Ġvarious Ġdomains Ġby Ġ10 Ġtimes Ġto Ġachieve Ġbetter Ġdomain Ġbalance . ĠTo Ġreduce Ġthe Ġpossibility Ġof Ġalignment Ġerrors , Ġparallel Ġsentences Ġin Ġthe Ġcorpor a Ġthat Ġare Ġlonger Ġthan Ġ85 Ġwords Ġin Ġeither ĠChinese Ġ( after Ġword Ġsegment ation ) Ġor ĠEnglish Ġare Ġdiscarded . ĠIn Ġthe Ġend , Ġthe Ġfinal Ġparallel Ġtext Ġconsists Ġof Ġaround Ġ8 . 8 M Ġsentence Ġpairs , Ġ228 M ĠChinese Ġtokens , Ġand Ġ254 M ĠEnglish Ġtokens Ġ( a Ġtoken Ġcan Ġbe Ġa Ġword Ġor Ġpunct uation Ġsymbol ). ĠWe Ġalso Ġadded Ġtwo Ġdiction aries Ġby Ġconc aten ating Ġthem Ġto Ġour Ġtraining Ġparallel Ġtext . ĠThe Ġtotal Ġnumber Ġof Ġwords Ġin Ġthese Ġtwo Ġcorpor a Ġis Ġ1 . 81 M Ġfor ĠChinese Ġand Ġ2 . 03 M Ġfor ĠEnglish . </s> All ĠChinese Ġsentences Ġin Ġthe Ġtraining , Ġdevelopment , Ġand Ġtest Ġdata Ġare Ġfirst Ġword - se gment ed Ġusing Ġa Ġmaximum Ġentropy - based ĠChinese Ġword Ġsegment er ĠB IB REF 19 Ġtrained Ġon Ġthe ĠChinese ĠTree bank Ġ( CT B ) Ġscheme . ĠThen Ġthe Ġparallel Ġcorpus Ġis Ġword - aligned Ġby ĠG IZ A ++ ĠB IB REF 20 Ġusing ĠIBM ĠModels Ġ1 , Ġ3 , Ġand Ġ4 ĠB IB REF 21 Ġ. ĠFor Ġbuilding Ġthe Ġphrase Ġtable , Ġwhich Ġfollows Ġword Ġalignment , Ġthe Ġmaximum Ġlength Ġof Ġa Ġphrase Ġpair Ġis Ġset Ġto Ġ7 Ġwords Ġfor Ġboth Ġthe Ġsource Ġand Ġtarget Ġsides . </s> The Ġlanguage Ġmodel Ġ( LM ) Ġis Ġa Ġ5 - gram Ġmodel Ġtrained Ġon Ġthe ĠEnglish Ġside Ġof Ġthe ĠFB IS Ġparallel Ġcorpus Ġ( L DC 2003 E 14 ) Ġand Ġthe Ġmon oling ual Ġcorpus ĠEnglish ĠGig aw ord Ġversion Ġ4 Ġ( L DC 2009 T 13 ), Ġconsisting Ġof Ġ107 M Ġsentences Ġand Ġ3 . 8 G Ġtokens Ġaltogether . ĠEach Ġindividual ĠGig aw ord Ġsub - cor p us Ġis Ġused Ġto Ġtrain Ġa Ġseparate Ġlanguage Ġmodel Ġand Ġso Ġis Ġthe ĠEnglish Ġside Ġof ĠFB IS . ĠThese Ġindividual Ġlanguage Ġmodels Ġare Ġthen Ġinterpol ated Ġto Ġbuild Ġone Ġsingle Ġlarge ĠLM , Ġvia Ġperplex ity Ġtuning Ġon Ġthe Ġdevelopment Ġset . </s> Our Ġtranslation Ġdevelopment Ġset Ġis ĠM TC Ġcorpus Ġversion Ġ1 Ġ( L DC 2002 T 01 ) Ġand Ġversion Ġ3 Ġ( L DC 2004 T 07 ). ĠThis Ġdevelopment Ġset Ġhas Ġ1 , 9 28 Ġsentence Ġpairs Ġin Ġtotal , Ġ49 K ĠChinese Ġtokens Ġand Ġ58 K ĠEnglish Ġtokens Ġon Ġaverage Ġacross Ġthe Ġfour Ġreference Ġtranslations . ĠWeight Ġtuning Ġis Ġdone Ġby Ġusing Ġthe Ġpair wise Ġranked Ġoptimization Ġ( PRO ) Ġalgorithm ĠB IB REF 22 Ġ, Ġwhich Ġis Ġalso Ġused Ġto Ġobtain Ġweights Ġof Ġthe Ġsparse Ġfeatures Ġto Ġhelp Ġdetermine Ġthe Ġre ordering . </s> For Ġdependency Ġsparse Ġfeatures , Ġwe Ġparse Ġthe ĠChinese Ġside Ġof Ġour Ġdevelopment Ġand Ġtest Ġsets Ġby Ġthe ĠMate Ġparser , Ġwhich Ġjointly Ġperforms ĠPOS Ġtagging Ġand Ġdependency Ġparsing ĠB IB REF 23 Ġ, Ġtrained Ġon ĠChinese ĠTree bank Ġ( CT B ) Ġversion Ġ8 . 0 Ġ( L DC 2013 T 21 ). </s> Our Ġtest Ġset Ġconsists Ġof Ġthe ĠN IST ĠMT Ġevaluation Ġsets Ġfrom Ġ2002 Ġto Ġ2006 Ġand Ġ2008 Ġ( L DC 2010 T 10 , ĠL DC 2010 T 11 , ĠL DC 2010 T 12 , ĠL DC 2010 T 14 , ĠL DC 2010 T 17 , ĠL DC 2010 T 21 ). </s> Bas eline ĠSystem </s> UTF 8 g bs n ĠWe Ġbuild Ġa Ġphrase - based Ġbaseline ĠSM T Ġsystem , Ġwhich Ġuses Ġnon - s parse Ġphrase - based Ġlex ical ized Ġre ordering Ġ( P BL R ), Ġin Ġwhich Ġthe Ġre ordering Ġprobability Ġdepends Ġon Ġthe Ġphrase Ġbeing Ġtranslated Ġand Ġits Ġposition Ġwith Ġrespect Ġto Ġthe Ġsource Ġposition Ġof Ġthe Ġpreviously Ġtranslated Ġphrase ĠB IB REF 1 Ġ, ĠB IB REF 24 Ġ, Ġand Ġnon - s parse Ġhierarchical Ġre ordering Ġ( HR ), Ġin Ġwhich Ġthe Ġprevious Ġunit Ġis Ġnot Ġonly Ġthe Ġprevious Ġphrase , Ġbut Ġthe Ġlargest Ġchunk Ġof Ġcontiguous Ġsource Ġphrases Ġhaving Ġcontiguous Ġtranslation ĠB IB REF 3 Ġ. ĠIn Ġaddition , Ġa Ġdistortion Ġlimit Ġis Ġset Ġsuch Ġthat Ġthe Ġre ordering Ġcannot Ġbe Ġlonger Ġthan Ġa Ġcertain Ġdistance . ĠWe Ġset Ġpunct uation Ġsymbols Ġas Ġre ordering Ġconstraint Ġacross Ġwhich Ġphrases Ġcannot Ġbe Ġre ordered , Ġas Ġthey Ġform Ġthe Ġnatural Ġboundaries Ġbetween Ġdifferent Ġclauses . ĠWe Ġalso Ġuse Ġn - best ĠMinimum ĠBay es ĠRisk Ġ( MB R ) Ġdecoding ĠB IB REF 25 Ġinstead Ġof Ġthe Ġdefault Ġmaximum Ġa - p oster ior i Ġ( MAP ) Ġdecoding . </s> Our ĠApproach </s> To Ġaccommodate Ġour Ġsparse Ġfeature Ġfunctions , Ġour ĠMoses Ġcode Ġhas Ġbeen Ġmodified Ġto Ġread Ġdependency - p ars ed Ġinput Ġsentences Ġand Ġincorporate Ġadditional Ġdecoding Ġfeatures Ġon Ġtop Ġof Ġour Ġbaseline , Ġnamely Ġdependency Ġdistortion Ġpenalty Ġ( D DP ) ĠB IB REF 13 Ġ, Ġsparse Ġdependency Ġpath Ġfeatures Ġ( Path ) ĠB IB REF 15 Ġ, Ġsparse Ġre ordering Ġorientation Ġfollowing Ġhierarchical Ġre ordering Ġorientation Ġ( SH R ) ĠB IB REF 4 Ġ, Ġand Ġour Ġsparse Ġdependency Ġswap Ġfeatures Ġ( DS ). ĠD DP Ġfeature Ġis Ġa Ġsingle Ġpenalty Ġfeature Ġsimilar Ġto Ġthe Ġdistortion Ġpenalty Ġfor Ġdistance - based Ġre ordering Ġmodel ĠB IB REF 0 Ġ, Ġwhile ĠPath , ĠSH R , Ġand ĠDS Ġare Ġsparse Ġfeatures , Ġeach Ġinstance Ġof Ġwhich Ġcaptures Ġa Ġspecific Ġphenomenon Ġduring Ġtranslation ĠB IB REF 17 Ġ. </s> We Ġalways Ġcouple ĠDS Ġwith ĠD DP . ĠHowever , Ġas Ġthe Ġoriginal ĠPath Ġfeatures Ġdid Ġnot Ġuse ĠD DP , Ġwe Ġexperiment Ġwith ĠPath Ġfeatures Ġin Ġone Ġsetting Ġthat Ġdoes Ġnot Ġincorporate ĠD DP Ġand Ġanother Ġthat Ġdoes . ĠThe ĠSH R Ġfeatures Ġare Ġnot Ġcoupled Ġwith ĠD DP , Ġfollowing Ġthe Ġoriginal Ġdesign , Ġas Ġthe Ġfeatures Ġare Ġnot Ġdefined Ġon Ġa Ġdependency - p ars ed Ġinput Ġsentence . </s> Exper imental ĠResults </s> The Ġtranslation Ġquality Ġof Ġthe Ġsystem Ġoutputs Ġis Ġmeasured Ġby Ġcase - ins ensitive ĠB LE U ĠB IB REF 26 Ġ, Ġfor Ġwhich Ġthe Ġbre vity Ġpenalty Ġis Ġcomputed Ġbased Ġon Ġthe Ġshortest Ġreference Ġ( N IST - BLE U ). ĠStatistical Ġsignificance Ġtesting Ġbetween Ġsystems Ġis Ġconducted Ġby Ġboot strap Ġres am pling ĠB IB REF 27 Ġ. </s> Table Ġ1 Ġshows Ġthe Ġexperimental Ġresults . ĠThe Ġdistortion Ġlimit Ġof Ġall Ġthe Ġsystems Ġis Ġset Ġto Ġ14 , Ġwhich Ġyields Ġthe Ġbest Ġresult Ġon Ġthe Ġdevelopment Ġset Ġfor Ġthe Ġbaseline Ġsystem . ĠAs Ġshown Ġin Ġthe Ġtable , Ġthe Ġsystem Ġwith Ġour ĠDS Ġfeatures Ġand ĠD DP Ġon Ġtop Ġof Ġthe Ġbaseline Ġis Ġable Ġto Ġimprove Ġover Ġthe Ġbaseline Ġsystem Ġwithout Ġand Ġwith ĠD DP , Ġby Ġ+ 1 . 09 Ġand Ġ+ 0 . 93 ĠB LE U Ġpoints Ġrespectively . ĠThe Ġindividual Ġcontribution Ġof Ġthe Ġother Ġdependency - based Ġfeatures Ġ( Path ), Ġwithout Ġor Ġwith ĠD DP , Ġis Ġinferior Ġto Ġour ĠDS Ġfeatures . ĠNevertheless , Ġcoupling Ġour ĠDS Ġfeatures Ġwith ĠPath Ġfeatures Ġyields Ġthe Ġbest Ġresult Ġ(+ 1 . 21 Ġand Ġ+ 1 . 05 ĠB LE U Ġpoints Ġover Ġthe Ġbaseline Ġwithout Ġand Ġwith ĠD DP ). </s> The ĠSH R Ġfeatures Ġyield Ġmore Ġimprovement Ġthan ĠPath Ġfeatures Ġwithout ĠD DP , Ġand Ġare Ġcomparable Ġto Ġthe ĠPath Ġfeatures Ġwith ĠD DP . ĠHowever , Ġour ĠDS Ġfeatures Ġyield Ġmore Ġimprovement Ġthan ĠSH R Ġfeatures . ĠOur Ġpreliminary Ġexperiments Ġindicate Ġthat Ġadding Ġthese Ġfeatures Ġon Ġtop Ġof Ġthe Ġsystem Ġwith ĠDS Ġdoes Ġnot Ġimprove Ġover Ġit . </s> Disc ussions </s> The Ġre ordering Ġorientation Ġmodels , Ġi . e ., ĠP BL R Ġand ĠHR , Ġonly Ġtake Ġinto Ġaccount Ġthe Ġphrase Ġpair Ġgenerated Ġby Ġa Ġhypothesis Ġand Ġnot Ġthe Ġrelated Ġword Ġproperties . ĠSp arse Ġre ordering Ġorientation Ġfeatures ĠB IB REF 4 Ġleverage Ġthis Ġby Ġcapturing Ġthe Ġprevious Ġphrase Ġ( or Ġcontiguous Ġchunk ) Ġproperties . ĠTherefore , Ġthey Ġare Ġable Ġto Ġimprove Ġover Ġthe Ġbaseline . ĠHowever , Ġas Ġthe Ġresults Ġsuggest , Ġdependency Ġparse Ġprovides Ġa Ġmore Ġuseful Ġguidance Ġto Ġre order Ġa Ġsource Ġsentence . </s> As Ġshown Ġin ĠFigure Ġ3 Ġ, Ġthe Ġbaseline Ġphrase - based ĠSM T Ġsystem Ġwith Ġthe Ġtwo Ġre ordering Ġorientation Ġmodels Ġ( P BL R Ġand ĠHR ) Ġproduces Ġan Ġincorrect Ġtranslation Ġoutput Ġthat ĠâĢ ľ the Ġexport Ġis Ġthe Ġmain Ġmarket âĢ Ŀ , Ġwhich Ġis Ġnot Ġwhat Ġthe Ġsource Ġsentence Ġmeans . ĠThis Ġis Ġalso Ġthe Ġcase Ġwith Ġthe Ġsystem Ġadded Ġwith Ġprior Ġre ordering Ġapproaches , Ġwhich Ġincludes ĠD DP ĠB IB REF 13 Ġ, ĠPath ĠB IB REF 15 Ġ, Ġtheir Ġcombination ĠD DP + Path , Ġand ĠSH R ĠB IB REF 4 Ġ. ĠMeanwhile , Ġour Ġre ordering Ġapproach Ġwith ĠDS Ġfeatures Ġis Ġable Ġto Ġoutput Ġthe Ġcorrect Ġtranslation ĠâĢ ľ the Ġmain Ġmarkets Ġfor Ġthe Ġexport âĢ Ŀ , Ġas Ġit Ġpenal izes Ġthe Ġswap Ġbetween Ġthe Ġsubject Ġhead ĠâĢ ľ market âĢ Ŀ Ġand Ġthe Ġcop ula , Ġwhich Ġshould Ġnot Ġbe Ġswapped . </s> The Ġdependency Ġparse Ġof Ġa Ġsentence Ġcan Ġcapture Ġthe Ġrelationship Ġamong Ġthe Ġwords Ġ( n odes ) Ġin Ġit . ĠBetween Ġeach Ġrelated Ġword Ġpair , Ġthere Ġis Ġa Ġdependency Ġlabel Ġwhich Ġspecifies Ġthe Ġhead - mod ifier Ġrelationship . ĠWhile Ġthe Ġhead - mod ifier Ġrelationships Ġin Ġa Ġsentence Ġhold Ġacross Ġlanguages , Ġtheir Ġordering Ġmay Ġdiffer . ĠFor Ġexample , Ġin ĠChinese , Ġpre pos itional Ġphrase Ġ( mod ifier ) Ġcomes Ġbefore Ġthe Ġpredicate Ġ( head ) Ġverb , Ġwhile Ġin ĠEnglish , Ġthey Ġcome Ġin Ġthe Ġother Ġway Ġround . ĠThis Ġparticular Ġclue , Ġprovided Ġby Ġthe Ġsource Ġdependency Ġparse , Ġis Ġuseful Ġin Ġdeciding Ġthe Ġword Ġorder Ġin Ġthe Ġtranslation Ġoutput , Ġcorresponding Ġto Ġa Ġword Ġin Ġthe Ġsource Ġsentence . </s> Comb ining Ġour Ġsparse Ġdependency Ġswap Ġfeatures Ġwith Ġsparse Ġdependency Ġpath Ġfeatures ĠB IB REF 15 Ġachieves Ġthe Ġbest Ġexperimental Ġresult . ĠThis Ġcan Ġbe Ġattributed Ġto Ġthe Ġcomplementary Ġnature Ġof Ġboth Ġtypes Ġof Ġsparse Ġfeatures . ĠPath Ġfeatures Ġare Ġable Ġto Ġcapture Ġthe Ġsubsequ ence Ġof Ġtranslation Ġoutput Ġphrases , Ġi . e ., Ġwhich Ġphrase Ġfollows Ġanother Ġbut Ġnot Ġthe Ġrelative Ġposition Ġ( right Ġor Ġleft ) Ġof Ġa Ġsource Ġdependency Ġword Ġwith Ġrespect Ġto Ġall Ġits Ġrelated Ġwords , Ġwhile Ġour Ġswap Ġfeatures Ġare Ġdesigned Ġto Ġcapture Ġthem . </s> Related ĠWork </s> Source Ġdependency Ġtrees Ġhave Ġbeen Ġexploited Ġin Ġphrase - based ĠSM T , Ġby Ġmodeling Ġtransition Ġamong Ġdependency Ġsubt rees Ġduring Ġtranslation ĠB IB REF 14 Ġ. ĠHowever , Ġthis Ġdoes Ġnot Ġtake Ġinto Ġaccount Ġthe Ġdependency Ġlabel Ġand Ġthe ĠPOS Ġtag . ĠAnother Ġwork Ġexploits Ġsource Ġand Ġtarget Ġdependency Ġtrees Ġfor Ġphrase - based ĠMT Ġoutput Ġre ranking ĠB IB REF 28 Ġ, Ġinstead Ġof Ġfor Ġtranslation Ġdecoding . </s> BI B REF 15 Ġchang disc rim in ative 2009 Ġintroduced Ġdependency Ġpath Ġas Ġa Ġsoft Ġconstraint Ġbased Ġon Ġthe Ġsequence Ġof Ġsource Ġdependency Ġlinks Ġtravers ed Ġin Ġphrase - based Ġtranslation . ĠIt Ġis Ġused Ġas Ġfeatures Ġon Ġa Ġmaximum Ġentropy Ġphrase Ġorientation Ġclass ifier , Ġwhose Ġprobability Ġoutput Ġis Ġused Ġas Ġa Ġdecoding Ġfeature Ġfunction . ĠAs Ġthe Ġpath Ġcan Ġbe Ġarbitrarily Ġlong , Ġit Ġmay Ġnot Ġbe Ġrepresented Ġsufficiently Ġin Ġthe Ġtraining Ġsamples . ĠOur Ġsparse Ġfeature Ġdefinition Ġcan Ġalleviate Ġthis Ġas Ġfeatures Ġare Ġdefined Ġon Ġtwo Ġwords . ĠIn Ġaddition , Ġcapturing Ġword Ġpairs Ġinstead Ġof Ġpaths Ġenables Ġincorporation Ġof Ġother Ġword Ġproperties Ġsuch Ġas ĠPOS Ġtags . </s> BI B REF 29 Ġhunt ere x pl o iting 2010 Ġproposed Ġa Ġprobability Ġmodel Ġto Ġcapture Ġthe Ġoffset Ġof Ġa Ġword Ġwith Ġrespect Ġto Ġits Ġhead Ġposition Ġin Ġphrase - based ĠMT . ĠTheir Ġmodel Ġdoes Ġnot Ġtake Ġinto Ġaccount Ġtwo Ġsibling Ġwords Ġsharing Ġthe Ġsame Ġhead . ĠThey Ġreported Ġnegative Ġresult . </s> Meanwhile , ĠB IB REF 30 Ġg aos oft 2011 Ġdefined Ġsoft Ġconstraints Ġon Ġhierarchical Ġphrase - based ĠMT , Ġwhich Ġproduce Ġtranslation Ġby Ġbottom - up Ġconstituency Ġparsing Ġalgorithm Ġinstead Ġof Ġbeam Ġsearch . ĠTheir Ġsoft Ġconstraint Ġis Ġalso Ġdefined Ġon Ġa Ġmaximum Ġentropy Ġclass ifier Ġinstead Ġof Ġsparse Ġfeatures . </s> Prior Ġwork Ġhas Ġalso Ġused Ġconstituency Ġparse Ġto Ġguide Ġre ordering , Ġby Ġmanually Ġdefining Ġpre - ordering Ġrules Ġto Ġre order Ġinput Ġsentences Ġinto Ġthe Ġtarget Ġlanguage Ġorder Ġbefore Ġtranslation ĠB IB REF 31 Ġ, ĠB IB REF 32 Ġ, Ġor Ġto Ġautomatically Ġlearn Ġthose Ġrules ĠB IB REF 33 Ġ, ĠB IB REF 34 Ġ. ĠEach Ġconstituency Ġparse Ġtree Ġnode Ġrepresents Ġthe Ġphrase Ġnesting Ġinstead Ġof Ġa Ġword , Ġresulting Ġin Ġa Ġdeeper Ġstructure , Ġwhich Ġis Ġgenerally Ġslower Ġto Ġproduce . </s> Conclusion </s> We Ġhave Ġpresented Ġa Ġre ordering Ġapproach Ġfor Ġphrase - based ĠSM T , Ġguided Ġby Ġsparse Ġdependency Ġswap Ġfeatures . ĠWe Ġhave Ġcontributed Ġa Ġnew Ġapproach Ġfor Ġlearning Ġand Ġperforming Ġre ordering Ġin Ġphrase - based ĠMT Ġby Ġthe Ġincorporation Ġof Ġdependency - based Ġfeatures . ĠFrom Ġour Ġexperiments , Ġwe Ġhave Ġshown Ġthat Ġutilizing Ġsource Ġdependency Ġparse Ġfor Ġre ordering Ġsentences Ġhelps Ġto Ġsignificantly Ġimprove Ġtranslation Ġquality Ġover Ġa Ġphrase - based Ġbaseline Ġsystem Ġwith Ġstate - of - the - art Ġre ordering Ġorientation Ġmodels . </s> Acknowled gments </s> This Ġresearch Ġis Ġsupported Ġby Ġthe ĠSingapore ĠNational ĠResearch ĠFoundation Ġunder Ġits ĠInternational ĠResearch ĠCentre Ġ@ ĠSingapore ĠFunding ĠInitiative Ġand Ġadministered Ġby Ġthe ĠID M ĠProgramme ĠOffice .
Introduction Reordering in machine translation (MT) is a crucial process to get the correct translation output word order given an input source sentence, as word order reflects meaning. It remains a major challenge, especially for language pairs with a significant word order difference. Phrase-based MT systems BIBREF0 generally adopt a reordering model that predicts reordering based on the span of a phrase and that of the adjacent phrase BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 . The above methods do not explicitly preserve the relationship between words in the source sentence, which reflects the sentence meaning. Word relationship in a sentence can be captured by its dependency parse tree, in which each word $w$ is a tree node connected to its head node $h_w$ , another word, indicating that the former is a dependent (child) of the latter. Dependency parsing has been used for reordering in statistical machine translation (SMT). Its usage is well-known in the pre-ordering approach, where a source sentence is reordered before the actual translation. Dependency-based pre-ordering can be performed either by a rule-based approach based on manually specified human linguistic knowledge BIBREF5 , BIBREF6 , or by a learning approach BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 . Dependency parsing has also been used in reordering approaches integrated with decoding to determine the next source phrase to translate after translating the current source phrase BIBREF13 , BIBREF14 , BIBREF15 . In this paper, we propose a novel reordering approach integrated with translation. We propose sparse feature functions based on the pre-ordering rules of BIBREF6 . However, in contrast to the manual rule-based pre-ordering approach of BIBREF6 , the weights of our sparse feature functions are automatically learned and used during the actual translation process, without an explicit pre-ordering step. Our approach detects and exploits the reordering of each dependency word pair in the source sentence during phrase-based decoding. Dependency Word Pair Features We define a set of sparse features based on dependency tree word pairs to be learned and used in a phrase-based SMT beam search decoding algorithm. Capturing Word Pair Ordering in Phrase-Based Beam Search The phrase-based approach is a state-of-the-art approach for SMT, taking phrases, defined as a sequence of one or more words, as its translation units. It is performed by a beam search algorithm BIBREF16 , in which the search process produces translation from left to right in the translation output. The search is organized into hypotheses, each of which represents an input sentence phrase covered and its possible translation. As the beam search can choose input phrases in any order, the target-language phrase sequence in the translation output may not follow the original source sentence order. The sequence determines the translation output order and enables translation output reordering for language pairs with differences in word order, such as Chinese and English. When a word $f_i$ in a source sentence ${\bf f}=f_{1}^{N}$ is covered by a hypothesis, it is known that the words $\lbrace f_{l}|f_{l} \in f_{1}^{i-1} \wedge \lnot {translated(f_l)}\rbrace $ on the left of $f_i$ and the words $\lbrace f_{r}|f_{r} \in f_{i+1}^{N} \wedge \lnot {translated(f_r)}\rbrace $ on the right of $f_i$ that have not been translated will be translated after (appearing on the right of) the translation of $f_i$ . As $f_l$ is before $f_i$ in the source sentence, but the translation of $f_l$ is after the translation of ${\bf f}=f_{1}^{N}$0 , the translations of ${\bf f}=f_{1}^{N}$1 and ${\bf f}=f_{1}^{N}$2 are swapped. Meanwhile, ${\bf f}=f_{1}^{N}$3 is after ${\bf f}=f_{1}^{N}$4 both in the source sentence and in the translation, therefore the translations of ${\bf f}=f_{1}^{N}$5 and ${\bf f}=f_{1}^{N}$6 are in-order. Internally within a phrase, the ordering of each of its words in the translation depends on the phrasal word alignment, which is stored in the phrase table. As each word in the source sentence is a node of the source dependency parse tree, the above notion can be used for reordering based on the source dependency parse tree. Instead of capturing all pairwise relations, we are only interested in the relations between a word and its related words, defined collectively as its head, sibling, and child words in the dependency parse tree. Dependency Swap Features We define our dependency swap features, following the rule template definition for dependency swap rules BIBREF6 , which defines rule instances based on the word pairs with head-child or sibling relationship. However, the difference is that our approach does not require manually specifying which dependency labels are the conditions to swap words, but learns them automatically. In our approach, each rule instance based on the above template becomes a Boolean sparse feature function BIBREF17 . The function parameters are the word pair specification and output order. While BIBREF6 caidependency-based2014 defined the rules only by the dependency labels, we define our feature functions for each word pair by the dependency labels, the POS tags, and the combination of both, resulting in a group of four feature functions for every word pair ordering. The dependency link label of a word $x$ is defined as the label of the link connecting $x$ to its head word. Henceforth for each word $x$ , $L(x)$ and $T(x)$ denote the dependency link label and POS tag of $x$ respectively. Following the dependency swap rule template, we define two types of feature function templates, namely head-child and sibling. The head-child feature functions are equal to 1 if a head word $x_h$ and its child word $x_c$ (where $x_h$ is on the $p\in \lbrace left,right\rbrace $ of $x_c$ in the source sentence) take a certain ordering $o$ (which can be in-order ( $io$ ) or swapped ( $sw$ )) in the translation output, and 0 otherwise. This group of feature functions is defined as $$H_{hc}(x_h,x_c,p,o) = \left[\begin{array}{c} h_{hc}(L(x_h),L(x_c),p,o) \\ h_{hc}(T(x_h),T(x_c),p,o) \\ h_{hc}(L(x_h),T(x_c),p,o) \\ h_{hc}(T(x_h),L(x_c),p,o) \end{array}\right]$$   (Eq. 6) Similarly, the sibling feature functions are equal to 1 if siblings $x_l$ and $x_r$ ( $x_l$ on the left of $x_r$ in the source sentence) take a certain ordering $o$ in the translation output, and 0 otherwise. This group of feature functions is defined as $$H_{sib}(x_l,x_r,o) = \left[\begin{array}{c} h_{sib}(L(x_l),L(x_r),o) \\ h_{sib}(T(x_l),T(x_r),o) \\ h_{sib}(L(x_l),T(x_r),o) \\ h_{sib}(T(x_l),L(x_r),o) \end{array}\right]$$   (Eq. 7) UTF8gbsn Each dependency swap feature that is set to 1 when a hypothesis is generated captures two source words, one is covered by the current hypothesis, while the other has not yet been translated. These word pairs have either head-child or sibling relationship as defined above. Figure 1 is an illustration of how word order is detected during beam search decoding and the swap features involved. When “发表 讲话 $\rightarrow $ made a speech” is translated after “佐科威 $\rightarrow $ Jokowi”, it is known that the head “发表” is before the child “讲话” ( $p=left$ ) and their translations also follow the same word order, setting the value of the following four features to 1: $H_{hc}(\text{发表},\text{讲话},left,io) = \left[\begin{array}{c} h_{hc}(root,dobj,left,io) \\ h_{hc}(VV,NN,left,io) \\ h_{hc}(root,NN,left,io) \\ h_{hc}(VV,dobj,left,io) \end{array}\right]$ On the other hand, the head “发表” is after the child “在” in the source sentence ( $p=right$ ), but “在” has not been translated. Therefore, the translation of “发表” will be swapped with that of “在”, setting the four features in $H_{hc}(\text{发表},\text{在},right,sw)$ to 1. Similarly, “讲话” is after its sibling “在” in the source sentence, but “在” has not been translated, resulting in the translation of the two words being swapped and setting the four features in $H_{sib}(\text{在},\text{讲话},sw)$ to 1. The features for “佐科威” such as $H_{hc}(\text{发表},\text{佐科威},right,io)$ and $H_{sib}(\text{佐科威},\text{讲话},io)$ are not set to 1 when the hypothesis “发表 讲话 $\rightarrow $ made a speech” is generated. Those features were set to 1 with the previous hypothesis “佐科威 $\rightarrow $ Jokowi”. Dependency Distortion Penalty To encourage the translation output that conforms to the dependency parse structure, we impose a penalty feature that discourages translation candidates not conforming to the dependency parse subtree BIBREF13 . This assigns a penalty if the translation of the current phrase results in a source dependency parse subtree to be split in the output translation. UTF8gbsn The translation output shown in Figure 1 has each dependency parse subtree grouped together, therefore incurring no penalty. However, there can be a case when the translation has produced “Jokowi made a speech in”, then translate “昨天” to “yesterday”. This translation will break the cohesion of the source dependency parse subtree “在 北京” and is bad. This is the case when a dependency distortion penalty is incurred. Modified Future Cost Phrase-based SMT beam search decoding involves future cost, not accumulated over the translation hypotheses but used to reduce search space BIBREF16 . This predicts the cost to translate the remaining untranslated phrases. Based on our dependency swap features, we incorporate the future cost for each untranslated dependency word pairs. The future cost assumes that the untranslated words will be ordered in the most likely ordering. However, if an untranslated word $x$ is an ancestor (in the dependency parse tree) of a word covered by the current hypothesis, the future cost assumes that $x$ precedes all of its related words. This follows the assumption that each subtree will have a contiguous translation BIBREF13 . Other Sparse Features This section describes other sparse features from previous work to compare to our method. Sparse Reordering Orientation Features We incorporate sparse reordering orientation features BIBREF4 . The features are derived from the reordering orientation model, capturing the source position of the current phrase being translated with respect to the previously translated phrase(s), for which three orientation types are defined: Monotone (M), if the current phrase and the previously translated unit are adjacent in the input sentence and the former follows the latter in it, e.g., “Beijing” with respect to the previous phrase “in” in Figure 1 . Swapped (S), if the current phrase and the previously translated unit are adjacent in the input sentence but the former precedes the latter in it (example below). Discontinuous (D), if the current phrase and the previously translated unit are not adjacent in the input sentence, e.g., “made a speech” with respect to the previous phrase “Jokowi” in Figure 1 . UTF8gbsn BIBREF4 cherryimproved2013 designed his sparse reordering orientation model following the hierarchical reordering (HR) model BIBREF3 , capturing the relative position of the current phrase (covered by the current hypothesis) with respect to the largest chunk of contiguous source phrases that form a contiguous translation before this phrase. Therefore, in Figure 1 , when the decoding produces a phrase “yesterday” after “Beijing”, the orientation of “yesterday” is swapped instead of discontinuous, as “made a speech in Beijing” is formed by contiguous phrases “在 北京 发表 讲话”, which is adjacent to “昨天”. While the original (non-sparse) reordering orientation model is based on the phrase orientation probability in the parallel training data and defines a single feature function on it, the sparse model defines one feature function for each reordering phenomenon during decoding. We define the sparse feature functions taking into account the phrase orientation $o\in \lbrace M,S,D\rbrace $ during decoding and important locations $loc$ pertaining to the current phrase and the previous phrase with the following template: $$h_{s\_hr}(loc:=rep(loc),o)$$   (Eq. 14) where locations $loc$ are the first and last words of the current source phrase ( $s_{first}$ , $s_{last}$ ), the previously translated unit ( $p_{first}$ , $p_{last}$ ), i.e., the largest contiguous chunk of phrases forming a contiguous translation, and the span between the current and the previous source phrase, or gap ( $g_{first}$ , $g_{last}$ ) only for discontinuous orientation. Each word in $loc$ is represented, $rep(loc)$ , by its POS tag, and the surface lexical form if it belongs to the 80-most frequent words in the training data. UTF8gbsn Assuming that only “在” belongs to the top-80 words, when the phrase “yesterday” is generated after “Beijing”, the sparse reordering orientation features that are equal to 1 are shown in Figure 2 . While the approach leverages the reordering orientation model by defining features on the context information, not just the current phrase, it does not capture dependency relation, by which the important relation between words in the source sentence is captured. Therefore, we introduce our sparse dependency swap features to enable the translation system to arrive at reordering decisions based on the source dependency relations. Dependency Path Features We also utilize dependency path features BIBREF15 for phrase-based SMT, defined over the shortest path of dependency parse tree links bridging the current and the previous source phrase. BIBREF15 changdiscriminative2009 defined the dependency path features on a maximum entropy phrase orientation classifier, trained on their word-aligned parallel text and labeled by the two possible phrase orderings in the translation output: in-order and swapped. Meanwhile, we use the features as sparse decoding features, with the following template: $$h_{path}(shortest\_path(p_{last},s_{first});o)$$   (Eq. 18) where $o \in \lbrace in\_order,swapped\rbrace $ denotes the orientation of the two phrases in the translation. Given a source sentence and its translation output, a path is defined between the last word of the previous source phrase $p_{last}$ and the first word of the current source phrase $s_{first}$ . Path traversal is always from left to right based on the source sentence word position. Therefore, if the current source phrase being translated is to the right of the previous source phrase, the traversal is from $p_{last}$ to $s_{first}$ . Otherwise, if the current source phrase is to the left of the previous source phrase, it is from $s_{first}$ to $p_{last}$ . In addition, path edges going against the dependency label arrow are distinguished from those following the arrow. UTF8gbsn As an example, in Figure 1 , when the translation generates “发表 讲话 $\rightarrow $ made a speech” after “佐科威 $\rightarrow $ Jokowi”, the path is from “佐科威” to “发表”, consisting of a direct link following the arrow of $nsubj$ , resulting in the feature $h_{path}(nsubj,in\_order)=1$ . However, when the translation generates “昨天 $\rightarrow $ yesterday” after “北京 $\rightarrow $ Beijing”, as “昨天” is before “北京” in the source sentence, the traversal is from “昨天” to “北京”, consisting of the link sequence $tmod,prepR,pobjR$ . As it goes against the arrows of $prep$ and $pobj$ , the suffix $R$ is added to distinguish it. This results in the feature $\rightarrow $0 . The approach leverages the phrase-based reordering by guiding the ordering of two adjacent phrases using dependency parse. However, the features do not capture the pairwise ordering of every word with its related word, as the features are induced only when the words are used in the two adjacent translation phrases. Data Set and Toolkits We built a phrase-based Chinese-to-English SMT system by using Moses BIBREF18 . Our parallel training text is a collection of parallel corpora from LDC, which we divide into older corpora and newer corpora. Due to the dominant older data, we duplicate the newer corpora of various domains by 10 times to achieve better domain balance. To reduce the possibility of alignment errors, parallel sentences in the corpora that are longer than 85 words in either Chinese (after word segmentation) or English are discarded. In the end, the final parallel text consists of around 8.8M sentence pairs, 228M Chinese tokens, and 254M English tokens (a token can be a word or punctuation symbol). We also added two dictionaries by concatenating them to our training parallel text. The total number of words in these two corpora is 1.81M for Chinese and 2.03M for English. All Chinese sentences in the training, development, and test data are first word-segmented using a maximum entropy-based Chinese word segmenter BIBREF19 trained on the Chinese Treebank (CTB) scheme. Then the parallel corpus is word-aligned by GIZA++ BIBREF20 using IBM Models 1, 3, and 4 BIBREF21 . For building the phrase table, which follows word alignment, the maximum length of a phrase pair is set to 7 words for both the source and target sides. The language model (LM) is a 5-gram model trained on the English side of the FBIS parallel corpus (LDC2003E14) and the monolingual corpus English Gigaword version 4 (LDC2009T13), consisting of 107M sentences and 3.8G tokens altogether. Each individual Gigaword sub-corpus is used to train a separate language model and so is the English side of FBIS. These individual language models are then interpolated to build one single large LM, via perplexity tuning on the development set. Our translation development set is MTC corpus version 1 (LDC2002T01) and version 3 (LDC2004T07). This development set has 1,928 sentence pairs in total, 49K Chinese tokens and 58K English tokens on average across the four reference translations. Weight tuning is done by using the pairwise ranked optimization (PRO) algorithm BIBREF22 , which is also used to obtain weights of the sparse features to help determine the reordering. For dependency sparse features, we parse the Chinese side of our development and test sets by the Mate parser, which jointly performs POS tagging and dependency parsing BIBREF23 , trained on Chinese Treebank (CTB) version 8.0 (LDC2013T21). Our test set consists of the NIST MT evaluation sets from 2002 to 2006 and 2008 (LDC2010T10, LDC2010T11, LDC2010T12, LDC2010T14, LDC2010T17, LDC2010T21). Baseline System UTF8gbsn We build a phrase-based baseline SMT system, which uses non-sparse phrase-based lexicalized reordering (PBLR), in which the reordering probability depends on the phrase being translated and its position with respect to the source position of the previously translated phrase BIBREF1 , BIBREF24 , and non-sparse hierarchical reordering (HR), in which the previous unit is not only the previous phrase, but the largest chunk of contiguous source phrases having contiguous translation BIBREF3 . In addition, a distortion limit is set such that the reordering cannot be longer than a certain distance. We set punctuation symbols as reordering constraint across which phrases cannot be reordered, as they form the natural boundaries between different clauses. We also use n-best Minimum Bayes Risk (MBR) decoding BIBREF25 instead of the default maximum a-posteriori (MAP) decoding. Our Approach To accommodate our sparse feature functions, our Moses code has been modified to read dependency-parsed input sentences and incorporate additional decoding features on top of our baseline, namely dependency distortion penalty (DDP) BIBREF13 , sparse dependency path features (Path) BIBREF15 , sparse reordering orientation following hierarchical reordering orientation (SHR) BIBREF4 , and our sparse dependency swap features (DS). DDP feature is a single penalty feature similar to the distortion penalty for distance-based reordering model BIBREF0 , while Path, SHR, and DS are sparse features, each instance of which captures a specific phenomenon during translation BIBREF17 . We always couple DS with DDP. However, as the original Path features did not use DDP, we experiment with Path features in one setting that does not incorporate DDP and another that does. The SHR features are not coupled with DDP, following the original design, as the features are not defined on a dependency-parsed input sentence. Experimental Results The translation quality of the system outputs is measured by case-insensitive BLEU BIBREF26 , for which the brevity penalty is computed based on the shortest reference (NIST-BLEU). Statistical significance testing between systems is conducted by bootstrap resampling BIBREF27 . Table 1 shows the experimental results. The distortion limit of all the systems is set to 14, which yields the best result on the development set for the baseline system. As shown in the table, the system with our DS features and DDP on top of the baseline is able to improve over the baseline system without and with DDP, by +1.09 and +0.93 BLEU points respectively. The individual contribution of the other dependency-based features (Path), without or with DDP, is inferior to our DS features. Nevertheless, coupling our DS features with Path features yields the best result (+1.21 and +1.05 BLEU points over the baseline without and with DDP). The SHR features yield more improvement than Path features without DDP, and are comparable to the Path features with DDP. However, our DS features yield more improvement than SHR features. Our preliminary experiments indicate that adding these features on top of the system with DS does not improve over it. Discussions The reordering orientation models, i.e., PBLR and HR, only take into account the phrase pair generated by a hypothesis and not the related word properties. Sparse reordering orientation features BIBREF4 leverage this by capturing the previous phrase (or contiguous chunk) properties. Therefore, they are able to improve over the baseline. However, as the results suggest, dependency parse provides a more useful guidance to reorder a source sentence. As shown in Figure 3 , the baseline phrase-based SMT system with the two reordering orientation models (PBLR and HR) produces an incorrect translation output that “the export is the main market”, which is not what the source sentence means. This is also the case with the system added with prior reordering approaches, which includes DDP BIBREF13 , Path BIBREF15 , their combination DDP+Path, and SHR BIBREF4 . Meanwhile, our reordering approach with DS features is able to output the correct translation “the main markets for the export”, as it penalizes the swap between the subject head “market” and the copula, which should not be swapped. The dependency parse of a sentence can capture the relationship among the words (nodes) in it. Between each related word pair, there is a dependency label which specifies the head-modifier relationship. While the head-modifier relationships in a sentence hold across languages, their ordering may differ. For example, in Chinese, prepositional phrase (modifier) comes before the predicate (head) verb, while in English, they come in the other way round. This particular clue, provided by the source dependency parse, is useful in deciding the word order in the translation output, corresponding to a word in the source sentence. Combining our sparse dependency swap features with sparse dependency path features BIBREF15 achieves the best experimental result. This can be attributed to the complementary nature of both types of sparse features. Path features are able to capture the subsequence of translation output phrases, i.e., which phrase follows another but not the relative position (right or left) of a source dependency word with respect to all its related words, while our swap features are designed to capture them. Related Work Source dependency trees have been exploited in phrase-based SMT, by modeling transition among dependency subtrees during translation BIBREF14 . However, this does not take into account the dependency label and the POS tag. Another work exploits source and target dependency trees for phrase-based MT output reranking BIBREF28 , instead of for translation decoding. BIBREF15 changdiscriminative2009 introduced dependency path as a soft constraint based on the sequence of source dependency links traversed in phrase-based translation. It is used as features on a maximum entropy phrase orientation classifier, whose probability output is used as a decoding feature function. As the path can be arbitrarily long, it may not be represented sufficiently in the training samples. Our sparse feature definition can alleviate this as features are defined on two words. In addition, capturing word pairs instead of paths enables incorporation of other word properties such as POS tags. BIBREF29 hunterexploiting2010 proposed a probability model to capture the offset of a word with respect to its head position in phrase-based MT. Their model does not take into account two sibling words sharing the same head. They reported negative result. Meanwhile, BIBREF30 gaosoft2011 defined soft constraints on hierarchical phrase-based MT, which produce translation by bottom-up constituency parsing algorithm instead of beam search. Their soft constraint is also defined on a maximum entropy classifier instead of sparse features. Prior work has also used constituency parse to guide reordering, by manually defining pre-ordering rules to reorder input sentences into the target language order before translation BIBREF31 , BIBREF32 , or to automatically learn those rules BIBREF33 , BIBREF34 . Each constituency parse tree node represents the phrase nesting instead of a word, resulting in a deeper structure, which is generally slower to produce. Conclusion We have presented a reordering approach for phrase-based SMT, guided by sparse dependency swap features. We have contributed a new approach for learning and performing reordering in phrase-based MT by the incorporation of dependency-based features. From our experiments, we have shown that utilizing source dependency parse for reordering sentences helps to significantly improve translation quality over a phrase-based baseline system with state-of-the-art reordering orientation models. Acknowledgments This research is supported by the Singapore National Research Foundation under its International Research Centre @ Singapore Funding Initiative and administered by the IDM Programme Office.
Did they only experiment with one language pair?
1f07e837574519f2b696f3d6fa3230af0b931e5d
Did Ġthey Ġonly Ġexperiment Ġwith Ġone Ġlanguage Ġpair ?
Yes
<s> Did Ġthey Ġonly Ġexperiment Ġwith Ġone Ġlanguage Ġpair ? </s> Introduction </s> Re ordering Ġin Ġmachine Ġtranslation Ġ( MT ) Ġis Ġa Ġcrucial Ġprocess Ġto Ġget Ġthe Ġcorrect Ġtranslation Ġoutput Ġword Ġorder Ġgiven Ġan Ġinput Ġsource Ġsentence , Ġas Ġword Ġorder Ġreflects Ġmeaning . ĠIt Ġremains Ġa Ġmajor Ġchallenge , Ġespecially Ġfor Ġlanguage Ġpairs Ġwith Ġa Ġsignificant Ġword Ġorder Ġdifference . ĠPh rase - based ĠMT Ġsystems ĠB IB REF 0 Ġgenerally Ġadopt Ġa Ġre ordering Ġmodel Ġthat Ġpredicts Ġre ordering Ġbased Ġon Ġthe Ġspan Ġof Ġa Ġphrase Ġand Ġthat Ġof Ġthe Ġadjacent Ġphrase ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ. </s> The Ġabove Ġmethods Ġdo Ġnot Ġexplicitly Ġpreserve Ġthe Ġrelationship Ġbetween Ġwords Ġin Ġthe Ġsource Ġsentence , Ġwhich Ġreflects Ġthe Ġsentence Ġmeaning . ĠWord Ġrelationship Ġin Ġa Ġsentence Ġcan Ġbe Ġcaptured Ġby Ġits Ġdependency Ġparse Ġtree , Ġin Ġwhich Ġeach Ġword Ġ$ w $ Ġis Ġa Ġtree Ġnode Ġconnected Ġto Ġits Ġhead Ġnode Ġ$ h _ w $ Ġ, Ġanother Ġword , Ġindicating Ġthat Ġthe Ġformer Ġis Ġa Ġdependent Ġ( child ) Ġof Ġthe Ġlatter . </s> D epend ency Ġparsing Ġhas Ġbeen Ġused Ġfor Ġre ordering Ġin Ġstatistical Ġmachine Ġtranslation Ġ( SM T ). ĠIts Ġusage Ġis Ġwell - known Ġin Ġthe Ġpre - ordering Ġapproach , Ġwhere Ġa Ġsource Ġsentence Ġis Ġre ordered Ġbefore Ġthe Ġactual Ġtranslation . ĠDepend ency - based Ġpre - ordering Ġcan Ġbe Ġperformed Ġeither Ġby Ġa Ġrule - based Ġapproach Ġbased Ġon Ġmanually Ġspecified Ġhuman Ġlinguistic Ġknowledge ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, Ġor Ġby Ġa Ġlearning Ġapproach ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġ, ĠB IB REF 12 Ġ. ĠDepend ency Ġparsing Ġhas Ġalso Ġbeen Ġused Ġin Ġre ordering Ġapproaches Ġintegrated Ġwith Ġdecoding Ġto Ġdetermine Ġthe Ġnext Ġsource Ġphrase Ġto Ġtranslate Ġafter Ġtranslating Ġthe Ġcurrent Ġsource Ġphrase ĠB IB REF 13 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ. </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġnovel Ġre ordering Ġapproach Ġintegrated Ġwith Ġtranslation . ĠWe Ġpropose Ġsparse Ġfeature Ġfunctions Ġbased Ġon Ġthe Ġpre - ordering Ġrules Ġof ĠB IB REF 6 Ġ. ĠHowever , Ġin Ġcontrast Ġto Ġthe Ġmanual Ġrule - based Ġpre - ordering Ġapproach Ġof ĠB IB REF 6 Ġ, Ġthe Ġweights Ġof Ġour Ġsparse Ġfeature Ġfunctions Ġare Ġautomatically Ġlearned Ġand Ġused Ġduring Ġthe Ġactual Ġtranslation Ġprocess , Ġwithout Ġan Ġexplicit Ġpre - ordering Ġstep . ĠOur Ġapproach Ġdetects Ġand Ġexploits Ġthe Ġre ordering Ġof Ġeach Ġdependency Ġword Ġpair Ġin Ġthe Ġsource Ġsentence Ġduring Ġphrase - based Ġdecoding . </s> D epend ency ĠWord ĠPair ĠFeatures </s> We Ġdefine Ġa Ġset Ġof Ġsparse Ġfeatures Ġbased Ġon Ġdependency Ġtree Ġword Ġpairs Ġto Ġbe Ġlearned Ġand Ġused Ġin Ġa Ġphrase - based ĠSM T Ġbeam Ġsearch Ġdecoding Ġalgorithm . </s> Capt uring ĠWord ĠPair ĠOrder ing Ġin ĠPh rase - Based ĠBeam ĠSearch </s> The Ġphrase - based Ġapproach Ġis Ġa Ġstate - of - the - art Ġapproach Ġfor ĠSM T , Ġtaking Ġphrases , Ġdefined Ġas Ġa Ġsequence Ġof Ġone Ġor Ġmore Ġwords , Ġas Ġits Ġtranslation Ġunits . ĠIt Ġis Ġperformed Ġby Ġa Ġbeam Ġsearch Ġalgorithm ĠB IB REF 16 Ġ, Ġin Ġwhich Ġthe Ġsearch Ġprocess Ġproduces Ġtranslation Ġfrom Ġleft Ġto Ġright Ġin Ġthe Ġtranslation Ġoutput . ĠThe Ġsearch Ġis Ġorganized Ġinto Ġhypotheses , Ġeach Ġof Ġwhich Ġrepresents Ġan Ġinput Ġsentence Ġphrase Ġcovered Ġand Ġits Ġpossible Ġtranslation . </s> As Ġthe Ġbeam Ġsearch Ġcan Ġchoose Ġinput Ġphrases Ġin Ġany Ġorder , Ġthe Ġtarget - language Ġphrase Ġsequence Ġin Ġthe Ġtranslation Ġoutput Ġmay Ġnot Ġfollow Ġthe Ġoriginal Ġsource Ġsentence Ġorder . ĠThe Ġsequence Ġdetermines Ġthe Ġtranslation Ġoutput Ġorder Ġand Ġenables Ġtranslation Ġoutput Ġre ordering Ġfor Ġlanguage Ġpairs Ġwith Ġdifferences Ġin Ġword Ġorder , Ġsuch Ġas ĠChinese Ġand ĠEnglish . </s> When Ġa Ġword Ġ$ f _ i $ Ġin Ġa Ġsource Ġsentence Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ Ġis Ġcovered Ġby Ġa Ġhypothesis , Ġit Ġis Ġknown Ġthat Ġthe Ġwords Ġ$\ l brace Ġf _{ l } | f _{ l } Ġ\ in Ġf _{ 1 } ^{ i - 1 } Ġ\ w edge Ġ\ l not Ġ{ trans lated ( f _ l ) }\ r brace Ġ$ Ġon Ġthe Ġleft Ġof Ġ$ f _ i $ Ġand Ġthe Ġwords Ġ$\ l brace Ġf _{ r } | f _{ r } Ġ\ in Ġf _{ i + 1 } ^{ N } Ġ\ w edge Ġ\ l not Ġ{ trans lated ( f _ r ) }\ r brace Ġ$ Ġon Ġthe Ġright Ġof Ġ$ f _ i $ Ġthat Ġhave Ġnot Ġbeen Ġtranslated Ġwill Ġbe Ġtranslated Ġafter Ġ( app earing Ġon Ġthe Ġright Ġof ) Ġthe Ġtranslation Ġof Ġ$ f _ i $ Ġ. ĠAs Ġ$ f _ l $ Ġis Ġbefore Ġ$ f _ i $ Ġin Ġthe Ġsource Ġsentence , Ġbut Ġthe Ġtranslation Ġof Ġ$ f _ l $ Ġis Ġafter Ġthe Ġtranslation Ġof Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 0 Ġ, Ġthe Ġtranslations Ġof Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 1 Ġand Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 2 Ġare Ġswapped . ĠMeanwhile , Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 3 Ġis Ġafter Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 4 Ġboth Ġin Ġthe Ġsource Ġsentence Ġand Ġin Ġthe Ġtranslation , Ġtherefore Ġthe Ġtranslations Ġof Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 5 Ġand Ġ${ \ bf Ġf } = f _{ 1 } ^{ N } $ 6 Ġare Ġin - order . ĠIntern ally Ġwithin Ġa Ġphrase , Ġthe Ġordering Ġof Ġeach Ġof Ġits Ġwords Ġin Ġthe Ġtranslation Ġdepends Ġon Ġthe Ġphr as al Ġword Ġalignment , Ġwhich Ġis Ġstored Ġin Ġthe Ġphrase Ġtable . </s> As Ġeach Ġword Ġin Ġthe Ġsource Ġsentence Ġis Ġa Ġnode Ġof Ġthe Ġsource Ġdependency Ġparse Ġtree , Ġthe Ġabove Ġnotion Ġcan Ġbe Ġused Ġfor Ġre ordering Ġbased Ġon Ġthe Ġsource Ġdependency Ġparse Ġtree . ĠInstead Ġof Ġcapturing Ġall Ġpair wise Ġrelations , Ġwe Ġare Ġonly Ġinterested Ġin Ġthe Ġrelations Ġbetween Ġa Ġword Ġand Ġits Ġrelated Ġwords , Ġdefined Ġcollectively Ġas Ġits Ġhead , Ġsibling , Ġand Ġchild Ġwords Ġin Ġthe Ġdependency Ġparse Ġtree . </s> D epend ency ĠSwap ĠFeatures </s> We Ġdefine Ġour Ġdependency Ġswap Ġfeatures , Ġfollowing Ġthe Ġrule Ġtemplate Ġdefinition Ġfor Ġdependency Ġswap Ġrules ĠB IB REF 6 Ġ, Ġwhich Ġdefines Ġrule Ġinstances Ġbased Ġon Ġthe Ġword Ġpairs Ġwith Ġhead - child Ġor Ġsibling Ġrelationship . ĠHowever , Ġthe Ġdifference Ġis Ġthat Ġour Ġapproach Ġdoes Ġnot Ġrequire Ġmanually Ġspecifying Ġwhich Ġdependency Ġlabels Ġare Ġthe Ġconditions Ġto Ġswap Ġwords , Ġbut Ġlearns Ġthem Ġautomatically . </s> In Ġour Ġapproach , Ġeach Ġrule Ġinstance Ġbased Ġon Ġthe Ġabove Ġtemplate Ġbecomes Ġa ĠBoolean Ġsparse Ġfeature Ġfunction ĠB IB REF 17 Ġ. ĠThe Ġfunction Ġparameters Ġare Ġthe Ġword Ġpair Ġspecification Ġand Ġoutput Ġorder . ĠWhile ĠB IB REF 6 Ġca ide pend ency - based 2014 Ġdefined Ġthe Ġrules Ġonly Ġby Ġthe Ġdependency Ġlabels , Ġwe Ġdefine Ġour Ġfeature Ġfunctions Ġfor Ġeach Ġword Ġpair Ġby Ġthe Ġdependency Ġlabels , Ġthe ĠPOS Ġtags , Ġand Ġthe Ġcombination Ġof Ġboth , Ġresulting Ġin Ġa Ġgroup Ġof Ġfour Ġfeature Ġfunctions Ġfor Ġevery Ġword Ġpair Ġordering . ĠThe Ġdependency Ġlink Ġlabel Ġof Ġa Ġword Ġ$ x $ Ġis Ġdefined Ġas Ġthe Ġlabel Ġof Ġthe Ġlink Ġconnecting Ġ$ x $ Ġto Ġits Ġhead Ġword . ĠHence forth Ġfor Ġeach Ġword Ġ$ x $ Ġ, Ġ$ L ( x ) $ Ġand Ġ$ T ( x ) $ Ġdenote Ġthe Ġdependency Ġlink Ġlabel Ġand ĠPOS Ġtag Ġof Ġ$ x $ Ġrespectively . </s> Following Ġthe Ġdependency Ġswap Ġrule Ġtemplate , Ġwe Ġdefine Ġtwo Ġtypes Ġof Ġfeature Ġfunction Ġtemplates , Ġnamely Ġhead - child Ġand Ġsibling . ĠThe Ġhead - child Ġfeature Ġfunctions Ġare Ġequal Ġto Ġ1 Ġif Ġa Ġhead Ġword Ġ$ x _ h $ Ġand Ġits Ġchild Ġword Ġ$ x _ c $ Ġ( where Ġ$ x _ h $ Ġis Ġon Ġthe Ġ$ p \ in Ġ\ l brace Ġleft , right \ r brace Ġ$ Ġof Ġ$ x _ c $ Ġin Ġthe Ġsource Ġsentence ) Ġtake Ġa Ġcertain Ġordering Ġ$ o $ Ġ( which Ġcan Ġbe Ġin - order Ġ( Ġ$ io $ Ġ) Ġor Ġswapped Ġ( Ġ$ sw $ Ġ)) Ġin Ġthe Ġtranslation Ġoutput , Ġand Ġ0 Ġotherwise . ĠThis Ġgroup Ġof Ġfeature Ġfunctions Ġis Ġdefined Ġas </s> $$ H _{ h c } ( x _ h , x _ c , p , o ) Ġ= Ġ\ left [ \ begin { array }{ c } Ġh _{ h c } ( L ( x _ h ), L ( x _ c ), p , o ) Ġ\\ Ġh _{ h c } ( T ( x _ h ), T ( x _ c ), p , o ) Ġ\\ Ġh _{ h c } ( L ( x _ h ), T ( x _ c ), p , o ) Ġ\\ Ġh _{ h c } ( T ( x _ h ), L ( x _ c ), p , o ) Ġ\ end { array }\ right ] $$ Ġ Ġ Ġ( E q . Ġ6 ) </s> Similarly , Ġthe Ġsibling Ġfeature Ġfunctions Ġare Ġequal Ġto Ġ1 Ġif Ġsiblings Ġ$ x _ l $ Ġand Ġ$ x _ r $ Ġ( Ġ$ x _ l $ Ġon Ġthe Ġleft Ġof Ġ$ x _ r $ Ġin Ġthe Ġsource Ġsentence ) Ġtake Ġa Ġcertain Ġordering Ġ$ o $ Ġin Ġthe Ġtranslation Ġoutput , Ġand Ġ0 Ġotherwise . ĠThis Ġgroup Ġof Ġfeature Ġfunctions Ġis Ġdefined Ġas </s> $$ H _{ s ib } ( x _ l , x _ r , o ) Ġ= Ġ\ left [ \ begin { array }{ c } Ġh _{ s ib } ( L ( x _ l ), L ( x _ r ), o ) Ġ\\ Ġh _{ s ib } ( T ( x _ l ), T ( x _ r ), o ) Ġ\\ Ġh _{ s ib } ( L ( x _ l ), T ( x _ r ), o ) Ġ\\ Ġh _{ s ib } ( T ( x _ l ), L ( x _ r ), o ) Ġ\ end { array }\ right ] $$ Ġ Ġ Ġ( E q . Ġ7 ) </s> UTF 8 g bs n ĠEach Ġdependency Ġswap Ġfeature Ġthat Ġis Ġset Ġto Ġ1 Ġwhen Ġa Ġhypothesis Ġis Ġgenerated Ġcaptures Ġtwo Ġsource Ġwords , Ġone Ġis Ġcovered Ġby Ġthe Ġcurrent Ġhypothesis , Ġwhile Ġthe Ġother Ġhas Ġnot Ġyet Ġbeen Ġtranslated . ĠThese Ġword Ġpairs Ġhave Ġeither Ġhead - child Ġor Ġsibling Ġrelationship Ġas Ġdefined Ġabove . ĠFigure Ġ1 Ġis Ġan Ġillustration Ġof Ġhow Ġword Ġorder Ġis Ġdetected Ġduring Ġbeam Ġsearch Ġdecoding Ġand Ġthe Ġswap Ġfeatures Ġinvolved . ĠWhen ĠâĢ ľ åı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ Ġ$\ right arrow Ġ$ Ġmade Ġa Ġspeech âĢ Ŀ Ġis Ġtranslated Ġafter ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ Ġ$\ right arrow Ġ$ ĠJ ok ow i âĢ Ŀ , Ġit Ġis Ġknown Ġthat Ġthe Ġhead ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ Ġis Ġbefore Ġthe Ġchild ĠâĢ ľ è ® ² è¯ Ŀ âĢ Ŀ Ġ( Ġ$ p = left $ Ġ) Ġand Ġtheir Ġtranslations Ġalso Ġfollow Ġthe Ġsame Ġword Ġorder , Ġsetting Ġthe Ġvalue Ġof Ġthe Ġfollowing Ġfour Ġfeatures Ġto Ġ1 : Ġ$ H _{ h c } (\ text { åı ĳ è¡ ¨ }, \ text { è ® ² è¯ Ŀ }, left , io ) Ġ= Ġ\ left [ \ begin { array }{ c } Ġh _{ h c } ( root , d obj , left , io ) Ġ\\ Ġh _{ h c } ( V V , NN , left , io ) Ġ\\ Ġh _{ h c } ( root , NN , left , io ) Ġ\\ Ġh _{ h c } ( V V , d obj , left , io ) Ġ\ end { array }\ right ] $ </s> On Ġthe Ġother Ġhand , Ġthe Ġhead ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ Ġis Ġafter Ġthe Ġchild ĠâĢ ľ åľ ¨ âĢ Ŀ Ġin Ġthe Ġsource Ġsentence Ġ( Ġ$ p = right $ Ġ), Ġbut ĠâĢ ľ åľ ¨ âĢ Ŀ Ġhas Ġnot Ġbeen Ġtranslated . ĠTherefore , Ġthe Ġtranslation Ġof ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ Ġwill Ġbe Ġswapped Ġwith Ġthat Ġof ĠâĢ ľ åľ ¨ âĢ Ŀ , Ġsetting Ġthe Ġfour Ġfeatures Ġin Ġ$ H _{ h c } (\ text { åı ĳ è¡ ¨ }, \ text { åľ ¨ }, right , sw ) $ Ġto Ġ1 . ĠSimilarly , ĠâĢ ľ è ® ² è¯ Ŀ âĢ Ŀ Ġis Ġafter Ġits Ġsibling ĠâĢ ľ åľ ¨ âĢ Ŀ Ġin Ġthe Ġsource Ġsentence , Ġbut ĠâĢ ľ åľ ¨ âĢ Ŀ Ġhas Ġnot Ġbeen Ġtranslated , Ġresulting Ġin Ġthe Ġtranslation Ġof Ġthe Ġtwo Ġwords Ġbeing Ġswapped Ġand Ġsetting Ġthe Ġfour Ġfeatures Ġin Ġ$ H _{ s ib } (\ text { åľ ¨ }, \ text { è ® ² è¯ Ŀ }, sw ) $ Ġto Ġ1 . ĠThe Ġfeatures Ġfor ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ âĢ Ŀ Ġsuch Ġas Ġ$ H _{ h c } (\ text { åı ĳ è¡ ¨ }, \ text { ä½ Ĳ ç § ĳ å ¨ ģ }, right , io ) $ Ġand Ġ$ H _{ s ib } (\ text { ä½ Ĳ ç § ĳ å ¨ ģ }, \ text { è ® ² è¯ Ŀ }, io ) $ Ġare Ġnot Ġset Ġto Ġ1 Ġwhen Ġthe Ġhypothesis ĠâĢ ľ åı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ Ġ$\ right arrow Ġ$ Ġmade Ġa Ġspeech âĢ Ŀ Ġis Ġgenerated . ĠThose Ġfeatures Ġwere Ġset Ġto Ġ1 Ġwith Ġthe Ġprevious Ġhypothesis ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ Ġ$\ right arrow Ġ$ ĠJ ok ow i âĢ Ŀ . </s> D epend ency ĠDist ortion ĠPenalty </s> To Ġencourage Ġthe Ġtranslation Ġoutput Ġthat Ġconform s Ġto Ġthe Ġdependency Ġparse Ġstructure , Ġwe Ġimpose Ġa Ġpenalty Ġfeature Ġthat Ġdiscour ages Ġtranslation Ġcandidates Ġnot Ġcon forming Ġto Ġthe Ġdependency Ġparse Ġsubt ree ĠB IB REF 13 Ġ. ĠThis Ġassigns Ġa Ġpenalty Ġif Ġthe Ġtranslation Ġof Ġthe Ġcurrent Ġphrase Ġresults Ġin Ġa Ġsource Ġdependency Ġparse Ġsubt ree Ġto Ġbe Ġsplit Ġin Ġthe Ġoutput Ġtranslation . </s> UTF 8 g bs n ĠThe Ġtranslation Ġoutput Ġshown Ġin ĠFigure Ġ1 Ġhas Ġeach Ġdependency Ġparse Ġsubt ree Ġgrouped Ġtogether , Ġtherefore Ġinc urring Ġno Ġpenalty . ĠHowever , Ġthere Ġcan Ġbe Ġa Ġcase Ġwhen Ġthe Ġtranslation Ġhas Ġproduced ĠâĢ ľ J ok ow i Ġmade Ġa Ġspeech Ġin âĢ Ŀ , Ġthen Ġtranslate ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ Ġto ĠâĢ ľ yes terday âĢ Ŀ . ĠThis Ġtranslation Ġwill Ġbreak Ġthe Ġcohesion Ġof Ġthe Ġsource Ġdependency Ġparse Ġsubt ree ĠâĢ ľ åľ ¨ Ġå Į Ĺ äº ¬ âĢ Ŀ Ġand Ġis Ġbad . ĠThis Ġis Ġthe Ġcase Ġwhen Ġa Ġdependency Ġdistortion Ġpenalty Ġis Ġincurred . </s> Mod ified ĠFuture ĠCost </s> Ph rase - based ĠSM T Ġbeam Ġsearch Ġdecoding Ġinvolves Ġfuture Ġcost , Ġnot Ġaccumulated Ġover Ġthe Ġtranslation Ġhypotheses Ġbut Ġused Ġto Ġreduce Ġsearch Ġspace ĠB IB REF 16 Ġ. ĠThis Ġpredicts Ġthe Ġcost Ġto Ġtranslate Ġthe Ġremaining Ġunt rans lated Ġphrases . </s> Based Ġon Ġour Ġdependency Ġswap Ġfeatures , Ġwe Ġincorporate Ġthe Ġfuture Ġcost Ġfor Ġeach Ġunt rans lated Ġdependency Ġword Ġpairs . ĠThe Ġfuture Ġcost Ġassumes Ġthat Ġthe Ġunt rans lated Ġwords Ġwill Ġbe Ġordered Ġin Ġthe Ġmost Ġlikely Ġordering . ĠHowever , Ġif Ġan Ġunt rans lated Ġword Ġ$ x $ Ġis Ġan Ġancestor Ġ( in Ġthe Ġdependency Ġparse Ġtree ) Ġof Ġa Ġword Ġcovered Ġby Ġthe Ġcurrent Ġhypothesis , Ġthe Ġfuture Ġcost Ġassumes Ġthat Ġ$ x $ Ġpreced es Ġall Ġof Ġits Ġrelated Ġwords . ĠThis Ġfollows Ġthe Ġassumption Ġthat Ġeach Ġsubt ree Ġwill Ġhave Ġa Ġcontiguous Ġtranslation ĠB IB REF 13 Ġ. </s> Other ĠSp arse ĠFeatures </s> This Ġsection Ġdescribes Ġother Ġsparse Ġfeatures Ġfrom Ġprevious Ġwork Ġto Ġcompare Ġto Ġour Ġmethod . </s> S parse ĠRe ordering ĠOrient ation ĠFeatures </s> We Ġincorporate Ġsparse Ġre ordering Ġorientation Ġfeatures ĠB IB REF 4 Ġ. ĠThe Ġfeatures Ġare Ġderived Ġfrom Ġthe Ġre ordering Ġorientation Ġmodel , Ġcapturing Ġthe Ġsource Ġposition Ġof Ġthe Ġcurrent Ġphrase Ġbeing Ġtranslated Ġwith Ġrespect Ġto Ġthe Ġpreviously Ġtranslated Ġphrase ( s ), Ġfor Ġwhich Ġthree Ġorientation Ġtypes Ġare Ġdefined : </s> Mon ot one Ġ( M ), Ġif Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġpreviously Ġtranslated Ġunit Ġare Ġadjacent Ġin Ġthe Ġinput Ġsentence Ġand Ġthe Ġformer Ġfollows Ġthe Ġlatter Ġin Ġit , Ġe . g ., ĠâĢ ľ Be ijing âĢ Ŀ Ġwith Ġrespect Ġto Ġthe Ġprevious Ġphrase ĠâĢ ľ in âĢ Ŀ Ġin ĠFigure Ġ1 Ġ. </s> Sw apped Ġ( S ), Ġif Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġpreviously Ġtranslated Ġunit Ġare Ġadjacent Ġin Ġthe Ġinput Ġsentence Ġbut Ġthe Ġformer Ġpreced es Ġthe Ġlatter Ġin Ġit Ġ( example Ġbelow ). </s> Disc ont in uous Ġ( D ), Ġif Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġpreviously Ġtranslated Ġunit Ġare Ġnot Ġadjacent Ġin Ġthe Ġinput Ġsentence , Ġe . g ., ĠâĢ ľ made Ġa Ġspeech âĢ Ŀ Ġwith Ġrespect Ġto Ġthe Ġprevious Ġphrase ĠâĢ ľ J ok ow i âĢ Ŀ Ġin ĠFigure Ġ1 Ġ. </s> UTF 8 g bs n ĠB IB REF 4 Ġcherry impro ved 2013 Ġdesigned Ġhis Ġsparse Ġre ordering Ġorientation Ġmodel Ġfollowing Ġthe Ġhierarchical Ġre ordering Ġ( HR ) Ġmodel ĠB IB REF 3 Ġ, Ġcapturing Ġthe Ġrelative Ġposition Ġof Ġthe Ġcurrent Ġphrase Ġ( covered Ġby Ġthe Ġcurrent Ġhypothesis ) Ġwith Ġrespect Ġto Ġthe Ġlargest Ġchunk Ġof Ġcontiguous Ġsource Ġphrases Ġthat Ġform Ġa Ġcontiguous Ġtranslation Ġbefore Ġthis Ġphrase . ĠTherefore , Ġin ĠFigure Ġ1 Ġ, Ġwhen Ġthe Ġdecoding Ġproduces Ġa Ġphrase ĠâĢ ľ yes terday âĢ Ŀ Ġafter ĠâĢ ľ Be ijing âĢ Ŀ , Ġthe Ġorientation Ġof ĠâĢ ľ yes terday âĢ Ŀ Ġis Ġswapped Ġinstead Ġof Ġdiscontin uous , Ġas ĠâĢ ľ made Ġa Ġspeech Ġin ĠBeijing âĢ Ŀ Ġis Ġformed Ġby Ġcontiguous Ġphrases ĠâĢ ľ åľ ¨ Ġå Į Ĺ äº ¬ Ġå ı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ âĢ Ŀ , Ġwhich Ġis Ġadjacent Ġto ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ . </s> While Ġthe Ġoriginal Ġ( non - s parse ) Ġre ordering Ġorientation Ġmodel Ġis Ġbased Ġon Ġthe Ġphrase Ġorientation Ġprobability Ġin Ġthe Ġparallel Ġtraining Ġdata Ġand Ġdefines Ġa Ġsingle Ġfeature Ġfunction Ġon Ġit , Ġthe Ġsparse Ġmodel Ġdefines Ġone Ġfeature Ġfunction Ġfor Ġeach Ġre ordering Ġphenomenon Ġduring Ġdecoding . ĠWe Ġdefine Ġthe Ġsparse Ġfeature Ġfunctions Ġtaking Ġinto Ġaccount Ġthe Ġphrase Ġorientation Ġ$ o \ in Ġ\ l brace ĠM , S , D \ r brace Ġ$ Ġduring Ġdecoding Ġand Ġimportant Ġlocations Ġ$ loc $ Ġpertaining Ġto Ġthe Ġcurrent Ġphrase Ġand Ġthe Ġprevious Ġphrase Ġwith Ġthe Ġfollowing Ġtemplate : </s> $$ h _{ s \ _ hr } ( loc : = rep ( loc ), o ) $$ Ġ Ġ Ġ( E q . Ġ14 ) </s> where Ġlocations Ġ$ loc $ Ġare Ġthe Ġfirst Ġand Ġlast Ġwords Ġof Ġthe Ġcurrent Ġsource Ġphrase Ġ( Ġ$ s _{ first } $ Ġ, Ġ$ s _{ last } $ Ġ), Ġthe Ġpreviously Ġtranslated Ġunit Ġ( Ġ$ p _{ first } $ Ġ, Ġ$ p _{ last } $ Ġ), Ġi . e ., Ġthe Ġlargest Ġcontiguous Ġchunk Ġof Ġphrases Ġforming Ġa Ġcontiguous Ġtranslation , Ġand Ġthe Ġspan Ġbetween Ġthe Ġcurrent Ġand Ġthe Ġprevious Ġsource Ġphrase , Ġor Ġgap Ġ( Ġ$ g _{ first } $ Ġ, Ġ$ g _{ last } $ Ġ) Ġonly Ġfor Ġdiscontin uous Ġorientation . ĠEach Ġword Ġin Ġ$ loc $ Ġis Ġrepresented , Ġ$ rep ( loc ) $ Ġ, Ġby Ġits ĠPOS Ġtag , Ġand Ġthe Ġsurface Ġlex ical Ġform Ġif Ġit Ġbelongs Ġto Ġthe Ġ80 - most Ġfrequent Ġwords Ġin Ġthe Ġtraining Ġdata . </s> UTF 8 g bs n ĠAssuming Ġthat Ġonly ĠâĢ ľ åľ ¨ âĢ Ŀ Ġbelongs Ġto Ġthe Ġtop - 80 Ġwords , Ġwhen Ġthe Ġphrase ĠâĢ ľ yes terday âĢ Ŀ Ġis Ġgenerated Ġafter ĠâĢ ľ Be ijing âĢ Ŀ , Ġthe Ġsparse Ġre ordering Ġorientation Ġfeatures Ġthat Ġare Ġequal Ġto Ġ1 Ġare Ġshown Ġin ĠFigure Ġ2 Ġ. </s> While Ġthe Ġapproach Ġlever ages Ġthe Ġre ordering Ġorientation Ġmodel Ġby Ġdefining Ġfeatures Ġon Ġthe Ġcontext Ġinformation , Ġnot Ġjust Ġthe Ġcurrent Ġphrase , Ġit Ġdoes Ġnot Ġcapture Ġdependency Ġrelation , Ġby Ġwhich Ġthe Ġimportant Ġrelation Ġbetween Ġwords Ġin Ġthe Ġsource Ġsentence Ġis Ġcaptured . ĠTherefore , Ġwe Ġintroduce Ġour Ġsparse Ġdependency Ġswap Ġfeatures Ġto Ġenable Ġthe Ġtranslation Ġsystem Ġto Ġarrive Ġat Ġre ordering Ġdecisions Ġbased Ġon Ġthe Ġsource Ġdependency Ġrelations . </s> D epend ency ĠPath ĠFeatures </s> We Ġalso Ġutilize Ġdependency Ġpath Ġfeatures ĠB IB REF 15 Ġfor Ġphrase - based ĠSM T , Ġdefined Ġover Ġthe Ġshortest Ġpath Ġof Ġdependency Ġparse Ġtree Ġlinks Ġbrid ging Ġthe Ġcurrent Ġand Ġthe Ġprevious Ġsource Ġphrase . ĠB IB REF 15 Ġchang disc rim in ative 2009 Ġdefined Ġthe Ġdependency Ġpath Ġfeatures Ġon Ġa Ġmaximum Ġentropy Ġphrase Ġorientation Ġclass ifier , Ġtrained Ġon Ġtheir Ġword - aligned Ġparallel Ġtext Ġand Ġlabeled Ġby Ġthe Ġtwo Ġpossible Ġphrase Ġorder ings Ġin Ġthe Ġtranslation Ġoutput : Ġin - order Ġand Ġswapped . ĠMeanwhile , Ġwe Ġuse Ġthe Ġfeatures Ġas Ġsparse Ġdecoding Ġfeatures , Ġwith Ġthe Ġfollowing Ġtemplate : </s> $$ h _{ path } ( short est \ _ path ( p _{ last }, s _{ first }); o ) $$ Ġ Ġ Ġ( E q . Ġ18 ) </s> where Ġ$ o Ġ\ in Ġ\ l brace Ġin \ _ order , sw apped \ r brace Ġ$ Ġdenotes Ġthe Ġorientation Ġof Ġthe Ġtwo Ġphrases Ġin Ġthe Ġtranslation . </s> Given Ġa Ġsource Ġsentence Ġand Ġits Ġtranslation Ġoutput , Ġa Ġpath Ġis Ġdefined Ġbetween Ġthe Ġlast Ġword Ġof Ġthe Ġprevious Ġsource Ġphrase Ġ$ p _{ last } $ Ġand Ġthe Ġfirst Ġword Ġof Ġthe Ġcurrent Ġsource Ġphrase Ġ$ s _{ first } $ Ġ. ĠPath Ġtravers al Ġis Ġalways Ġfrom Ġleft Ġto Ġright Ġbased Ġon Ġthe Ġsource Ġsentence Ġword Ġposition . ĠTherefore , Ġif Ġthe Ġcurrent Ġsource Ġphrase Ġbeing Ġtranslated Ġis Ġto Ġthe Ġright Ġof Ġthe Ġprevious Ġsource Ġphrase , Ġthe Ġtravers al Ġis Ġfrom Ġ$ p _{ last } $ Ġto Ġ$ s _{ first } $ Ġ. ĠOtherwise , Ġif Ġthe Ġcurrent Ġsource Ġphrase Ġis Ġto Ġthe Ġleft Ġof Ġthe Ġprevious Ġsource Ġphrase , Ġit Ġis Ġfrom Ġ$ s _{ first } $ Ġto Ġ$ p _{ last } $ Ġ. ĠIn Ġaddition , Ġpath Ġedges Ġgoing Ġagainst Ġthe Ġdependency Ġlabel Ġarrow Ġare Ġdistinguished Ġfrom Ġthose Ġfollowing Ġthe Ġarrow . </s> UTF 8 g bs n ĠAs Ġan Ġexample , Ġin ĠFigure Ġ1 Ġ, Ġwhen Ġthe Ġtranslation Ġgenerates ĠâĢ ľ åı ĳ è¡ ¨ Ġè ® ² è¯ Ŀ Ġ$\ right arrow Ġ$ Ġmade Ġa Ġspeech âĢ Ŀ Ġafter ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ Ġ$\ right arrow Ġ$ ĠJ ok ow i âĢ Ŀ , Ġthe Ġpath Ġis Ġfrom ĠâĢ ľ ä½ Ĳ ç § ĳ å ¨ ģ âĢ Ŀ Ġto ĠâĢ ľ åı ĳ è¡ ¨ âĢ Ŀ , Ġconsisting Ġof Ġa Ġdirect Ġlink Ġfollowing Ġthe Ġarrow Ġof Ġ$ ns ub j $ Ġ, Ġresulting Ġin Ġthe Ġfeature Ġ$ h _{ path } ( ns ub j , in \ _ order )= 1 $ Ġ. ĠHowever , Ġwhen Ġthe Ġtranslation Ġgenerates ĠâĢ ľ æĺ ¨ å¤© Ġ$\ right arrow Ġ$ Ġyesterday âĢ Ŀ Ġafter ĠâĢ ľ åĮ Ĺ äº ¬ Ġ$\ right arrow Ġ$ ĠBeijing âĢ Ŀ , Ġas ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ Ġis Ġbefore ĠâĢ ľ åĮ Ĺ äº ¬ âĢ Ŀ Ġin Ġthe Ġsource Ġsentence , Ġthe Ġtravers al Ġis Ġfrom ĠâĢ ľ æĺ ¨ å¤© âĢ Ŀ Ġto ĠâĢ ľ åĮ Ĺ äº ¬ âĢ Ŀ , Ġconsisting Ġof Ġthe Ġlink Ġsequence Ġ$ t mod , prep R , p obj R $ Ġ. ĠAs Ġit Ġgoes Ġagainst Ġthe Ġarrows Ġof Ġ$ prep $ Ġand Ġ$ p obj $ Ġ, Ġthe Ġsuffix Ġ$ R $ Ġis Ġadded Ġto Ġdistinguish Ġit . ĠThis Ġresults Ġin Ġthe Ġfeature Ġ$\ right arrow Ġ$ 0 Ġ. </s> The Ġapproach Ġlever ages Ġthe Ġphrase - based Ġre ordering Ġby Ġguiding Ġthe Ġordering Ġof Ġtwo Ġadjacent Ġphrases Ġusing Ġdependency Ġparse . ĠHowever , Ġthe Ġfeatures Ġdo Ġnot Ġcapture Ġthe Ġpair wise Ġordering Ġof Ġevery Ġword Ġwith Ġits Ġrelated Ġword , Ġas Ġthe Ġfeatures Ġare Ġinduced Ġonly Ġwhen Ġthe Ġwords Ġare Ġused Ġin Ġthe Ġtwo Ġadjacent Ġtranslation Ġphrases . </s> Data ĠSet Ġand ĠTool k its </s> We Ġbuilt Ġa Ġphrase - based ĠChinese - to - English ĠSM T Ġsystem Ġby Ġusing ĠMoses ĠB IB REF 18 Ġ. ĠOur Ġparallel Ġtraining Ġtext Ġis Ġa Ġcollection Ġof Ġparallel Ġcorpor a Ġfrom ĠL DC , Ġwhich Ġwe Ġdivide Ġinto Ġolder Ġcorpor a Ġand Ġnewer Ġcorpor a . ĠDue Ġto Ġthe Ġdominant Ġolder Ġdata , Ġwe Ġduplicate Ġthe Ġnewer Ġcorpor a Ġof Ġvarious Ġdomains Ġby Ġ10 Ġtimes Ġto Ġachieve Ġbetter Ġdomain Ġbalance . ĠTo Ġreduce Ġthe Ġpossibility Ġof Ġalignment Ġerrors , Ġparallel Ġsentences Ġin Ġthe Ġcorpor a Ġthat Ġare Ġlonger Ġthan Ġ85 Ġwords Ġin Ġeither ĠChinese Ġ( after Ġword Ġsegment ation ) Ġor ĠEnglish Ġare Ġdiscarded . ĠIn Ġthe Ġend , Ġthe Ġfinal Ġparallel Ġtext Ġconsists Ġof Ġaround Ġ8 . 8 M Ġsentence Ġpairs , Ġ228 M ĠChinese Ġtokens , Ġand Ġ254 M ĠEnglish Ġtokens Ġ( a Ġtoken Ġcan Ġbe Ġa Ġword Ġor Ġpunct uation Ġsymbol ). ĠWe Ġalso Ġadded Ġtwo Ġdiction aries Ġby Ġconc aten ating Ġthem Ġto Ġour Ġtraining Ġparallel Ġtext . ĠThe Ġtotal Ġnumber Ġof Ġwords Ġin Ġthese Ġtwo Ġcorpor a Ġis Ġ1 . 81 M Ġfor ĠChinese Ġand Ġ2 . 03 M Ġfor ĠEnglish . </s> All ĠChinese Ġsentences Ġin Ġthe Ġtraining , Ġdevelopment , Ġand Ġtest Ġdata Ġare Ġfirst Ġword - se gment ed Ġusing Ġa Ġmaximum Ġentropy - based ĠChinese Ġword Ġsegment er ĠB IB REF 19 Ġtrained Ġon Ġthe ĠChinese ĠTree bank Ġ( CT B ) Ġscheme . ĠThen Ġthe Ġparallel Ġcorpus Ġis Ġword - aligned Ġby ĠG IZ A ++ ĠB IB REF 20 Ġusing ĠIBM ĠModels Ġ1 , Ġ3 , Ġand Ġ4 ĠB IB REF 21 Ġ. ĠFor Ġbuilding Ġthe Ġphrase Ġtable , Ġwhich Ġfollows Ġword Ġalignment , Ġthe Ġmaximum Ġlength Ġof Ġa Ġphrase Ġpair Ġis Ġset Ġto Ġ7 Ġwords Ġfor Ġboth Ġthe Ġsource Ġand Ġtarget Ġsides . </s> The Ġlanguage Ġmodel Ġ( LM ) Ġis Ġa Ġ5 - gram Ġmodel Ġtrained Ġon Ġthe ĠEnglish Ġside Ġof Ġthe ĠFB IS Ġparallel Ġcorpus Ġ( L DC 2003 E 14 ) Ġand Ġthe Ġmon oling ual Ġcorpus ĠEnglish ĠGig aw ord Ġversion Ġ4 Ġ( L DC 2009 T 13 ), Ġconsisting Ġof Ġ107 M Ġsentences Ġand Ġ3 . 8 G Ġtokens Ġaltogether . ĠEach Ġindividual ĠGig aw ord Ġsub - cor p us Ġis Ġused Ġto Ġtrain Ġa Ġseparate Ġlanguage Ġmodel Ġand Ġso Ġis Ġthe ĠEnglish Ġside Ġof ĠFB IS . ĠThese Ġindividual Ġlanguage Ġmodels Ġare Ġthen Ġinterpol ated Ġto Ġbuild Ġone Ġsingle Ġlarge ĠLM , Ġvia Ġperplex ity Ġtuning Ġon Ġthe Ġdevelopment Ġset . </s> Our Ġtranslation Ġdevelopment Ġset Ġis ĠM TC Ġcorpus Ġversion Ġ1 Ġ( L DC 2002 T 01 ) Ġand Ġversion Ġ3 Ġ( L DC 2004 T 07 ). ĠThis Ġdevelopment Ġset Ġhas Ġ1 , 9 28 Ġsentence Ġpairs Ġin Ġtotal , Ġ49 K ĠChinese Ġtokens Ġand Ġ58 K ĠEnglish Ġtokens Ġon Ġaverage Ġacross Ġthe Ġfour Ġreference Ġtranslations . ĠWeight Ġtuning Ġis Ġdone Ġby Ġusing Ġthe Ġpair wise Ġranked Ġoptimization Ġ( PRO ) Ġalgorithm ĠB IB REF 22 Ġ, Ġwhich Ġis Ġalso Ġused Ġto Ġobtain Ġweights Ġof Ġthe Ġsparse Ġfeatures Ġto Ġhelp Ġdetermine Ġthe Ġre ordering . </s> For Ġdependency Ġsparse Ġfeatures , Ġwe Ġparse Ġthe ĠChinese Ġside Ġof Ġour Ġdevelopment Ġand Ġtest Ġsets Ġby Ġthe ĠMate Ġparser , Ġwhich Ġjointly Ġperforms ĠPOS Ġtagging Ġand Ġdependency Ġparsing ĠB IB REF 23 Ġ, Ġtrained Ġon ĠChinese ĠTree bank Ġ( CT B ) Ġversion Ġ8 . 0 Ġ( L DC 2013 T 21 ). </s> Our Ġtest Ġset Ġconsists Ġof Ġthe ĠN IST ĠMT Ġevaluation Ġsets Ġfrom Ġ2002 Ġto Ġ2006 Ġand Ġ2008 Ġ( L DC 2010 T 10 , ĠL DC 2010 T 11 , ĠL DC 2010 T 12 , ĠL DC 2010 T 14 , ĠL DC 2010 T 17 , ĠL DC 2010 T 21 ). </s> Bas eline ĠSystem </s> UTF 8 g bs n ĠWe Ġbuild Ġa Ġphrase - based Ġbaseline ĠSM T Ġsystem , Ġwhich Ġuses Ġnon - s parse Ġphrase - based Ġlex ical ized Ġre ordering Ġ( P BL R ), Ġin Ġwhich Ġthe Ġre ordering Ġprobability Ġdepends Ġon Ġthe Ġphrase Ġbeing Ġtranslated Ġand Ġits Ġposition Ġwith Ġrespect Ġto Ġthe Ġsource Ġposition Ġof Ġthe Ġpreviously Ġtranslated Ġphrase ĠB IB REF 1 Ġ, ĠB IB REF 24 Ġ, Ġand Ġnon - s parse Ġhierarchical Ġre ordering Ġ( HR ), Ġin Ġwhich Ġthe Ġprevious Ġunit Ġis Ġnot Ġonly Ġthe Ġprevious Ġphrase , Ġbut Ġthe Ġlargest Ġchunk Ġof Ġcontiguous Ġsource Ġphrases Ġhaving Ġcontiguous Ġtranslation ĠB IB REF 3 Ġ. ĠIn Ġaddition , Ġa Ġdistortion Ġlimit Ġis Ġset Ġsuch Ġthat Ġthe Ġre ordering Ġcannot Ġbe Ġlonger Ġthan Ġa Ġcertain Ġdistance . ĠWe Ġset Ġpunct uation Ġsymbols Ġas Ġre ordering Ġconstraint Ġacross Ġwhich Ġphrases Ġcannot Ġbe Ġre ordered , Ġas Ġthey Ġform Ġthe Ġnatural Ġboundaries Ġbetween Ġdifferent Ġclauses . ĠWe Ġalso Ġuse Ġn - best ĠMinimum ĠBay es ĠRisk Ġ( MB R ) Ġdecoding ĠB IB REF 25 Ġinstead Ġof Ġthe Ġdefault Ġmaximum Ġa - p oster ior i Ġ( MAP ) Ġdecoding . </s> Our ĠApproach </s> To Ġaccommodate Ġour Ġsparse Ġfeature Ġfunctions , Ġour ĠMoses Ġcode Ġhas Ġbeen Ġmodified Ġto Ġread Ġdependency - p ars ed Ġinput Ġsentences Ġand Ġincorporate Ġadditional Ġdecoding Ġfeatures Ġon Ġtop Ġof Ġour Ġbaseline , Ġnamely Ġdependency Ġdistortion Ġpenalty Ġ( D DP ) ĠB IB REF 13 Ġ, Ġsparse Ġdependency Ġpath Ġfeatures Ġ( Path ) ĠB IB REF 15 Ġ, Ġsparse Ġre ordering Ġorientation Ġfollowing Ġhierarchical Ġre ordering Ġorientation Ġ( SH R ) ĠB IB REF 4 Ġ, Ġand Ġour Ġsparse Ġdependency Ġswap Ġfeatures Ġ( DS ). ĠD DP Ġfeature Ġis Ġa Ġsingle Ġpenalty Ġfeature Ġsimilar Ġto Ġthe Ġdistortion Ġpenalty Ġfor Ġdistance - based Ġre ordering Ġmodel ĠB IB REF 0 Ġ, Ġwhile ĠPath , ĠSH R , Ġand ĠDS Ġare Ġsparse Ġfeatures , Ġeach Ġinstance Ġof Ġwhich Ġcaptures Ġa Ġspecific Ġphenomenon Ġduring Ġtranslation ĠB IB REF 17 Ġ. </s> We Ġalways Ġcouple ĠDS Ġwith ĠD DP . ĠHowever , Ġas Ġthe Ġoriginal ĠPath Ġfeatures Ġdid Ġnot Ġuse ĠD DP , Ġwe Ġexperiment Ġwith ĠPath Ġfeatures Ġin Ġone Ġsetting Ġthat Ġdoes Ġnot Ġincorporate ĠD DP Ġand Ġanother Ġthat Ġdoes . ĠThe ĠSH R Ġfeatures Ġare Ġnot Ġcoupled Ġwith ĠD DP , Ġfollowing Ġthe Ġoriginal Ġdesign , Ġas Ġthe Ġfeatures Ġare Ġnot Ġdefined Ġon Ġa Ġdependency - p ars ed Ġinput Ġsentence . </s> Exper imental ĠResults </s> The Ġtranslation Ġquality Ġof Ġthe Ġsystem Ġoutputs Ġis Ġmeasured Ġby Ġcase - ins ensitive ĠB LE U ĠB IB REF 26 Ġ, Ġfor Ġwhich Ġthe Ġbre vity Ġpenalty Ġis Ġcomputed Ġbased Ġon Ġthe Ġshortest Ġreference Ġ( N IST - BLE U ). ĠStatistical Ġsignificance Ġtesting Ġbetween Ġsystems Ġis Ġconducted Ġby Ġboot strap Ġres am pling ĠB IB REF 27 Ġ. </s> Table Ġ1 Ġshows Ġthe Ġexperimental Ġresults . ĠThe Ġdistortion Ġlimit Ġof Ġall Ġthe Ġsystems Ġis Ġset Ġto Ġ14 , Ġwhich Ġyields Ġthe Ġbest Ġresult Ġon Ġthe Ġdevelopment Ġset Ġfor Ġthe Ġbaseline Ġsystem . ĠAs Ġshown Ġin Ġthe Ġtable , Ġthe Ġsystem Ġwith Ġour ĠDS Ġfeatures Ġand ĠD DP Ġon Ġtop Ġof Ġthe Ġbaseline Ġis Ġable Ġto Ġimprove Ġover Ġthe Ġbaseline Ġsystem Ġwithout Ġand Ġwith ĠD DP , Ġby Ġ+ 1 . 09 Ġand Ġ+ 0 . 93 ĠB LE U Ġpoints Ġrespectively . ĠThe Ġindividual Ġcontribution Ġof Ġthe Ġother Ġdependency - based Ġfeatures Ġ( Path ), Ġwithout Ġor Ġwith ĠD DP , Ġis Ġinferior Ġto Ġour ĠDS Ġfeatures . ĠNevertheless , Ġcoupling Ġour ĠDS Ġfeatures Ġwith ĠPath Ġfeatures Ġyields Ġthe Ġbest Ġresult Ġ(+ 1 . 21 Ġand Ġ+ 1 . 05 ĠB LE U Ġpoints Ġover Ġthe Ġbaseline Ġwithout Ġand Ġwith ĠD DP ). </s> The ĠSH R Ġfeatures Ġyield Ġmore Ġimprovement Ġthan ĠPath Ġfeatures Ġwithout ĠD DP , Ġand Ġare Ġcomparable Ġto Ġthe ĠPath Ġfeatures Ġwith ĠD DP . ĠHowever , Ġour ĠDS Ġfeatures Ġyield Ġmore Ġimprovement Ġthan ĠSH R Ġfeatures . ĠOur Ġpreliminary Ġexperiments Ġindicate Ġthat Ġadding Ġthese Ġfeatures Ġon Ġtop Ġof Ġthe Ġsystem Ġwith ĠDS Ġdoes Ġnot Ġimprove Ġover Ġit . </s> Disc ussions </s> The Ġre ordering Ġorientation Ġmodels , Ġi . e ., ĠP BL R Ġand ĠHR , Ġonly Ġtake Ġinto Ġaccount Ġthe Ġphrase Ġpair Ġgenerated Ġby Ġa Ġhypothesis Ġand Ġnot Ġthe Ġrelated Ġword Ġproperties . ĠSp arse Ġre ordering Ġorientation Ġfeatures ĠB IB REF 4 Ġleverage Ġthis Ġby Ġcapturing Ġthe Ġprevious Ġphrase Ġ( or Ġcontiguous Ġchunk ) Ġproperties . ĠTherefore , Ġthey Ġare Ġable Ġto Ġimprove Ġover Ġthe Ġbaseline . ĠHowever , Ġas Ġthe Ġresults Ġsuggest , Ġdependency Ġparse Ġprovides Ġa Ġmore Ġuseful Ġguidance Ġto Ġre order Ġa Ġsource Ġsentence . </s> As Ġshown Ġin ĠFigure Ġ3 Ġ, Ġthe Ġbaseline Ġphrase - based ĠSM T Ġsystem Ġwith Ġthe Ġtwo Ġre ordering Ġorientation Ġmodels Ġ( P BL R Ġand ĠHR ) Ġproduces Ġan Ġincorrect Ġtranslation Ġoutput Ġthat ĠâĢ ľ the Ġexport Ġis Ġthe Ġmain Ġmarket âĢ Ŀ , Ġwhich Ġis Ġnot Ġwhat Ġthe Ġsource Ġsentence Ġmeans . ĠThis Ġis Ġalso Ġthe Ġcase Ġwith Ġthe Ġsystem Ġadded Ġwith Ġprior Ġre ordering Ġapproaches , Ġwhich Ġincludes ĠD DP ĠB IB REF 13 Ġ, ĠPath ĠB IB REF 15 Ġ, Ġtheir Ġcombination ĠD DP + Path , Ġand ĠSH R ĠB IB REF 4 Ġ. ĠMeanwhile , Ġour Ġre ordering Ġapproach Ġwith ĠDS Ġfeatures Ġis Ġable Ġto Ġoutput Ġthe Ġcorrect Ġtranslation ĠâĢ ľ the Ġmain Ġmarkets Ġfor Ġthe Ġexport âĢ Ŀ , Ġas Ġit Ġpenal izes Ġthe Ġswap Ġbetween Ġthe Ġsubject Ġhead ĠâĢ ľ market âĢ Ŀ Ġand Ġthe Ġcop ula , Ġwhich Ġshould Ġnot Ġbe Ġswapped . </s> The Ġdependency Ġparse Ġof Ġa Ġsentence Ġcan Ġcapture Ġthe Ġrelationship Ġamong Ġthe Ġwords Ġ( n odes ) Ġin Ġit . ĠBetween Ġeach Ġrelated Ġword Ġpair , Ġthere Ġis Ġa Ġdependency Ġlabel Ġwhich Ġspecifies Ġthe Ġhead - mod ifier Ġrelationship . ĠWhile Ġthe Ġhead - mod ifier Ġrelationships Ġin Ġa Ġsentence Ġhold Ġacross Ġlanguages , Ġtheir Ġordering Ġmay Ġdiffer . ĠFor Ġexample , Ġin ĠChinese , Ġpre pos itional Ġphrase Ġ( mod ifier ) Ġcomes Ġbefore Ġthe Ġpredicate Ġ( head ) Ġverb , Ġwhile Ġin ĠEnglish , Ġthey Ġcome Ġin Ġthe Ġother Ġway Ġround . ĠThis Ġparticular Ġclue , Ġprovided Ġby Ġthe Ġsource Ġdependency Ġparse , Ġis Ġuseful Ġin Ġdeciding Ġthe Ġword Ġorder Ġin Ġthe Ġtranslation Ġoutput , Ġcorresponding Ġto Ġa Ġword Ġin Ġthe Ġsource Ġsentence . </s> Comb ining Ġour Ġsparse Ġdependency Ġswap Ġfeatures Ġwith Ġsparse Ġdependency Ġpath Ġfeatures ĠB IB REF 15 Ġachieves Ġthe Ġbest Ġexperimental Ġresult . ĠThis Ġcan Ġbe Ġattributed Ġto Ġthe Ġcomplementary Ġnature Ġof Ġboth Ġtypes Ġof Ġsparse Ġfeatures . ĠPath Ġfeatures Ġare Ġable Ġto Ġcapture Ġthe Ġsubsequ ence Ġof Ġtranslation Ġoutput Ġphrases , Ġi . e ., Ġwhich Ġphrase Ġfollows Ġanother Ġbut Ġnot Ġthe Ġrelative Ġposition Ġ( right Ġor Ġleft ) Ġof Ġa Ġsource Ġdependency Ġword Ġwith Ġrespect Ġto Ġall Ġits Ġrelated Ġwords , Ġwhile Ġour Ġswap Ġfeatures Ġare Ġdesigned Ġto Ġcapture Ġthem . </s> Related ĠWork </s> Source Ġdependency Ġtrees Ġhave Ġbeen Ġexploited Ġin Ġphrase - based ĠSM T , Ġby Ġmodeling Ġtransition Ġamong Ġdependency Ġsubt rees Ġduring Ġtranslation ĠB IB REF 14 Ġ. ĠHowever , Ġthis Ġdoes Ġnot Ġtake Ġinto Ġaccount Ġthe Ġdependency Ġlabel Ġand Ġthe ĠPOS Ġtag . ĠAnother Ġwork Ġexploits Ġsource Ġand Ġtarget Ġdependency Ġtrees Ġfor Ġphrase - based ĠMT Ġoutput Ġre ranking ĠB IB REF 28 Ġ, Ġinstead Ġof Ġfor Ġtranslation Ġdecoding . </s> BI B REF 15 Ġchang disc rim in ative 2009 Ġintroduced Ġdependency Ġpath Ġas Ġa Ġsoft Ġconstraint Ġbased Ġon Ġthe Ġsequence Ġof Ġsource Ġdependency Ġlinks Ġtravers ed Ġin Ġphrase - based Ġtranslation . ĠIt Ġis Ġused Ġas Ġfeatures Ġon Ġa Ġmaximum Ġentropy Ġphrase Ġorientation Ġclass ifier , Ġwhose Ġprobability Ġoutput Ġis Ġused Ġas Ġa Ġdecoding Ġfeature Ġfunction . ĠAs Ġthe Ġpath Ġcan Ġbe Ġarbitrarily Ġlong , Ġit Ġmay Ġnot Ġbe Ġrepresented Ġsufficiently Ġin Ġthe Ġtraining Ġsamples . ĠOur Ġsparse Ġfeature Ġdefinition Ġcan Ġalleviate Ġthis Ġas Ġfeatures Ġare Ġdefined Ġon Ġtwo Ġwords . ĠIn Ġaddition , Ġcapturing Ġword Ġpairs Ġinstead Ġof Ġpaths Ġenables Ġincorporation Ġof Ġother Ġword Ġproperties Ġsuch Ġas ĠPOS Ġtags . </s> BI B REF 29 Ġhunt ere x pl o iting 2010 Ġproposed Ġa Ġprobability Ġmodel Ġto Ġcapture Ġthe Ġoffset Ġof Ġa Ġword Ġwith Ġrespect Ġto Ġits Ġhead Ġposition Ġin Ġphrase - based ĠMT . ĠTheir Ġmodel Ġdoes Ġnot Ġtake Ġinto Ġaccount Ġtwo Ġsibling Ġwords Ġsharing Ġthe Ġsame Ġhead . ĠThey Ġreported Ġnegative Ġresult . </s> Meanwhile , ĠB IB REF 30 Ġg aos oft 2011 Ġdefined Ġsoft Ġconstraints Ġon Ġhierarchical Ġphrase - based ĠMT , Ġwhich Ġproduce Ġtranslation Ġby Ġbottom - up Ġconstituency Ġparsing Ġalgorithm Ġinstead Ġof Ġbeam Ġsearch . ĠTheir Ġsoft Ġconstraint Ġis Ġalso Ġdefined Ġon Ġa Ġmaximum Ġentropy Ġclass ifier Ġinstead Ġof Ġsparse Ġfeatures . </s> Prior Ġwork Ġhas Ġalso Ġused Ġconstituency Ġparse Ġto Ġguide Ġre ordering , Ġby Ġmanually Ġdefining Ġpre - ordering Ġrules Ġto Ġre order Ġinput Ġsentences Ġinto Ġthe Ġtarget Ġlanguage Ġorder Ġbefore Ġtranslation ĠB IB REF 31 Ġ, ĠB IB REF 32 Ġ, Ġor Ġto Ġautomatically Ġlearn Ġthose Ġrules ĠB IB REF 33 Ġ, ĠB IB REF 34 Ġ. ĠEach Ġconstituency Ġparse Ġtree Ġnode Ġrepresents Ġthe Ġphrase Ġnesting Ġinstead Ġof Ġa Ġword , Ġresulting Ġin Ġa Ġdeeper Ġstructure , Ġwhich Ġis Ġgenerally Ġslower Ġto Ġproduce . </s> Conclusion </s> We Ġhave Ġpresented Ġa Ġre ordering Ġapproach Ġfor Ġphrase - based ĠSM T , Ġguided Ġby Ġsparse Ġdependency Ġswap Ġfeatures . ĠWe Ġhave Ġcontributed Ġa Ġnew Ġapproach Ġfor Ġlearning Ġand Ġperforming Ġre ordering Ġin Ġphrase - based ĠMT Ġby Ġthe Ġincorporation Ġof Ġdependency - based Ġfeatures . ĠFrom Ġour Ġexperiments , Ġwe Ġhave Ġshown Ġthat Ġutilizing Ġsource Ġdependency Ġparse Ġfor Ġre ordering Ġsentences Ġhelps Ġto Ġsignificantly Ġimprove Ġtranslation Ġquality Ġover Ġa Ġphrase - based Ġbaseline Ġsystem Ġwith Ġstate - of - the - art Ġre ordering Ġorientation Ġmodels . </s> Acknowled gments </s> This Ġresearch Ġis Ġsupported Ġby Ġthe ĠSingapore ĠNational ĠResearch ĠFoundation Ġunder Ġits ĠInternational ĠResearch ĠCentre Ġ@ ĠSingapore ĠFunding ĠInitiative Ġand Ġadministered Ġby Ġthe ĠID M ĠProgramme ĠOffice .