ancient Chinese history records in several dynasties (about 1000BC-200BC) and articles written by celebrities of that era
EXTRACTIVE
Ancient Chinese history records in several dynasties and articles written by celebrities during 1000BC-200BC collected from the internet 
ABSTRACTIVE
Data Collection. To build the large ancient-modern Chinese dataset, we collected 1.7K bilingual ancient-modern Chinese articles from the internet. More specifically, a large part of the ancient Chinese data we used come from ancient Chinese history records in several dynasties (about 1000BC-200BC) and articles written by celebrities of that era. They used plain and accurate words to express what happened at that time, and thus ensure the generality of the translated materials.
Data Collection. To build the large ancient-modern Chinese dataset, we collected 1.7K bilingual ancient-modern Chinese articles from the internet. More specifically, a large part of the ancient Chinese data we used come from ancient Chinese history records in several dynasties (about 1000BC-200BC) and articles written by celebrities of that era. They used plain and accurate words to express what happened at that time, and thus ensure the generality of the translated materials.
1808.03738
Introduction </s> Ancient ĠChinese Ġis Ġthe Ġwriting Ġlanguage Ġin Ġancient ĠChina . ĠIt Ġis Ġa Ġtreasure Ġof ĠChinese Ġculture Ġwhich Ġbrings Ġtogether Ġthe Ġwisdom Ġand Ġideas Ġof Ġthe ĠChinese Ġnation Ġand Ġchron icles Ġthe Ġancient Ġcultural Ġheritage Ġof ĠChina . ĠLearning Ġancient ĠChinese Ġnot Ġonly Ġhelps Ġpeople Ġto Ġunderstand Ġand Ġinherit Ġthe Ġwisdom Ġof Ġthe Ġan cients , Ġbut Ġalso Ġpromotes Ġpeople Ġto Ġabsorb Ġand Ġdevelop ĠChinese Ġculture . </s> However , Ġit Ġis Ġdifficult Ġfor Ġmodern Ġpeople Ġto Ġread Ġancient ĠChinese . ĠFirstly , Ġcompared Ġwith Ġmodern ĠChinese , Ġancient ĠChinese Ġis Ġmore Ġconcise Ġand Ġshorter . ĠThe Ġgram matical Ġorder Ġof Ġmodern ĠChinese Ġis Ġalso Ġquite Ġdifferent Ġfrom Ġthat Ġof Ġancient ĠChinese . ĠSecondly , Ġmost Ġmodern ĠChinese Ġwords Ġare Ġdouble Ġsyll ables , Ġwhile Ġthe Ġmost Ġof Ġthe Ġancient ĠChinese Ġwords Ġare Ġmon os yll ab ic . ĠThird ly , Ġthere Ġis Ġmore Ġthan Ġone Ġpoly sem ous Ġphenomenon Ġin Ġancient ĠChinese . ĠIn Ġaddition , Ġmanual Ġtranslation Ġhas Ġa Ġhigh Ġcost . ĠTherefore , Ġit Ġis Ġmeaningful Ġand Ġuseful Ġto Ġstudy Ġthe Ġautomatic Ġtranslation Ġfrom Ġancient ĠChinese Ġto Ġmodern ĠChinese . ĠThrough Ġancient - modern ĠChinese Ġtranslation , Ġthe Ġwisdom , Ġtalent Ġand Ġaccumulated Ġexperience Ġof Ġthe Ġpredecessors Ġcan Ġbe Ġpassed Ġon Ġto Ġmore Ġpeople . </s> Ne ural Ġmachine Ġtranslation Ġ( N MT ) ĠB IB REF 0 Ġ, ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġhas Ġachieved Ġremarkable Ġperformance Ġon Ġmany Ġbilingual Ġtranslation Ġtasks . ĠIt Ġis Ġan Ġend - to - end Ġlearning Ġapproach Ġfor Ġmachine Ġtranslation , Ġwith Ġthe Ġpotential Ġto Ġshow Ġgreat Ġadvantages Ġover Ġthe Ġstatistic Ġmachine Ġtranslation Ġ( SM T ) Ġsystems . ĠHowever , ĠN MT Ġapproach Ġhas Ġnot Ġbeen Ġwidely Ġapplied Ġto Ġthe Ġancient - modern ĠChinese Ġtranslation Ġtask . ĠOne Ġof Ġthe Ġmain Ġreasons Ġis Ġthe Ġlimited Ġhigh - quality Ġparallel Ġdata Ġresource . </s> The Ġmost Ġpopular Ġmethod Ġof Ġacquiring Ġtranslation Ġexamples Ġis Ġbilingual Ġtext Ġalignment ĠB IB REF 5 Ġ. ĠThis Ġkind Ġof Ġmethod Ġcan Ġbe Ġclassified Ġinto Ġtwo Ġtypes : Ġlex ical - based Ġand Ġstatistical - based . ĠThe Ġlex ical - based Ġapproaches ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġfocus Ġon Ġlex ical Ġinformation , Ġwhich Ġutilize Ġthe Ġbilingual Ġdictionary ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġor Ġlex ical Ġfeatures . ĠMeanwhile , Ġthe Ġstatistical - based Ġapproaches ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġrely Ġon Ġstatistical Ġinformation , Ġsuch Ġas Ġsentence Ġlength Ġratio Ġin Ġtwo Ġlanguages Ġand Ġalign Ġmode Ġprobability . </s> However , Ġthese Ġmethods Ġare Ġdesigned Ġfor Ġother Ġbilingual Ġlanguage Ġpairs Ġthat Ġare Ġwritten Ġin Ġdifferent Ġlanguage Ġcharacters Ġ( e . g . ĠEnglish - French , ĠChinese - Japanese ). ĠThe Ġancient - modern ĠChinese Ġhas Ġsome Ġcharacteristics Ġthat Ġare Ġquite Ġdifferent Ġfrom Ġother Ġlanguage Ġpairs . ĠFor Ġexample , Ġancient Ġand Ġmodern ĠChinese Ġare Ġboth Ġwritten Ġin ĠChinese Ġcharacters , Ġbut Ġancient ĠChinese Ġis Ġhighly Ġconcise Ġand Ġits Ġsynt act ical Ġstructure Ġis Ġdifferent Ġfrom Ġmodern ĠChinese . ĠThe Ġtraditional Ġmethods Ġdo Ġnot Ġtake Ġthese Ġcharacteristics Ġinto Ġaccount . ĠIn Ġthis Ġpaper , Ġwe Ġpropose Ġan Ġeffective Ġancient - modern ĠChinese Ġtext Ġalignment Ġmethod Ġat Ġthe Ġlevel Ġof Ġclause Ġbased Ġon Ġthe Ġcharacteristics Ġof Ġthese Ġtwo Ġlanguages . ĠThe Ġproposed Ġmethod Ġcombines Ġboth Ġlex ical - based Ġinformation Ġand Ġstatistical - based Ġinformation , Ġwhich Ġachieves Ġ94 . 2 ĠF 1 - score Ġon ĠTest Ġset . ĠRecently , Ġa Ġsimple Ġlongest Ġcommon Ġsubsequ ence Ġbased Ġapproach Ġfor Ġancient - modern ĠChinese Ġsentence Ġalignment Ġis Ġproposed Ġin ĠB IB REF 12 Ġ. ĠOur Ġexperiments Ġshowed Ġthat Ġour Ġproposed Ġalignment Ġapproach Ġperforms Ġmuch Ġbetter Ġthan Ġtheir Ġmethod . </s> We Ġapply Ġthe Ġproposed Ġmethod Ġto Ġcreate Ġa Ġlarge Ġtranslation Ġparallel Ġcorpus Ġwhich Ġcontains ĠIN LINE FORM 0 Ġ1 . 24 M Ġbilingual Ġsentence Ġpairs . ĠTo Ġour Ġbest Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġlarge Ġhigh - quality Ġancient - modern ĠChinese Ġdataset . ĠFurthermore , Ġwe Ġtest ĠSM T Ġmodels Ġand Ġvarious ĠN MT Ġmodels Ġon Ġthe Ġcreated Ġdataset Ġand Ġprovide Ġa Ġstrong Ġbaseline Ġfor Ġthis Ġtask . </s> Overview </s> There Ġare Ġfour Ġsteps Ġto Ġbuild Ġthe Ġancient - modern ĠChinese Ġtranslation Ġdataset : Ġ( i ) ĠThe Ġparallel Ġcorpus Ġcrawling Ġand Ġcleaning . Ġ( ii ) ĠThe Ġparagraph Ġalignment . Ġ( iii ) ĠThe Ġclause Ġalignment Ġbased Ġon Ġaligned Ġparagraphs . Ġ( iv ) ĠAug ment ing Ġdata Ġby Ġmerging Ġaligned Ġadjacent Ġclauses . ĠThe Ġmost Ġcritical Ġstep Ġis Ġthe Ġthird Ġstep . </s> Cl ause ĠAl ignment </s> In Ġthe Ġclause Ġalignment Ġstep , Ġwe Ġcombine Ġboth Ġstatistical - based Ġand Ġlex ical - based Ġinformation Ġto Ġmeasure Ġthe Ġscore Ġfor Ġeach Ġpossible Ġclause Ġalignment Ġbetween Ġancient Ġand Ġmodern ĠChinese Ġstrings . ĠThe Ġdynamic Ġprogramming Ġis Ġemployed Ġto Ġfurther Ġfind Ġoverall Ġoptimal Ġalignment Ġparagraph Ġby Ġparagraph . ĠAccording Ġto Ġthe Ġcharacteristics Ġof Ġthe Ġancient Ġand Ġmodern ĠChinese Ġlanguages , Ġwe Ġconsider Ġthe Ġfollowing Ġfactors Ġto Ġmeasure Ġthe Ġalignment Ġscore ĠIN LINE FORM 0 Ġbetween Ġa Ġbilingual Ġclause Ġpair : </s> Lex ical ĠMatch ing . ĠThe Ġlex ical Ġmatching Ġscore Ġis Ġused Ġto Ġcalculate Ġthe Ġmatching Ġcoverage Ġof Ġthe Ġancient Ġclause ĠIN LINE FORM 0 Ġ. ĠIt Ġcontains Ġtwo Ġparts : Ġexact Ġmatching Ġand Ġdictionary Ġmatching . ĠAn Ġancient ĠChinese Ġcharacter Ġusually Ġcorresponds Ġto Ġone Ġor Ġmore Ġmodern ĠChinese Ġwords . ĠIn Ġthe Ġfirst Ġpart , Ġwe Ġcarry Ġout ĠChinese ĠWord Ġsegment ation Ġto Ġthe Ġmodern ĠChinese Ġclause ĠIN LINE FORM 1 Ġ. ĠThen Ġwe Ġmatch Ġthe Ġancient Ġcharacters Ġand Ġmodern Ġwords Ġin Ġthe Ġorder Ġfrom Ġleft Ġto Ġright . ĠIn Ġfurther Ġmatching , Ġthe Ġwords Ġthat Ġhave Ġbeen Ġmatched Ġwill Ġbe Ġdeleted Ġfrom Ġthe Ġoriginal Ġclauses . </s> However , Ġsome Ġancient Ġcharacters Ġdo Ġnot Ġappear Ġin Ġits Ġcorresponding Ġmodern ĠChinese Ġwords . ĠAn Ġancient ĠChinese Ġdictionary Ġis Ġemployed Ġto Ġaddress Ġthis Ġissue . ĠWe Ġpre process Ġthe Ġancient ĠChinese Ġdictionary Ġand Ġremove Ġthe Ġstop Ġwords . ĠIn Ġthis Ġdictionary Ġmatching Ġstep , Ġwe Ġretrieve Ġthe Ġdictionary Ġdefinition Ġof Ġeach Ġunmatched Ġancient Ġcharacter Ġand Ġuse Ġit Ġto Ġmatch Ġthe Ġremaining Ġmodern ĠChinese Ġwords . ĠTo Ġreduce Ġthe Ġimpact Ġof Ġuniversal Ġword Ġmatching , Ġwe Ġuse ĠIn verse ĠDocument ĠFrequency Ġ( ID F ) Ġto Ġweight Ġthe Ġmatching Ġwords . ĠThe Ġlex ical Ġmatching Ġscore Ġis Ġcalculated Ġas : ĠDIS PLAY FORM 0 </s> The Ġabove Ġequation Ġis Ġused Ġto Ġcalculate Ġthe Ġmatching Ġcoverage Ġof Ġthe Ġancient Ġclause ĠIN LINE FORM 0 Ġ. ĠThe Ġfirst Ġterm Ġof Ġequation Ġ( ĠEQ REF 8 Ġ) Ġrepresents Ġexact Ġmatching Ġscore . ĠIN LINE FORM 1 Ġdenotes Ġthe Ġlength Ġof ĠIN LINE FORM 2 Ġ, ĠIN LINE FORM 3 Ġdenotes Ġeach Ġancient Ġcharacter Ġin ĠIN LINE FORM 4 Ġ, Ġand Ġthe Ġindicator Ġfunction ĠIN LINE FORM 5 Ġindicates Ġwhether Ġthe Ġcharacter ĠIN LINE FORM 6 Ġcan Ġmatch Ġthe Ġwords Ġin Ġthe Ġclause ĠIN LINE FORM 7 Ġ. ĠThe Ġsecond Ġterm Ġis Ġdictionary Ġmatching Ġscore . ĠHere ĠIN LINE FORM 8 Ġand ĠIN LINE FORM 9 Ġrepresent Ġthe Ġremaining Ġunmatched Ġstrings Ġof ĠIN LINE FORM 10 Ġand ĠIN LINE FORM 11 Ġ, Ġrespectively . ĠIN LINE FORM 12 Ġdenotes Ġthe ĠIN LINE FORM 13 Ġ- th Ġcharacter Ġin Ġthe Ġdictionary Ġdefinition Ġof Ġthe ĠIN LINE FORM 14 Ġand Ġits ĠIDF Ġscore Ġis Ġden oted Ġas ĠIN LINE FORM 15 Ġ. ĠThe ĠIN LINE FORM 16 Ġis Ġa Ġpred efined Ġparameter Ġwhich Ġis Ġused Ġto Ġnormal ize Ġthe ĠIDF Ġscore . ĠWe Ġtuned Ġthe Ġvalue Ġof Ġthis Ġparameter Ġon Ġthe ĠDev Ġset . </s> Stat istical ĠInformation . ĠSimilar Ġto ĠB IB REF 11 Ġand ĠB IB REF 6 Ġ, Ġthe Ġstatistical Ġinformation Ġcontains Ġalignment Ġmode Ġand Ġlength Ġinformation . ĠThere Ġare Ġmany Ġalignment Ġmodes Ġbetween Ġancient Ġand Ġmodern ĠChinese Ġlanguages . ĠIf Ġone Ġancient ĠChinese Ġclause Ġalign s Ġtwo Ġadjacent Ġmodern ĠChinese Ġclauses , Ġwe Ġcall Ġthis Ġalignment Ġas Ġ1 - 2 Ġalignment Ġmode . ĠWe Ġshow Ġsome Ġexamples Ġof Ġdifferent Ġalignment Ġmodes Ġin ĠFigure ĠFIG REF 9 Ġ. ĠIn Ġthis Ġpaper , Ġwe Ġonly Ġconsider Ġ1 - 0 , Ġ0 - 1 , Ġ1 - 1 , Ġ1 - 2 , Ġ2 - 1 Ġand Ġ2 - 2 Ġalignment Ġmodes Ġwhich Ġaccount Ġfor ĠIN LINE FORM 0 Ġof Ġthe ĠDev Ġset . ĠWe Ġestimate Ġthe Ġprobability ĠPr ĠIN LINE FORM 1 Ġn - m ĠIN LINE FORM 2 Ġof Ġeach Ġalignment Ġmode Ġn - m Ġon Ġthe ĠDev Ġset . ĠTo Ġutilize Ġlength Ġinformation , Ġwe Ġmake Ġan Ġinvestigation Ġon Ġlength Ġcorrelation Ġbetween Ġthese Ġtwo Ġlanguages . ĠBased Ġon Ġthe Ġassumption Ġof ĠB IB REF 11 Ġthat Ġeach Ġcharacter Ġin Ġone Ġlanguage Ġgives Ġrise Ġto Ġa Ġrandom Ġnumber Ġof Ġcharacters Ġin Ġthe Ġother Ġlanguage Ġand Ġthose Ġrandom Ġvariables ĠIN LINE FORM 3 Ġare Ġindependent Ġand Ġident ically Ġdistributed Ġwith Ġa Ġnormal Ġdistribution , Ġwe Ġestimate Ġthe Ġmean ĠIN LINE FORM 4 Ġand Ġstandard Ġdeviation ĠIN LINE FORM 5 Ġfrom Ġthe Ġparagraph Ġaligned Ġparallel Ġcorpus . ĠGiven Ġa Ġclause Ġpair ĠIN LINE FORM 6 Ġ, Ġthe Ġstatistical Ġinformation Ġscore Ġcan Ġbe Ġcalculated Ġby : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġdenotes Ġthe Ġnormal Ġdistribution Ġprobability Ġdensity Ġfunction . </s> Edit ĠDistance . ĠBecause Ġancient Ġand Ġmodern ĠChinese Ġare Ġboth Ġwritten Ġin ĠChinese Ġcharacters , Ġwe Ġalso Ġconsider Ġusing Ġthe Ġedit Ġdistance . ĠIt Ġis Ġa Ġway Ġof Ġquant ifying Ġthe Ġdiss imilar ity Ġbetween Ġtwo Ġstrings Ġby Ġcounting Ġthe Ġminimum Ġnumber Ġof Ġoperations Ġ( insert ion , Ġdeletion , Ġand Ġsubstitution ) Ġrequired Ġto Ġtransform Ġone Ġstring Ġinto Ġthe Ġother . ĠHere Ġwe Ġdefine Ġthe Ġedit Ġdistance Ġscore Ġas : ĠDIS PLAY FORM 0 </s> Dynamic ĠProgramming . ĠThe Ġoverall Ġalignment Ġscore Ġfor Ġeach Ġpossible Ġclause Ġalignment Ġis Ġas Ġfollows : ĠDIS PLAY FORM 0 </s> Here ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġare Ġpre - defined Ġinterpol ation Ġfactors . ĠWe Ġuse Ġdynamic Ġprogramming Ġto Ġfind Ġthe Ġoverall Ġoptimal Ġalignment Ġparagraph Ġby Ġparagraph . ĠLet ĠIN LINE FORM 2 Ġbe Ġtotal Ġalignment Ġscores Ġof Ġalign ing Ġthe Ġfirst Ġto ĠIN LINE FORM 3 Ġ- th Ġancient ĠChinese Ġclauses Ġwith Ġthe Ġfirst Ġto Ġto ĠIN LINE FORM 4 Ġ- th Ġmodern ĠChinese Ġclauses , Ġand Ġthe Ġrec urrence Ġthen Ġcan Ġbe Ġdescribed Ġas Ġfollows : ĠDIS PLAY FORM 0 </s> Where ĠIN LINE FORM 0 Ġdenotes Ġconc aten ate Ġclause ĠIN LINE FORM 1 Ġto Ġclause ĠIN LINE FORM 2 Ġ. ĠAs Ġwe Ġdiscussed Ġabove , Ġhere Ġwe Ġonly Ġconsider Ġ1 - 0 , Ġ0 - 1 , Ġ1 - 1 , Ġ1 - 2 , Ġ2 - 1 Ġand Ġ2 - 2 Ġalignment Ġmodes . </s> Ancient - Modern ĠChinese ĠDat as et </s> Data ĠCollection . ĠTo Ġbuild Ġthe Ġlarge Ġancient - modern ĠChinese Ġdataset , Ġwe Ġcollected Ġ1 . 7 K Ġbilingual Ġancient - modern ĠChinese Ġarticles Ġfrom Ġthe Ġinternet . ĠMore Ġspecifically , Ġa Ġlarge Ġpart Ġof Ġthe Ġancient ĠChinese Ġdata Ġwe Ġused Ġcome Ġfrom Ġancient ĠChinese Ġhistory Ġrecords Ġin Ġseveral Ġdyn ast ies Ġ( about Ġ1000 BC - 200 BC ) Ġand Ġarticles Ġwritten Ġby Ġcelebrities Ġof Ġthat Ġera . ĠThey Ġused Ġplain Ġand Ġaccurate Ġwords Ġto Ġexpress Ġwhat Ġhappened Ġat Ġthat Ġtime , Ġand Ġthus Ġensure Ġthe Ġgener ality Ġof Ġthe Ġtranslated Ġmaterials . </s> Par agraph ĠAl ignment . ĠTo Ġfurther Ġensure Ġthe Ġquality Ġof Ġthe Ġnew Ġdataset , Ġthe Ġwork Ġof Ġparagraph Ġalignment Ġis Ġmanually Ġcompleted . ĠAfter Ġdata Ġcleaning Ġand Ġmanual Ġparagraph Ġalignment , Ġwe Ġobtained Ġ35 K Ġaligned Ġbilingual Ġparagraphs . </s> Cl ause ĠAl ignment . ĠWe Ġapplied Ġour Ġclause Ġalignment Ġalgorithm Ġon Ġthe Ġ35 K Ġaligned Ġbilingual Ġparagraphs Ġand Ġobtained Ġ5 17 K Ġaligned Ġbilingual Ġclauses . ĠThe Ġreason Ġwe Ġuse Ġclause Ġalignment Ġalgorithm Ġinstead Ġof Ġsentence Ġalignment Ġis Ġbecause Ġwe Ġcan Ġconstruct Ġmore Ġaligned Ġsentences Ġmore Ġflex ibly Ġand Ġconveniently . ĠTo Ġbe Ġspecific , Ġwe Ġcan Ġget Ġmultiple Ġadditional Ġsentence Ġlevel Ġbilingual Ġpairs Ġby ĠâĢ ľ data Ġaug mentation âĢ Ŀ . </s> Data ĠAug mentation . ĠWe Ġaugmented Ġthe Ġdata Ġin Ġthe Ġfollowing Ġway : ĠGiven Ġan Ġaligned Ġclause Ġpair , Ġwe Ġmerged Ġits Ġadjacent Ġclause Ġpairs Ġas Ġa Ġnew Ġsample Ġpair . ĠFor Ġexample , Ġsuppose Ġwe Ġhave Ġthree Ġadjacent Ġclause Ġlevel Ġbilingual Ġpairs : Ġ( ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġ), Ġ( ĠIN LINE FORM 2 Ġ, ĠIN LINE FORM 3 Ġ), Ġand Ġ( ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġ). ĠWe Ġcan Ġget Ġsome Ġadditional Ġsentence Ġlevel Ġbilingual Ġpairs , Ġsuch Ġas : Ġ( ĠIN LINE FORM 6 Ġ, ĠIN LINE FORM 7 Ġ) Ġand Ġ( ĠIN LINE FORM 8 Ġ, ĠIN LINE FORM 9 Ġ). ĠHere ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġ, Ġand ĠIN LINE FORM 12 Ġare Ġadjacent Ġclauses Ġin Ġthe Ġoriginal Ġparagraph , Ġand ĠIN LINE FORM 13 Ġdenotes Ġconc aten ate Ġclause ĠIN LINE FORM 14 Ġto Ġclause ĠIN LINE FORM 15 Ġ. ĠThe Ġadvantage Ġof Ġusing Ġthis Ġdata Ġaug mentation Ġmethod Ġis Ġthat Ġcompared Ġwith Ġonly Ġusing Ġ( ĠIN LINE FORM 16 Ġ, ĠIN LINE FORM 17 Ġ) Ġas Ġthe Ġtraining Ġdata , Ġwe Ġcan Ġalso Ġuse Ġ( ĠIN LINE FORM 18 Ġ, ĠIN LINE FORM 19 Ġ) Ġand Ġ( ĠIN LINE FORM 20 Ġ, ĠIN LINE FORM 21 Ġ) Ġas Ġthe Ġtraining Ġdata , Ġwhich Ġcan Ġprovide Ġricher Ġsupervision Ġinformation Ġfor Ġthe Ġmodel Ġand Ġmake Ġthe Ġmodel Ġlearn Ġthe Ġalign Ġinformation Ġbetween Ġthe Ġsource Ġlanguage Ġand Ġthe Ġtarget Ġlanguage Ġbetter . ĠAfter Ġthe Ġdata Ġaug mentation , Ġwe Ġfiltered Ġthe Ġsentences Ġwhich Ġare Ġlonger Ġthan Ġ50 Ġor Ġcontain Ġmore Ġthan Ġfour Ġclause Ġpairs . </s> Dat as et ĠCreation . ĠFinally , Ġwe Ġsplit Ġthe Ġdataset Ġinto Ġthree Ġsets : Ġtraining Ġ( Train ), Ġdevelopment Ġ( Dev ) Ġand Ġtesting Ġ( Test ). ĠNote Ġthat Ġthe Ġun au gment ed Ġdataset Ġcontains Ġ5 17 K Ġaligned Ġbilingual Ġclause Ġpairs Ġfrom Ġ35 K Ġaligned Ġbilingual Ġparagraphs . ĠTo Ġkeep Ġall Ġthe Ġsentences Ġin Ġdifferent Ġsets Ġcome Ġfrom Ġdifferent Ġarticles , Ġwe Ġsplit Ġthe Ġ35 K Ġaligned Ġbilingual Ġparagraphs Ġinto ĠTrain , ĠDev Ġand ĠTest Ġsets Ġfollowing Ġthese Ġratios Ġrespectively : Ġ80 %, Ġ10 %, Ġ10 %. ĠBefore Ġdata Ġaug mentation , Ġthe Ġun au gment ed ĠTrain Ġset Ġcontains ĠIN LINE FORM 0 Ġaligned Ġbilingual Ġclause Ġpairs Ġfrom Ġ28 K Ġaligned Ġbilingual Ġparagraphs . ĠThen Ġwe Ġaugmented Ġthe ĠTrain , ĠDev Ġand ĠTest Ġsets Ġrespectively . ĠNote Ġthat Ġthe Ġaugmented ĠTrain , ĠDev Ġand ĠTest Ġsets Ġalso Ġcontain Ġthe Ġun au gment ed Ġdata . ĠThe Ġstatistical Ġinformation Ġof Ġthe Ġthree Ġdata Ġsets Ġis Ġshown Ġin ĠTable ĠT AB REF 17 Ġ. ĠWe Ġshow Ġsome Ġexamples Ġof Ġdata Ġin ĠFigure ĠFIG REF 14 Ġ. </s> R NN - based ĠN MT Ġmodel </s> We Ġfirst Ġbriefly Ġintroduce Ġthe ĠR NN Ġbased ĠNeural ĠMachine ĠTranslation Ġ( R NN - based ĠN MT ) Ġmodel . ĠThe ĠR NN - based ĠN MT Ġwith Ġattention Ġmechanism ĠB IB REF 0 Ġhas Ġachieved Ġremarkable Ġperformance Ġon Ġmany Ġtranslation Ġtasks . ĠIt Ġconsists Ġof Ġenc oder Ġand Ġdec oder Ġpart . </s> We Ġfirst ly Ġintroduce Ġthe Ġenc oder Ġpart . ĠThe Ġinput Ġword Ġsequence Ġof Ġsource Ġlanguage Ġare Ġindividually Ġmapped Ġinto Ġa ĠIN LINE FORM 0 Ġ- dimensional Ġvector Ġspace ĠIN LINE FORM 1 Ġ. ĠThen Ġa Ġbi - direction al ĠR NN ĠB IB REF 15 Ġwith ĠGR U ĠB IB REF 16 Ġor ĠL ST M ĠB IB REF 17 Ġcell Ġconverts Ġthese Ġvectors Ġinto Ġa Ġsequences Ġof Ġhidden Ġstates ĠIN LINE FORM 2 Ġ. </s> For Ġthe Ġdec oder Ġpart , Ġanother ĠR NN Ġis Ġused Ġto Ġgenerate Ġtarget Ġsequence ĠIN LINE FORM 0 Ġ. ĠThe Ġattention Ġmechanism ĠB IB REF 0 Ġ, ĠB IB REF 18 Ġis Ġemployed Ġto Ġallow Ġthe Ġdec oder Ġto Ġrefer Ġback Ġto Ġthe Ġhidden Ġstate Ġsequence Ġand Ġfocus Ġon Ġa Ġparticular Ġsegment . ĠThe ĠIN LINE FORM 1 Ġ- th Ġhidden Ġstate ĠIN LINE FORM 2 Ġof Ġdec oder Ġpart Ġis Ġcalculated Ġas : ĠDIS PLAY FORM 0 </s> Here Ġg ĠIN LINE FORM 0 Ġis Ġa Ġlinear Ġcombination Ġof Ġattended Ġcontext Ġvector Ġc ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġis Ġthe Ġword Ġembed ding Ġof Ġ( i - 1 )- th Ġtarget Ġword : ĠDIS PLAY FORM 0 </s> The Ġattended Ġcontext Ġvector Ġc ĠIN LINE FORM 0 Ġis Ġcomputed Ġas Ġa Ġweighted Ġsum Ġof Ġthe Ġhidden Ġstates Ġof Ġthe Ġenc oder : ĠDIS PLAY FORM 0 </s> The Ġprobability Ġdistribution Ġvector Ġof Ġthe Ġnext Ġword ĠIN LINE FORM 0 Ġis Ġgenerated Ġaccording Ġto Ġthe Ġfollowing : ĠDIS PLAY FORM 0 </s> We Ġtake Ġthis Ġmodel Ġas Ġthe Ġbasic ĠR NN - based ĠN MT Ġmodel Ġin Ġthe Ġfollowing Ġexperiments . </s> Trans former - N MT </s> Recently , Ġthe ĠTrans former Ġmodel ĠB IB REF 4 Ġhas Ġmade Ġremarkable Ġprogress Ġin Ġmachine Ġtranslation . ĠThis Ġmodel Ġcontains Ġa Ġmulti - head Ġself - att ention Ġenc oder Ġand Ġa Ġmulti - head Ġself - att ention Ġdec oder . </s> As Ġproposed Ġby ĠB IB REF 4 Ġ, Ġan Ġattention Ġfunction Ġmaps Ġa Ġquery Ġand Ġa Ġset Ġof Ġkey - value Ġpairs Ġto Ġan Ġoutput , Ġwhere Ġthe Ġqueries ĠIN LINE FORM 0 Ġ, Ġkeys ĠIN LINE FORM 1 Ġ, Ġand Ġvalues ĠIN LINE FORM 2 Ġare Ġall Ġvectors . ĠThe Ġinput Ġconsists Ġof Ġqueries Ġand Ġkeys Ġof Ġdimension ĠIN LINE FORM 3 Ġ, Ġand Ġvalues Ġof Ġdimension ĠIN LINE FORM 4 Ġ. ĠThe Ġattention Ġfunction Ġis Ġgiven Ġby : ĠDIS PLAY FORM 0 </s> Multi - head Ġattention Ġmechanism Ġprojects Ġqueries , Ġkeys Ġand Ġvalues Ġto ĠIN LINE FORM 0 Ġdifferent Ġrepresentation Ġsub sp aces Ġand Ġcalculates Ġcorresponding Ġattention . ĠThe Ġattention Ġfunction Ġoutputs Ġare Ġconc aten ated Ġand Ġprojected Ġagain Ġbefore Ġgiving Ġthe Ġfinal Ġoutput . ĠMulti - head Ġattention Ġallows Ġthe Ġmodel Ġto Ġattend Ġto Ġmultiple Ġfeatures Ġat Ġdifferent Ġpositions . </s> The Ġenc oder Ġis Ġcomposed Ġof Ġa Ġstack Ġof ĠIN LINE FORM 0 Ġidentical Ġlayers . ĠEach Ġlayer Ġhas Ġtwo Ġsub - l ayers : Ġmulti - head Ġself - att ention Ġmechanism Ġand Ġposition - wise Ġfully Ġconnected Ġfeed - forward Ġnetwork . ĠSimilarly , Ġthe Ġdec oder Ġis Ġalso Ġcomposed Ġof Ġa Ġstack Ġof ĠIN LINE FORM 1 Ġidentical Ġlayers . ĠIn Ġaddition Ġto Ġthe Ġtwo Ġsub - l ayers Ġin Ġeach Ġenc oder Ġlayer , Ġthe Ġdec oder Ġcontains Ġa Ġthird Ġsub - layer Ġwhich Ġperforms Ġmulti - head Ġattention Ġover Ġthe Ġoutput Ġof Ġthe Ġenc oder Ġstack Ġ( see Ġmore Ġdetails Ġin ĠB IB REF 4 Ġ). </s> Exper iments </s> Our Ġexperiments Ġrev olve Ġaround Ġthe Ġfollowing Ġquestions : ĠQ 1 : ĠAs Ġwe Ġconsider Ġthree Ġfactors Ġfor Ġclause Ġalignment , Ġdo Ġall Ġthese Ġfactors Ġhelp ? ĠHow Ġdoes Ġour Ġmethod Ġcompare Ġwith Ġprevious Ġmethods ? ĠQ 2 : ĠHow Ġdoes Ġthe ĠN MT Ġand ĠSM T Ġmodels Ġperform Ġon Ġthis Ġnew Ġdataset Ġwe Ġbuild ? </s> Cl ause ĠAl ignment ĠResults Ġ( Q 1 ) </s> In Ġorder Ġto Ġevaluate Ġour Ġclause Ġalignment Ġalgorithm , Ġwe Ġmanually Ġaligned Ġbilingual Ġclauses Ġfrom Ġ37 Ġbilingual Ġancient - modern ĠChinese Ġarticles , Ġand Ġfinally Ġgot Ġ4 K Ġaligned Ġbilingual Ġclauses Ġas Ġthe ĠTest Ġset Ġand Ġ2 K Ġclauses Ġas Ġthe ĠDev Ġset . </s> Met rics . ĠWe Ġused ĠF 1 - score Ġand Ġprecision Ġscore Ġas Ġthe Ġevaluation Ġmetrics . ĠSuppose Ġthat Ġwe Ġget ĠIN LINE FORM 0 Ġbilingual Ġclause Ġpairs Ġafter Ġrunning Ġthe Ġalgorithm Ġon Ġthe ĠTest Ġset , Ġand Ġthere Ġare ĠIN LINE FORM 1 Ġbilingual Ġclause Ġpairs Ġof Ġthese ĠIN LINE FORM 2 Ġpairs Ġare Ġin Ġthe Ġground Ġtruth Ġof Ġthe ĠTest Ġset , Ġthe Ġprecision Ġscore Ġis Ġdefined Ġas ĠIN LINE FORM 3 Ġ( the Ġalgorithm Ġgives ĠIN LINE FORM 4 Ġoutputs , ĠIN LINE FORM 5 Ġof Ġwhich Ġare Ġcorrect ). ĠAnd Ġsuppose Ġthat Ġthe Ġground Ġtruth Ġof Ġthe ĠTest Ġset Ġcontains ĠIN LINE FORM 6 Ġbilingual Ġclause Ġpairs , Ġthe Ġrecall Ġscore Ġis ĠIN LINE FORM 7 Ġ( there Ġare ĠIN LINE FORM 8 Ġground Ġtruth Ġsamples , ĠIN LINE FORM 9 Ġof Ġwhich Ġare Ġoutput Ġby Ġthe Ġalgorithm ), Ġthen Ġthe ĠF 1 - score Ġis ĠIN LINE FORM 10 Ġ. </s> Bas elines . ĠSince Ġthe Ġrelated Ġwork ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġcan Ġbe Ġseen Ġas Ġthe Ġab lation Ġcases Ġof Ġour Ġmethod Ġ( only Ġstatistical Ġscore ĠIN LINE FORM 0 Ġwith Ġdynamic Ġprogramming ), Ġwe Ġcompared Ġthe Ġfull Ġproposed Ġmethod Ġwith Ġits Ġvariants Ġon Ġthe ĠTest Ġset Ġfor Ġab lation Ġstudy . ĠIn Ġaddition , Ġwe Ġalso Ġcompared Ġour Ġmethod Ġwith Ġthe Ġlongest Ġcommon Ġsubsequ ence Ġ( LCS ) Ġbased Ġapproach Ġproposed Ġby ĠB IB REF 12 Ġ. ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , ĠB IB REF 12 Ġis Ġthe Ġlatest Ġrelated Ġwork Ġwhich Ġare Ġdesigned Ġfor ĠAncient - Modern ĠChinese Ġalignment . </s> Hyper - param eters . ĠFor Ġthe Ġproposed Ġmethod , Ġwe Ġestimated ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġon Ġall Ġaligned Ġparagraphs . ĠThe Ġprobability ĠPr ĠIN LINE FORM 2 Ġn - m ĠIN LINE FORM 3 Ġof Ġeach Ġalignment Ġmode Ġn - m Ġwas Ġestimated Ġon Ġthe ĠDev Ġset . ĠFor Ġthe Ġhyper - param eters ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġand ĠIN LINE FORM 6 Ġ, Ġthe Ġgrid Ġsearch Ġwas Ġapplied Ġto Ġtune Ġthem Ġon Ġthe ĠDev Ġset . ĠIn Ġorder Ġto Ġshow Ġthe Ġeffect Ġof Ġhyper - param eters ĠIN LINE FORM 7 Ġ, ĠIN LINE FORM 8 Ġ, Ġand ĠIN LINE FORM 9 Ġ, Ġwe Ġreported Ġthe Ġresults Ġof Ġvarious Ġhyper - param eters Ġon Ġthe ĠDev Ġset Ġin ĠTable ĠT AB REF 26 Ġ. ĠBased Ġon Ġthe Ġresults Ġof Ġgrid Ġsearch Ġon Ġthe ĠDev Ġset , Ġwe Ġset ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġ, Ġand ĠIN LINE FORM 12 Ġin Ġthe Ġfollowing Ġexperiment . ĠThe ĠJ ie ba ĠChinese Ġtext Ġsegment ation Ġis Ġemployed Ġfor Ġmodern ĠChinese Ġword Ġsegment ation . </s> Results . ĠThe Ġresults Ġon Ġthe ĠTest Ġset Ġare Ġshown Ġin ĠTable ĠT AB REF 28 Ġ, Ġthe Ġabbre viation Ġw / o Ġmeans Ġremoving Ġa Ġparticular Ġpart Ġfrom Ġthe Ġsetting . ĠFrom Ġthe Ġresults , Ġwe Ġcan Ġsee Ġthat Ġthe Ġlex ical Ġmatching Ġscore Ġis Ġthe Ġmost Ġimportant Ġamong Ġthese Ġthree Ġfactors , Ġand Ġstatistical Ġinformation Ġscore Ġis Ġmore Ġimportant Ġthan Ġedit Ġdistance Ġscore . ĠMoreover , Ġthe Ġdictionary Ġterm Ġin Ġlex ical Ġmatching Ġscore Ġsignificantly Ġimproves Ġthe Ġperformance . ĠFrom Ġthese Ġresults , Ġwe Ġobtain Ġthe Ġbest Ġsetting Ġthat Ġinvolves Ġall Ġthese Ġthree Ġfactors . ĠWe Ġused Ġthis Ġsetting Ġfor Ġdataset Ġcreation . ĠFurthermore , Ġthe Ġproposed Ġmethod Ġperforms Ġmuch Ġbetter Ġthan ĠLCS ĠB IB REF 12 Ġ. </s> Translation ĠResults Ġ( Q 2 ) </s> In Ġthis Ġexperiment , Ġwe Ġanalyzed Ġand Ġcompared Ġthe Ġperformance Ġof Ġthe ĠSM T Ġand Ġvarious ĠN MT Ġmodels Ġon Ġour Ġbuilt Ġdataset . ĠTo Ġverify Ġthe Ġeffectiveness Ġof Ġour Ġdata Ġaugmented Ġmethod . ĠWe Ġtrained Ġthe ĠN MT Ġand ĠSM T Ġmodels Ġon Ġboth Ġun au gment ed Ġdataset Ġ( including Ġ0 . 46 M Ġtraining Ġpairs ) Ġand Ġaugmented Ġdataset , Ġand Ġtest Ġall Ġthe Ġmodels Ġon Ġthe Ġsame ĠTest Ġset Ġwhich Ġis Ġaugmented . ĠThe Ġmodels Ġto Ġbe Ġtested Ġand Ġtheir Ġconfigurations Ġare Ġas Ġfollows : </s> SM T . ĠThe Ġstate - of - art ĠMoses Ġtool kit ĠB IB REF 19 Ġwas Ġused Ġto Ġtrain ĠSM T Ġmodel . ĠWe Ġused ĠKen LM ĠB IB REF 20 Ġto Ġtrain Ġa Ġ5 - gram Ġlanguage Ġmodel , Ġand Ġthe ĠG IZ A ++ Ġtool kit Ġto Ġalign Ġthe Ġdata . </s> R NN - based ĠN MT . ĠThe Ġbasic ĠR NN - based ĠN MT Ġmodel Ġis Ġbased Ġon ĠB IB REF 0 Ġwhich Ġis Ġintroduced Ġabove . ĠBoth Ġthe Ġenc oder Ġand Ġdec oder Ġused Ġ2 - layer ĠR NN Ġwith Ġ1024 ĠL ST M Ġcells , Ġand Ġthe Ġenc oder Ġis Ġa Ġbi - direction al ĠR NN . ĠThe Ġbatch Ġsize , Ġthreshold Ġof Ġelement - wise Ġgradient Ġclipping Ġand Ġinitial Ġlearning Ġrate Ġof ĠAdam Ġoptim izer ĠB IB REF 21 Ġwere Ġset Ġto Ġ128 , Ġ5 . 0 Ġand Ġ0 . 001 . ĠWhen Ġtrained Ġthe Ġmodel Ġon Ġaugmented Ġdataset , Ġwe Ġused Ġ4 - layer ĠR NN . ĠSeveral Ġtechniques Ġwere Ġinvestigated Ġto Ġtrain Ġthe Ġmodel , Ġincluding Ġlayer - normal ization ĠB IB REF 22 Ġ, ĠR NN - drop out ĠB IB REF 23 Ġ, Ġand Ġlearning Ġrate Ġdecay ĠB IB REF 1 Ġ. ĠThe Ġhyper - param eters Ġwere Ġchosen Ġempir ically Ġand Ġadjusted Ġin Ġthe ĠDev Ġset . ĠFurthermore , Ġwe Ġtested Ġthe Ġbasic ĠN MT Ġmodel Ġwith Ġseveral Ġtechniques , Ġsuch Ġas Ġtarget Ġlanguage Ġreversal ĠB IB REF 24 Ġ( re vers ing Ġthe Ġorder Ġof Ġthe Ġwords Ġin Ġall Ġtarget Ġsentences , Ġbut Ġnot Ġsource Ġsentences ), Ġresidual Ġconnection ĠB IB REF 25 Ġand Ġpre - trained Ġword 2 vec ĠB IB REF 26 Ġ. ĠFor Ġword Ġembed ding Ġpre - training , Ġwe Ġcollected Ġan Ġexternal Ġancient Ġcorpus Ġwhich Ġcontains ĠIN LINE FORM 0 Ġ134 M Ġtokens . </s> Trans former - N MT . ĠWe Ġalso Ġtrained Ġthe ĠTrans former Ġmodel ĠB IB REF 4 Ġwhich Ġis Ġa Ġstrong Ġbaseline Ġof ĠN MT Ġon Ġboth Ġaugmented Ġand Ġun au gment ed Ġparallel Ġcorpus . ĠThe Ġtraining Ġconfiguration Ġof Ġthe ĠTrans former Ġmodel Ġis Ġshown Ġin ĠTable ĠT AB REF 32 Ġ. ĠThe Ġhyper - param eters Ġare Ġset Ġbased Ġon Ġthe Ġsettings Ġin Ġthe Ġpaper ĠB IB REF 4 Ġand Ġthe Ġsizes Ġof Ġour Ġtraining Ġsets . </s> For Ġthe Ġevaluation , Ġwe Ġused Ġthe Ġaverage Ġof Ġ1 Ġto Ġ4 Ġgram ĠB LE Us Ġmultiplied Ġby Ġa Ġbre vity Ġpenalty ĠB IB REF 27 Ġwhich Ġcomputed Ġby Ġmulti - ble u . per l Ġin ĠMoses Ġas Ġmetrics . ĠThe Ġresults Ġare Ġreported Ġin ĠTable ĠT AB REF 34 Ġ. ĠFor ĠR NN - based ĠN MT , Ġwe Ġcan Ġsee Ġthat Ġtarget Ġlanguage Ġreversal , Ġresidual Ġconnection , Ġand Ġword 2 vec Ġcan Ġfurther Ġimprove Ġthe Ġperformance Ġof Ġthe Ġbasic ĠR NN - based ĠN MT Ġmodel . ĠHowever , Ġwe Ġfind Ġthat Ġword 2 vec Ġand Ġreversal Ġtricks Ġseem Ġno Ġobvious Ġimprovement Ġwhen Ġtrained Ġthe ĠR NN - based ĠN MT Ġand ĠTrans former Ġmodels Ġon Ġaugmented Ġparallel Ġcorpus . ĠFor ĠSM T , Ġit Ġperforms Ġbetter Ġthan ĠN MT Ġmodels Ġwhen Ġthey Ġwere Ġtrained Ġon Ġthe Ġun au gment ed Ġdataset . ĠNevertheless , Ġwhen Ġtrained Ġon Ġthe Ġaugmented Ġdataset , Ġboth Ġthe ĠR NN - based ĠN MT Ġmodel Ġand ĠTrans former Ġbased ĠN MT Ġmodel Ġoutper form Ġthe ĠSM T Ġmodel . ĠIn Ġaddition , Ġas Ġwith Ġother Ġtranslation Ġtasks ĠB IB REF 4 Ġ, Ġthe ĠTrans former Ġalso Ġperforms Ġbetter Ġthan ĠR NN - based ĠN MT . </s> Because Ġthe ĠTest Ġset Ġcontains Ġboth Ġaugmented Ġand Ġun au gment ed Ġdata , Ġit Ġis Ġnot Ġsurprising Ġthat Ġthe ĠR NN - based ĠN MT Ġmodel Ġand ĠTrans former Ġbased ĠN MT Ġmodel Ġtrained Ġon Ġun au gment ed Ġdata Ġwould Ġperform Ġpoorly . ĠIn Ġorder Ġto Ġfurther Ġverify Ġthe Ġeffect Ġof Ġdata Ġaug mentation , Ġwe Ġreport Ġthe Ġtest Ġresults Ġof Ġthe Ġmodels Ġon Ġonly Ġun au gment ed Ġtest Ġdata Ġ( including Ġ48 K Ġtest Ġpairs ) Ġin ĠTable ĠT AB REF 35 Ġ. ĠFrom Ġthe Ġresults , Ġit Ġcan Ġbe Ġseen Ġthat Ġthe Ġdata Ġaug mentation Ġcan Ġstill Ġimprove Ġthe Ġmodels . </s> Analysis </s> The Ġgenerated Ġsamples Ġof Ġvarious Ġmodels Ġare Ġshown Ġin ĠFigure ĠFIG REF 36 Ġ. ĠBesides ĠB LE U Ġscores , Ġwe Ġanalyze Ġthese Ġexamples Ġfrom Ġa Ġhuman Ġperspective Ġand Ġdraw Ġsome Ġconclusions . ĠAt Ġthe Ġsame Ġtime , Ġwe Ġdesign Ġdifferent Ġmetrics Ġand Ġevaluate Ġon Ġthe Ġwhole ĠTest Ġset Ġto Ġsupport Ġour Ġconclusions Ġas Ġfollows : </s> On Ġthe Ġone Ġhand , Ġwe Ġfurther Ġcompare Ġthe Ġtranslation Ġresults Ġfrom Ġthe Ġperspective Ġof Ġpeople . ĠWe Ġfind Ġthat Ġalthough Ġthe Ġoriginal Ġmeaning Ġcan Ġbe Ġbasically Ġtranslated Ġby ĠSM T , Ġits Ġtranslation Ġresults Ġare Ġless Ġsmooth Ġwhen Ġcompared Ġwith Ġthe Ġother Ġtwo ĠN MT Ġmodels Ġ( R NN - based ĠN MT Ġand ĠTrans former ). ĠFor Ġexample , Ġthe Ġtranslations Ġof ĠSM T Ġare Ġusually Ġlack Ġof Ġauxiliary Ġwords , Ġconj unctions Ġand Ġfunction Ġwords , Ġwhich Ġis Ġnot Ġconsistent Ġwith Ġhuman Ġtranslation Ġhabits . ĠTo Ġfurther Ġconfirm Ġthis Ġconclusion , Ġthe Ġaverage Ġlength Ġof Ġthe Ġtranslation Ġresults Ġof Ġthe Ġthree Ġmodels Ġare Ġmeasured Ġ( R NN - based ĠN MT : 17 . 12 , ĠSM T : 15 . 50 , ĠTrans former : 16 . 78 , ĠReference : 16 . 47 ). ĠWe Ġcan Ġsee Ġthat Ġthe Ġaverage Ġlength Ġof Ġthe ĠSM T Ġoutputs Ġis Ġshortest , Ġand Ġthe Ġlength Ġgaps Ġbetween Ġthe ĠSM T Ġoutputs Ġand Ġthe Ġreferences Ġare Ġlargest . ĠMeanwhile , Ġthe Ġaverage Ġlength Ġof Ġthe Ġsentences Ġtranslated Ġby ĠTrans former Ġis Ġclosest Ġto Ġthe Ġaverage Ġlength Ġof Ġreferences . ĠThese Ġresults Ġindirectly Ġverify Ġour Ġpoint Ġof Ġview , Ġand Ġshow Ġthat Ġthe ĠN MT Ġmodels Ġperform Ġbetter Ġthan ĠSM T Ġin Ġthis Ġtask . </s> On Ġthe Ġother Ġhand , Ġthere Ġstill Ġexists Ġsome Ġproblems Ġto Ġbe Ġsolved . ĠWe Ġobserve Ġthat Ġtranslating Ġproper Ġnoun s Ġand Ġpersonal Ġpronouns Ġ( such Ġas Ġnames , Ġplace Ġnames Ġand Ġancient - specific Ġappell ations ) Ġis Ġvery Ġdifficult Ġfor Ġall Ġof Ġthese Ġmodels . ĠFor Ġinstance , Ġthe Ġancient ĠChinese Ġappell ation Ġ` Z hen ' Ġshould Ġbe Ġtranslated Ġinto Ġ` Wo ' Ġin Ġmodern ĠChinese . ĠUnfortunately , Ġwe Ġcalculate Ġthe Ġaccurate Ġrate Ġof Ġsome Ġspecial Ġwords Ġ( such Ġas Ġ` Z hen ', ` C hen ' Ġand Ġ` G ua '), Ġand Ġfind Ġthat Ġthis Ġrate Ġis Ġvery Ġlow Ġ( the Ġaccurate Ġrate Ġof Ġtranslating Ġ` Z hen ' Ġare : ĠR NN - based ĠN MT : 0 . 14 , ĠSM T : 0 . 16 , ĠTrans former : 0 . 05 ). ĠWe Ġwill Ġfocus Ġon Ġthis Ġissue Ġin Ġthe Ġfuture . </s> Conclusion Ġand ĠFuture ĠWork </s> We Ġpropose Ġan Ġeffective Ġancient - modern ĠChinese Ġclause Ġalignment Ġmethod Ġwhich Ġachieves Ġ94 . 2 ĠF 1 - score Ġon ĠTest Ġset . ĠBased Ġon Ġit , Ġwe Ġbuild Ġa Ġlarge Ġscale Ġparallel Ġcorpus Ġwhich Ġcontains ĠIN LINE FORM 0 Ġ1 . 24 M Ġbilingual Ġsentence Ġpairs . ĠTo Ġour Ġbest Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġlarge Ġhigh - quality Ġancient - modern ĠChinese Ġdataset . ĠIn Ġaddition , Ġwe Ġtest Ġthe Ġperformance Ġof Ġthe ĠSM T Ġand Ġvarious ĠN MT Ġmodels Ġon Ġour Ġbuilt Ġdataset Ġand Ġprovide Ġa Ġstrong ĠN MT Ġbaseline Ġfor Ġthis Ġtask Ġwhich Ġachieves Ġ27 . 16 ĠB LE U Ġscore Ġ( 4 - gram ). ĠWe Ġfurther Ġanalyze Ġthe Ġperformance Ġof Ġthe ĠSM T Ġand Ġvarious ĠN MT Ġmodels Ġand Ġsummarize Ġsome Ġspecific Ġproblems Ġthat Ġmachine Ġtranslation Ġmodels Ġwill Ġencounter Ġwhen Ġtranslating Ġancient ĠChinese . </s> For Ġthe Ġfuture Ġwork , Ġfirst ly , Ġwe Ġare Ġgoing Ġto Ġexpand Ġthe Ġdataset Ġusing Ġthe Ġproposed Ġmethod Ġcontinually . ĠSecondly , Ġwe Ġwill Ġfocus Ġon Ġsolving Ġthe Ġproblem Ġof Ġproper Ġnoun Ġtranslation Ġand Ġimprove Ġthe Ġtranslation Ġsystem Ġaccording Ġto Ġthe Ġfeatures Ġof Ġancient ĠChinese Ġtranslation . ĠFinally , Ġwe Ġplan Ġto Ġintroduce Ġsome Ġtechniques Ġof Ġstatistical Ġtranslation Ġinto Ġneural Ġmachine Ġtranslation Ġto Ġimprove Ġthe Ġperformance . </s> This Ġwork Ġis Ġsupported Ġby ĠNational ĠNatural ĠScience ĠFund Ġfor ĠDist inguished ĠYoung ĠScholar Ġ( Grant ĠNo . Ġ6 16 25 204 ) Ġand Ġpartially Ġsupported Ġby Ġthe ĠState ĠKey ĠProgram Ġof ĠNational ĠScience ĠFoundation Ġof ĠChina Ġ( Grant ĠNos . Ġ6 18 36 006 Ġand Ġ6 143 2014 ).
Introduction Ancient Chinese is the writing language in ancient China. It is a treasure of Chinese culture which brings together the wisdom and ideas of the Chinese nation and chronicles the ancient cultural heritage of China. Learning ancient Chinese not only helps people to understand and inherit the wisdom of the ancients, but also promotes people to absorb and develop Chinese culture. However, it is difficult for modern people to read ancient Chinese. Firstly, compared with modern Chinese, ancient Chinese is more concise and shorter. The grammatical order of modern Chinese is also quite different from that of ancient Chinese. Secondly, most modern Chinese words are double syllables, while the most of the ancient Chinese words are monosyllabic. Thirdly, there is more than one polysemous phenomenon in ancient Chinese. In addition, manual translation has a high cost. Therefore, it is meaningful and useful to study the automatic translation from ancient Chinese to modern Chinese. Through ancient-modern Chinese translation, the wisdom, talent and accumulated experience of the predecessors can be passed on to more people. Neural machine translation (NMT) BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 has achieved remarkable performance on many bilingual translation tasks. It is an end-to-end learning approach for machine translation, with the potential to show great advantages over the statistic machine translation (SMT) systems. However, NMT approach has not been widely applied to the ancient-modern Chinese translation task. One of the main reasons is the limited high-quality parallel data resource. The most popular method of acquiring translation examples is bilingual text alignment BIBREF5 . This kind of method can be classified into two types: lexical-based and statistical-based. The lexical-based approaches BIBREF6 , BIBREF7 focus on lexical information, which utilize the bilingual dictionary BIBREF8 , BIBREF9 or lexical features. Meanwhile, the statistical-based approaches BIBREF10 , BIBREF11 rely on statistical information, such as sentence length ratio in two languages and align mode probability. However, these methods are designed for other bilingual language pairs that are written in different language characters (e.g. English-French, Chinese-Japanese). The ancient-modern Chinese has some characteristics that are quite different from other language pairs. For example, ancient and modern Chinese are both written in Chinese characters, but ancient Chinese is highly concise and its syntactical structure is different from modern Chinese. The traditional methods do not take these characteristics into account. In this paper, we propose an effective ancient-modern Chinese text alignment method at the level of clause based on the characteristics of these two languages. The proposed method combines both lexical-based information and statistical-based information, which achieves 94.2 F1-score on Test set. Recently, a simple longest common subsequence based approach for ancient-modern Chinese sentence alignment is proposed in BIBREF12 . Our experiments showed that our proposed alignment approach performs much better than their method. We apply the proposed method to create a large translation parallel corpus which contains INLINEFORM0 1.24M bilingual sentence pairs. To our best knowledge, this is the first large high-quality ancient-modern Chinese dataset. Furthermore, we test SMT models and various NMT models on the created dataset and provide a strong baseline for this task. Overview There are four steps to build the ancient-modern Chinese translation dataset: (i) The parallel corpus crawling and cleaning. (ii) The paragraph alignment. (iii) The clause alignment based on aligned paragraphs. (iv) Augmenting data by merging aligned adjacent clauses. The most critical step is the third step. Clause Alignment In the clause alignment step, we combine both statistical-based and lexical-based information to measure the score for each possible clause alignment between ancient and modern Chinese strings. The dynamic programming is employed to further find overall optimal alignment paragraph by paragraph. According to the characteristics of the ancient and modern Chinese languages, we consider the following factors to measure the alignment score INLINEFORM0 between a bilingual clause pair: Lexical Matching. The lexical matching score is used to calculate the matching coverage of the ancient clause INLINEFORM0 . It contains two parts: exact matching and dictionary matching. An ancient Chinese character usually corresponds to one or more modern Chinese words. In the first part, we carry out Chinese Word segmentation to the modern Chinese clause INLINEFORM1 . Then we match the ancient characters and modern words in the order from left to right. In further matching, the words that have been matched will be deleted from the original clauses. However, some ancient characters do not appear in its corresponding modern Chinese words. An ancient Chinese dictionary is employed to address this issue. We preprocess the ancient Chinese dictionary and remove the stop words. In this dictionary matching step, we retrieve the dictionary definition of each unmatched ancient character and use it to match the remaining modern Chinese words. To reduce the impact of universal word matching, we use Inverse Document Frequency (IDF) to weight the matching words. The lexical matching score is calculated as: DISPLAYFORM0 The above equation is used to calculate the matching coverage of the ancient clause INLINEFORM0 . The first term of equation ( EQREF8 ) represents exact matching score. INLINEFORM1 denotes the length of INLINEFORM2 , INLINEFORM3 denotes each ancient character in INLINEFORM4 , and the indicator function INLINEFORM5 indicates whether the character INLINEFORM6 can match the words in the clause INLINEFORM7 . The second term is dictionary matching score. Here INLINEFORM8 and INLINEFORM9 represent the remaining unmatched strings of INLINEFORM10 and INLINEFORM11 , respectively. INLINEFORM12 denotes the INLINEFORM13 -th character in the dictionary definition of the INLINEFORM14 and its IDF score is denoted as INLINEFORM15 . The INLINEFORM16 is a predefined parameter which is used to normalize the IDF score. We tuned the value of this parameter on the Dev set. Statistical Information. Similar to BIBREF11 and BIBREF6 , the statistical information contains alignment mode and length information. There are many alignment modes between ancient and modern Chinese languages. If one ancient Chinese clause aligns two adjacent modern Chinese clauses, we call this alignment as 1-2 alignment mode. We show some examples of different alignment modes in Figure FIGREF9 . In this paper, we only consider 1-0, 0-1, 1-1, 1-2, 2-1 and 2-2 alignment modes which account for INLINEFORM0 of the Dev set. We estimate the probability Pr INLINEFORM1 n-m INLINEFORM2 of each alignment mode n-m on the Dev set. To utilize length information, we make an investigation on length correlation between these two languages. Based on the assumption of BIBREF11 that each character in one language gives rise to a random number of characters in the other language and those random variables INLINEFORM3 are independent and identically distributed with a normal distribution, we estimate the mean INLINEFORM4 and standard deviation INLINEFORM5 from the paragraph aligned parallel corpus. Given a clause pair INLINEFORM6 , the statistical information score can be calculated by: DISPLAYFORM0 where INLINEFORM0 denotes the normal distribution probability density function. Edit Distance. Because ancient and modern Chinese are both written in Chinese characters, we also consider using the edit distance. It is a way of quantifying the dissimilarity between two strings by counting the minimum number of operations (insertion, deletion, and substitution) required to transform one string into the other. Here we define the edit distance score as: DISPLAYFORM0 Dynamic Programming. The overall alignment score for each possible clause alignment is as follows: DISPLAYFORM0 Here INLINEFORM0 and INLINEFORM1 are pre-defined interpolation factors. We use dynamic programming to find the overall optimal alignment paragraph by paragraph. Let INLINEFORM2 be total alignment scores of aligning the first to INLINEFORM3 -th ancient Chinese clauses with the first to to INLINEFORM4 -th modern Chinese clauses, and the recurrence then can be described as follows: DISPLAYFORM0 Where INLINEFORM0 denotes concatenate clause INLINEFORM1 to clause INLINEFORM2 . As we discussed above, here we only consider 1-0, 0-1, 1-1, 1-2, 2-1 and 2-2 alignment modes. Ancient-Modern Chinese Dataset Data Collection. To build the large ancient-modern Chinese dataset, we collected 1.7K bilingual ancient-modern Chinese articles from the internet. More specifically, a large part of the ancient Chinese data we used come from ancient Chinese history records in several dynasties (about 1000BC-200BC) and articles written by celebrities of that era. They used plain and accurate words to express what happened at that time, and thus ensure the generality of the translated materials. Paragraph Alignment. To further ensure the quality of the new dataset, the work of paragraph alignment is manually completed. After data cleaning and manual paragraph alignment, we obtained 35K aligned bilingual paragraphs. Clause Alignment. We applied our clause alignment algorithm on the 35K aligned bilingual paragraphs and obtained 517K aligned bilingual clauses. The reason we use clause alignment algorithm instead of sentence alignment is because we can construct more aligned sentences more flexibly and conveniently. To be specific, we can get multiple additional sentence level bilingual pairs by “data augmentation”. Data Augmentation. We augmented the data in the following way: Given an aligned clause pair, we merged its adjacent clause pairs as a new sample pair. For example, suppose we have three adjacent clause level bilingual pairs: ( INLINEFORM0 , INLINEFORM1 ), ( INLINEFORM2 , INLINEFORM3 ), and ( INLINEFORM4 , INLINEFORM5 ). We can get some additional sentence level bilingual pairs, such as: ( INLINEFORM6 , INLINEFORM7 ) and ( INLINEFORM8 , INLINEFORM9 ). Here INLINEFORM10 , INLINEFORM11 , and INLINEFORM12 are adjacent clauses in the original paragraph, and INLINEFORM13 denotes concatenate clause INLINEFORM14 to clause INLINEFORM15 . The advantage of using this data augmentation method is that compared with only using ( INLINEFORM16 , INLINEFORM17 ) as the training data, we can also use ( INLINEFORM18 , INLINEFORM19 ) and ( INLINEFORM20 , INLINEFORM21 ) as the training data, which can provide richer supervision information for the model and make the model learn the align information between the source language and the target language better. After the data augmentation, we filtered the sentences which are longer than 50 or contain more than four clause pairs. Dataset Creation. Finally, we split the dataset into three sets: training (Train), development (Dev) and testing (Test). Note that the unaugmented dataset contains 517K aligned bilingual clause pairs from 35K aligned bilingual paragraphs. To keep all the sentences in different sets come from different articles, we split the 35K aligned bilingual paragraphs into Train, Dev and Test sets following these ratios respectively: 80%, 10%, 10%. Before data augmentation, the unaugmented Train set contains INLINEFORM0 aligned bilingual clause pairs from 28K aligned bilingual paragraphs. Then we augmented the Train, Dev and Test sets respectively. Note that the augmented Train, Dev and Test sets also contain the unaugmented data. The statistical information of the three data sets is shown in Table TABREF17 . We show some examples of data in Figure FIGREF14 . RNN-based NMT model We first briefly introduce the RNN based Neural Machine Translation (RNN-based NMT) model. The RNN-based NMT with attention mechanism BIBREF0 has achieved remarkable performance on many translation tasks. It consists of encoder and decoder part. We firstly introduce the encoder part. The input word sequence of source language are individually mapped into a INLINEFORM0 -dimensional vector space INLINEFORM1 . Then a bi-directional RNN BIBREF15 with GRU BIBREF16 or LSTM BIBREF17 cell converts these vectors into a sequences of hidden states INLINEFORM2 . For the decoder part, another RNN is used to generate target sequence INLINEFORM0 . The attention mechanism BIBREF0 , BIBREF18 is employed to allow the decoder to refer back to the hidden state sequence and focus on a particular segment. The INLINEFORM1 -th hidden state INLINEFORM2 of decoder part is calculated as: DISPLAYFORM0 Here g INLINEFORM0 is a linear combination of attended context vector c INLINEFORM1 and INLINEFORM2 is the word embedding of (i-1)-th target word: DISPLAYFORM0 The attended context vector c INLINEFORM0 is computed as a weighted sum of the hidden states of the encoder: DISPLAYFORM0 The probability distribution vector of the next word INLINEFORM0 is generated according to the following: DISPLAYFORM0 We take this model as the basic RNN-based NMT model in the following experiments. Transformer-NMT Recently, the Transformer model BIBREF4 has made remarkable progress in machine translation. This model contains a multi-head self-attention encoder and a multi-head self-attention decoder. As proposed by BIBREF4 , an attention function maps a query and a set of key-value pairs to an output, where the queries INLINEFORM0 , keys INLINEFORM1 , and values INLINEFORM2 are all vectors. The input consists of queries and keys of dimension INLINEFORM3 , and values of dimension INLINEFORM4 . The attention function is given by: DISPLAYFORM0 Multi-head attention mechanism projects queries, keys and values to INLINEFORM0 different representation subspaces and calculates corresponding attention. The attention function outputs are concatenated and projected again before giving the final output. Multi-head attention allows the model to attend to multiple features at different positions. The encoder is composed of a stack of INLINEFORM0 identical layers. Each layer has two sub-layers: multi-head self-attention mechanism and position-wise fully connected feed-forward network. Similarly, the decoder is also composed of a stack of INLINEFORM1 identical layers. In addition to the two sub-layers in each encoder layer, the decoder contains a third sub-layer which performs multi-head attention over the output of the encoder stack (see more details in BIBREF4 ). Experiments Our experiments revolve around the following questions: Q1: As we consider three factors for clause alignment, do all these factors help? How does our method compare with previous methods? Q2: How does the NMT and SMT models perform on this new dataset we build? Clause Alignment Results (Q1) In order to evaluate our clause alignment algorithm, we manually aligned bilingual clauses from 37 bilingual ancient-modern Chinese articles, and finally got 4K aligned bilingual clauses as the Test set and 2K clauses as the Dev set. Metrics. We used F1-score and precision score as the evaluation metrics. Suppose that we get INLINEFORM0 bilingual clause pairs after running the algorithm on the Test set, and there are INLINEFORM1 bilingual clause pairs of these INLINEFORM2 pairs are in the ground truth of the Test set, the precision score is defined as INLINEFORM3 (the algorithm gives INLINEFORM4 outputs, INLINEFORM5 of which are correct). And suppose that the ground truth of the Test set contains INLINEFORM6 bilingual clause pairs, the recall score is INLINEFORM7 (there are INLINEFORM8 ground truth samples, INLINEFORM9 of which are output by the algorithm), then the F1-score is INLINEFORM10 . Baselines. Since the related work BIBREF10 , BIBREF11 can be seen as the ablation cases of our method (only statistical score INLINEFORM0 with dynamic programming), we compared the full proposed method with its variants on the Test set for ablation study. In addition, we also compared our method with the longest common subsequence (LCS) based approach proposed by BIBREF12 . To the best of our knowledge, BIBREF12 is the latest related work which are designed for Ancient-Modern Chinese alignment. Hyper-parameters. For the proposed method, we estimated INLINEFORM0 and INLINEFORM1 on all aligned paragraphs. The probability Pr INLINEFORM2 n-m INLINEFORM3 of each alignment mode n-m was estimated on the Dev set. For the hyper-parameters INLINEFORM4 , INLINEFORM5 and INLINEFORM6 , the grid search was applied to tune them on the Dev set. In order to show the effect of hyper-parameters INLINEFORM7 , INLINEFORM8 , and INLINEFORM9 , we reported the results of various hyper-parameters on the Dev set in Table TABREF26 . Based on the results of grid search on the Dev set, we set INLINEFORM10 , INLINEFORM11 , and INLINEFORM12 in the following experiment. The Jieba Chinese text segmentation is employed for modern Chinese word segmentation. Results. The results on the Test set are shown in Table TABREF28 , the abbreviation w/o means removing a particular part from the setting. From the results, we can see that the lexical matching score is the most important among these three factors, and statistical information score is more important than edit distance score. Moreover, the dictionary term in lexical matching score significantly improves the performance. From these results, we obtain the best setting that involves all these three factors. We used this setting for dataset creation. Furthermore, the proposed method performs much better than LCS BIBREF12 . Translation Results (Q2) In this experiment, we analyzed and compared the performance of the SMT and various NMT models on our built dataset. To verify the effectiveness of our data augmented method. We trained the NMT and SMT models on both unaugmented dataset (including 0.46M training pairs) and augmented dataset, and test all the models on the same Test set which is augmented. The models to be tested and their configurations are as follows: SMT. The state-of-art Moses toolkit BIBREF19 was used to train SMT model. We used KenLM BIBREF20 to train a 5-gram language model, and the GIZA++ toolkit to align the data. RNN-based NMT. The basic RNN-based NMT model is based on BIBREF0 which is introduced above. Both the encoder and decoder used 2-layer RNN with 1024 LSTM cells, and the encoder is a bi-directional RNN. The batch size, threshold of element-wise gradient clipping and initial learning rate of Adam optimizer BIBREF21 were set to 128, 5.0 and 0.001. When trained the model on augmented dataset, we used 4-layer RNN. Several techniques were investigated to train the model, including layer-normalization BIBREF22 , RNN-dropout BIBREF23 , and learning rate decay BIBREF1 . The hyper-parameters were chosen empirically and adjusted in the Dev set. Furthermore, we tested the basic NMT model with several techniques, such as target language reversal BIBREF24 (reversing the order of the words in all target sentences, but not source sentences), residual connection BIBREF25 and pre-trained word2vec BIBREF26 . For word embedding pre-training, we collected an external ancient corpus which contains INLINEFORM0 134M tokens. Transformer-NMT. We also trained the Transformer model BIBREF4 which is a strong baseline of NMT on both augmented and unaugmented parallel corpus. The training configuration of the Transformer model is shown in Table TABREF32 . The hyper-parameters are set based on the settings in the paper BIBREF4 and the sizes of our training sets. For the evaluation, we used the average of 1 to 4 gram BLEUs multiplied by a brevity penalty BIBREF27 which computed by multi-bleu.perl in Moses as metrics. The results are reported in Table TABREF34 . For RNN-based NMT, we can see that target language reversal, residual connection, and word2vec can further improve the performance of the basic RNN-based NMT model. However, we find that word2vec and reversal tricks seem no obvious improvement when trained the RNN-based NMT and Transformer models on augmented parallel corpus. For SMT, it performs better than NMT models when they were trained on the unaugmented dataset. Nevertheless, when trained on the augmented dataset, both the RNN-based NMT model and Transformer based NMT model outperform the SMT model. In addition, as with other translation tasks BIBREF4 , the Transformer also performs better than RNN-based NMT. Because the Test set contains both augmented and unaugmented data, it is not surprising that the RNN-based NMT model and Transformer based NMT model trained on unaugmented data would perform poorly. In order to further verify the effect of data augmentation, we report the test results of the models on only unaugmented test data (including 48K test pairs) in Table TABREF35 . From the results, it can be seen that the data augmentation can still improve the models. Analysis The generated samples of various models are shown in Figure FIGREF36 . Besides BLEU scores, we analyze these examples from a human perspective and draw some conclusions. At the same time, we design different metrics and evaluate on the whole Test set to support our conclusions as follows: On the one hand, we further compare the translation results from the perspective of people. We find that although the original meaning can be basically translated by SMT, its translation results are less smooth when compared with the other two NMT models (RNN-based NMT and Transformer). For example, the translations of SMT are usually lack of auxiliary words, conjunctions and function words, which is not consistent with human translation habits. To further confirm this conclusion, the average length of the translation results of the three models are measured (RNN-based NMT:17.12, SMT:15.50, Transformer:16.78, Reference:16.47). We can see that the average length of the SMT outputs is shortest, and the length gaps between the SMT outputs and the references are largest. Meanwhile, the average length of the sentences translated by Transformer is closest to the average length of references. These results indirectly verify our point of view, and show that the NMT models perform better than SMT in this task. On the other hand, there still exists some problems to be solved. We observe that translating proper nouns and personal pronouns (such as names, place names and ancient-specific appellations) is very difficult for all of these models. For instance, the ancient Chinese appellation `Zhen' should be translated into `Wo' in modern Chinese. Unfortunately, we calculate the accurate rate of some special words (such as `Zhen',`Chen' and `Gua'), and find that this rate is very low (the accurate rate of translating `Zhen' are: RNN-based NMT:0.14, SMT:0.16, Transformer:0.05). We will focus on this issue in the future. Conclusion and Future Work We propose an effective ancient-modern Chinese clause alignment method which achieves 94.2 F1-score on Test set. Based on it, we build a large scale parallel corpus which contains INLINEFORM0 1.24M bilingual sentence pairs. To our best knowledge, this is the first large high-quality ancient-modern Chinese dataset. In addition, we test the performance of the SMT and various NMT models on our built dataset and provide a strong NMT baseline for this task which achieves 27.16 BLEU score (4-gram). We further analyze the performance of the SMT and various NMT models and summarize some specific problems that machine translation models will encounter when translating ancient Chinese. For the future work, firstly, we are going to expand the dataset using the proposed method continually. Secondly, we will focus on solving the problem of proper noun translation and improve the translation system according to the features of ancient Chinese translation. Finally, we plan to introduce some techniques of statistical translation into neural machine translation to improve the performance. This work is supported by National Natural Science Fund for Distinguished Young Scholar (Grant No. 61625204) and partially supported by the State Key Program of National Science Foundation of China (Grant Nos. 61836006 and 61432014).
Where does the ancient Chinese dataset come from?
b9d07757e2d2c4be41823dd1ea3b9c7f115b5f72
Where Ġdoes Ġthe Ġancient ĠChinese Ġdataset Ġcome Ġfrom ?
an cient ĠChinese Ġhistory Ġrecords Ġin Ġseveral Ġdyn ast ies Ġ( about Ġ1000 BC - 200 BC ) Ġand Ġarticles Ġwritten Ġby Ġcelebrities Ġof Ġthat Ġera
<s> Where Ġdoes Ġthe Ġancient ĠChinese Ġdataset Ġcome Ġfrom ? </s> Introduction </s> Ancient ĠChinese Ġis Ġthe Ġwriting Ġlanguage Ġin Ġancient ĠChina . ĠIt Ġis Ġa Ġtreasure Ġof ĠChinese Ġculture Ġwhich Ġbrings Ġtogether Ġthe Ġwisdom Ġand Ġideas Ġof Ġthe ĠChinese Ġnation Ġand Ġchron icles Ġthe Ġancient Ġcultural Ġheritage Ġof ĠChina . ĠLearning Ġancient ĠChinese Ġnot Ġonly Ġhelps Ġpeople Ġto Ġunderstand Ġand Ġinherit Ġthe Ġwisdom Ġof Ġthe Ġan cients , Ġbut Ġalso Ġpromotes Ġpeople Ġto Ġabsorb Ġand Ġdevelop ĠChinese Ġculture . </s> However , Ġit Ġis Ġdifficult Ġfor Ġmodern Ġpeople Ġto Ġread Ġancient ĠChinese . ĠFirstly , Ġcompared Ġwith Ġmodern ĠChinese , Ġancient ĠChinese Ġis Ġmore Ġconcise Ġand Ġshorter . ĠThe Ġgram matical Ġorder Ġof Ġmodern ĠChinese Ġis Ġalso Ġquite Ġdifferent Ġfrom Ġthat Ġof Ġancient ĠChinese . ĠSecondly , Ġmost Ġmodern ĠChinese Ġwords Ġare Ġdouble Ġsyll ables , Ġwhile Ġthe Ġmost Ġof Ġthe Ġancient ĠChinese Ġwords Ġare Ġmon os yll ab ic . ĠThird ly , Ġthere Ġis Ġmore Ġthan Ġone Ġpoly sem ous Ġphenomenon Ġin Ġancient ĠChinese . ĠIn Ġaddition , Ġmanual Ġtranslation Ġhas Ġa Ġhigh Ġcost . ĠTherefore , Ġit Ġis Ġmeaningful Ġand Ġuseful Ġto Ġstudy Ġthe Ġautomatic Ġtranslation Ġfrom Ġancient ĠChinese Ġto Ġmodern ĠChinese . ĠThrough Ġancient - modern ĠChinese Ġtranslation , Ġthe Ġwisdom , Ġtalent Ġand Ġaccumulated Ġexperience Ġof Ġthe Ġpredecessors Ġcan Ġbe Ġpassed Ġon Ġto Ġmore Ġpeople . </s> Ne ural Ġmachine Ġtranslation Ġ( N MT ) ĠB IB REF 0 Ġ, ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġhas Ġachieved Ġremarkable Ġperformance Ġon Ġmany Ġbilingual Ġtranslation Ġtasks . ĠIt Ġis Ġan Ġend - to - end Ġlearning Ġapproach Ġfor Ġmachine Ġtranslation , Ġwith Ġthe Ġpotential Ġto Ġshow Ġgreat Ġadvantages Ġover Ġthe Ġstatistic Ġmachine Ġtranslation Ġ( SM T ) Ġsystems . ĠHowever , ĠN MT Ġapproach Ġhas Ġnot Ġbeen Ġwidely Ġapplied Ġto Ġthe Ġancient - modern ĠChinese Ġtranslation Ġtask . ĠOne Ġof Ġthe Ġmain Ġreasons Ġis Ġthe Ġlimited Ġhigh - quality Ġparallel Ġdata Ġresource . </s> The Ġmost Ġpopular Ġmethod Ġof Ġacquiring Ġtranslation Ġexamples Ġis Ġbilingual Ġtext Ġalignment ĠB IB REF 5 Ġ. ĠThis Ġkind Ġof Ġmethod Ġcan Ġbe Ġclassified Ġinto Ġtwo Ġtypes : Ġlex ical - based Ġand Ġstatistical - based . ĠThe Ġlex ical - based Ġapproaches ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġfocus Ġon Ġlex ical Ġinformation , Ġwhich Ġutilize Ġthe Ġbilingual Ġdictionary ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġor Ġlex ical Ġfeatures . ĠMeanwhile , Ġthe Ġstatistical - based Ġapproaches ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġrely Ġon Ġstatistical Ġinformation , Ġsuch Ġas Ġsentence Ġlength Ġratio Ġin Ġtwo Ġlanguages Ġand Ġalign Ġmode Ġprobability . </s> However , Ġthese Ġmethods Ġare Ġdesigned Ġfor Ġother Ġbilingual Ġlanguage Ġpairs Ġthat Ġare Ġwritten Ġin Ġdifferent Ġlanguage Ġcharacters Ġ( e . g . ĠEnglish - French , ĠChinese - Japanese ). ĠThe Ġancient - modern ĠChinese Ġhas Ġsome Ġcharacteristics Ġthat Ġare Ġquite Ġdifferent Ġfrom Ġother Ġlanguage Ġpairs . ĠFor Ġexample , Ġancient Ġand Ġmodern ĠChinese Ġare Ġboth Ġwritten Ġin ĠChinese Ġcharacters , Ġbut Ġancient ĠChinese Ġis Ġhighly Ġconcise Ġand Ġits Ġsynt act ical Ġstructure Ġis Ġdifferent Ġfrom Ġmodern ĠChinese . ĠThe Ġtraditional Ġmethods Ġdo Ġnot Ġtake Ġthese Ġcharacteristics Ġinto Ġaccount . ĠIn Ġthis Ġpaper , Ġwe Ġpropose Ġan Ġeffective Ġancient - modern ĠChinese Ġtext Ġalignment Ġmethod Ġat Ġthe Ġlevel Ġof Ġclause Ġbased Ġon Ġthe Ġcharacteristics Ġof Ġthese Ġtwo Ġlanguages . ĠThe Ġproposed Ġmethod Ġcombines Ġboth Ġlex ical - based Ġinformation Ġand Ġstatistical - based Ġinformation , Ġwhich Ġachieves Ġ94 . 2 ĠF 1 - score Ġon ĠTest Ġset . ĠRecently , Ġa Ġsimple Ġlongest Ġcommon Ġsubsequ ence Ġbased Ġapproach Ġfor Ġancient - modern ĠChinese Ġsentence Ġalignment Ġis Ġproposed Ġin ĠB IB REF 12 Ġ. ĠOur Ġexperiments Ġshowed Ġthat Ġour Ġproposed Ġalignment Ġapproach Ġperforms Ġmuch Ġbetter Ġthan Ġtheir Ġmethod . </s> We Ġapply Ġthe Ġproposed Ġmethod Ġto Ġcreate Ġa Ġlarge Ġtranslation Ġparallel Ġcorpus Ġwhich Ġcontains ĠIN LINE FORM 0 Ġ1 . 24 M Ġbilingual Ġsentence Ġpairs . ĠTo Ġour Ġbest Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġlarge Ġhigh - quality Ġancient - modern ĠChinese Ġdataset . ĠFurthermore , Ġwe Ġtest ĠSM T Ġmodels Ġand Ġvarious ĠN MT Ġmodels Ġon Ġthe Ġcreated Ġdataset Ġand Ġprovide Ġa Ġstrong Ġbaseline Ġfor Ġthis Ġtask . </s> Overview </s> There Ġare Ġfour Ġsteps Ġto Ġbuild Ġthe Ġancient - modern ĠChinese Ġtranslation Ġdataset : Ġ( i ) ĠThe Ġparallel Ġcorpus Ġcrawling Ġand Ġcleaning . Ġ( ii ) ĠThe Ġparagraph Ġalignment . Ġ( iii ) ĠThe Ġclause Ġalignment Ġbased Ġon Ġaligned Ġparagraphs . Ġ( iv ) ĠAug ment ing Ġdata Ġby Ġmerging Ġaligned Ġadjacent Ġclauses . ĠThe Ġmost Ġcritical Ġstep Ġis Ġthe Ġthird Ġstep . </s> Cl ause ĠAl ignment </s> In Ġthe Ġclause Ġalignment Ġstep , Ġwe Ġcombine Ġboth Ġstatistical - based Ġand Ġlex ical - based Ġinformation Ġto Ġmeasure Ġthe Ġscore Ġfor Ġeach Ġpossible Ġclause Ġalignment Ġbetween Ġancient Ġand Ġmodern ĠChinese Ġstrings . ĠThe Ġdynamic Ġprogramming Ġis Ġemployed Ġto Ġfurther Ġfind Ġoverall Ġoptimal Ġalignment Ġparagraph Ġby Ġparagraph . ĠAccording Ġto Ġthe Ġcharacteristics Ġof Ġthe Ġancient Ġand Ġmodern ĠChinese Ġlanguages , Ġwe Ġconsider Ġthe Ġfollowing Ġfactors Ġto Ġmeasure Ġthe Ġalignment Ġscore ĠIN LINE FORM 0 Ġbetween Ġa Ġbilingual Ġclause Ġpair : </s> Lex ical ĠMatch ing . ĠThe Ġlex ical Ġmatching Ġscore Ġis Ġused Ġto Ġcalculate Ġthe Ġmatching Ġcoverage Ġof Ġthe Ġancient Ġclause ĠIN LINE FORM 0 Ġ. ĠIt Ġcontains Ġtwo Ġparts : Ġexact Ġmatching Ġand Ġdictionary Ġmatching . ĠAn Ġancient ĠChinese Ġcharacter Ġusually Ġcorresponds Ġto Ġone Ġor Ġmore Ġmodern ĠChinese Ġwords . ĠIn Ġthe Ġfirst Ġpart , Ġwe Ġcarry Ġout ĠChinese ĠWord Ġsegment ation Ġto Ġthe Ġmodern ĠChinese Ġclause ĠIN LINE FORM 1 Ġ. ĠThen Ġwe Ġmatch Ġthe Ġancient Ġcharacters Ġand Ġmodern Ġwords Ġin Ġthe Ġorder Ġfrom Ġleft Ġto Ġright . ĠIn Ġfurther Ġmatching , Ġthe Ġwords Ġthat Ġhave Ġbeen Ġmatched Ġwill Ġbe Ġdeleted Ġfrom Ġthe Ġoriginal Ġclauses . </s> However , Ġsome Ġancient Ġcharacters Ġdo Ġnot Ġappear Ġin Ġits Ġcorresponding Ġmodern ĠChinese Ġwords . ĠAn Ġancient ĠChinese Ġdictionary Ġis Ġemployed Ġto Ġaddress Ġthis Ġissue . ĠWe Ġpre process Ġthe Ġancient ĠChinese Ġdictionary Ġand Ġremove Ġthe Ġstop Ġwords . ĠIn Ġthis Ġdictionary Ġmatching Ġstep , Ġwe Ġretrieve Ġthe Ġdictionary Ġdefinition Ġof Ġeach Ġunmatched Ġancient Ġcharacter Ġand Ġuse Ġit Ġto Ġmatch Ġthe Ġremaining Ġmodern ĠChinese Ġwords . ĠTo Ġreduce Ġthe Ġimpact Ġof Ġuniversal Ġword Ġmatching , Ġwe Ġuse ĠIn verse ĠDocument ĠFrequency Ġ( ID F ) Ġto Ġweight Ġthe Ġmatching Ġwords . ĠThe Ġlex ical Ġmatching Ġscore Ġis Ġcalculated Ġas : ĠDIS PLAY FORM 0 </s> The Ġabove Ġequation Ġis Ġused Ġto Ġcalculate Ġthe Ġmatching Ġcoverage Ġof Ġthe Ġancient Ġclause ĠIN LINE FORM 0 Ġ. ĠThe Ġfirst Ġterm Ġof Ġequation Ġ( ĠEQ REF 8 Ġ) Ġrepresents Ġexact Ġmatching Ġscore . ĠIN LINE FORM 1 Ġdenotes Ġthe Ġlength Ġof ĠIN LINE FORM 2 Ġ, ĠIN LINE FORM 3 Ġdenotes Ġeach Ġancient Ġcharacter Ġin ĠIN LINE FORM 4 Ġ, Ġand Ġthe Ġindicator Ġfunction ĠIN LINE FORM 5 Ġindicates Ġwhether Ġthe Ġcharacter ĠIN LINE FORM 6 Ġcan Ġmatch Ġthe Ġwords Ġin Ġthe Ġclause ĠIN LINE FORM 7 Ġ. ĠThe Ġsecond Ġterm Ġis Ġdictionary Ġmatching Ġscore . ĠHere ĠIN LINE FORM 8 Ġand ĠIN LINE FORM 9 Ġrepresent Ġthe Ġremaining Ġunmatched Ġstrings Ġof ĠIN LINE FORM 10 Ġand ĠIN LINE FORM 11 Ġ, Ġrespectively . ĠIN LINE FORM 12 Ġdenotes Ġthe ĠIN LINE FORM 13 Ġ- th Ġcharacter Ġin Ġthe Ġdictionary Ġdefinition Ġof Ġthe ĠIN LINE FORM 14 Ġand Ġits ĠIDF Ġscore Ġis Ġden oted Ġas ĠIN LINE FORM 15 Ġ. ĠThe ĠIN LINE FORM 16 Ġis Ġa Ġpred efined Ġparameter Ġwhich Ġis Ġused Ġto Ġnormal ize Ġthe ĠIDF Ġscore . ĠWe Ġtuned Ġthe Ġvalue Ġof Ġthis Ġparameter Ġon Ġthe ĠDev Ġset . </s> Stat istical ĠInformation . ĠSimilar Ġto ĠB IB REF 11 Ġand ĠB IB REF 6 Ġ, Ġthe Ġstatistical Ġinformation Ġcontains Ġalignment Ġmode Ġand Ġlength Ġinformation . ĠThere Ġare Ġmany Ġalignment Ġmodes Ġbetween Ġancient Ġand Ġmodern ĠChinese Ġlanguages . ĠIf Ġone Ġancient ĠChinese Ġclause Ġalign s Ġtwo Ġadjacent Ġmodern ĠChinese Ġclauses , Ġwe Ġcall Ġthis Ġalignment Ġas Ġ1 - 2 Ġalignment Ġmode . ĠWe Ġshow Ġsome Ġexamples Ġof Ġdifferent Ġalignment Ġmodes Ġin ĠFigure ĠFIG REF 9 Ġ. ĠIn Ġthis Ġpaper , Ġwe Ġonly Ġconsider Ġ1 - 0 , Ġ0 - 1 , Ġ1 - 1 , Ġ1 - 2 , Ġ2 - 1 Ġand Ġ2 - 2 Ġalignment Ġmodes Ġwhich Ġaccount Ġfor ĠIN LINE FORM 0 Ġof Ġthe ĠDev Ġset . ĠWe Ġestimate Ġthe Ġprobability ĠPr ĠIN LINE FORM 1 Ġn - m ĠIN LINE FORM 2 Ġof Ġeach Ġalignment Ġmode Ġn - m Ġon Ġthe ĠDev Ġset . ĠTo Ġutilize Ġlength Ġinformation , Ġwe Ġmake Ġan Ġinvestigation Ġon Ġlength Ġcorrelation Ġbetween Ġthese Ġtwo Ġlanguages . ĠBased Ġon Ġthe Ġassumption Ġof ĠB IB REF 11 Ġthat Ġeach Ġcharacter Ġin Ġone Ġlanguage Ġgives Ġrise Ġto Ġa Ġrandom Ġnumber Ġof Ġcharacters Ġin Ġthe Ġother Ġlanguage Ġand Ġthose Ġrandom Ġvariables ĠIN LINE FORM 3 Ġare Ġindependent Ġand Ġident ically Ġdistributed Ġwith Ġa Ġnormal Ġdistribution , Ġwe Ġestimate Ġthe Ġmean ĠIN LINE FORM 4 Ġand Ġstandard Ġdeviation ĠIN LINE FORM 5 Ġfrom Ġthe Ġparagraph Ġaligned Ġparallel Ġcorpus . ĠGiven Ġa Ġclause Ġpair ĠIN LINE FORM 6 Ġ, Ġthe Ġstatistical Ġinformation Ġscore Ġcan Ġbe Ġcalculated Ġby : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġdenotes Ġthe Ġnormal Ġdistribution Ġprobability Ġdensity Ġfunction . </s> Edit ĠDistance . ĠBecause Ġancient Ġand Ġmodern ĠChinese Ġare Ġboth Ġwritten Ġin ĠChinese Ġcharacters , Ġwe Ġalso Ġconsider Ġusing Ġthe Ġedit Ġdistance . ĠIt Ġis Ġa Ġway Ġof Ġquant ifying Ġthe Ġdiss imilar ity Ġbetween Ġtwo Ġstrings Ġby Ġcounting Ġthe Ġminimum Ġnumber Ġof Ġoperations Ġ( insert ion , Ġdeletion , Ġand Ġsubstitution ) Ġrequired Ġto Ġtransform Ġone Ġstring Ġinto Ġthe Ġother . ĠHere Ġwe Ġdefine Ġthe Ġedit Ġdistance Ġscore Ġas : ĠDIS PLAY FORM 0 </s> Dynamic ĠProgramming . ĠThe Ġoverall Ġalignment Ġscore Ġfor Ġeach Ġpossible Ġclause Ġalignment Ġis Ġas Ġfollows : ĠDIS PLAY FORM 0 </s> Here ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġare Ġpre - defined Ġinterpol ation Ġfactors . ĠWe Ġuse Ġdynamic Ġprogramming Ġto Ġfind Ġthe Ġoverall Ġoptimal Ġalignment Ġparagraph Ġby Ġparagraph . ĠLet ĠIN LINE FORM 2 Ġbe Ġtotal Ġalignment Ġscores Ġof Ġalign ing Ġthe Ġfirst Ġto ĠIN LINE FORM 3 Ġ- th Ġancient ĠChinese Ġclauses Ġwith Ġthe Ġfirst Ġto Ġto ĠIN LINE FORM 4 Ġ- th Ġmodern ĠChinese Ġclauses , Ġand Ġthe Ġrec urrence Ġthen Ġcan Ġbe Ġdescribed Ġas Ġfollows : ĠDIS PLAY FORM 0 </s> Where ĠIN LINE FORM 0 Ġdenotes Ġconc aten ate Ġclause ĠIN LINE FORM 1 Ġto Ġclause ĠIN LINE FORM 2 Ġ. ĠAs Ġwe Ġdiscussed Ġabove , Ġhere Ġwe Ġonly Ġconsider Ġ1 - 0 , Ġ0 - 1 , Ġ1 - 1 , Ġ1 - 2 , Ġ2 - 1 Ġand Ġ2 - 2 Ġalignment Ġmodes . </s> Ancient - Modern ĠChinese ĠDat as et </s> Data ĠCollection . ĠTo Ġbuild Ġthe Ġlarge Ġancient - modern ĠChinese Ġdataset , Ġwe Ġcollected Ġ1 . 7 K Ġbilingual Ġancient - modern ĠChinese Ġarticles Ġfrom Ġthe Ġinternet . ĠMore Ġspecifically , Ġa Ġlarge Ġpart Ġof Ġthe Ġancient ĠChinese Ġdata Ġwe Ġused Ġcome Ġfrom Ġancient ĠChinese Ġhistory Ġrecords Ġin Ġseveral Ġdyn ast ies Ġ( about Ġ1000 BC - 200 BC ) Ġand Ġarticles Ġwritten Ġby Ġcelebrities Ġof Ġthat Ġera . ĠThey Ġused Ġplain Ġand Ġaccurate Ġwords Ġto Ġexpress Ġwhat Ġhappened Ġat Ġthat Ġtime , Ġand Ġthus Ġensure Ġthe Ġgener ality Ġof Ġthe Ġtranslated Ġmaterials . </s> Par agraph ĠAl ignment . ĠTo Ġfurther Ġensure Ġthe Ġquality Ġof Ġthe Ġnew Ġdataset , Ġthe Ġwork Ġof Ġparagraph Ġalignment Ġis Ġmanually Ġcompleted . ĠAfter Ġdata Ġcleaning Ġand Ġmanual Ġparagraph Ġalignment , Ġwe Ġobtained Ġ35 K Ġaligned Ġbilingual Ġparagraphs . </s> Cl ause ĠAl ignment . ĠWe Ġapplied Ġour Ġclause Ġalignment Ġalgorithm Ġon Ġthe Ġ35 K Ġaligned Ġbilingual Ġparagraphs Ġand Ġobtained Ġ5 17 K Ġaligned Ġbilingual Ġclauses . ĠThe Ġreason Ġwe Ġuse Ġclause Ġalignment Ġalgorithm Ġinstead Ġof Ġsentence Ġalignment Ġis Ġbecause Ġwe Ġcan Ġconstruct Ġmore Ġaligned Ġsentences Ġmore Ġflex ibly Ġand Ġconveniently . ĠTo Ġbe Ġspecific , Ġwe Ġcan Ġget Ġmultiple Ġadditional Ġsentence Ġlevel Ġbilingual Ġpairs Ġby ĠâĢ ľ data Ġaug mentation âĢ Ŀ . </s> Data ĠAug mentation . ĠWe Ġaugmented Ġthe Ġdata Ġin Ġthe Ġfollowing Ġway : ĠGiven Ġan Ġaligned Ġclause Ġpair , Ġwe Ġmerged Ġits Ġadjacent Ġclause Ġpairs Ġas Ġa Ġnew Ġsample Ġpair . ĠFor Ġexample , Ġsuppose Ġwe Ġhave Ġthree Ġadjacent Ġclause Ġlevel Ġbilingual Ġpairs : Ġ( ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġ), Ġ( ĠIN LINE FORM 2 Ġ, ĠIN LINE FORM 3 Ġ), Ġand Ġ( ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġ). ĠWe Ġcan Ġget Ġsome Ġadditional Ġsentence Ġlevel Ġbilingual Ġpairs , Ġsuch Ġas : Ġ( ĠIN LINE FORM 6 Ġ, ĠIN LINE FORM 7 Ġ) Ġand Ġ( ĠIN LINE FORM 8 Ġ, ĠIN LINE FORM 9 Ġ). ĠHere ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġ, Ġand ĠIN LINE FORM 12 Ġare Ġadjacent Ġclauses Ġin Ġthe Ġoriginal Ġparagraph , Ġand ĠIN LINE FORM 13 Ġdenotes Ġconc aten ate Ġclause ĠIN LINE FORM 14 Ġto Ġclause ĠIN LINE FORM 15 Ġ. ĠThe Ġadvantage Ġof Ġusing Ġthis Ġdata Ġaug mentation Ġmethod Ġis Ġthat Ġcompared Ġwith Ġonly Ġusing Ġ( ĠIN LINE FORM 16 Ġ, ĠIN LINE FORM 17 Ġ) Ġas Ġthe Ġtraining Ġdata , Ġwe Ġcan Ġalso Ġuse Ġ( ĠIN LINE FORM 18 Ġ, ĠIN LINE FORM 19 Ġ) Ġand Ġ( ĠIN LINE FORM 20 Ġ, ĠIN LINE FORM 21 Ġ) Ġas Ġthe Ġtraining Ġdata , Ġwhich Ġcan Ġprovide Ġricher Ġsupervision Ġinformation Ġfor Ġthe Ġmodel Ġand Ġmake Ġthe Ġmodel Ġlearn Ġthe Ġalign Ġinformation Ġbetween Ġthe Ġsource Ġlanguage Ġand Ġthe Ġtarget Ġlanguage Ġbetter . ĠAfter Ġthe Ġdata Ġaug mentation , Ġwe Ġfiltered Ġthe Ġsentences Ġwhich Ġare Ġlonger Ġthan Ġ50 Ġor Ġcontain Ġmore Ġthan Ġfour Ġclause Ġpairs . </s> Dat as et ĠCreation . ĠFinally , Ġwe Ġsplit Ġthe Ġdataset Ġinto Ġthree Ġsets : Ġtraining Ġ( Train ), Ġdevelopment Ġ( Dev ) Ġand Ġtesting Ġ( Test ). ĠNote Ġthat Ġthe Ġun au gment ed Ġdataset Ġcontains Ġ5 17 K Ġaligned Ġbilingual Ġclause Ġpairs Ġfrom Ġ35 K Ġaligned Ġbilingual Ġparagraphs . ĠTo Ġkeep Ġall Ġthe Ġsentences Ġin Ġdifferent Ġsets Ġcome Ġfrom Ġdifferent Ġarticles , Ġwe Ġsplit Ġthe Ġ35 K Ġaligned Ġbilingual Ġparagraphs Ġinto ĠTrain , ĠDev Ġand ĠTest Ġsets Ġfollowing Ġthese Ġratios Ġrespectively : Ġ80 %, Ġ10 %, Ġ10 %. ĠBefore Ġdata Ġaug mentation , Ġthe Ġun au gment ed ĠTrain Ġset Ġcontains ĠIN LINE FORM 0 Ġaligned Ġbilingual Ġclause Ġpairs Ġfrom Ġ28 K Ġaligned Ġbilingual Ġparagraphs . ĠThen Ġwe Ġaugmented Ġthe ĠTrain , ĠDev Ġand ĠTest Ġsets Ġrespectively . ĠNote Ġthat Ġthe Ġaugmented ĠTrain , ĠDev Ġand ĠTest Ġsets Ġalso Ġcontain Ġthe Ġun au gment ed Ġdata . ĠThe Ġstatistical Ġinformation Ġof Ġthe Ġthree Ġdata Ġsets Ġis Ġshown Ġin ĠTable ĠT AB REF 17 Ġ. ĠWe Ġshow Ġsome Ġexamples Ġof Ġdata Ġin ĠFigure ĠFIG REF 14 Ġ. </s> R NN - based ĠN MT Ġmodel </s> We Ġfirst Ġbriefly Ġintroduce Ġthe ĠR NN Ġbased ĠNeural ĠMachine ĠTranslation Ġ( R NN - based ĠN MT ) Ġmodel . ĠThe ĠR NN - based ĠN MT Ġwith Ġattention Ġmechanism ĠB IB REF 0 Ġhas Ġachieved Ġremarkable Ġperformance Ġon Ġmany Ġtranslation Ġtasks . ĠIt Ġconsists Ġof Ġenc oder Ġand Ġdec oder Ġpart . </s> We Ġfirst ly Ġintroduce Ġthe Ġenc oder Ġpart . ĠThe Ġinput Ġword Ġsequence Ġof Ġsource Ġlanguage Ġare Ġindividually Ġmapped Ġinto Ġa ĠIN LINE FORM 0 Ġ- dimensional Ġvector Ġspace ĠIN LINE FORM 1 Ġ. ĠThen Ġa Ġbi - direction al ĠR NN ĠB IB REF 15 Ġwith ĠGR U ĠB IB REF 16 Ġor ĠL ST M ĠB IB REF 17 Ġcell Ġconverts Ġthese Ġvectors Ġinto Ġa Ġsequences Ġof Ġhidden Ġstates ĠIN LINE FORM 2 Ġ. </s> For Ġthe Ġdec oder Ġpart , Ġanother ĠR NN Ġis Ġused Ġto Ġgenerate Ġtarget Ġsequence ĠIN LINE FORM 0 Ġ. ĠThe Ġattention Ġmechanism ĠB IB REF 0 Ġ, ĠB IB REF 18 Ġis Ġemployed Ġto Ġallow Ġthe Ġdec oder Ġto Ġrefer Ġback Ġto Ġthe Ġhidden Ġstate Ġsequence Ġand Ġfocus Ġon Ġa Ġparticular Ġsegment . ĠThe ĠIN LINE FORM 1 Ġ- th Ġhidden Ġstate ĠIN LINE FORM 2 Ġof Ġdec oder Ġpart Ġis Ġcalculated Ġas : ĠDIS PLAY FORM 0 </s> Here Ġg ĠIN LINE FORM 0 Ġis Ġa Ġlinear Ġcombination Ġof Ġattended Ġcontext Ġvector Ġc ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġis Ġthe Ġword Ġembed ding Ġof Ġ( i - 1 )- th Ġtarget Ġword : ĠDIS PLAY FORM 0 </s> The Ġattended Ġcontext Ġvector Ġc ĠIN LINE FORM 0 Ġis Ġcomputed Ġas Ġa Ġweighted Ġsum Ġof Ġthe Ġhidden Ġstates Ġof Ġthe Ġenc oder : ĠDIS PLAY FORM 0 </s> The Ġprobability Ġdistribution Ġvector Ġof Ġthe Ġnext Ġword ĠIN LINE FORM 0 Ġis Ġgenerated Ġaccording Ġto Ġthe Ġfollowing : ĠDIS PLAY FORM 0 </s> We Ġtake Ġthis Ġmodel Ġas Ġthe Ġbasic ĠR NN - based ĠN MT Ġmodel Ġin Ġthe Ġfollowing Ġexperiments . </s> Trans former - N MT </s> Recently , Ġthe ĠTrans former Ġmodel ĠB IB REF 4 Ġhas Ġmade Ġremarkable Ġprogress Ġin Ġmachine Ġtranslation . ĠThis Ġmodel Ġcontains Ġa Ġmulti - head Ġself - att ention Ġenc oder Ġand Ġa Ġmulti - head Ġself - att ention Ġdec oder . </s> As Ġproposed Ġby ĠB IB REF 4 Ġ, Ġan Ġattention Ġfunction Ġmaps Ġa Ġquery Ġand Ġa Ġset Ġof Ġkey - value Ġpairs Ġto Ġan Ġoutput , Ġwhere Ġthe Ġqueries ĠIN LINE FORM 0 Ġ, Ġkeys ĠIN LINE FORM 1 Ġ, Ġand Ġvalues ĠIN LINE FORM 2 Ġare Ġall Ġvectors . ĠThe Ġinput Ġconsists Ġof Ġqueries Ġand Ġkeys Ġof Ġdimension ĠIN LINE FORM 3 Ġ, Ġand Ġvalues Ġof Ġdimension ĠIN LINE FORM 4 Ġ. ĠThe Ġattention Ġfunction Ġis Ġgiven Ġby : ĠDIS PLAY FORM 0 </s> Multi - head Ġattention Ġmechanism Ġprojects Ġqueries , Ġkeys Ġand Ġvalues Ġto ĠIN LINE FORM 0 Ġdifferent Ġrepresentation Ġsub sp aces Ġand Ġcalculates Ġcorresponding Ġattention . ĠThe Ġattention Ġfunction Ġoutputs Ġare Ġconc aten ated Ġand Ġprojected Ġagain Ġbefore Ġgiving Ġthe Ġfinal Ġoutput . ĠMulti - head Ġattention Ġallows Ġthe Ġmodel Ġto Ġattend Ġto Ġmultiple Ġfeatures Ġat Ġdifferent Ġpositions . </s> The Ġenc oder Ġis Ġcomposed Ġof Ġa Ġstack Ġof ĠIN LINE FORM 0 Ġidentical Ġlayers . ĠEach Ġlayer Ġhas Ġtwo Ġsub - l ayers : Ġmulti - head Ġself - att ention Ġmechanism Ġand Ġposition - wise Ġfully Ġconnected Ġfeed - forward Ġnetwork . ĠSimilarly , Ġthe Ġdec oder Ġis Ġalso Ġcomposed Ġof Ġa Ġstack Ġof ĠIN LINE FORM 1 Ġidentical Ġlayers . ĠIn Ġaddition Ġto Ġthe Ġtwo Ġsub - l ayers Ġin Ġeach Ġenc oder Ġlayer , Ġthe Ġdec oder Ġcontains Ġa Ġthird Ġsub - layer Ġwhich Ġperforms Ġmulti - head Ġattention Ġover Ġthe Ġoutput Ġof Ġthe Ġenc oder Ġstack Ġ( see Ġmore Ġdetails Ġin ĠB IB REF 4 Ġ). </s> Exper iments </s> Our Ġexperiments Ġrev olve Ġaround Ġthe Ġfollowing Ġquestions : ĠQ 1 : ĠAs Ġwe Ġconsider Ġthree Ġfactors Ġfor Ġclause Ġalignment , Ġdo Ġall Ġthese Ġfactors Ġhelp ? ĠHow Ġdoes Ġour Ġmethod Ġcompare Ġwith Ġprevious Ġmethods ? ĠQ 2 : ĠHow Ġdoes Ġthe ĠN MT Ġand ĠSM T Ġmodels Ġperform Ġon Ġthis Ġnew Ġdataset Ġwe Ġbuild ? </s> Cl ause ĠAl ignment ĠResults Ġ( Q 1 ) </s> In Ġorder Ġto Ġevaluate Ġour Ġclause Ġalignment Ġalgorithm , Ġwe Ġmanually Ġaligned Ġbilingual Ġclauses Ġfrom Ġ37 Ġbilingual Ġancient - modern ĠChinese Ġarticles , Ġand Ġfinally Ġgot Ġ4 K Ġaligned Ġbilingual Ġclauses Ġas Ġthe ĠTest Ġset Ġand Ġ2 K Ġclauses Ġas Ġthe ĠDev Ġset . </s> Met rics . ĠWe Ġused ĠF 1 - score Ġand Ġprecision Ġscore Ġas Ġthe Ġevaluation Ġmetrics . ĠSuppose Ġthat Ġwe Ġget ĠIN LINE FORM 0 Ġbilingual Ġclause Ġpairs Ġafter Ġrunning Ġthe Ġalgorithm Ġon Ġthe ĠTest Ġset , Ġand Ġthere Ġare ĠIN LINE FORM 1 Ġbilingual Ġclause Ġpairs Ġof Ġthese ĠIN LINE FORM 2 Ġpairs Ġare Ġin Ġthe Ġground Ġtruth Ġof Ġthe ĠTest Ġset , Ġthe Ġprecision Ġscore Ġis Ġdefined Ġas ĠIN LINE FORM 3 Ġ( the Ġalgorithm Ġgives ĠIN LINE FORM 4 Ġoutputs , ĠIN LINE FORM 5 Ġof Ġwhich Ġare Ġcorrect ). ĠAnd Ġsuppose Ġthat Ġthe Ġground Ġtruth Ġof Ġthe ĠTest Ġset Ġcontains ĠIN LINE FORM 6 Ġbilingual Ġclause Ġpairs , Ġthe Ġrecall Ġscore Ġis ĠIN LINE FORM 7 Ġ( there Ġare ĠIN LINE FORM 8 Ġground Ġtruth Ġsamples , ĠIN LINE FORM 9 Ġof Ġwhich Ġare Ġoutput Ġby Ġthe Ġalgorithm ), Ġthen Ġthe ĠF 1 - score Ġis ĠIN LINE FORM 10 Ġ. </s> Bas elines . ĠSince Ġthe Ġrelated Ġwork ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġcan Ġbe Ġseen Ġas Ġthe Ġab lation Ġcases Ġof Ġour Ġmethod Ġ( only Ġstatistical Ġscore ĠIN LINE FORM 0 Ġwith Ġdynamic Ġprogramming ), Ġwe Ġcompared Ġthe Ġfull Ġproposed Ġmethod Ġwith Ġits Ġvariants Ġon Ġthe ĠTest Ġset Ġfor Ġab lation Ġstudy . ĠIn Ġaddition , Ġwe Ġalso Ġcompared Ġour Ġmethod Ġwith Ġthe Ġlongest Ġcommon Ġsubsequ ence Ġ( LCS ) Ġbased Ġapproach Ġproposed Ġby ĠB IB REF 12 Ġ. ĠTo Ġthe Ġbest Ġof Ġour Ġknowledge , ĠB IB REF 12 Ġis Ġthe Ġlatest Ġrelated Ġwork Ġwhich Ġare Ġdesigned Ġfor ĠAncient - Modern ĠChinese Ġalignment . </s> Hyper - param eters . ĠFor Ġthe Ġproposed Ġmethod , Ġwe Ġestimated ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġon Ġall Ġaligned Ġparagraphs . ĠThe Ġprobability ĠPr ĠIN LINE FORM 2 Ġn - m ĠIN LINE FORM 3 Ġof Ġeach Ġalignment Ġmode Ġn - m Ġwas Ġestimated Ġon Ġthe ĠDev Ġset . ĠFor Ġthe Ġhyper - param eters ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġand ĠIN LINE FORM 6 Ġ, Ġthe Ġgrid Ġsearch Ġwas Ġapplied Ġto Ġtune Ġthem Ġon Ġthe ĠDev Ġset . ĠIn Ġorder Ġto Ġshow Ġthe Ġeffect Ġof Ġhyper - param eters ĠIN LINE FORM 7 Ġ, ĠIN LINE FORM 8 Ġ, Ġand ĠIN LINE FORM 9 Ġ, Ġwe Ġreported Ġthe Ġresults Ġof Ġvarious Ġhyper - param eters Ġon Ġthe ĠDev Ġset Ġin ĠTable ĠT AB REF 26 Ġ. ĠBased Ġon Ġthe Ġresults Ġof Ġgrid Ġsearch Ġon Ġthe ĠDev Ġset , Ġwe Ġset ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġ, Ġand ĠIN LINE FORM 12 Ġin Ġthe Ġfollowing Ġexperiment . ĠThe ĠJ ie ba ĠChinese Ġtext Ġsegment ation Ġis Ġemployed Ġfor Ġmodern ĠChinese Ġword Ġsegment ation . </s> Results . ĠThe Ġresults Ġon Ġthe ĠTest Ġset Ġare Ġshown Ġin ĠTable ĠT AB REF 28 Ġ, Ġthe Ġabbre viation Ġw / o Ġmeans Ġremoving Ġa Ġparticular Ġpart Ġfrom Ġthe Ġsetting . ĠFrom Ġthe Ġresults , Ġwe Ġcan Ġsee Ġthat Ġthe Ġlex ical Ġmatching Ġscore Ġis Ġthe Ġmost Ġimportant Ġamong Ġthese Ġthree Ġfactors , Ġand Ġstatistical Ġinformation Ġscore Ġis Ġmore Ġimportant Ġthan Ġedit Ġdistance Ġscore . ĠMoreover , Ġthe Ġdictionary Ġterm Ġin Ġlex ical Ġmatching Ġscore Ġsignificantly Ġimproves Ġthe Ġperformance . ĠFrom Ġthese Ġresults , Ġwe Ġobtain Ġthe Ġbest Ġsetting Ġthat Ġinvolves Ġall Ġthese Ġthree Ġfactors . ĠWe Ġused Ġthis Ġsetting Ġfor Ġdataset Ġcreation . ĠFurthermore , Ġthe Ġproposed Ġmethod Ġperforms Ġmuch Ġbetter Ġthan ĠLCS ĠB IB REF 12 Ġ. </s> Translation ĠResults Ġ( Q 2 ) </s> In Ġthis Ġexperiment , Ġwe Ġanalyzed Ġand Ġcompared Ġthe Ġperformance Ġof Ġthe ĠSM T Ġand Ġvarious ĠN MT Ġmodels Ġon Ġour Ġbuilt Ġdataset . ĠTo Ġverify Ġthe Ġeffectiveness Ġof Ġour Ġdata Ġaugmented Ġmethod . ĠWe Ġtrained Ġthe ĠN MT Ġand ĠSM T Ġmodels Ġon Ġboth Ġun au gment ed Ġdataset Ġ( including Ġ0 . 46 M Ġtraining Ġpairs ) Ġand Ġaugmented Ġdataset , Ġand Ġtest Ġall Ġthe Ġmodels Ġon Ġthe Ġsame ĠTest Ġset Ġwhich Ġis Ġaugmented . ĠThe Ġmodels Ġto Ġbe Ġtested Ġand Ġtheir Ġconfigurations Ġare Ġas Ġfollows : </s> SM T . ĠThe Ġstate - of - art ĠMoses Ġtool kit ĠB IB REF 19 Ġwas Ġused Ġto Ġtrain ĠSM T Ġmodel . ĠWe Ġused ĠKen LM ĠB IB REF 20 Ġto Ġtrain Ġa Ġ5 - gram Ġlanguage Ġmodel , Ġand Ġthe ĠG IZ A ++ Ġtool kit Ġto Ġalign Ġthe Ġdata . </s> R NN - based ĠN MT . ĠThe Ġbasic ĠR NN - based ĠN MT Ġmodel Ġis Ġbased Ġon ĠB IB REF 0 Ġwhich Ġis Ġintroduced Ġabove . ĠBoth Ġthe Ġenc oder Ġand Ġdec oder Ġused Ġ2 - layer ĠR NN Ġwith Ġ1024 ĠL ST M Ġcells , Ġand Ġthe Ġenc oder Ġis Ġa Ġbi - direction al ĠR NN . ĠThe Ġbatch Ġsize , Ġthreshold Ġof Ġelement - wise Ġgradient Ġclipping Ġand Ġinitial Ġlearning Ġrate Ġof ĠAdam Ġoptim izer ĠB IB REF 21 Ġwere Ġset Ġto Ġ128 , Ġ5 . 0 Ġand Ġ0 . 001 . ĠWhen Ġtrained Ġthe Ġmodel Ġon Ġaugmented Ġdataset , Ġwe Ġused Ġ4 - layer ĠR NN . ĠSeveral Ġtechniques Ġwere Ġinvestigated Ġto Ġtrain Ġthe Ġmodel , Ġincluding Ġlayer - normal ization ĠB IB REF 22 Ġ, ĠR NN - drop out ĠB IB REF 23 Ġ, Ġand Ġlearning Ġrate Ġdecay ĠB IB REF 1 Ġ. ĠThe Ġhyper - param eters Ġwere Ġchosen Ġempir ically Ġand Ġadjusted Ġin Ġthe ĠDev Ġset . ĠFurthermore , Ġwe Ġtested Ġthe Ġbasic ĠN MT Ġmodel Ġwith Ġseveral Ġtechniques , Ġsuch Ġas Ġtarget Ġlanguage Ġreversal ĠB IB REF 24 Ġ( re vers ing Ġthe Ġorder Ġof Ġthe Ġwords Ġin Ġall Ġtarget Ġsentences , Ġbut Ġnot Ġsource Ġsentences ), Ġresidual Ġconnection ĠB IB REF 25 Ġand Ġpre - trained Ġword 2 vec ĠB IB REF 26 Ġ. ĠFor Ġword Ġembed ding Ġpre - training , Ġwe Ġcollected Ġan Ġexternal Ġancient Ġcorpus Ġwhich Ġcontains ĠIN LINE FORM 0 Ġ134 M Ġtokens . </s> Trans former - N MT . ĠWe Ġalso Ġtrained Ġthe ĠTrans former Ġmodel ĠB IB REF 4 Ġwhich Ġis Ġa Ġstrong Ġbaseline Ġof ĠN MT Ġon Ġboth Ġaugmented Ġand Ġun au gment ed Ġparallel Ġcorpus . ĠThe Ġtraining Ġconfiguration Ġof Ġthe ĠTrans former Ġmodel Ġis Ġshown Ġin ĠTable ĠT AB REF 32 Ġ. ĠThe Ġhyper - param eters Ġare Ġset Ġbased Ġon Ġthe Ġsettings Ġin Ġthe Ġpaper ĠB IB REF 4 Ġand Ġthe Ġsizes Ġof Ġour Ġtraining Ġsets . </s> For Ġthe Ġevaluation , Ġwe Ġused Ġthe Ġaverage Ġof Ġ1 Ġto Ġ4 Ġgram ĠB LE Us Ġmultiplied Ġby Ġa Ġbre vity Ġpenalty ĠB IB REF 27 Ġwhich Ġcomputed Ġby Ġmulti - ble u . per l Ġin ĠMoses Ġas Ġmetrics . ĠThe Ġresults Ġare Ġreported Ġin ĠTable ĠT AB REF 34 Ġ. ĠFor ĠR NN - based ĠN MT , Ġwe Ġcan Ġsee Ġthat Ġtarget Ġlanguage Ġreversal , Ġresidual Ġconnection , Ġand Ġword 2 vec Ġcan Ġfurther Ġimprove Ġthe Ġperformance Ġof Ġthe Ġbasic ĠR NN - based ĠN MT Ġmodel . ĠHowever , Ġwe Ġfind Ġthat Ġword 2 vec Ġand Ġreversal Ġtricks Ġseem Ġno Ġobvious Ġimprovement Ġwhen Ġtrained Ġthe ĠR NN - based ĠN MT Ġand ĠTrans former Ġmodels Ġon Ġaugmented Ġparallel Ġcorpus . ĠFor ĠSM T , Ġit Ġperforms Ġbetter Ġthan ĠN MT Ġmodels Ġwhen Ġthey Ġwere Ġtrained Ġon Ġthe Ġun au gment ed Ġdataset . ĠNevertheless , Ġwhen Ġtrained Ġon Ġthe Ġaugmented Ġdataset , Ġboth Ġthe ĠR NN - based ĠN MT Ġmodel Ġand ĠTrans former Ġbased ĠN MT Ġmodel Ġoutper form Ġthe ĠSM T Ġmodel . ĠIn Ġaddition , Ġas Ġwith Ġother Ġtranslation Ġtasks ĠB IB REF 4 Ġ, Ġthe ĠTrans former Ġalso Ġperforms Ġbetter Ġthan ĠR NN - based ĠN MT . </s> Because Ġthe ĠTest Ġset Ġcontains Ġboth Ġaugmented Ġand Ġun au gment ed Ġdata , Ġit Ġis Ġnot Ġsurprising Ġthat Ġthe ĠR NN - based ĠN MT Ġmodel Ġand ĠTrans former Ġbased ĠN MT Ġmodel Ġtrained Ġon Ġun au gment ed Ġdata Ġwould Ġperform Ġpoorly . ĠIn Ġorder Ġto Ġfurther Ġverify Ġthe Ġeffect Ġof Ġdata Ġaug mentation , Ġwe Ġreport Ġthe Ġtest Ġresults Ġof Ġthe Ġmodels Ġon Ġonly Ġun au gment ed Ġtest Ġdata Ġ( including Ġ48 K Ġtest Ġpairs ) Ġin ĠTable ĠT AB REF 35 Ġ. ĠFrom Ġthe Ġresults , Ġit Ġcan Ġbe Ġseen Ġthat Ġthe Ġdata Ġaug mentation Ġcan Ġstill Ġimprove Ġthe Ġmodels . </s> Analysis </s> The Ġgenerated Ġsamples Ġof Ġvarious Ġmodels Ġare Ġshown Ġin ĠFigure ĠFIG REF 36 Ġ. ĠBesides ĠB LE U Ġscores , Ġwe Ġanalyze Ġthese Ġexamples Ġfrom Ġa Ġhuman Ġperspective Ġand Ġdraw Ġsome Ġconclusions . ĠAt Ġthe Ġsame Ġtime , Ġwe Ġdesign Ġdifferent Ġmetrics Ġand Ġevaluate Ġon Ġthe Ġwhole ĠTest Ġset Ġto Ġsupport Ġour Ġconclusions Ġas Ġfollows : </s> On Ġthe Ġone Ġhand , Ġwe Ġfurther Ġcompare Ġthe Ġtranslation Ġresults Ġfrom Ġthe Ġperspective Ġof Ġpeople . ĠWe Ġfind Ġthat Ġalthough Ġthe Ġoriginal Ġmeaning Ġcan Ġbe Ġbasically Ġtranslated Ġby ĠSM T , Ġits Ġtranslation Ġresults Ġare Ġless Ġsmooth Ġwhen Ġcompared Ġwith Ġthe Ġother Ġtwo ĠN MT Ġmodels Ġ( R NN - based ĠN MT Ġand ĠTrans former ). ĠFor Ġexample , Ġthe Ġtranslations Ġof ĠSM T Ġare Ġusually Ġlack Ġof Ġauxiliary Ġwords , Ġconj unctions Ġand Ġfunction Ġwords , Ġwhich Ġis Ġnot Ġconsistent Ġwith Ġhuman Ġtranslation Ġhabits . ĠTo Ġfurther Ġconfirm Ġthis Ġconclusion , Ġthe Ġaverage Ġlength Ġof Ġthe Ġtranslation Ġresults Ġof Ġthe Ġthree Ġmodels Ġare Ġmeasured Ġ( R NN - based ĠN MT : 17 . 12 , ĠSM T : 15 . 50 , ĠTrans former : 16 . 78 , ĠReference : 16 . 47 ). ĠWe Ġcan Ġsee Ġthat Ġthe Ġaverage Ġlength Ġof Ġthe ĠSM T Ġoutputs Ġis Ġshortest , Ġand Ġthe Ġlength Ġgaps Ġbetween Ġthe ĠSM T Ġoutputs Ġand Ġthe Ġreferences Ġare Ġlargest . ĠMeanwhile , Ġthe Ġaverage Ġlength Ġof Ġthe Ġsentences Ġtranslated Ġby ĠTrans former Ġis Ġclosest Ġto Ġthe Ġaverage Ġlength Ġof Ġreferences . ĠThese Ġresults Ġindirectly Ġverify Ġour Ġpoint Ġof Ġview , Ġand Ġshow Ġthat Ġthe ĠN MT Ġmodels Ġperform Ġbetter Ġthan ĠSM T Ġin Ġthis Ġtask . </s> On Ġthe Ġother Ġhand , Ġthere Ġstill Ġexists Ġsome Ġproblems Ġto Ġbe Ġsolved . ĠWe Ġobserve Ġthat Ġtranslating Ġproper Ġnoun s Ġand Ġpersonal Ġpronouns Ġ( such Ġas Ġnames , Ġplace Ġnames Ġand Ġancient - specific Ġappell ations ) Ġis Ġvery Ġdifficult Ġfor Ġall Ġof Ġthese Ġmodels . ĠFor Ġinstance , Ġthe Ġancient ĠChinese Ġappell ation Ġ` Z hen ' Ġshould Ġbe Ġtranslated Ġinto Ġ` Wo ' Ġin Ġmodern ĠChinese . ĠUnfortunately , Ġwe Ġcalculate Ġthe Ġaccurate Ġrate Ġof Ġsome Ġspecial Ġwords Ġ( such Ġas Ġ` Z hen ', ` C hen ' Ġand Ġ` G ua '), Ġand Ġfind Ġthat Ġthis Ġrate Ġis Ġvery Ġlow Ġ( the Ġaccurate Ġrate Ġof Ġtranslating Ġ` Z hen ' Ġare : ĠR NN - based ĠN MT : 0 . 14 , ĠSM T : 0 . 16 , ĠTrans former : 0 . 05 ). ĠWe Ġwill Ġfocus Ġon Ġthis Ġissue Ġin Ġthe Ġfuture . </s> Conclusion Ġand ĠFuture ĠWork </s> We Ġpropose Ġan Ġeffective Ġancient - modern ĠChinese Ġclause Ġalignment Ġmethod Ġwhich Ġachieves Ġ94 . 2 ĠF 1 - score Ġon ĠTest Ġset . ĠBased Ġon Ġit , Ġwe Ġbuild Ġa Ġlarge Ġscale Ġparallel Ġcorpus Ġwhich Ġcontains ĠIN LINE FORM 0 Ġ1 . 24 M Ġbilingual Ġsentence Ġpairs . ĠTo Ġour Ġbest Ġknowledge , Ġthis Ġis Ġthe Ġfirst Ġlarge Ġhigh - quality Ġancient - modern ĠChinese Ġdataset . ĠIn Ġaddition , Ġwe Ġtest Ġthe Ġperformance Ġof Ġthe ĠSM T Ġand Ġvarious ĠN MT Ġmodels Ġon Ġour Ġbuilt Ġdataset Ġand Ġprovide Ġa Ġstrong ĠN MT Ġbaseline Ġfor Ġthis Ġtask Ġwhich Ġachieves Ġ27 . 16 ĠB LE U Ġscore Ġ( 4 - gram ). ĠWe Ġfurther Ġanalyze Ġthe Ġperformance Ġof Ġthe ĠSM T Ġand Ġvarious ĠN MT Ġmodels Ġand Ġsummarize Ġsome Ġspecific Ġproblems Ġthat Ġmachine Ġtranslation Ġmodels Ġwill Ġencounter Ġwhen Ġtranslating Ġancient ĠChinese . </s> For Ġthe Ġfuture Ġwork , Ġfirst ly , Ġwe Ġare Ġgoing Ġto Ġexpand Ġthe Ġdataset Ġusing Ġthe Ġproposed Ġmethod Ġcontinually . ĠSecondly , Ġwe Ġwill Ġfocus Ġon Ġsolving Ġthe Ġproblem Ġof Ġproper Ġnoun Ġtranslation Ġand Ġimprove Ġthe Ġtranslation Ġsystem Ġaccording Ġto Ġthe Ġfeatures Ġof Ġancient ĠChinese Ġtranslation . ĠFinally , Ġwe Ġplan Ġto Ġintroduce Ġsome Ġtechniques Ġof Ġstatistical Ġtranslation Ġinto Ġneural Ġmachine Ġtranslation Ġto Ġimprove Ġthe Ġperformance . </s> This Ġwork Ġis Ġsupported Ġby ĠNational ĠNatural ĠScience ĠFund Ġfor ĠDist inguished ĠYoung ĠScholar Ġ( Grant ĠNo . Ġ6 16 25 204 ) Ġand Ġpartially Ġsupported Ġby Ġthe ĠState ĠKey ĠProgram Ġof ĠNational ĠScience ĠFoundation Ġof ĠChina Ġ( Grant ĠNos . Ġ6 18 36 006 Ġand Ġ6 143 2014 ).