No
BOOLEAN
1909.00326
Introduction </s> Ne ural Ġmachine Ġtranslation Ġ( N MT ) Ġhas Ġachieved Ġthe Ġstate - of - the - art Ġresults Ġon Ġa Ġmass Ġof Ġlanguage Ġpairs Ġwith Ġvarying Ġstructural Ġdifferences , Ġsuch Ġas ĠEnglish - French ĠB IB REF 0 , ĠB IB REF 1 Ġand ĠChinese - English ĠB IB REF 2 . ĠHowever , Ġso Ġfar Ġnot Ġmuch Ġis Ġknown Ġabout Ġhow Ġand Ġwhy ĠN MT Ġworks , Ġwhich Ġpose Ġgreat Ġchallenges Ġfor Ġdebugging ĠN MT Ġmodels Ġand Ġdesigning Ġoptimal Ġarchitectures . </s> The Ġunderstanding Ġof ĠN MT Ġmodels Ġhas Ġbeen Ġapproached Ġprimarily Ġfrom Ġtwo Ġcomplementary Ġperspectives . ĠThe Ġfirst Ġthread Ġof Ġwork Ġaims Ġto Ġunderstand Ġthe Ġimportance Ġof Ġrepresentations Ġby Ġanalyzing Ġthe Ġlinguistic Ġinformation Ġembedded Ġin Ġrepresentation Ġvectors ĠB IB REF 3 , ĠB IB REF 4 Ġor Ġhidden Ġunits ĠB IB REF 5 , ĠB IB REF 6 . ĠAnother Ġdirection Ġfocuses Ġon Ġunderstanding Ġthe Ġimportance Ġof Ġinput Ġwords Ġby Ġinterpreting Ġthe Ġinput - output Ġbehavior Ġof ĠN MT Ġmodels . ĠPrevious Ġwork ĠB IB REF 7 Ġtreats ĠN MT Ġmodels Ġas Ġblack - boxes Ġand Ġprovides Ġexplanations Ġthat Ġclosely Ġresemble Ġthe Ġattention Ġscores Ġin ĠN MT Ġmodels . ĠHowever , Ġrecent Ġstudies Ġreveal Ġthat Ġattention Ġdoes Ġnot Ġprovide Ġmeaningful Ġexplanations Ġsince Ġthe Ġrelationship Ġbetween Ġattention Ġscores Ġand Ġmodel Ġoutput Ġis Ġunclear ĠB IB REF 8 . </s> In Ġthis Ġpaper , Ġwe Ġfocus Ġon Ġthe Ġsecond Ġthread Ġand Ġtry Ġto Ġopen Ġthe Ġblack - box Ġby Ġexploiting Ġthe Ġgrad ients Ġin ĠN MT Ġgeneration , Ġwhich Ġaims Ġto Ġestimate Ġthe Ġword Ġimportance Ġbetter . ĠSpecifically , Ġwe Ġemploy Ġthe Ġintegrated Ġgrad ients Ġmethod ĠB IB REF 9 Ġto Ġattribute Ġthe Ġoutput Ġto Ġthe Ġinput Ġwords Ġwith Ġthe Ġintegration Ġof Ġfirst - order Ġderivatives . ĠWe Ġjustify Ġthe Ġgradient - based Ġapproach Ġvia Ġquantitative Ġcomparison Ġwith Ġblack - box Ġmethods Ġon Ġa Ġcouple Ġof Ġpert urb ation Ġoperations , Ġseveral Ġlanguage Ġpairs , Ġand Ġtwo Ġrepresentative Ġmodel Ġarchitectures , Ġdemonstrating Ġits Ġsuperiority Ġon Ġestimating Ġword Ġimportance . </s> We Ġanalyze Ġthe Ġlinguistic Ġbehaviors Ġof Ġwords Ġwith Ġthe Ġimportance Ġand Ġshow Ġits Ġpotential Ġto Ġimprove ĠN MT Ġmodels . ĠFirst , Ġwe Ġleverage Ġthe Ġword Ġimportance Ġto Ġidentify Ġinput Ġwords Ġthat Ġare Ġunder - trans lated Ġby ĠN MT Ġmodels . ĠExperimental Ġresults Ġshow Ġthat Ġthe Ġgradient - based Ġapproach Ġoutper forms Ġboth Ġthe Ġbest Ġblack - box Ġmethod Ġand Ġother Ġcomparative Ġmethods . ĠSecond , Ġwe Ġanalyze Ġthe Ġlinguistic Ġroles Ġof Ġidentified Ġimportant Ġwords , Ġand Ġfind Ġthat Ġwords Ġof Ġcertain Ġsynt actic Ġcategories Ġhave Ġhigher Ġimportance Ġwhile Ġthe Ġcategories Ġvary Ġacross Ġlanguage . ĠFor Ġexample , Ġnoun s Ġare Ġmore Ġimportant Ġfor ĠChinese $ \ Right arrow Ġ$ English Ġtranslation , Ġwhile Ġpre pos itions Ġare Ġmore Ġimportant Ġfor ĠEnglish - French Ġand Ġ- Japanese Ġtranslation . ĠThis Ġfinding Ġcan Ġinspire Ġbetter Ġdesign Ġprinciples Ġof ĠN MT Ġarchitectures Ġfor Ġdifferent Ġlanguage Ġpairs . ĠFor Ġinstance , Ġa Ġbetter Ġarchitecture Ġfor Ġa Ġgiven Ġlanguage Ġpair Ġshould Ġconsider Ġits Ġown Ġlanguage Ġcharacteristics . </s> Introduction Ġ: :: ĠContributions </s> Our Ġmain Ġcontributions Ġare : </s> Our Ġstudy Ġdemonstrates Ġthe Ġnecessity Ġand Ġeffectiveness Ġof Ġexploiting Ġthe Ġintermediate Ġgrad ients Ġfor Ġestimating Ġword Ġimportance . </s> We Ġfind Ġthat Ġword Ġimportance Ġis Ġuseful Ġfor Ġunderstanding ĠN MT Ġby Ġidentifying Ġunder - trans lated Ġwords . </s> We Ġprovide Ġempirical Ġsupport Ġfor Ġthe Ġdesign Ġprinciple Ġof ĠN MT Ġarchitectures : Ġessential Ġinduct ive Ġbias Ġ( e . g ., Ġlanguage Ġcharacteristics ) Ġshould Ġbe Ġconsidered Ġfor Ġmodel Ġdesign . </s> Related ĠWork Ġ: :: ĠInter pre ting ĠSe q 2 Se q ĠModels </s> Inter pret ability Ġof ĠSe q 2 Se q Ġmodels Ġhas Ġrecently Ġbeen Ġexplored Ġmainly Ġfrom Ġtwo Ġperspectives : Ġinterpreting Ġinternal Ġrepresentations Ġand Ġunderstanding Ġinput - output Ġbehaviors . ĠMost Ġof Ġthe Ġexisting Ġwork Ġfocus Ġon Ġthe Ġformer Ġthread , Ġwhich Ġanaly zes Ġthe Ġlinguistic Ġinformation Ġembed ed Ġin Ġthe Ġlearned Ġrepresentations ĠB IB REF 3 , ĠB IB REF 4 , ĠB IB REF 10 Ġor Ġthe Ġhidden Ġunits ĠB IB REF 6 , ĠB IB REF 5 . ĠSeveral Ġresearchers Ġturn Ġto Ġexpose Ġsystematic Ġdifferences Ġbetween Ġhuman Ġand ĠN MT Ġtranslations ĠB IB REF 11 , ĠB IB REF 12 , Ġindicating Ġthe Ġlinguistic Ġproperties Ġworthy Ġof Ġinvestigating . ĠHowever , Ġthe Ġlearned Ġrepresentations Ġmay Ġdepend Ġon Ġthe Ġmodel Ġimplementation , Ġwhich Ġpotentially Ġlimit Ġthe Ġapplic ability Ġof Ġthese Ġmethods Ġto Ġa Ġbroader Ġrange Ġof Ġmodel Ġarchitectures . ĠAccordingly , Ġwe Ġfocus Ġon Ġunderstanding Ġthe Ġinput - output Ġbehaviors , Ġand Ġvalidate Ġon Ġdifferent Ġarchitectures Ġto Ġdemonstrate Ġthe Ġunivers ality Ġof Ġour Ġfindings . </s> Con cerning Ġinterpreting Ġthe Ġinput - output Ġbehavior , Ġprevious Ġwork Ġgenerally Ġtreats ĠSe q 2 Se q Ġmodels Ġas Ġblack - boxes ĠB IB REF 13 , ĠB IB REF 7 . ĠFor Ġexample , Ġal v arez 2017 ca usal Ġmeasure Ġthe Ġrelevance Ġbetween Ġtwo Ġinput - output Ġtokens Ġby Ġpert ur bing Ġthe Ġinput Ġsequence . ĠHowever , Ġthey Ġdo Ġnot Ġexploit Ġany Ġintermediate Ġinformation Ġsuch Ġas Ġgrad ients , Ġand Ġthe Ġrelevance Ġscore Ġonly Ġresembles Ġattention Ġscores . ĠRecently , ĠJ ain 2019 Att ention IN Ġshow Ġthat Ġattention Ġscores Ġare Ġin Ġweak Ġcorrelation Ġwith Ġthe Ġfeature Ġimportance . ĠStarting Ġfrom Ġthis Ġobservation , Ġwe Ġexploit Ġthe Ġintermediate Ġgrad ients Ġto Ġbetter Ġestimate Ġword Ġimportance , Ġwhich Ġconsistently Ġoutper forms Ġits Ġattention Ġcounterpart Ġacross Ġmodel Ġarchitectures Ġand Ġlanguage Ġpairs . </s> Related ĠWork Ġ: :: ĠExpl o iting ĠGrad ients Ġfor ĠModel ĠInterpret ation </s> The Ġintermediate Ġgrad ients Ġhave Ġproven Ġto Ġbe Ġuseful Ġin Ġinterpreting Ġdeep Ġlearning Ġmodels , Ġsuch Ġas ĠN LP Ġmodels ĠB IB REF 14 , ĠB IB REF 15 Ġand Ġcomputer Ġvision Ġmodels ĠB IB REF 16 , ĠB IB REF 9 . ĠAmong Ġall Ġgradient - based Ġapproaches , Ġthe Ġintegrated Ġgrad ients ĠB IB REF 9 Ġis Ġappealing Ġsince Ġit Ġdoes Ġnot Ġneed Ġany Ġinstrument ation Ġof Ġthe Ġarchitecture Ġand Ġcan Ġbe Ġcomputed Ġeasily Ġby Ġcalling Ġgradient Ġoperations . ĠIn Ġthis Ġwork , Ġwe Ġemploy Ġthe ĠIG Ġmethod Ġto Ġinterpret ĠN MT Ġmodels Ġand Ġreveal Ġseveral Ġinteresting Ġfindings , Ġwhich Ġcan Ġpotentially Ġhelp Ġdebug ĠN MT Ġmodels Ġand Ġdesign Ġbetter Ġarchitectures Ġfor Ġspecific Ġlanguage Ġpairs . </s> App roach Ġ: :: ĠNeural ĠMachine ĠTranslation </s> In Ġmachine Ġtranslation Ġtask , Ġa ĠN MT Ġmodel Ġ$ F $ : Ġ$\ text bf Ġ{ x } Ġ\ right arrow Ġ\ text bf Ġ{ y } $ Ġmaxim izes Ġthe Ġprobability Ġof Ġa Ġtarget Ġsequence Ġ$\ text bf Ġ{ y } Ġ= Ġ\ l brace Ġy _ 1 ,... , y _ N \ r brace Ġ$ Ġgiven Ġa Ġsource Ġsentence Ġ$\ text bf Ġ{ x } Ġ= Ġ\ l brace Ġx _ 1 ,... , x _ M \ r brace Ġ$ : </s> where Ġ$\ math bf Ġ{\ the ta Ġ} $ Ġis Ġthe Ġmodel Ġparameter Ġand Ġ$\ text bf Ġ{ y } _{ < n } $ Ġis Ġa Ġpartial Ġtranslation . ĠAt Ġeach Ġtime Ġstep Ġn , Ġthe Ġmodel Ġgenerates Ġan Ġoutput Ġword Ġof Ġthe Ġhighest Ġprobability Ġbased Ġon Ġthe Ġsource Ġsentence Ġ$\ text bf Ġ{ x } $ Ġand Ġthe Ġpartial Ġtranslation Ġ$\ text bf Ġ{ y } _{ < n } $. ĠThe Ġtraining Ġobjective Ġis Ġto Ġminimize Ġthe Ġnegative Ġlog - like lihood Ġloss Ġon Ġthe Ġtraining Ġcorpus . ĠDuring Ġthe Ġinference , Ġbeam Ġsearch Ġis Ġemployed Ġto Ġdecode Ġa Ġmore Ġoptimal Ġtranslation . ĠIn Ġthis Ġstudy , Ġwe Ġinvestigate Ġthe Ġcontribution Ġof Ġeach Ġinput Ġword Ġ$ x _ m $ Ġto Ġthe Ġtranslated Ġsentence Ġ${ \ bf Ġy } $. </s> App roach Ġ: :: ĠWord ĠImport ance </s> In Ġthis Ġwork , Ġthe Ġnotion Ġof ĠâĢ ľ word Ġimportance âĢ Ŀ Ġis Ġemployed Ġto Ġquantify Ġthe Ġcontribution Ġthat Ġa Ġword Ġin Ġthe Ġinput Ġsentence Ġmakes Ġto Ġthe ĠN MT Ġgenerations . ĠWe Ġcategor ize Ġthe Ġmethods Ġof Ġword Ġimportance Ġestimation Ġinto Ġtwo Ġtypes : Ġblack - box Ġmethods Ġwithout Ġthe Ġknowledge Ġof Ġthe Ġmodel Ġand Ġwhite - box Ġmethods Ġthat Ġhave Ġaccess Ġto Ġthe Ġmodel Ġinternal Ġinformation Ġ( e . g ., Ġparameters Ġand Ġgrad ients ). ĠPrevious Ġstudies Ġmostly Ġfall Ġinto Ġthe Ġformer Ġtype , Ġand Ġin Ġthis Ġstudy , Ġwe Ġinvestigate Ġseveral Ġrepresentative Ġblack - box Ġmethods : </s> Content ĠWords : ĠIn Ġlingu istics , Ġall Ġwords Ġcan Ġbe Ġcategorized Ġas Ġeither Ġcontent Ġor Ġcontent - free Ġwords . ĠContent Ġwords Ġconsist Ġmostly Ġof Ġnoun s , Ġverbs , Ġand Ġadject ives , Ġwhich Ġcarry Ġdescriptive Ġmeanings Ġof Ġthe Ġsentence Ġand Ġthereby Ġare Ġoften Ġconsidered Ġas Ġimportant . </s> F requent ĠWords : ĠWe Ġrank Ġthe Ġrelative Ġimportance Ġof Ġinput Ġwords Ġaccording Ġto Ġtheir Ġfrequency Ġin Ġthe Ġtraining Ġcorpus . ĠWe Ġdo Ġnot Ġconsider Ġthe Ġtop Ġ50 Ġmost Ġfrequent Ġwords Ġsince Ġthey Ġare Ġmostly Ġpunct uation Ġand Ġstop Ġwords . </s> Ca usal ĠModel ĠB IB REF 7 : ĠSince Ġthe Ġcausal Ġmodel Ġis Ġcomplicated Ġto Ġimplement Ġand Ġits Ġscores Ġclosely Ġresemble Ġattention Ġscores Ġin ĠN MT Ġmodels . ĠIn Ġthis Ġstudy , Ġwe Ġuse ĠAttention Ġscores Ġto Ġsimulate Ġthe Ġcausal Ġmodel . </s> Our Ġapproach Ġbelongs Ġto Ġthe Ġwhite - box Ġcategory Ġby Ġexploiting Ġthe Ġintermediate Ġgrad ients , Ġwhich Ġwill Ġbe Ġdescribed Ġin Ġthe Ġnext Ġsection . </s> App roach Ġ: :: ĠIntegrated ĠGrad ients </s> In Ġthis Ġwork , Ġwe Ġresort Ġto Ġa Ġgradient - based Ġmethod , Ġintegrated Ġgrad ients ĠB IB REF 9 Ġ( IG ), Ġwhich Ġwas Ġoriginally Ġproposed Ġto Ġattribute Ġthe Ġmodel Ġpredictions Ġto Ġinput Ġfeatures . ĠIt Ġexploits Ġthe Ġhandy Ġmodel Ġgradient Ġinformation Ġby Ġintegrating Ġfirst - order Ġderivatives . ĠIG Ġis Ġimplementation Ġinvari ant Ġand Ġdoes Ġnot Ġrequire Ġneural Ġmodels Ġto Ġbe Ġdifferent iable Ġor Ġsmooth , Ġthereby Ġis Ġsuitable Ġfor Ġcomplex Ġneural Ġnetworks Ġlike ĠTrans former . ĠIn Ġthis Ġwork , Ġwe Ġuse ĠIG Ġto Ġestimate Ġthe Ġword Ġimportance Ġin Ġan Ġinput Ġsentence Ġprecisely . </s> Form ally , Ġlet Ġ$\ text bf Ġ{ x } Ġ= Ġ( x _ 1 , Ġ... , Ġx _ M ) $ Ġbe Ġthe Ġinput Ġsentence Ġand Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġbe Ġa Ġbaseline Ġinput . Ġ$ F $ Ġis Ġa Ġwell - trained ĠN MT Ġmodel , Ġand Ġ$ F (\ text bf Ġ{ x }) _ n $ Ġis Ġthe Ġmodel Ġoutput Ġ( i . e ., Ġ$ P ( y _ n | \ text bf Ġ{ y } _{ < n }, \ text bf Ġ{ x }) $ ) Ġat Ġtime Ġstep Ġ$ n $. ĠIntegrated Ġgrad ients Ġis Ġthen Ġdefined Ġas Ġthe Ġintegral Ġof Ġgrad ients Ġalong Ġthe Ġstraight line Ġpath Ġfrom Ġthe Ġbaseline Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġto Ġthe Ġinput Ġ$\ text bf Ġ{ x } $. ĠIn Ġdetail , Ġthe Ġcontribution Ġof Ġthe Ġ$ m ^{ th } $ Ġword Ġin Ġ$\ text bf Ġ{ x } $ Ġto Ġthe Ġprediction Ġof Ġ$ F (\ text bf Ġ{ x }) _ n $ Ġis Ġdefined Ġas Ġfollows . </s> where Ġ$\ frac {\ partial Ġ{ F (\ text bf Ġ{ x }) _ n }} {\ partial Ġ{\ text bf Ġ{ x } _ m }} $ Ġis Ġthe Ġgradient Ġof Ġ$ F (\ text bf Ġ{ x }) _ n $ Ġw . r . t . Ġthe Ġembed ding Ġof Ġthe Ġ$ m ^{ th } $ Ġword . ĠIn Ġthis Ġpaper , Ġas Ġsuggested , Ġthe Ġbaseline Ġinput Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġis Ġset Ġas Ġa Ġsequence Ġof Ġzero Ġembed d ings Ġthat Ġhas Ġthe Ġsame Ġsequence Ġlength Ġ$ M $. ĠIn Ġthis Ġway , Ġwe Ġcan Ġcompute Ġthe Ġcontribution Ġof Ġa Ġspecific Ġinput Ġword Ġto Ġa Ġdesignated Ġoutput Ġword . ĠSince Ġthe Ġabove Ġformula Ġis Ġint ract able Ġfor Ġdeep Ġneural Ġmodels , Ġwe Ġapproximate Ġit Ġby Ġsum ming Ġthe Ġgrad ients Ġalong Ġa Ġmulti - step Ġpath Ġfrom Ġbaseline Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġto Ġthe Ġinput Ġx . </s> where Ġ$ S $ Ġdenotes Ġthe Ġnumber Ġof Ġsteps Ġthat Ġare Ġuniformly Ġdistributed Ġalong Ġthe Ġpath . ĠThe ĠIG Ġwill Ġbe Ġmore Ġaccurate Ġif Ġa Ġlarger ĠS Ġis Ġused . ĠIn Ġour Ġpreliminary Ġexperiments , Ġwe Ġvaried Ġthe Ġsteps Ġand Ġfound Ġ300 Ġsteps Ġyielding Ġfairly Ġgood Ġperformance . </s> Following Ġthe Ġformula , Ġwe Ġcan Ġcalculate Ġthe Ġcontribution Ġof Ġevery Ġinput Ġword Ġmakes Ġto Ġevery Ġoutput Ġword , Ġforming Ġa Ġcontribution Ġmatrix Ġof Ġsize Ġ$ M Ġ\ times ĠN $, Ġwhere Ġ$ N $ Ġis Ġthe Ġoutput Ġsentence Ġlength . ĠGiven Ġthe Ġcontribution Ġmatrix , Ġwe Ġcan Ġobtain Ġthe Ġword Ġimportance Ġof Ġeach Ġinput Ġword Ġto Ġthe Ġentire Ġoutput Ġsentence . ĠTo Ġthis Ġend , Ġfor Ġeach Ġinput Ġword , Ġwe Ġfirst Ġaggregate Ġits Ġcontribution Ġvalues Ġto Ġall Ġoutput Ġwords Ġby Ġthe Ġsum Ġoperation , Ġand Ġthen Ġnormal ize Ġall Ġsums Ġthrough Ġthe ĠSoft max Ġfunction . ĠFigure ĠFIG REF 13 Ġillustrates Ġan Ġexample Ġof Ġthe Ġcalculated Ġword Ġimportance Ġand Ġthe Ġcontribution Ġmatrix , Ġwhere Ġan ĠEnglish Ġsentence Ġis Ġtranslated Ġinto Ġa ĠFrench Ġsentence Ġusing Ġthe ĠTrans former Ġmodel . ĠA Ġnegative Ġcontribution Ġvalue Ġindicates Ġthat Ġthe Ġinput Ġword Ġhas Ġnegative Ġeffects Ġon Ġthe Ġoutput Ġword . </s> Exper iment Ġ: :: ĠData </s> To Ġmake Ġthe Ġconclusion Ġconvincing , Ġwe Ġfirst Ġchoose Ġtwo Ġlarge - scale Ġdatasets Ġthat Ġare Ġpublicly Ġavailable , Ġi . e ., ĠChinese - English Ġand ĠEnglish - French . ĠSince ĠEnglish , ĠFrench , Ġand ĠChinese Ġall Ġbelong Ġto Ġthe Ġsubject - verb - object Ġ( S VO ) Ġfamily , Ġwe Ġchoose Ġanother Ġvery Ġdifferent Ġsubject - object - verb Ġ( S OV ) Ġlanguage , ĠJapanese , Ġwhich Ġmight Ġbring Ġsome Ġinteresting Ġlinguistic Ġbehaviors Ġin ĠEnglish - Japanese Ġtranslation . </s> For ĠChinese - English Ġtask , Ġwe Ġuse ĠW MT 17 ĠChinese - English Ġdataset Ġthat Ġconsists Ġof Ġ$ 20 . 6 $ M Ġsentence Ġpairs . ĠFor ĠEnglish - French Ġtask , Ġwe Ġuse ĠW MT 14 ĠEnglish - French Ġdataset Ġthat Ġcomprises Ġ$ 35 . 5 $ M Ġsentence Ġpairs . ĠFor ĠEnglish - Japanese Ġtask , Ġwe Ġfollow ĠB IB REF 17 Ġto Ġuse Ġthe Ġfirst Ġtwo Ġsections Ġof ĠW AT 17 ĠEnglish - Japanese Ġdataset Ġthat Ġconsists Ġof Ġ$ 1 . 9 $ M Ġsentence Ġpairs . ĠFollowing Ġthe Ġstandard ĠN MT Ġprocedure , Ġwe Ġadopt Ġthe Ġstandard Ġbyte Ġpair Ġencoding Ġ( B PE ) ĠB IB REF 18 Ġwith Ġ32 K Ġmerge Ġoperations Ġfor Ġall Ġlanguage Ġpairs . ĠWe Ġbelieve Ġthat Ġthese Ġdatasets Ġare Ġlarge Ġenough Ġto Ġconfirm Ġthe Ġrationality Ġand Ġvalidity Ġof Ġour Ġexperimental Ġanalyses . </s> Exper iment Ġ: :: ĠImplementation </s> We Ġchoose Ġthe Ġstate - of - the - art ĠTrans former ĠB IB REF 1 Ġmodel Ġand Ġthe Ġconventional ĠR NN - Search Ġmodel ĠB IB REF 0 Ġas Ġour Ġtest Ġbed . ĠWe Ġimplement Ġthe ĠAttribution Ġmethod Ġbased Ġon Ġthe ĠFair seq - py ĠB IB REF 19 Ġframework Ġfor Ġthe Ġabove Ġmodels . ĠAll Ġmodels Ġare Ġtrained Ġon Ġthe Ġtraining Ġcorpus Ġfor Ġ100 k Ġsteps Ġunder Ġthe Ġstandard Ġsettings , Ġwhich Ġachieve Ġcomparable Ġtranslation Ġresults . ĠAll Ġthe Ġfollowing Ġexperiments Ġare Ġconducted Ġon Ġthe Ġtest Ġdataset , Ġand Ġwe Ġestimate Ġthe Ġinput Ġword Ġimportance Ġusing Ġthe Ġmodel Ġgenerated Ġhypotheses . </s> In Ġthe Ġfollowing Ġexperiments , Ġwe Ġcompare ĠIG Ġ( Att ribution ) Ġwith Ġseveral Ġblack - box Ġmethods Ġ( i . e ., ĠContent , ĠFrequency , ĠAttention ) Ġas Ġintroduced Ġin ĠSection ĠSEC REF 8 . ĠIn ĠSection ĠSEC REF 21 , Ġto Ġensure Ġthat Ġthe Ġtranslation Ġperformance Ġdecrease Ġattributes Ġto Ġthe Ġselected Ġwords Ġinstead Ġof Ġthe Ġpert urb ation Ġoperations , Ġwe Ġrandomly Ġselect Ġthe Ġsame Ġnumber Ġof Ġwords Ġto Ġpert urb Ġ( Random ), Ġwhich Ġserves Ġas Ġa Ġbaseline . ĠSince Ġthere Ġis Ġno Ġranking Ġfor Ġcontent Ġwords , Ġwe Ġrandomly Ġselect Ġa Ġset Ġof Ġcontent Ġwords Ġas Ġimportant Ġwords . ĠTo Ġavoid Ġthe Ġpotential Ġbias Ġintroduced Ġby Ġrandom ness Ġ( i . e ., ĠRandom Ġand ĠContent ), Ġwe Ġrepeat Ġthe Ġexperiments Ġfor Ġ10 Ġtimes Ġand Ġreport Ġthe Ġaveraged Ġresults . ĠWe Ġcalculate Ġthe ĠAttention Ġimportance Ġin Ġa Ġsimilar Ġmanner Ġas Ġthe ĠAttribution , Ġexcept Ġthat Ġthe Ġattention Ġscores Ġuse Ġa Ġmax Ġoperation Ġdue Ġto Ġthe Ġbetter Ġperformance . </s> Exper iment Ġ: :: ĠEvaluation </s> We Ġevaluate Ġthe Ġeffectiveness Ġof Ġestimating Ġword Ġimportance Ġby Ġthe Ġtranslation Ġperformance Ġdecrease . ĠMore Ġspecifically , Ġunlike Ġthe Ġusual Ġway , Ġwe Ġmeasure Ġthe Ġdecrease Ġof Ġtranslation Ġperformance Ġwhen Ġpert ur bing Ġa Ġset Ġof Ġimportant Ġwords Ġthat Ġare Ġof Ġtop - most Ġword Ġimportance Ġin Ġa Ġsentence . ĠThe Ġmore Ġtranslation Ġperformance Ġde grades , Ġthe Ġmore Ġimportant Ġthe Ġword Ġis . </s> We Ġuse Ġthe Ġstandard ĠB LE U Ġscore Ġas Ġthe Ġevaluation Ġmetric Ġfor Ġtranslation Ġperformance . ĠTo Ġmake Ġthe Ġconclusion Ġmore Ġconvincing , Ġwe Ġconduct Ġexperiments Ġon Ġdifferent Ġtypes Ġof Ġsynthetic Ġpert urb ations Ġ( Section ĠSEC REF 21 ), Ġas Ġwell Ġas Ġdifferent ĠN MT Ġarchitectures Ġand Ġlanguage Ġpairs Ġ( Section ĠSEC REF 27 ). ĠIn Ġaddition , Ġwe Ġcompare Ġwith Ġa Ġsupervised Ġer asure Ġmethod , Ġwhich Ġrequires Ġground - truth Ġtranslations Ġfor Ġscoring Ġword Ġimportance Ġ( Section ĠSEC REF 30 ). </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠP ert urb ations </s> In Ġthis Ġexperiment , Ġwe Ġinvestigate Ġthe Ġeffectiveness Ġof Ġword Ġimportance Ġestimation Ġmethods Ġunder Ġdifferent Ġsynthetic Ġpert urb ations . ĠSince Ġthe Ġpert urb ation Ġon Ġtext Ġis Ġnotoriously Ġhard ĠB IB REF 20 Ġdue Ġto Ġthe Ġsemantic Ġshifting Ġproblem , Ġin Ġthis Ġexperiment , Ġwe Ġinvestigate Ġthree Ġtypes Ġof Ġpert urb ations Ġto Ġavoid Ġthe Ġpotential Ġbias Ġ: </s> De let ion Ġpert urb ation Ġremoves Ġthe Ġselected Ġwords Ġfrom Ġthe Ġinput Ġsentence , Ġand Ġit Ġcan Ġbe Ġregarded Ġas Ġa Ġspecific Ġinstant iation Ġof Ġsentence Ġcompression ĠB IB REF 21 . </s> Mask Ġpert urb ation Ġreplaces Ġembed ding Ġvectors Ġof Ġthe Ġselected Ġwords Ġwith Ġall - zero Ġvectors ĠB IB REF 22 , Ġwhich Ġis Ġsimilar Ġto ĠDe let ion Ġpert urb ation Ġexcept Ġthat Ġit Ġretains Ġthe Ġplaceholder . </s> G ram matical ĠReplacement Ġpert urb ation Ġreplaces Ġa Ġword Ġby Ġanother Ġword Ġof Ġthe Ġsame Ġlinguistic Ġrole Ġ( i . e ., ĠPOS Ġtags ), Ġyielding Ġa Ġsentence Ġthat Ġis Ġgram matically Ġcorrect Ġbut Ġsem antically Ġnonsensical ĠB IB REF 23 , ĠB IB REF 24 , Ġsuch Ġas ĠâĢ ľ color less Ġgreen Ġideas Ġsleep Ġfuriously âĢ Ŀ . </s> Figure ĠFIG REF 19 Ġillustrates Ġthe Ġexperimental Ġresults Ġon ĠChinese $ \ Right arrow Ġ$ English Ġtranslation Ġwith ĠTrans former . ĠIt Ġshows Ġthat ĠAttribution Ġmethod Ġconsistently Ġoutper forms Ġother Ġmethods Ġagainst Ġdifferent Ġpert urb ations Ġon Ġa Ġvarious Ġnumber Ġof Ġoperations . ĠHere Ġthe Ġoperation Ġnumber Ġdenotes Ġthe Ġnumber Ġof Ġpert urbed Ġwords Ġin Ġa Ġsentence . ĠSpecifically , Ġwe Ġcan Ġmake Ġthe Ġfollowing Ġobservations . </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠP ert urb ations Ġ: :: ĠImportant Ġwords Ġare Ġmore Ġinfluential Ġon Ġtranslation Ġperformance Ġthan Ġthe Ġothers . </s> Under Ġthree Ġdifferent Ġpert urb ations , Ġpert ur bing Ġwords Ġof Ġtop - most Ġimportance Ġleads Ġto Ġlower ĠB LE U Ġscores Ġthan ĠRandom Ġselected Ġwords . ĠIt Ġconfirms Ġthe Ġexistence Ġof Ġimportant Ġwords , Ġwhich Ġhave Ġgreater Ġimpacts Ġon Ġtranslation Ġperformance . ĠFurthermore , Ġpert ur bing Ġimportant Ġwords Ġidentified Ġby ĠAttribution Ġoutper forms Ġthe ĠRandom Ġmethod Ġby Ġa Ġlarge Ġmargin Ġ( more Ġthan Ġ4 . 0 ĠB LE U Ġunder Ġ5 Ġoperations ). </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠP ert urb ations Ġ: :: ĠThe Ġgradient - based Ġmethod Ġis Ġsuperior Ġto Ġcomparative Ġmethods Ġ( e . g ., ĠAttention ) Ġin Ġestimating Ġword Ġimportance . </s> Figure ĠFIG REF 19 Ġshows Ġthat Ġtwo Ġblack - box Ġmethods Ġ( i . e ., ĠContent , ĠFrequency ) Ġperform Ġonly Ġslightly Ġbetter Ġthan Ġthe ĠRandom Ġmethod . ĠSpecifically , Ġthe ĠFrequency Ġmethod Ġdemonstrates Ġeven Ġworse Ġperformances Ġunder Ġthe ĠMask Ġpert urb ation . ĠTherefore , Ġlinguistic Ġproperties Ġ( such Ġas ĠPOS Ġtags ) Ġand Ġthe Ġword Ġfrequency Ġcan Ġonly Ġpartially Ġhelp Ġidentify Ġthe Ġimportant Ġwords , Ġbut Ġit Ġis Ġnot Ġas Ġaccurate Ġas Ġwe Ġthought . ĠIn Ġthe Ġmeanwhile , Ġit Ġis Ġintriguing Ġto Ġexplore Ġwhat Ġexact Ġlinguistic Ġcharacteristics Ġthese Ġimportant Ġwords Ġreveal , Ġwhich Ġwill Ġbe Ġintroduced Ġin ĠSection ĠSEC REF 5 . </s> We Ġalso Ġevaluate Ġthe ĠAttention Ġmethod , Ġwhich Ġbases Ġon Ġthe Ġenc oder - dec oder Ġattention Ġscores Ġat Ġthe Ġlast Ġlayer Ġof ĠTrans former . ĠNote Ġthat Ġthe ĠAttention Ġmethod Ġis Ġalso Ġused Ġto Ġsimulate Ġthe Ġbest Ġblack - box Ġmethod ĠSOC R AT , Ġand Ġthe Ġresults Ġshow Ġthat Ġit Ġis Ġmore Ġeffective Ġthan Ġblack - box Ġmethods Ġand Ġthe ĠRandom Ġbaseline . ĠGiven Ġthe Ġpowerful ĠAttention Ġmethod , ĠAttribution Ġmethod Ġstill Ġachieves Ġbest Ġperformances Ġunder Ġall Ġthree Ġpert urb ations . ĠFurthermore , Ġwe Ġfind Ġthat Ġthe Ġgap Ġbetween ĠAttribution Ġand ĠAttention Ġis Ġnotably Ġlarge Ġ( around Ġ$ 1 . 0 + $ ĠB LE U Ġdifference ). ĠAttention Ġmethod Ġdoes Ġnot Ġprovide Ġas Ġaccurate Ġword Ġimportance Ġas Ġthe ĠAttribution , Ġwhich Ġexhibits Ġthe Ġsuperiority Ġof Ġgradient - based Ġmethods Ġand Ġconsists Ġwith Ġthe Ġconclusion Ġreported Ġin Ġthe Ġprevious Ġstudy ĠB IB REF 8 . </s> In Ġaddition , Ġas Ġshown Ġin ĠFigure ĠFIG REF 19 , Ġthe Ġpert urb ation Ġeffectiveness Ġof ĠDe let ion , ĠMask , Ġand ĠGram matical ĠReplacement Ġvaries Ġfrom Ġstrong Ġto Ġweak . ĠIn Ġthe Ġfollowing Ġexperiments , Ġwe Ġchoose ĠMask Ġas Ġthe Ġrepresentative Ġpert urb ation Ġoperation Ġfor Ġits Ġmoderate Ġpert urb ation Ġperformance , Ġbased Ġon Ġwhich Ġwe Ġcompare Ġtwo Ġmost Ġeffective Ġmethods ĠAttribution Ġand ĠAttention . </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠN MT ĠArchitecture Ġand ĠLanguage ĠP airs Ġ: :: ĠDifferent ĠN MT ĠArchitecture </s> We Ġvalidate Ġthe Ġeffectiveness Ġof Ġthe Ġproposed Ġapproach Ġusing Ġa Ġdifferent ĠN MT Ġarchitecture ĠR NN - Search Ġon Ġthe ĠChinese $ \ Right arrow Ġ$ English Ġtranslation Ġtask . ĠThe Ġresults Ġare Ġshown Ġin ĠFigure ĠFIG REF 20 ( a ). ĠWe Ġobserve Ġthat Ġthe ĠAttribution Ġmethod Ġstill Ġoutper forms Ġboth ĠAttention Ġmethod Ġand ĠRandom Ġmethod Ġby Ġa Ġdecent Ġmargin . ĠBy Ġcomparing Ġto ĠTrans former , Ġthe Ġresults Ġalso Ġreveal Ġthat Ġthe ĠR NN - Search Ġmodel Ġis Ġless Ġrobust Ġto Ġthese Ġpert urb ations . ĠTo Ġbe Ġspecific , Ġunder Ġthe Ġsetting Ġof Ġfive Ġoperations Ġand ĠAttribution Ġmethod , ĠTrans former Ġshows Ġa Ġrelative Ġdecrease Ġof Ġ$ 55 \ % $ Ġon ĠB LE U Ġscores Ġwhile Ġthe Ġdecline Ġof ĠR NN - Search Ġmodel Ġis Ġ$ 64 \ % $. </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠN MT ĠArchitecture Ġand ĠLanguage ĠP airs Ġ: :: ĠDifferent ĠLanguage ĠP airs Ġand ĠDirections </s> We Ġfurther Ġconduct Ġexperiments Ġon Ġanother Ġtwo Ġlanguage Ġpairs Ġ( i . e ., ĠEnglish $ \ Right arrow Ġ$ French , ĠEnglish $ \ Right arrow Ġ$ Japanese Ġin ĠFigures ĠFIG REF 20 ( b , Ġc )) Ġas Ġwell Ġas Ġthe Ġreverse Ġdirections Ġ( Fig ures ĠFIG REF 20 ( d , Ġe , Ġf )) Ġusing ĠTrans former Ġunder Ġthe ĠMask Ġpert urb ation . ĠIn Ġall Ġthe Ġcases , ĠAttribution Ġshows Ġthe Ġbest Ġperformance Ġwhile ĠRandom Ġachieves Ġthe Ġworst Ġresult . ĠMore Ġspecifically , ĠAttribution Ġmethod Ġshows Ġsimilar Ġtranslation Ġquality Ġdegradation Ġon Ġall Ġthree Ġlanguage - p airs , Ġwhich Ġdeclines Ġto Ġaround Ġthe Ġhalf Ġof Ġthe Ġoriginal ĠB LE U Ġscore Ġwith Ġfive Ġoperations . </s> Exper iment Ġ: :: ĠComparison Ġwith ĠSuper vised ĠEr asure </s> There Ġexists Ġanother Ġstraightforward Ġmethod , ĠEr asure ĠB IB REF 7 , ĠB IB REF 22 , ĠB IB REF 25 , Ġwhich Ġdirectly Ġevaluates Ġthe Ġword Ġimportance Ġby Ġmeasuring Ġthe Ġtranslation Ġperformance Ġdegradation Ġof Ġeach Ġword . ĠSpecifically , Ġit Ġer ases Ġ( i . e ., ĠMask ) Ġone Ġword Ġfrom Ġthe Ġinput Ġsentence Ġeach Ġtime Ġand Ġuses Ġthe ĠB LE U Ġscore Ġchanges Ġto Ġdenote Ġthe Ġword Ġimportance Ġ( after Ġnormal ization ). </s> In ĠFigure ĠFIG REF 31 , Ġwe Ġcompare ĠEr asure Ġmethod Ġwith ĠAttribution Ġmethod Ġunder Ġthe ĠMask Ġpert urb ation . ĠThe Ġresults Ġshow Ġthat ĠAttribution Ġmethod Ġis Ġless Ġeffective Ġthan ĠEr asure Ġmethod Ġwhen Ġonly Ġone Ġword Ġis Ġpert urbed . ĠBut Ġit Ġoutper forms Ġthe ĠEr asure Ġmethod Ġwhen Ġpert ur bing Ġ2 Ġor Ġmore Ġwords . ĠThe Ġresults Ġreveal Ġthat Ġthe Ġimportance Ġcalculated Ġby Ġer asing Ġonly Ġone Ġword Ġcannot Ġbe Ġgeneralized Ġto Ġmultiple - words Ġscenarios Ġvery Ġwell . ĠBesides , Ġthe ĠEr asure Ġmethod Ġis Ġa Ġsupervised Ġmethod Ġwhich Ġrequires Ġground - truth Ġreferences , Ġand Ġfinding Ġa Ġbetter Ġwords Ġcombination Ġis Ġcomputation Ġinf eas ible Ġwhen Ġer asing Ġmultiple Ġwords . </s> We Ġclose Ġthis Ġsection Ġby Ġpointing Ġout Ġthat Ġour Ġgradient - based Ġmethod Ġconsistently Ġoutper forms Ġits Ġblack - box Ġcounterparts Ġin Ġvarious Ġsettings , Ġdemonstrating Ġthe Ġeffectiveness Ġand Ġunivers ality Ġof Ġexploiting Ġgrad ients Ġfor Ġestimating Ġword Ġimportance . ĠIn Ġaddition , Ġour Ġapproach Ġis Ġon Ġpar Ġwith Ġor Ġeven Ġoutper forms Ġthe Ġsupervised Ġer asure Ġmethod Ġ( on Ġmultiple - word Ġpert urb ations ). ĠThis Ġis Ġencouraging Ġsince Ġour Ġapproach Ġdoes Ġnot Ġrequire Ġany Ġexternal Ġresource Ġand Ġis Ġfully Ġun super vised . </s> Analysis </s> In Ġthis Ġsection , Ġwe Ġconduct Ġanalyses Ġon Ġtwo Ġpotential Ġus ages Ġof Ġword Ġimportance , Ġwhich Ġcan Ġhelp Ġdebug ĠN MT Ġmodels Ġ( Section ĠSEC REF 33 ) Ġand Ġdesign Ġbetter Ġarchitectures Ġfor Ġspecific Ġlanguages Ġ( Section ĠSEC REF 37 ). ĠDue Ġto Ġthe Ġspace Ġlimitation , Ġwe Ġonly Ġanalyze Ġthe Ġresults Ġof ĠChinese $ \ Right arrow Ġ$ English , ĠEnglish $ \ Right arrow Ġ$ French , Ġand ĠEnglish $ \ Right arrow Ġ$ Japanese . ĠWe Ġlist Ġthe Ġresults Ġon Ġthe Ġreverse Ġdirections Ġin ĠAppendix , Ġin Ġwhich Ġthe Ġgeneral Ġconclusions Ġalso Ġhold . </s> Analysis Ġ: :: ĠEffect Ġon ĠDetect ing ĠTranslation ĠErrors </s> In Ġthis Ġexperiment , Ġwe Ġpropose Ġto Ġuse Ġthe Ġestimated Ġword Ġimportance Ġto Ġdetect Ġthe Ġunder - trans lated Ġwords Ġby ĠN MT Ġmodels . ĠInt uitive ly , Ġunder - trans lated Ġinput Ġwords Ġshould Ġcontribute Ġlittle Ġto Ġthe ĠN MT Ġoutputs , Ġyielding Ġmuch Ġsmaller Ġword Ġimportance . ĠGiven Ġ500 ĠChinese $ \ Right arrow Ġ$ English Ġsentence Ġpairs Ġtranslated Ġby Ġthe ĠTrans former Ġmodel Ġ( BLE U Ġ23 . 57 ), Ġwe Ġask Ġten Ġhuman Ġannot ators Ġto Ġmanually Ġlabel Ġthe Ġunder - trans lated Ġinput Ġwords , Ġand Ġat Ġleast Ġtwo Ġannot ators Ġlabel Ġeach Ġinput - hyp ot hesis Ġpair . ĠThese Ġannot ators Ġhave Ġat Ġleast Ġsix Ġyears Ġof ĠEnglish Ġstudy Ġexperience , Ġwhose Ġnative Ġlanguage Ġis ĠChinese . ĠAmong Ġthese Ġsentences , Ġ178 Ġsentences Ġhave Ġunder - translation Ġerrors Ġwith Ġ5 53 Ġunder - trans lated Ġwords Ġin Ġtotal . </s> Table ĠT AB REF 32 Ġlists Ġthe Ġaccuracy Ġof Ġdetecting Ġunder - translation Ġerrors Ġby Ġcomparing Ġwords Ġof Ġleast Ġimportance Ġand Ġhuman - annot ated Ġunder - trans lated Ġwords . ĠAs Ġseen , Ġour ĠAttribution Ġmethod Ġconsistently Ġand Ġsignificantly Ġoutper forms Ġboth ĠEr asure Ġand ĠAttention Ġapproaches . ĠBy Ġexploiting Ġthe Ġword Ġimportance Ġcalculated Ġby ĠAttribution Ġmethod , Ġwe Ġcan Ġidentify Ġthe Ġunder - translation Ġerrors Ġautomatically Ġwithout Ġthe Ġinvolvement Ġof Ġhuman Ġinterpre ters . ĠAlthough Ġthe Ġaccuracy Ġis Ġnot Ġhigh , Ġit Ġis Ġworth Ġnoting Ġthat Ġour Ġunder - translation Ġmethod Ġis Ġvery Ġsimple Ġand Ġstraightforward . ĠThis Ġis Ġpotentially Ġuseful Ġfor Ġdebugging ĠN MT Ġmodels , Ġe . g ., Ġautomatic Ġpost - ed iting Ġwith Ġconstraint Ġdecoding ĠB IB REF 26 , ĠB IB REF 27 . </s> Analysis Ġ: :: ĠAnalysis Ġon ĠL ingu istic ĠProperties </s> In Ġthis Ġsection , Ġwe Ġanalyze Ġthe Ġlinguistic Ġcharacteristics Ġof Ġimportant Ġwords Ġidentified Ġby Ġthe Ġattribution - based Ġapproach . ĠSpecifically , Ġwe Ġinvestigate Ġseveral Ġrepresentative Ġsets Ġof Ġlinguistic Ġproperties , Ġincluding ĠPOS Ġtags , Ġand Ġfertility , Ġand Ġdepth Ġin Ġa Ġsynt actic Ġparse Ġtree . ĠIn Ġthese Ġanalyses , Ġwe Ġmultiply Ġthe Ġword Ġimportance Ġwith Ġthe Ġcorresponding Ġsentence Ġlength Ġfor Ġfair Ġcomparison . ĠWe Ġuse Ġa Ġdecision Ġtree Ġbased Ġregression Ġmodel Ġto Ġcalculate Ġthe Ġcorrelation Ġbetween Ġthe Ġimportance Ġand Ġlinguistic Ġproperties . </s> Table ĠT AB REF 34 Ġlists Ġthe Ġcorrelations , Ġwhere Ġa Ġhigher Ġvalue Ġindicates Ġa Ġstronger Ġcorrelation . ĠWe Ġfind Ġthat Ġthe Ġsynt actic Ġinformation Ġis Ġalmost Ġindependent Ġof Ġthe Ġword Ġimportance Ġvalue . ĠInstead , Ġthe Ġword Ġimportance Ġstrongly Ġcorrelates Ġwith Ġthe ĠPOS Ġtags Ġand Ġfertility Ġfeatures , Ġand Ġthese Ġfeatures Ġin Ġtotal Ġcontribute Ġover Ġ95 %. ĠTherefore , Ġin Ġthe Ġfollowing Ġanalyses , Ġwe Ġmainly Ġfocus Ġon Ġthe ĠPOS Ġtags Ġ( Table ĠT AB REF 35 ) Ġand Ġfertility Ġproperties Ġ( Table ĠT AB REF 36 ). ĠFor Ġbetter Ġillustration , Ġwe Ġcalculate Ġthe Ġdistribution Ġover Ġthe Ġlinguistic Ġproperty Ġbased Ġon Ġboth Ġthe ĠAttribution Ġimportance Ġ( âĢ ľ Att r . âĢ Ŀ ) Ġand Ġthe Ġword Ġfrequency Ġ( âĢ ľ Count âĢ Ŀ ) Ġinside Ġa Ġsentence . ĠThe Ġlarger Ġthe Ġrelative Ġincrease Ġbetween Ġthese Ġtwo Ġvalues , Ġthe Ġmore Ġimportant Ġthe Ġlinguistic Ġproperty Ġis . </s> Analysis Ġ: :: ĠAnalysis Ġon ĠL ingu istic ĠProperties Ġ: :: ĠCertain Ġsynt actic Ġcategories Ġhave Ġhigher Ġimportance Ġwhile Ġthe Ġcategories Ġvary Ġacross Ġlanguage Ġpairs . </s> As Ġshown Ġin ĠTable ĠT AB REF 35 , Ġcontent Ġwords Ġare Ġmore Ġimportant Ġon ĠChinese $ \ Right arrow Ġ$ English Ġbut Ġcontent - free Ġwords Ġare Ġmore Ġimportant Ġon ĠEnglish $ \ Right arrow Ġ$ Japanese . ĠOn ĠEnglish $ \ Right arrow Ġ$ French , Ġthere Ġis Ġno Ġnotable Ġincrease Ġor Ġdecrease Ġof Ġthe Ġdistribution Ġsince ĠEnglish Ġand ĠFrench Ġare Ġin Ġessence Ġvery Ġsimilar . ĠWe Ġalso Ġobtain Ġsome Ġspecific Ġfindings Ġof Ġgreat Ġinterest . ĠFor Ġexample , Ġwe Ġfind Ġthat Ġnoun Ġis Ġmore Ġimportant Ġon ĠChinese $ \ Right arrow Ġ$ English Ġtranslation , Ġwhile Ġpre position Ġis Ġmore Ġimportant Ġon ĠEnglish $ \ Right arrow Ġ$ French Ġtranslation . ĠMore Ġinterestingly , ĠEnglish $ \ Right arrow Ġ$ Japanese Ġtranslation Ġshows Ġa Ġsubstantial Ġdiscrepancy Ġin Ġcontrast Ġto Ġthe Ġother Ġtwo Ġlanguage Ġpairs . ĠThe Ġresults Ġreveal Ġthat Ġpre position Ġand Ġpunct uation Ġare Ġvery Ġimportant Ġin ĠEnglish $ \ Right arrow Ġ$ Japanese Ġtranslation , Ġwhich Ġis Ġcounter - intuitive . </s> P unct uation Ġin ĠN MT Ġis Ġunder stud ied Ġsince Ġit Ġcarries Ġlittle Ġinformation Ġand Ġoften Ġdoes Ġnot Ġaffect Ġthe Ġunderstanding Ġof Ġa Ġsentence . ĠHowever , Ġwe Ġfind Ġthat Ġpunct uation Ġis Ġimportant Ġon ĠEnglish $ \ Right arrow Ġ$ Japanese Ġtranslation , Ġwhose Ġproportion Ġincreases Ġdramatically . ĠWe Ġconjecture Ġthat Ġit Ġis Ġbecause Ġthe Ġpunct uation Ġcould Ġaffect Ġthe Ġsense Ġgroups Ġin Ġa Ġsentence , Ġwhich Ġfurther Ġbenefits Ġthe Ġsynt actic Ġre ordering Ġin ĠJapanese . </s> Analysis Ġ: :: ĠAnalysis Ġon ĠL ingu istic ĠProperties Ġ: :: ĠWords Ġof Ġhigh Ġfertility Ġare Ġalways Ġimportant . </s> We Ġfurther Ġcompare Ġthe Ġfertility Ġdistribution Ġbased Ġon Ġword Ġimportance Ġand Ġthe Ġword Ġfrequency Ġon Ġthree Ġlanguage Ġpairs . ĠWe Ġhypothes ize Ġthat Ġa Ġsource Ġword Ġthat Ġcorresponds Ġto Ġmultiple Ġtarget Ġwords Ġshould Ġbe Ġmore Ġimportant Ġsince Ġit Ġcontributes Ġmore Ġto Ġboth Ġsentence Ġlength Ġand ĠB LE U Ġscore . </s> Table ĠT AB REF 36 Ġlists Ġthe Ġresults . ĠOverall Ġspeaking , Ġone - to - many Ġfertility Ġis Ġconsistently Ġmore Ġimportant Ġon Ġall Ġthree Ġlanguage Ġpairs , Ġwhich Ġconfirms Ġour Ġhypothesis . ĠOn Ġthe Ġcontrary , Ġnull - aligned Ġwords Ġreceive Ġmuch Ġless Ġattention , Ġwhich Ġshows Ġa Ġpersist ently Ġdecrease Ġon Ġthree Ġlanguage Ġpairs . ĠIt Ġis Ġalso Ġreasonable Ġsince Ġnull - aligned Ġinput Ġwords Ġcontribute Ġalmost Ġnothing Ġto Ġthe Ġtranslation Ġoutputs . </s> Discussion Ġand ĠConclusion </s> We Ġapproach Ġunderstanding ĠN MT Ġby Ġinvestigating Ġthe Ġword Ġimportance Ġvia Ġa Ġgradient - based Ġmethod , Ġwhich Ġbridges Ġthe Ġgap Ġbetween Ġword Ġimportance Ġand Ġtranslation Ġperformance . ĠEm pir ical Ġresults Ġshow Ġthat Ġthe Ġgradient - based Ġmethod Ġis Ġsuperior Ġto Ġseveral Ġblack - box Ġmethods Ġin Ġestimating Ġthe Ġword Ġimportance . ĠFurther Ġanalyses Ġshow Ġthat Ġimportant Ġwords Ġare Ġof Ġdistinct Ġsynt actic Ġcategories Ġon Ġdifferent Ġlanguage Ġpairs , Ġwhich Ġmight Ġsupport Ġthe Ġviewpoint Ġthat Ġessential Ġinduct ive Ġbias Ġshould Ġbe Ġintroduced Ġinto Ġthe Ġmodel Ġdesign ĠB IB REF 28 . ĠOur Ġstudy Ġalso Ġsuggests Ġthe Ġpossibility Ġof Ġdetecting Ġthe Ġnotorious Ġunder - translation Ġproblem Ġvia Ġthe Ġgradient - based Ġmethod . </s> This Ġpaper Ġis Ġan Ġinitiating Ġstep Ġtowards Ġthe Ġgeneral Ġunderstanding Ġof ĠN MT Ġmodels , Ġwhich Ġmay Ġbring Ġsome Ġpotential Ġimprovements , Ġsuch Ġas </s> Inter active ĠMT Ġand ĠCon str aint ĠDec oding ĠB IB REF 29 , ĠB IB REF 26 : ĠThe Ġmodel Ġpays Ġmore Ġattention Ġto Ġthe Ġdetected Ġun important Ġwords , Ġwhich Ġare Ġpossibly Ġunder - trans lated ; </s> Adapt ive ĠInput ĠEmb edd ing ĠB IB REF 30 : ĠWe Ġcan Ġextend Ġthe Ġadaptive Ġsoft max ĠB IB REF 31 Ġto Ġthe Ġinput Ġembed ding Ġof Ġvariable Ġcapacity ĠâĢĵ Ġmore Ġimportant Ġwords Ġare Ġassigned Ġwith Ġmore Ġcapacity ; </s> N MT ĠArchitecture ĠDesign : ĠThe Ġlanguage - specific Ġinduct ive Ġbias Ġ( e . g ., Ġdifferent Ġbehaviors Ġon ĠPOS ) Ġshould Ġbe Ġincorporated Ġinto Ġthe Ġmodel Ġdesign . </s> We Ġcan Ġalso Ġexplore Ġother Ġapplications Ġof Ġword Ġimportance Ġto Ġimprove ĠN MT Ġmodels , Ġsuch Ġas Ġmore Ġtailored Ġtraining Ġmethods . ĠIn Ġgeneral , Ġmodel Ġinterpret ability Ġcan Ġbuild Ġtrust Ġin Ġmodel Ġpredictions , Ġhelp Ġerror Ġdiagnosis Ġand Ġfacilitate Ġmodel Ġrefinement . ĠWe Ġexpect Ġour Ġwork Ġcould Ġshed Ġlight Ġon Ġthe ĠN MT Ġmodel Ġunderstanding Ġand Ġbenefit Ġthe Ġmodel Ġimprovement . </s> There Ġare Ġmany Ġpossible Ġways Ġto Ġimplement Ġthe Ġgeneral Ġidea Ġof Ġexploiting Ġgrad ients Ġfor Ġmodel Ġinterpretation . ĠThe Ġaim Ġof Ġthis Ġpaper Ġis Ġnot Ġto Ġexplore Ġthis Ġwhole Ġspace Ġbut Ġsimply Ġto Ġshow Ġthat Ġsome Ġfairly Ġstraightforward Ġimplementations Ġwork Ġwell . ĠOur Ġapproach Ġcan Ġbenefit Ġfrom Ġadvanced Ġexploitation Ġof Ġthe Ġgrad ients Ġor Ġother Ġuseful Ġintermediate Ġinformation , Ġwhich Ġwe Ġleave Ġto Ġthe Ġfuture Ġwork . </s> Acknowled gement </s> Sh il in ĠHe Ġand ĠMichael ĠR . ĠLy u Ġwere Ġsupported Ġby Ġthe ĠResearch ĠGrants ĠCouncil Ġof Ġthe ĠHong ĠKong ĠSpecial ĠAdministrative ĠRegion , ĠChina Ġ( No . ĠCU HK Ġ142 107 17 Ġof Ġthe ĠGeneral ĠResearch ĠFund ), Ġand ĠMicrosoft ĠResearch ĠAsia Ġ( 2018 ĠMicrosoft ĠResearch ĠAsia ĠCollabor ative ĠResearch ĠAward ). ĠWe Ġthank Ġthe Ġanonymous Ġreviewers Ġfor Ġtheir Ġinsightful Ġcomments Ġand Ġsuggestions . </s> An alyses Ġon ĠReverse ĠDirections </s> 2 </s> We Ġanalyze Ġthe Ġdistribution Ġof Ġsynt actic Ġcategories Ġand Ġword Ġfertility Ġon Ġthe Ġsame Ġlanguage Ġpairs Ġwith Ġreverse Ġdirections , Ġi . e ., ĠEnglish $ \ Right arrow Ġ$ Chinese , ĠFrench $ \ Right arrow Ġ$ English , Ġand ĠJapanese $ \ Right arrow Ġ$ English . ĠThe Ġresults Ġare Ġshown Ġin ĠTable ĠT AB REF 43 Ġand ĠTable ĠT AB REF 44 Ġrespectively , Ġwhere Ġwe Ġobserve Ġsimilar Ġfindings Ġas Ġbefore . ĠWe Ġuse Ġthe ĠStanford ĠPOS Ġtag ger Ġto Ġparse Ġthe ĠEnglish Ġand ĠFrench Ġinput Ġsentences , Ġand Ġuse Ġthe ĠKy te a Ġto Ġparse Ġthe ĠJapanese Ġinput Ġsentences . </s> An alyses Ġon ĠReverse ĠDirections Ġ: :: ĠSynt actic ĠCategories </s> On ĠEnglish $ \ Right arrow Ġ$ Chinese , Ġcontent Ġwords Ġare Ġmore Ġimportant Ġthan Ġcontent - free Ġwords , Ġwhile Ġthe Ġsituation Ġis Ġreversed Ġon Ġboth ĠFrench $ \ Right arrow Ġ$ English Ġand ĠJapanese $ \ Right arrow Ġ$ English Ġtranslations . ĠSince Ġthere Ġis Ġno Ġclear Ġboundary Ġbetween ĠPre position / D etermin er Ġand Ġother Ġcategories Ġin ĠJapanese , Ġwe Ġset Ġboth Ġcategories Ġto Ġbe Ġnone . ĠSimilarly , ĠPun ct uation Ġis Ġmore Ġimportant Ġon ĠJapanese $ \ Right arrow Ġ$ English , Ġwhich Ġis Ġin Ġline Ġwith Ġthe Ġfinding Ġon ĠEnglish $ \ Right arrow Ġ$ Japanese . ĠOverall Ġspeaking , Ġit Ġmight Ġindicate Ġthat Ġthe ĠSynt actic Ġdistribution Ġwith Ġword Ġimportance Ġis Ġlanguage - pair Ġrelated Ġinstead Ġof Ġthe Ġdirection . </s> An alyses Ġon ĠReverse ĠDirections Ġ: :: ĠWord ĠF ertility </s> The Ġword Ġfertility Ġalso Ġshows Ġsimilar Ġtrend Ġas Ġthe Ġpreviously Ġreported Ġresults , Ġwhere Ġone - to - many Ġfertility Ġis Ġmore Ġimportant Ġand Ġnull - aligned Ġfertility Ġis Ġless Ġimportant . ĠInterestingly , Ġmany - to - one Ġfertility Ġshows Ġan Ġincreasing Ġtrend Ġon ĠJapanese $ \ Right arrow Ġ$ English Ġtranslation , Ġbut Ġthe Ġproportion Ġis Ġrelatively Ġsmall . </s> In Ġsummary , Ġthe Ġfindings Ġon Ġlanguage Ġpairs Ġwith Ġreverse Ġdirections Ġstill Ġagree Ġwith Ġthe Ġfindings Ġin Ġthe Ġpaper , Ġwhich Ġfurther Ġconfirms Ġthe Ġgener ality Ġof Ġour Ġexperimental Ġfindings .
Introduction Neural machine translation (NMT) has achieved the state-of-the-art results on a mass of language pairs with varying structural differences, such as English-French BIBREF0, BIBREF1 and Chinese-English BIBREF2. However, so far not much is known about how and why NMT works, which pose great challenges for debugging NMT models and designing optimal architectures. The understanding of NMT models has been approached primarily from two complementary perspectives. The first thread of work aims to understand the importance of representations by analyzing the linguistic information embedded in representation vectors BIBREF3, BIBREF4 or hidden units BIBREF5, BIBREF6. Another direction focuses on understanding the importance of input words by interpreting the input-output behavior of NMT models. Previous work BIBREF7 treats NMT models as black-boxes and provides explanations that closely resemble the attention scores in NMT models. However, recent studies reveal that attention does not provide meaningful explanations since the relationship between attention scores and model output is unclear BIBREF8. In this paper, we focus on the second thread and try to open the black-box by exploiting the gradients in NMT generation, which aims to estimate the word importance better. Specifically, we employ the integrated gradients method BIBREF9 to attribute the output to the input words with the integration of first-order derivatives. We justify the gradient-based approach via quantitative comparison with black-box methods on a couple of perturbation operations, several language pairs, and two representative model architectures, demonstrating its superiority on estimating word importance. We analyze the linguistic behaviors of words with the importance and show its potential to improve NMT models. First, we leverage the word importance to identify input words that are under-translated by NMT models. Experimental results show that the gradient-based approach outperforms both the best black-box method and other comparative methods. Second, we analyze the linguistic roles of identified important words, and find that words of certain syntactic categories have higher importance while the categories vary across language. For example, nouns are more important for Chinese$\Rightarrow $English translation, while prepositions are more important for English-French and -Japanese translation. This finding can inspire better design principles of NMT architectures for different language pairs. For instance, a better architecture for a given language pair should consider its own language characteristics. Introduction ::: Contributions Our main contributions are: Our study demonstrates the necessity and effectiveness of exploiting the intermediate gradients for estimating word importance. We find that word importance is useful for understanding NMT by identifying under-translated words. We provide empirical support for the design principle of NMT architectures: essential inductive bias (e.g., language characteristics) should be considered for model design. Related Work ::: Interpreting Seq2Seq Models Interpretability of Seq2Seq models has recently been explored mainly from two perspectives: interpreting internal representations and understanding input-output behaviors. Most of the existing work focus on the former thread, which analyzes the linguistic information embeded in the learned representations BIBREF3, BIBREF4, BIBREF10 or the hidden units BIBREF6, BIBREF5. Several researchers turn to expose systematic differences between human and NMT translations BIBREF11, BIBREF12, indicating the linguistic properties worthy of investigating. However, the learned representations may depend on the model implementation, which potentially limit the applicability of these methods to a broader range of model architectures. Accordingly, we focus on understanding the input-output behaviors, and validate on different architectures to demonstrate the universality of our findings. Concerning interpreting the input-output behavior, previous work generally treats Seq2Seq models as black-boxes BIBREF13, BIBREF7. For example, alvarez2017causal measure the relevance between two input-output tokens by perturbing the input sequence. However, they do not exploit any intermediate information such as gradients, and the relevance score only resembles attention scores. Recently, Jain2019AttentionIN show that attention scores are in weak correlation with the feature importance. Starting from this observation, we exploit the intermediate gradients to better estimate word importance, which consistently outperforms its attention counterpart across model architectures and language pairs. Related Work ::: Exploiting Gradients for Model Interpretation The intermediate gradients have proven to be useful in interpreting deep learning models, such as NLP models BIBREF14, BIBREF15 and computer vision models BIBREF16, BIBREF9. Among all gradient-based approaches, the integrated gradients BIBREF9 is appealing since it does not need any instrumentation of the architecture and can be computed easily by calling gradient operations. In this work, we employ the IG method to interpret NMT models and reveal several interesting findings, which can potentially help debug NMT models and design better architectures for specific language pairs. Approach ::: Neural Machine Translation In machine translation task, a NMT model $F$: $\textbf {x} \rightarrow \textbf {y}$ maximizes the probability of a target sequence $\textbf {y} = \lbrace y_1,...,y_N\rbrace $ given a source sentence $\textbf {x} = \lbrace x_1,...,x_M\rbrace $: where $\mathbf {\theta }$ is the model parameter and $\textbf {y}_{<n}$ is a partial translation. At each time step n, the model generates an output word of the highest probability based on the source sentence $\textbf {x}$ and the partial translation $\textbf {y}_{<n}$. The training objective is to minimize the negative log-likelihood loss on the training corpus. During the inference, beam search is employed to decode a more optimal translation. In this study, we investigate the contribution of each input word $x_m$ to the translated sentence ${\bf y}$. Approach ::: Word Importance In this work, the notion of “word importance” is employed to quantify the contribution that a word in the input sentence makes to the NMT generations. We categorize the methods of word importance estimation into two types: black-box methods without the knowledge of the model and white-box methods that have access to the model internal information (e.g., parameters and gradients). Previous studies mostly fall into the former type, and in this study, we investigate several representative black-box methods: Content Words: In linguistics, all words can be categorized as either content or content-free words. Content words consist mostly of nouns, verbs, and adjectives, which carry descriptive meanings of the sentence and thereby are often considered as important. Frequent Words: We rank the relative importance of input words according to their frequency in the training corpus. We do not consider the top 50 most frequent words since they are mostly punctuation and stop words. Causal Model BIBREF7: Since the causal model is complicated to implement and its scores closely resemble attention scores in NMT models. In this study, we use Attention scores to simulate the causal model. Our approach belongs to the white-box category by exploiting the intermediate gradients, which will be described in the next section. Approach ::: Integrated Gradients In this work, we resort to a gradient-based method, integrated gradients BIBREF9 (IG), which was originally proposed to attribute the model predictions to input features. It exploits the handy model gradient information by integrating first-order derivatives. IG is implementation invariant and does not require neural models to be differentiable or smooth, thereby is suitable for complex neural networks like Transformer. In this work, we use IG to estimate the word importance in an input sentence precisely. Formally, let $\textbf {x} = (x_1, ..., x_M)$ be the input sentence and $\textbf {x}^{\prime }$ be a baseline input. $F$ is a well-trained NMT model, and $F(\textbf {x})_n$ is the model output (i.e., $P(y_n|\textbf {y}_{<n},\textbf {x})$) at time step $n$. Integrated gradients is then defined as the integral of gradients along the straightline path from the baseline $\textbf {x}^{\prime }$ to the input $\textbf {x}$. In detail, the contribution of the $m^{th}$ word in $\textbf {x}$ to the prediction of $F(\textbf {x})_n$ is defined as follows. where $\frac{\partial {F(\textbf {x})_n}}{\partial {\textbf {x}_m}}$ is the gradient of $F(\textbf {x})_n$ w.r.t. the embedding of the $m^{th}$ word. In this paper, as suggested, the baseline input $\textbf {x}^{\prime }$ is set as a sequence of zero embeddings that has the same sequence length $M$. In this way, we can compute the contribution of a specific input word to a designated output word. Since the above formula is intractable for deep neural models, we approximate it by summing the gradients along a multi-step path from baseline $\textbf {x}^{\prime }$ to the input x. where $S$ denotes the number of steps that are uniformly distributed along the path. The IG will be more accurate if a larger S is used. In our preliminary experiments, we varied the steps and found 300 steps yielding fairly good performance. Following the formula, we can calculate the contribution of every input word makes to every output word, forming a contribution matrix of size $M \times N$, where $N$ is the output sentence length. Given the contribution matrix, we can obtain the word importance of each input word to the entire output sentence. To this end, for each input word, we first aggregate its contribution values to all output words by the sum operation, and then normalize all sums through the Softmax function. Figure FIGREF13 illustrates an example of the calculated word importance and the contribution matrix, where an English sentence is translated into a French sentence using the Transformer model. A negative contribution value indicates that the input word has negative effects on the output word. Experiment ::: Data To make the conclusion convincing, we first choose two large-scale datasets that are publicly available, i.e., Chinese-English and English-French. Since English, French, and Chinese all belong to the subject-verb-object (SVO) family, we choose another very different subject-object-verb (SOV) language, Japanese, which might bring some interesting linguistic behaviors in English-Japanese translation. For Chinese-English task, we use WMT17 Chinese-English dataset that consists of $20.6$M sentence pairs. For English-French task, we use WMT14 English-French dataset that comprises $35.5$M sentence pairs. For English-Japanese task, we follow BIBREF17 to use the first two sections of WAT17 English-Japanese dataset that consists of $1.9$M sentence pairs. Following the standard NMT procedure, we adopt the standard byte pair encoding (BPE) BIBREF18 with 32K merge operations for all language pairs. We believe that these datasets are large enough to confirm the rationality and validity of our experimental analyses. Experiment ::: Implementation We choose the state-of-the-art Transformer BIBREF1 model and the conventional RNN-Search model BIBREF0 as our test bed. We implement the Attribution method based on the Fairseq-py BIBREF19 framework for the above models. All models are trained on the training corpus for 100k steps under the standard settings, which achieve comparable translation results. All the following experiments are conducted on the test dataset, and we estimate the input word importance using the model generated hypotheses. In the following experiments, we compare IG (Attribution) with several black-box methods (i.e., Content, Frequency, Attention) as introduced in Section SECREF8. In Section SECREF21, to ensure that the translation performance decrease attributes to the selected words instead of the perturbation operations, we randomly select the same number of words to perturb (Random), which serves as a baseline. Since there is no ranking for content words, we randomly select a set of content words as important words. To avoid the potential bias introduced by randomness (i.e., Random and Content), we repeat the experiments for 10 times and report the averaged results. We calculate the Attention importance in a similar manner as the Attribution, except that the attention scores use a max operation due to the better performance. Experiment ::: Evaluation We evaluate the effectiveness of estimating word importance by the translation performance decrease. More specifically, unlike the usual way, we measure the decrease of translation performance when perturbing a set of important words that are of top-most word importance in a sentence. The more translation performance degrades, the more important the word is. We use the standard BLEU score as the evaluation metric for translation performance. To make the conclusion more convincing, we conduct experiments on different types of synthetic perturbations (Section SECREF21), as well as different NMT architectures and language pairs (Section SECREF27). In addition, we compare with a supervised erasure method, which requires ground-truth translations for scoring word importance (Section SECREF30). Experiment ::: Results on Different Perturbations In this experiment, we investigate the effectiveness of word importance estimation methods under different synthetic perturbations. Since the perturbation on text is notoriously hard BIBREF20 due to the semantic shifting problem, in this experiment, we investigate three types of perturbations to avoid the potential bias : Deletion perturbation removes the selected words from the input sentence, and it can be regarded as a specific instantiation of sentence compression BIBREF21. Mask perturbation replaces embedding vectors of the selected words with all-zero vectors BIBREF22, which is similar to Deletion perturbation except that it retains the placeholder. Grammatical Replacement perturbation replaces a word by another word of the same linguistic role (i.e., POS tags), yielding a sentence that is grammatically correct but semantically nonsensical BIBREF23, BIBREF24, such as “colorless green ideas sleep furiously”. Figure FIGREF19 illustrates the experimental results on Chinese$\Rightarrow $English translation with Transformer. It shows that Attribution method consistently outperforms other methods against different perturbations on a various number of operations. Here the operation number denotes the number of perturbed words in a sentence. Specifically, we can make the following observations. Experiment ::: Results on Different Perturbations ::: Important words are more influential on translation performance than the others. Under three different perturbations, perturbing words of top-most importance leads to lower BLEU scores than Random selected words. It confirms the existence of important words, which have greater impacts on translation performance. Furthermore, perturbing important words identified by Attribution outperforms the Random method by a large margin (more than 4.0 BLEU under 5 operations). Experiment ::: Results on Different Perturbations ::: The gradient-based method is superior to comparative methods (e.g., Attention) in estimating word importance. Figure FIGREF19 shows that two black-box methods (i.e., Content, Frequency) perform only slightly better than the Random method. Specifically, the Frequency method demonstrates even worse performances under the Mask perturbation. Therefore, linguistic properties (such as POS tags) and the word frequency can only partially help identify the important words, but it is not as accurate as we thought. In the meanwhile, it is intriguing to explore what exact linguistic characteristics these important words reveal, which will be introduced in Section SECREF5. We also evaluate the Attention method, which bases on the encoder-decoder attention scores at the last layer of Transformer. Note that the Attention method is also used to simulate the best black-box method SOCRAT, and the results show that it is more effective than black-box methods and the Random baseline. Given the powerful Attention method, Attribution method still achieves best performances under all three perturbations. Furthermore, we find that the gap between Attribution and Attention is notably large (around $1.0+$ BLEU difference). Attention method does not provide as accurate word importance as the Attribution, which exhibits the superiority of gradient-based methods and consists with the conclusion reported in the previous study BIBREF8. In addition, as shown in Figure FIGREF19, the perturbation effectiveness of Deletion, Mask, and Grammatical Replacement varies from strong to weak. In the following experiments, we choose Mask as the representative perturbation operation for its moderate perturbation performance, based on which we compare two most effective methods Attribution and Attention. Experiment ::: Results on Different NMT Architecture and Language Pairs ::: Different NMT Architecture We validate the effectiveness of the proposed approach using a different NMT architecture RNN-Search on the Chinese$\Rightarrow $English translation task. The results are shown in Figure FIGREF20(a). We observe that the Attribution method still outperforms both Attention method and Random method by a decent margin. By comparing to Transformer, the results also reveal that the RNN-Search model is less robust to these perturbations. To be specific, under the setting of five operations and Attribution method, Transformer shows a relative decrease of $55\%$ on BLEU scores while the decline of RNN-Search model is $64\%$. Experiment ::: Results on Different NMT Architecture and Language Pairs ::: Different Language Pairs and Directions We further conduct experiments on another two language pairs (i.e., English$\Rightarrow $French, English$\Rightarrow $Japanese in Figures FIGREF20(b, c)) as well as the reverse directions (Figures FIGREF20(d, e, f)) using Transformer under the Mask perturbation. In all the cases, Attribution shows the best performance while Random achieves the worst result. More specifically, Attribution method shows similar translation quality degradation on all three language-pairs, which declines to around the half of the original BLEU score with five operations. Experiment ::: Comparison with Supervised Erasure There exists another straightforward method, Erasure BIBREF7, BIBREF22, BIBREF25, which directly evaluates the word importance by measuring the translation performance degradation of each word. Specifically, it erases (i.e., Mask) one word from the input sentence each time and uses the BLEU score changes to denote the word importance (after normalization). In Figure FIGREF31, we compare Erasure method with Attribution method under the Mask perturbation. The results show that Attribution method is less effective than Erasure method when only one word is perturbed. But it outperforms the Erasure method when perturbing 2 or more words. The results reveal that the importance calculated by erasing only one word cannot be generalized to multiple-words scenarios very well. Besides, the Erasure method is a supervised method which requires ground-truth references, and finding a better words combination is computation infeasible when erasing multiple words. We close this section by pointing out that our gradient-based method consistently outperforms its black-box counterparts in various settings, demonstrating the effectiveness and universality of exploiting gradients for estimating word importance. In addition, our approach is on par with or even outperforms the supervised erasure method (on multiple-word perturbations). This is encouraging since our approach does not require any external resource and is fully unsupervised. Analysis In this section, we conduct analyses on two potential usages of word importance, which can help debug NMT models (Section SECREF33) and design better architectures for specific languages (Section SECREF37). Due to the space limitation, we only analyze the results of Chinese$\Rightarrow $English, English$\Rightarrow $French, and English$\Rightarrow $Japanese. We list the results on the reverse directions in Appendix, in which the general conclusions also hold. Analysis ::: Effect on Detecting Translation Errors In this experiment, we propose to use the estimated word importance to detect the under-translated words by NMT models. Intuitively, under-translated input words should contribute little to the NMT outputs, yielding much smaller word importance. Given 500 Chinese$\Rightarrow $English sentence pairs translated by the Transformer model (BLEU 23.57), we ask ten human annotators to manually label the under-translated input words, and at least two annotators label each input-hypothesis pair. These annotators have at least six years of English study experience, whose native language is Chinese. Among these sentences, 178 sentences have under-translation errors with 553 under-translated words in total. Table TABREF32 lists the accuracy of detecting under-translation errors by comparing words of least importance and human-annotated under-translated words. As seen, our Attribution method consistently and significantly outperforms both Erasure and Attention approaches. By exploiting the word importance calculated by Attribution method, we can identify the under-translation errors automatically without the involvement of human interpreters. Although the accuracy is not high, it is worth noting that our under-translation method is very simple and straightforward. This is potentially useful for debugging NMT models, e.g., automatic post-editing with constraint decoding BIBREF26, BIBREF27. Analysis ::: Analysis on Linguistic Properties In this section, we analyze the linguistic characteristics of important words identified by the attribution-based approach. Specifically, we investigate several representative sets of linguistic properties, including POS tags, and fertility, and depth in a syntactic parse tree. In these analyses, we multiply the word importance with the corresponding sentence length for fair comparison. We use a decision tree based regression model to calculate the correlation between the importance and linguistic properties. Table TABREF34 lists the correlations, where a higher value indicates a stronger correlation. We find that the syntactic information is almost independent of the word importance value. Instead, the word importance strongly correlates with the POS tags and fertility features, and these features in total contribute over 95%. Therefore, in the following analyses, we mainly focus on the POS tags (Table TABREF35) and fertility properties (Table TABREF36). For better illustration, we calculate the distribution over the linguistic property based on both the Attribution importance (“Attr.”) and the word frequency (“Count”) inside a sentence. The larger the relative increase between these two values, the more important the linguistic property is. Analysis ::: Analysis on Linguistic Properties ::: Certain syntactic categories have higher importance while the categories vary across language pairs. As shown in Table TABREF35, content words are more important on Chinese$\Rightarrow $English but content-free words are more important on English$\Rightarrow $Japanese. On English$\Rightarrow $French, there is no notable increase or decrease of the distribution since English and French are in essence very similar. We also obtain some specific findings of great interest. For example, we find that noun is more important on Chinese$\Rightarrow $English translation, while preposition is more important on English$\Rightarrow $French translation. More interestingly, English$\Rightarrow $Japanese translation shows a substantial discrepancy in contrast to the other two language pairs. The results reveal that preposition and punctuation are very important in English$\Rightarrow $Japanese translation, which is counter-intuitive. Punctuation in NMT is understudied since it carries little information and often does not affect the understanding of a sentence. However, we find that punctuation is important on English$\Rightarrow $Japanese translation, whose proportion increases dramatically. We conjecture that it is because the punctuation could affect the sense groups in a sentence, which further benefits the syntactic reordering in Japanese. Analysis ::: Analysis on Linguistic Properties ::: Words of high fertility are always important. We further compare the fertility distribution based on word importance and the word frequency on three language pairs. We hypothesize that a source word that corresponds to multiple target words should be more important since it contributes more to both sentence length and BLEU score. Table TABREF36 lists the results. Overall speaking, one-to-many fertility is consistently more important on all three language pairs, which confirms our hypothesis. On the contrary, null-aligned words receive much less attention, which shows a persistently decrease on three language pairs. It is also reasonable since null-aligned input words contribute almost nothing to the translation outputs. Discussion and Conclusion We approach understanding NMT by investigating the word importance via a gradient-based method, which bridges the gap between word importance and translation performance. Empirical results show that the gradient-based method is superior to several black-box methods in estimating the word importance. Further analyses show that important words are of distinct syntactic categories on different language pairs, which might support the viewpoint that essential inductive bias should be introduced into the model design BIBREF28. Our study also suggests the possibility of detecting the notorious under-translation problem via the gradient-based method. This paper is an initiating step towards the general understanding of NMT models, which may bring some potential improvements, such as Interactive MT and Constraint Decoding BIBREF29, BIBREF26: The model pays more attention to the detected unimportant words, which are possibly under-translated; Adaptive Input Embedding BIBREF30: We can extend the adaptive softmax BIBREF31 to the input embedding of variable capacity – more important words are assigned with more capacity; NMT Architecture Design: The language-specific inductive bias (e.g., different behaviors on POS) should be incorporated into the model design. We can also explore other applications of word importance to improve NMT models, such as more tailored training methods. In general, model interpretability can build trust in model predictions, help error diagnosis and facilitate model refinement. We expect our work could shed light on the NMT model understanding and benefit the model improvement. There are many possible ways to implement the general idea of exploiting gradients for model interpretation. The aim of this paper is not to explore this whole space but simply to show that some fairly straightforward implementations work well. Our approach can benefit from advanced exploitation of the gradients or other useful intermediate information, which we leave to the future work. Acknowledgement Shilin He and Michael R. Lyu were supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (No. CUHK 14210717 of the General Research Fund), and Microsoft Research Asia (2018 Microsoft Research Asia Collaborative Research Award). We thank the anonymous reviewers for their insightful comments and suggestions. Analyses on Reverse Directions 2 We analyze the distribution of syntactic categories and word fertility on the same language pairs with reverse directions, i.e., English$\Rightarrow $Chinese, French$\Rightarrow $English, and Japanese$\Rightarrow $English. The results are shown in Table TABREF43 and Table TABREF44 respectively, where we observe similar findings as before. We use the Stanford POS tagger to parse the English and French input sentences, and use the Kytea to parse the Japanese input sentences. Analyses on Reverse Directions ::: Syntactic Categories On English$\Rightarrow $Chinese, content words are more important than content-free words, while the situation is reversed on both French$\Rightarrow $English and Japanese$\Rightarrow $English translations. Since there is no clear boundary between Preposition/Determiner and other categories in Japanese, we set both categories to be none. Similarly, Punctuation is more important on Japanese$\Rightarrow $English, which is in line with the finding on English$\Rightarrow $Japanese. Overall speaking, it might indicate that the Syntactic distribution with word importance is language-pair related instead of the direction. Analyses on Reverse Directions ::: Word Fertility The word fertility also shows similar trend as the previously reported results, where one-to-many fertility is more important and null-aligned fertility is less important. Interestingly, many-to-one fertility shows an increasing trend on Japanese$\Rightarrow $English translation, but the proportion is relatively small. In summary, the findings on language pairs with reverse directions still agree with the findings in the paper, which further confirms the generality of our experimental findings.
Does their model suffer exhibit performance drops when incorporating word importance?
384bf1f55c34b36cb03f916f50bbefade6c86a75
Does Ġtheir Ġmodel Ġsuffer Ġexhibit Ġperformance Ġdrops Ġwhen Ġincorporating Ġword Ġimportance ?
No
<s> Does Ġtheir Ġmodel Ġsuffer Ġexhibit Ġperformance Ġdrops Ġwhen Ġincorporating Ġword Ġimportance ? </s> Introduction </s> Ne ural Ġmachine Ġtranslation Ġ( N MT ) Ġhas Ġachieved Ġthe Ġstate - of - the - art Ġresults Ġon Ġa Ġmass Ġof Ġlanguage Ġpairs Ġwith Ġvarying Ġstructural Ġdifferences , Ġsuch Ġas ĠEnglish - French ĠB IB REF 0 , ĠB IB REF 1 Ġand ĠChinese - English ĠB IB REF 2 . ĠHowever , Ġso Ġfar Ġnot Ġmuch Ġis Ġknown Ġabout Ġhow Ġand Ġwhy ĠN MT Ġworks , Ġwhich Ġpose Ġgreat Ġchallenges Ġfor Ġdebugging ĠN MT Ġmodels Ġand Ġdesigning Ġoptimal Ġarchitectures . </s> The Ġunderstanding Ġof ĠN MT Ġmodels Ġhas Ġbeen Ġapproached Ġprimarily Ġfrom Ġtwo Ġcomplementary Ġperspectives . ĠThe Ġfirst Ġthread Ġof Ġwork Ġaims Ġto Ġunderstand Ġthe Ġimportance Ġof Ġrepresentations Ġby Ġanalyzing Ġthe Ġlinguistic Ġinformation Ġembedded Ġin Ġrepresentation Ġvectors ĠB IB REF 3 , ĠB IB REF 4 Ġor Ġhidden Ġunits ĠB IB REF 5 , ĠB IB REF 6 . ĠAnother Ġdirection Ġfocuses Ġon Ġunderstanding Ġthe Ġimportance Ġof Ġinput Ġwords Ġby Ġinterpreting Ġthe Ġinput - output Ġbehavior Ġof ĠN MT Ġmodels . ĠPrevious Ġwork ĠB IB REF 7 Ġtreats ĠN MT Ġmodels Ġas Ġblack - boxes Ġand Ġprovides Ġexplanations Ġthat Ġclosely Ġresemble Ġthe Ġattention Ġscores Ġin ĠN MT Ġmodels . ĠHowever , Ġrecent Ġstudies Ġreveal Ġthat Ġattention Ġdoes Ġnot Ġprovide Ġmeaningful Ġexplanations Ġsince Ġthe Ġrelationship Ġbetween Ġattention Ġscores Ġand Ġmodel Ġoutput Ġis Ġunclear ĠB IB REF 8 . </s> In Ġthis Ġpaper , Ġwe Ġfocus Ġon Ġthe Ġsecond Ġthread Ġand Ġtry Ġto Ġopen Ġthe Ġblack - box Ġby Ġexploiting Ġthe Ġgrad ients Ġin ĠN MT Ġgeneration , Ġwhich Ġaims Ġto Ġestimate Ġthe Ġword Ġimportance Ġbetter . ĠSpecifically , Ġwe Ġemploy Ġthe Ġintegrated Ġgrad ients Ġmethod ĠB IB REF 9 Ġto Ġattribute Ġthe Ġoutput Ġto Ġthe Ġinput Ġwords Ġwith Ġthe Ġintegration Ġof Ġfirst - order Ġderivatives . ĠWe Ġjustify Ġthe Ġgradient - based Ġapproach Ġvia Ġquantitative Ġcomparison Ġwith Ġblack - box Ġmethods Ġon Ġa Ġcouple Ġof Ġpert urb ation Ġoperations , Ġseveral Ġlanguage Ġpairs , Ġand Ġtwo Ġrepresentative Ġmodel Ġarchitectures , Ġdemonstrating Ġits Ġsuperiority Ġon Ġestimating Ġword Ġimportance . </s> We Ġanalyze Ġthe Ġlinguistic Ġbehaviors Ġof Ġwords Ġwith Ġthe Ġimportance Ġand Ġshow Ġits Ġpotential Ġto Ġimprove ĠN MT Ġmodels . ĠFirst , Ġwe Ġleverage Ġthe Ġword Ġimportance Ġto Ġidentify Ġinput Ġwords Ġthat Ġare Ġunder - trans lated Ġby ĠN MT Ġmodels . ĠExperimental Ġresults Ġshow Ġthat Ġthe Ġgradient - based Ġapproach Ġoutper forms Ġboth Ġthe Ġbest Ġblack - box Ġmethod Ġand Ġother Ġcomparative Ġmethods . ĠSecond , Ġwe Ġanalyze Ġthe Ġlinguistic Ġroles Ġof Ġidentified Ġimportant Ġwords , Ġand Ġfind Ġthat Ġwords Ġof Ġcertain Ġsynt actic Ġcategories Ġhave Ġhigher Ġimportance Ġwhile Ġthe Ġcategories Ġvary Ġacross Ġlanguage . ĠFor Ġexample , Ġnoun s Ġare Ġmore Ġimportant Ġfor ĠChinese $ \ Right arrow Ġ$ English Ġtranslation , Ġwhile Ġpre pos itions Ġare Ġmore Ġimportant Ġfor ĠEnglish - French Ġand Ġ- Japanese Ġtranslation . ĠThis Ġfinding Ġcan Ġinspire Ġbetter Ġdesign Ġprinciples Ġof ĠN MT Ġarchitectures Ġfor Ġdifferent Ġlanguage Ġpairs . ĠFor Ġinstance , Ġa Ġbetter Ġarchitecture Ġfor Ġa Ġgiven Ġlanguage Ġpair Ġshould Ġconsider Ġits Ġown Ġlanguage Ġcharacteristics . </s> Introduction Ġ: :: ĠContributions </s> Our Ġmain Ġcontributions Ġare : </s> Our Ġstudy Ġdemonstrates Ġthe Ġnecessity Ġand Ġeffectiveness Ġof Ġexploiting Ġthe Ġintermediate Ġgrad ients Ġfor Ġestimating Ġword Ġimportance . </s> We Ġfind Ġthat Ġword Ġimportance Ġis Ġuseful Ġfor Ġunderstanding ĠN MT Ġby Ġidentifying Ġunder - trans lated Ġwords . </s> We Ġprovide Ġempirical Ġsupport Ġfor Ġthe Ġdesign Ġprinciple Ġof ĠN MT Ġarchitectures : Ġessential Ġinduct ive Ġbias Ġ( e . g ., Ġlanguage Ġcharacteristics ) Ġshould Ġbe Ġconsidered Ġfor Ġmodel Ġdesign . </s> Related ĠWork Ġ: :: ĠInter pre ting ĠSe q 2 Se q ĠModels </s> Inter pret ability Ġof ĠSe q 2 Se q Ġmodels Ġhas Ġrecently Ġbeen Ġexplored Ġmainly Ġfrom Ġtwo Ġperspectives : Ġinterpreting Ġinternal Ġrepresentations Ġand Ġunderstanding Ġinput - output Ġbehaviors . ĠMost Ġof Ġthe Ġexisting Ġwork Ġfocus Ġon Ġthe Ġformer Ġthread , Ġwhich Ġanaly zes Ġthe Ġlinguistic Ġinformation Ġembed ed Ġin Ġthe Ġlearned Ġrepresentations ĠB IB REF 3 , ĠB IB REF 4 , ĠB IB REF 10 Ġor Ġthe Ġhidden Ġunits ĠB IB REF 6 , ĠB IB REF 5 . ĠSeveral Ġresearchers Ġturn Ġto Ġexpose Ġsystematic Ġdifferences Ġbetween Ġhuman Ġand ĠN MT Ġtranslations ĠB IB REF 11 , ĠB IB REF 12 , Ġindicating Ġthe Ġlinguistic Ġproperties Ġworthy Ġof Ġinvestigating . ĠHowever , Ġthe Ġlearned Ġrepresentations Ġmay Ġdepend Ġon Ġthe Ġmodel Ġimplementation , Ġwhich Ġpotentially Ġlimit Ġthe Ġapplic ability Ġof Ġthese Ġmethods Ġto Ġa Ġbroader Ġrange Ġof Ġmodel Ġarchitectures . ĠAccordingly , Ġwe Ġfocus Ġon Ġunderstanding Ġthe Ġinput - output Ġbehaviors , Ġand Ġvalidate Ġon Ġdifferent Ġarchitectures Ġto Ġdemonstrate Ġthe Ġunivers ality Ġof Ġour Ġfindings . </s> Con cerning Ġinterpreting Ġthe Ġinput - output Ġbehavior , Ġprevious Ġwork Ġgenerally Ġtreats ĠSe q 2 Se q Ġmodels Ġas Ġblack - boxes ĠB IB REF 13 , ĠB IB REF 7 . ĠFor Ġexample , Ġal v arez 2017 ca usal Ġmeasure Ġthe Ġrelevance Ġbetween Ġtwo Ġinput - output Ġtokens Ġby Ġpert ur bing Ġthe Ġinput Ġsequence . ĠHowever , Ġthey Ġdo Ġnot Ġexploit Ġany Ġintermediate Ġinformation Ġsuch Ġas Ġgrad ients , Ġand Ġthe Ġrelevance Ġscore Ġonly Ġresembles Ġattention Ġscores . ĠRecently , ĠJ ain 2019 Att ention IN Ġshow Ġthat Ġattention Ġscores Ġare Ġin Ġweak Ġcorrelation Ġwith Ġthe Ġfeature Ġimportance . ĠStarting Ġfrom Ġthis Ġobservation , Ġwe Ġexploit Ġthe Ġintermediate Ġgrad ients Ġto Ġbetter Ġestimate Ġword Ġimportance , Ġwhich Ġconsistently Ġoutper forms Ġits Ġattention Ġcounterpart Ġacross Ġmodel Ġarchitectures Ġand Ġlanguage Ġpairs . </s> Related ĠWork Ġ: :: ĠExpl o iting ĠGrad ients Ġfor ĠModel ĠInterpret ation </s> The Ġintermediate Ġgrad ients Ġhave Ġproven Ġto Ġbe Ġuseful Ġin Ġinterpreting Ġdeep Ġlearning Ġmodels , Ġsuch Ġas ĠN LP Ġmodels ĠB IB REF 14 , ĠB IB REF 15 Ġand Ġcomputer Ġvision Ġmodels ĠB IB REF 16 , ĠB IB REF 9 . ĠAmong Ġall Ġgradient - based Ġapproaches , Ġthe Ġintegrated Ġgrad ients ĠB IB REF 9 Ġis Ġappealing Ġsince Ġit Ġdoes Ġnot Ġneed Ġany Ġinstrument ation Ġof Ġthe Ġarchitecture Ġand Ġcan Ġbe Ġcomputed Ġeasily Ġby Ġcalling Ġgradient Ġoperations . ĠIn Ġthis Ġwork , Ġwe Ġemploy Ġthe ĠIG Ġmethod Ġto Ġinterpret ĠN MT Ġmodels Ġand Ġreveal Ġseveral Ġinteresting Ġfindings , Ġwhich Ġcan Ġpotentially Ġhelp Ġdebug ĠN MT Ġmodels Ġand Ġdesign Ġbetter Ġarchitectures Ġfor Ġspecific Ġlanguage Ġpairs . </s> App roach Ġ: :: ĠNeural ĠMachine ĠTranslation </s> In Ġmachine Ġtranslation Ġtask , Ġa ĠN MT Ġmodel Ġ$ F $ : Ġ$\ text bf Ġ{ x } Ġ\ right arrow Ġ\ text bf Ġ{ y } $ Ġmaxim izes Ġthe Ġprobability Ġof Ġa Ġtarget Ġsequence Ġ$\ text bf Ġ{ y } Ġ= Ġ\ l brace Ġy _ 1 ,... , y _ N \ r brace Ġ$ Ġgiven Ġa Ġsource Ġsentence Ġ$\ text bf Ġ{ x } Ġ= Ġ\ l brace Ġx _ 1 ,... , x _ M \ r brace Ġ$ : </s> where Ġ$\ math bf Ġ{\ the ta Ġ} $ Ġis Ġthe Ġmodel Ġparameter Ġand Ġ$\ text bf Ġ{ y } _{ < n } $ Ġis Ġa Ġpartial Ġtranslation . ĠAt Ġeach Ġtime Ġstep Ġn , Ġthe Ġmodel Ġgenerates Ġan Ġoutput Ġword Ġof Ġthe Ġhighest Ġprobability Ġbased Ġon Ġthe Ġsource Ġsentence Ġ$\ text bf Ġ{ x } $ Ġand Ġthe Ġpartial Ġtranslation Ġ$\ text bf Ġ{ y } _{ < n } $. ĠThe Ġtraining Ġobjective Ġis Ġto Ġminimize Ġthe Ġnegative Ġlog - like lihood Ġloss Ġon Ġthe Ġtraining Ġcorpus . ĠDuring Ġthe Ġinference , Ġbeam Ġsearch Ġis Ġemployed Ġto Ġdecode Ġa Ġmore Ġoptimal Ġtranslation . ĠIn Ġthis Ġstudy , Ġwe Ġinvestigate Ġthe Ġcontribution Ġof Ġeach Ġinput Ġword Ġ$ x _ m $ Ġto Ġthe Ġtranslated Ġsentence Ġ${ \ bf Ġy } $. </s> App roach Ġ: :: ĠWord ĠImport ance </s> In Ġthis Ġwork , Ġthe Ġnotion Ġof ĠâĢ ľ word Ġimportance âĢ Ŀ Ġis Ġemployed Ġto Ġquantify Ġthe Ġcontribution Ġthat Ġa Ġword Ġin Ġthe Ġinput Ġsentence Ġmakes Ġto Ġthe ĠN MT Ġgenerations . ĠWe Ġcategor ize Ġthe Ġmethods Ġof Ġword Ġimportance Ġestimation Ġinto Ġtwo Ġtypes : Ġblack - box Ġmethods Ġwithout Ġthe Ġknowledge Ġof Ġthe Ġmodel Ġand Ġwhite - box Ġmethods Ġthat Ġhave Ġaccess Ġto Ġthe Ġmodel Ġinternal Ġinformation Ġ( e . g ., Ġparameters Ġand Ġgrad ients ). ĠPrevious Ġstudies Ġmostly Ġfall Ġinto Ġthe Ġformer Ġtype , Ġand Ġin Ġthis Ġstudy , Ġwe Ġinvestigate Ġseveral Ġrepresentative Ġblack - box Ġmethods : </s> Content ĠWords : ĠIn Ġlingu istics , Ġall Ġwords Ġcan Ġbe Ġcategorized Ġas Ġeither Ġcontent Ġor Ġcontent - free Ġwords . ĠContent Ġwords Ġconsist Ġmostly Ġof Ġnoun s , Ġverbs , Ġand Ġadject ives , Ġwhich Ġcarry Ġdescriptive Ġmeanings Ġof Ġthe Ġsentence Ġand Ġthereby Ġare Ġoften Ġconsidered Ġas Ġimportant . </s> F requent ĠWords : ĠWe Ġrank Ġthe Ġrelative Ġimportance Ġof Ġinput Ġwords Ġaccording Ġto Ġtheir Ġfrequency Ġin Ġthe Ġtraining Ġcorpus . ĠWe Ġdo Ġnot Ġconsider Ġthe Ġtop Ġ50 Ġmost Ġfrequent Ġwords Ġsince Ġthey Ġare Ġmostly Ġpunct uation Ġand Ġstop Ġwords . </s> Ca usal ĠModel ĠB IB REF 7 : ĠSince Ġthe Ġcausal Ġmodel Ġis Ġcomplicated Ġto Ġimplement Ġand Ġits Ġscores Ġclosely Ġresemble Ġattention Ġscores Ġin ĠN MT Ġmodels . ĠIn Ġthis Ġstudy , Ġwe Ġuse ĠAttention Ġscores Ġto Ġsimulate Ġthe Ġcausal Ġmodel . </s> Our Ġapproach Ġbelongs Ġto Ġthe Ġwhite - box Ġcategory Ġby Ġexploiting Ġthe Ġintermediate Ġgrad ients , Ġwhich Ġwill Ġbe Ġdescribed Ġin Ġthe Ġnext Ġsection . </s> App roach Ġ: :: ĠIntegrated ĠGrad ients </s> In Ġthis Ġwork , Ġwe Ġresort Ġto Ġa Ġgradient - based Ġmethod , Ġintegrated Ġgrad ients ĠB IB REF 9 Ġ( IG ), Ġwhich Ġwas Ġoriginally Ġproposed Ġto Ġattribute Ġthe Ġmodel Ġpredictions Ġto Ġinput Ġfeatures . ĠIt Ġexploits Ġthe Ġhandy Ġmodel Ġgradient Ġinformation Ġby Ġintegrating Ġfirst - order Ġderivatives . ĠIG Ġis Ġimplementation Ġinvari ant Ġand Ġdoes Ġnot Ġrequire Ġneural Ġmodels Ġto Ġbe Ġdifferent iable Ġor Ġsmooth , Ġthereby Ġis Ġsuitable Ġfor Ġcomplex Ġneural Ġnetworks Ġlike ĠTrans former . ĠIn Ġthis Ġwork , Ġwe Ġuse ĠIG Ġto Ġestimate Ġthe Ġword Ġimportance Ġin Ġan Ġinput Ġsentence Ġprecisely . </s> Form ally , Ġlet Ġ$\ text bf Ġ{ x } Ġ= Ġ( x _ 1 , Ġ... , Ġx _ M ) $ Ġbe Ġthe Ġinput Ġsentence Ġand Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġbe Ġa Ġbaseline Ġinput . Ġ$ F $ Ġis Ġa Ġwell - trained ĠN MT Ġmodel , Ġand Ġ$ F (\ text bf Ġ{ x }) _ n $ Ġis Ġthe Ġmodel Ġoutput Ġ( i . e ., Ġ$ P ( y _ n | \ text bf Ġ{ y } _{ < n }, \ text bf Ġ{ x }) $ ) Ġat Ġtime Ġstep Ġ$ n $. ĠIntegrated Ġgrad ients Ġis Ġthen Ġdefined Ġas Ġthe Ġintegral Ġof Ġgrad ients Ġalong Ġthe Ġstraight line Ġpath Ġfrom Ġthe Ġbaseline Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġto Ġthe Ġinput Ġ$\ text bf Ġ{ x } $. ĠIn Ġdetail , Ġthe Ġcontribution Ġof Ġthe Ġ$ m ^{ th } $ Ġword Ġin Ġ$\ text bf Ġ{ x } $ Ġto Ġthe Ġprediction Ġof Ġ$ F (\ text bf Ġ{ x }) _ n $ Ġis Ġdefined Ġas Ġfollows . </s> where Ġ$\ frac {\ partial Ġ{ F (\ text bf Ġ{ x }) _ n }} {\ partial Ġ{\ text bf Ġ{ x } _ m }} $ Ġis Ġthe Ġgradient Ġof Ġ$ F (\ text bf Ġ{ x }) _ n $ Ġw . r . t . Ġthe Ġembed ding Ġof Ġthe Ġ$ m ^{ th } $ Ġword . ĠIn Ġthis Ġpaper , Ġas Ġsuggested , Ġthe Ġbaseline Ġinput Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġis Ġset Ġas Ġa Ġsequence Ġof Ġzero Ġembed d ings Ġthat Ġhas Ġthe Ġsame Ġsequence Ġlength Ġ$ M $. ĠIn Ġthis Ġway , Ġwe Ġcan Ġcompute Ġthe Ġcontribution Ġof Ġa Ġspecific Ġinput Ġword Ġto Ġa Ġdesignated Ġoutput Ġword . ĠSince Ġthe Ġabove Ġformula Ġis Ġint ract able Ġfor Ġdeep Ġneural Ġmodels , Ġwe Ġapproximate Ġit Ġby Ġsum ming Ġthe Ġgrad ients Ġalong Ġa Ġmulti - step Ġpath Ġfrom Ġbaseline Ġ$\ text bf Ġ{ x } ^ {\ prime Ġ} $ Ġto Ġthe Ġinput Ġx . </s> where Ġ$ S $ Ġdenotes Ġthe Ġnumber Ġof Ġsteps Ġthat Ġare Ġuniformly Ġdistributed Ġalong Ġthe Ġpath . ĠThe ĠIG Ġwill Ġbe Ġmore Ġaccurate Ġif Ġa Ġlarger ĠS Ġis Ġused . ĠIn Ġour Ġpreliminary Ġexperiments , Ġwe Ġvaried Ġthe Ġsteps Ġand Ġfound Ġ300 Ġsteps Ġyielding Ġfairly Ġgood Ġperformance . </s> Following Ġthe Ġformula , Ġwe Ġcan Ġcalculate Ġthe Ġcontribution Ġof Ġevery Ġinput Ġword Ġmakes Ġto Ġevery Ġoutput Ġword , Ġforming Ġa Ġcontribution Ġmatrix Ġof Ġsize Ġ$ M Ġ\ times ĠN $, Ġwhere Ġ$ N $ Ġis Ġthe Ġoutput Ġsentence Ġlength . ĠGiven Ġthe Ġcontribution Ġmatrix , Ġwe Ġcan Ġobtain Ġthe Ġword Ġimportance Ġof Ġeach Ġinput Ġword Ġto Ġthe Ġentire Ġoutput Ġsentence . ĠTo Ġthis Ġend , Ġfor Ġeach Ġinput Ġword , Ġwe Ġfirst Ġaggregate Ġits Ġcontribution Ġvalues Ġto Ġall Ġoutput Ġwords Ġby Ġthe Ġsum Ġoperation , Ġand Ġthen Ġnormal ize Ġall Ġsums Ġthrough Ġthe ĠSoft max Ġfunction . ĠFigure ĠFIG REF 13 Ġillustrates Ġan Ġexample Ġof Ġthe Ġcalculated Ġword Ġimportance Ġand Ġthe Ġcontribution Ġmatrix , Ġwhere Ġan ĠEnglish Ġsentence Ġis Ġtranslated Ġinto Ġa ĠFrench Ġsentence Ġusing Ġthe ĠTrans former Ġmodel . ĠA Ġnegative Ġcontribution Ġvalue Ġindicates Ġthat Ġthe Ġinput Ġword Ġhas Ġnegative Ġeffects Ġon Ġthe Ġoutput Ġword . </s> Exper iment Ġ: :: ĠData </s> To Ġmake Ġthe Ġconclusion Ġconvincing , Ġwe Ġfirst Ġchoose Ġtwo Ġlarge - scale Ġdatasets Ġthat Ġare Ġpublicly Ġavailable , Ġi . e ., ĠChinese - English Ġand ĠEnglish - French . ĠSince ĠEnglish , ĠFrench , Ġand ĠChinese Ġall Ġbelong Ġto Ġthe Ġsubject - verb - object Ġ( S VO ) Ġfamily , Ġwe Ġchoose Ġanother Ġvery Ġdifferent Ġsubject - object - verb Ġ( S OV ) Ġlanguage , ĠJapanese , Ġwhich Ġmight Ġbring Ġsome Ġinteresting Ġlinguistic Ġbehaviors Ġin ĠEnglish - Japanese Ġtranslation . </s> For ĠChinese - English Ġtask , Ġwe Ġuse ĠW MT 17 ĠChinese - English Ġdataset Ġthat Ġconsists Ġof Ġ$ 20 . 6 $ M Ġsentence Ġpairs . ĠFor ĠEnglish - French Ġtask , Ġwe Ġuse ĠW MT 14 ĠEnglish - French Ġdataset Ġthat Ġcomprises Ġ$ 35 . 5 $ M Ġsentence Ġpairs . ĠFor ĠEnglish - Japanese Ġtask , Ġwe Ġfollow ĠB IB REF 17 Ġto Ġuse Ġthe Ġfirst Ġtwo Ġsections Ġof ĠW AT 17 ĠEnglish - Japanese Ġdataset Ġthat Ġconsists Ġof Ġ$ 1 . 9 $ M Ġsentence Ġpairs . ĠFollowing Ġthe Ġstandard ĠN MT Ġprocedure , Ġwe Ġadopt Ġthe Ġstandard Ġbyte Ġpair Ġencoding Ġ( B PE ) ĠB IB REF 18 Ġwith Ġ32 K Ġmerge Ġoperations Ġfor Ġall Ġlanguage Ġpairs . ĠWe Ġbelieve Ġthat Ġthese Ġdatasets Ġare Ġlarge Ġenough Ġto Ġconfirm Ġthe Ġrationality Ġand Ġvalidity Ġof Ġour Ġexperimental Ġanalyses . </s> Exper iment Ġ: :: ĠImplementation </s> We Ġchoose Ġthe Ġstate - of - the - art ĠTrans former ĠB IB REF 1 Ġmodel Ġand Ġthe Ġconventional ĠR NN - Search Ġmodel ĠB IB REF 0 Ġas Ġour Ġtest Ġbed . ĠWe Ġimplement Ġthe ĠAttribution Ġmethod Ġbased Ġon Ġthe ĠFair seq - py ĠB IB REF 19 Ġframework Ġfor Ġthe Ġabove Ġmodels . ĠAll Ġmodels Ġare Ġtrained Ġon Ġthe Ġtraining Ġcorpus Ġfor Ġ100 k Ġsteps Ġunder Ġthe Ġstandard Ġsettings , Ġwhich Ġachieve Ġcomparable Ġtranslation Ġresults . ĠAll Ġthe Ġfollowing Ġexperiments Ġare Ġconducted Ġon Ġthe Ġtest Ġdataset , Ġand Ġwe Ġestimate Ġthe Ġinput Ġword Ġimportance Ġusing Ġthe Ġmodel Ġgenerated Ġhypotheses . </s> In Ġthe Ġfollowing Ġexperiments , Ġwe Ġcompare ĠIG Ġ( Att ribution ) Ġwith Ġseveral Ġblack - box Ġmethods Ġ( i . e ., ĠContent , ĠFrequency , ĠAttention ) Ġas Ġintroduced Ġin ĠSection ĠSEC REF 8 . ĠIn ĠSection ĠSEC REF 21 , Ġto Ġensure Ġthat Ġthe Ġtranslation Ġperformance Ġdecrease Ġattributes Ġto Ġthe Ġselected Ġwords Ġinstead Ġof Ġthe Ġpert urb ation Ġoperations , Ġwe Ġrandomly Ġselect Ġthe Ġsame Ġnumber Ġof Ġwords Ġto Ġpert urb Ġ( Random ), Ġwhich Ġserves Ġas Ġa Ġbaseline . ĠSince Ġthere Ġis Ġno Ġranking Ġfor Ġcontent Ġwords , Ġwe Ġrandomly Ġselect Ġa Ġset Ġof Ġcontent Ġwords Ġas Ġimportant Ġwords . ĠTo Ġavoid Ġthe Ġpotential Ġbias Ġintroduced Ġby Ġrandom ness Ġ( i . e ., ĠRandom Ġand ĠContent ), Ġwe Ġrepeat Ġthe Ġexperiments Ġfor Ġ10 Ġtimes Ġand Ġreport Ġthe Ġaveraged Ġresults . ĠWe Ġcalculate Ġthe ĠAttention Ġimportance Ġin Ġa Ġsimilar Ġmanner Ġas Ġthe ĠAttribution , Ġexcept Ġthat Ġthe Ġattention Ġscores Ġuse Ġa Ġmax Ġoperation Ġdue Ġto Ġthe Ġbetter Ġperformance . </s> Exper iment Ġ: :: ĠEvaluation </s> We Ġevaluate Ġthe Ġeffectiveness Ġof Ġestimating Ġword Ġimportance Ġby Ġthe Ġtranslation Ġperformance Ġdecrease . ĠMore Ġspecifically , Ġunlike Ġthe Ġusual Ġway , Ġwe Ġmeasure Ġthe Ġdecrease Ġof Ġtranslation Ġperformance Ġwhen Ġpert ur bing Ġa Ġset Ġof Ġimportant Ġwords Ġthat Ġare Ġof Ġtop - most Ġword Ġimportance Ġin Ġa Ġsentence . ĠThe Ġmore Ġtranslation Ġperformance Ġde grades , Ġthe Ġmore Ġimportant Ġthe Ġword Ġis . </s> We Ġuse Ġthe Ġstandard ĠB LE U Ġscore Ġas Ġthe Ġevaluation Ġmetric Ġfor Ġtranslation Ġperformance . ĠTo Ġmake Ġthe Ġconclusion Ġmore Ġconvincing , Ġwe Ġconduct Ġexperiments Ġon Ġdifferent Ġtypes Ġof Ġsynthetic Ġpert urb ations Ġ( Section ĠSEC REF 21 ), Ġas Ġwell Ġas Ġdifferent ĠN MT Ġarchitectures Ġand Ġlanguage Ġpairs Ġ( Section ĠSEC REF 27 ). ĠIn Ġaddition , Ġwe Ġcompare Ġwith Ġa Ġsupervised Ġer asure Ġmethod , Ġwhich Ġrequires Ġground - truth Ġtranslations Ġfor Ġscoring Ġword Ġimportance Ġ( Section ĠSEC REF 30 ). </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠP ert urb ations </s> In Ġthis Ġexperiment , Ġwe Ġinvestigate Ġthe Ġeffectiveness Ġof Ġword Ġimportance Ġestimation Ġmethods Ġunder Ġdifferent Ġsynthetic Ġpert urb ations . ĠSince Ġthe Ġpert urb ation Ġon Ġtext Ġis Ġnotoriously Ġhard ĠB IB REF 20 Ġdue Ġto Ġthe Ġsemantic Ġshifting Ġproblem , Ġin Ġthis Ġexperiment , Ġwe Ġinvestigate Ġthree Ġtypes Ġof Ġpert urb ations Ġto Ġavoid Ġthe Ġpotential Ġbias Ġ: </s> De let ion Ġpert urb ation Ġremoves Ġthe Ġselected Ġwords Ġfrom Ġthe Ġinput Ġsentence , Ġand Ġit Ġcan Ġbe Ġregarded Ġas Ġa Ġspecific Ġinstant iation Ġof Ġsentence Ġcompression ĠB IB REF 21 . </s> Mask Ġpert urb ation Ġreplaces Ġembed ding Ġvectors Ġof Ġthe Ġselected Ġwords Ġwith Ġall - zero Ġvectors ĠB IB REF 22 , Ġwhich Ġis Ġsimilar Ġto ĠDe let ion Ġpert urb ation Ġexcept Ġthat Ġit Ġretains Ġthe Ġplaceholder . </s> G ram matical ĠReplacement Ġpert urb ation Ġreplaces Ġa Ġword Ġby Ġanother Ġword Ġof Ġthe Ġsame Ġlinguistic Ġrole Ġ( i . e ., ĠPOS Ġtags ), Ġyielding Ġa Ġsentence Ġthat Ġis Ġgram matically Ġcorrect Ġbut Ġsem antically Ġnonsensical ĠB IB REF 23 , ĠB IB REF 24 , Ġsuch Ġas ĠâĢ ľ color less Ġgreen Ġideas Ġsleep Ġfuriously âĢ Ŀ . </s> Figure ĠFIG REF 19 Ġillustrates Ġthe Ġexperimental Ġresults Ġon ĠChinese $ \ Right arrow Ġ$ English Ġtranslation Ġwith ĠTrans former . ĠIt Ġshows Ġthat ĠAttribution Ġmethod Ġconsistently Ġoutper forms Ġother Ġmethods Ġagainst Ġdifferent Ġpert urb ations Ġon Ġa Ġvarious Ġnumber Ġof Ġoperations . ĠHere Ġthe Ġoperation Ġnumber Ġdenotes Ġthe Ġnumber Ġof Ġpert urbed Ġwords Ġin Ġa Ġsentence . ĠSpecifically , Ġwe Ġcan Ġmake Ġthe Ġfollowing Ġobservations . </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠP ert urb ations Ġ: :: ĠImportant Ġwords Ġare Ġmore Ġinfluential Ġon Ġtranslation Ġperformance Ġthan Ġthe Ġothers . </s> Under Ġthree Ġdifferent Ġpert urb ations , Ġpert ur bing Ġwords Ġof Ġtop - most Ġimportance Ġleads Ġto Ġlower ĠB LE U Ġscores Ġthan ĠRandom Ġselected Ġwords . ĠIt Ġconfirms Ġthe Ġexistence Ġof Ġimportant Ġwords , Ġwhich Ġhave Ġgreater Ġimpacts Ġon Ġtranslation Ġperformance . ĠFurthermore , Ġpert ur bing Ġimportant Ġwords Ġidentified Ġby ĠAttribution Ġoutper forms Ġthe ĠRandom Ġmethod Ġby Ġa Ġlarge Ġmargin Ġ( more Ġthan Ġ4 . 0 ĠB LE U Ġunder Ġ5 Ġoperations ). </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠP ert urb ations Ġ: :: ĠThe Ġgradient - based Ġmethod Ġis Ġsuperior Ġto Ġcomparative Ġmethods Ġ( e . g ., ĠAttention ) Ġin Ġestimating Ġword Ġimportance . </s> Figure ĠFIG REF 19 Ġshows Ġthat Ġtwo Ġblack - box Ġmethods Ġ( i . e ., ĠContent , ĠFrequency ) Ġperform Ġonly Ġslightly Ġbetter Ġthan Ġthe ĠRandom Ġmethod . ĠSpecifically , Ġthe ĠFrequency Ġmethod Ġdemonstrates Ġeven Ġworse Ġperformances Ġunder Ġthe ĠMask Ġpert urb ation . ĠTherefore , Ġlinguistic Ġproperties Ġ( such Ġas ĠPOS Ġtags ) Ġand Ġthe Ġword Ġfrequency Ġcan Ġonly Ġpartially Ġhelp Ġidentify Ġthe Ġimportant Ġwords , Ġbut Ġit Ġis Ġnot Ġas Ġaccurate Ġas Ġwe Ġthought . ĠIn Ġthe Ġmeanwhile , Ġit Ġis Ġintriguing Ġto Ġexplore Ġwhat Ġexact Ġlinguistic Ġcharacteristics Ġthese Ġimportant Ġwords Ġreveal , Ġwhich Ġwill Ġbe Ġintroduced Ġin ĠSection ĠSEC REF 5 . </s> We Ġalso Ġevaluate Ġthe ĠAttention Ġmethod , Ġwhich Ġbases Ġon Ġthe Ġenc oder - dec oder Ġattention Ġscores Ġat Ġthe Ġlast Ġlayer Ġof ĠTrans former . ĠNote Ġthat Ġthe ĠAttention Ġmethod Ġis Ġalso Ġused Ġto Ġsimulate Ġthe Ġbest Ġblack - box Ġmethod ĠSOC R AT , Ġand Ġthe Ġresults Ġshow Ġthat Ġit Ġis Ġmore Ġeffective Ġthan Ġblack - box Ġmethods Ġand Ġthe ĠRandom Ġbaseline . ĠGiven Ġthe Ġpowerful ĠAttention Ġmethod , ĠAttribution Ġmethod Ġstill Ġachieves Ġbest Ġperformances Ġunder Ġall Ġthree Ġpert urb ations . ĠFurthermore , Ġwe Ġfind Ġthat Ġthe Ġgap Ġbetween ĠAttribution Ġand ĠAttention Ġis Ġnotably Ġlarge Ġ( around Ġ$ 1 . 0 + $ ĠB LE U Ġdifference ). ĠAttention Ġmethod Ġdoes Ġnot Ġprovide Ġas Ġaccurate Ġword Ġimportance Ġas Ġthe ĠAttribution , Ġwhich Ġexhibits Ġthe Ġsuperiority Ġof Ġgradient - based Ġmethods Ġand Ġconsists Ġwith Ġthe Ġconclusion Ġreported Ġin Ġthe Ġprevious Ġstudy ĠB IB REF 8 . </s> In Ġaddition , Ġas Ġshown Ġin ĠFigure ĠFIG REF 19 , Ġthe Ġpert urb ation Ġeffectiveness Ġof ĠDe let ion , ĠMask , Ġand ĠGram matical ĠReplacement Ġvaries Ġfrom Ġstrong Ġto Ġweak . ĠIn Ġthe Ġfollowing Ġexperiments , Ġwe Ġchoose ĠMask Ġas Ġthe Ġrepresentative Ġpert urb ation Ġoperation Ġfor Ġits Ġmoderate Ġpert urb ation Ġperformance , Ġbased Ġon Ġwhich Ġwe Ġcompare Ġtwo Ġmost Ġeffective Ġmethods ĠAttribution Ġand ĠAttention . </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠN MT ĠArchitecture Ġand ĠLanguage ĠP airs Ġ: :: ĠDifferent ĠN MT ĠArchitecture </s> We Ġvalidate Ġthe Ġeffectiveness Ġof Ġthe Ġproposed Ġapproach Ġusing Ġa Ġdifferent ĠN MT Ġarchitecture ĠR NN - Search Ġon Ġthe ĠChinese $ \ Right arrow Ġ$ English Ġtranslation Ġtask . ĠThe Ġresults Ġare Ġshown Ġin ĠFigure ĠFIG REF 20 ( a ). ĠWe Ġobserve Ġthat Ġthe ĠAttribution Ġmethod Ġstill Ġoutper forms Ġboth ĠAttention Ġmethod Ġand ĠRandom Ġmethod Ġby Ġa Ġdecent Ġmargin . ĠBy Ġcomparing Ġto ĠTrans former , Ġthe Ġresults Ġalso Ġreveal Ġthat Ġthe ĠR NN - Search Ġmodel Ġis Ġless Ġrobust Ġto Ġthese Ġpert urb ations . ĠTo Ġbe Ġspecific , Ġunder Ġthe Ġsetting Ġof Ġfive Ġoperations Ġand ĠAttribution Ġmethod , ĠTrans former Ġshows Ġa Ġrelative Ġdecrease Ġof Ġ$ 55 \ % $ Ġon ĠB LE U Ġscores Ġwhile Ġthe Ġdecline Ġof ĠR NN - Search Ġmodel Ġis Ġ$ 64 \ % $. </s> Exper iment Ġ: :: ĠResults Ġon ĠDifferent ĠN MT ĠArchitecture Ġand ĠLanguage ĠP airs Ġ: :: ĠDifferent ĠLanguage ĠP airs Ġand ĠDirections </s> We Ġfurther Ġconduct Ġexperiments Ġon Ġanother Ġtwo Ġlanguage Ġpairs Ġ( i . e ., ĠEnglish $ \ Right arrow Ġ$ French , ĠEnglish $ \ Right arrow Ġ$ Japanese Ġin ĠFigures ĠFIG REF 20 ( b , Ġc )) Ġas Ġwell Ġas Ġthe Ġreverse Ġdirections Ġ( Fig ures ĠFIG REF 20 ( d , Ġe , Ġf )) Ġusing ĠTrans former Ġunder Ġthe ĠMask Ġpert urb ation . ĠIn Ġall Ġthe Ġcases , ĠAttribution Ġshows Ġthe Ġbest Ġperformance Ġwhile ĠRandom Ġachieves Ġthe Ġworst Ġresult . ĠMore Ġspecifically , ĠAttribution Ġmethod Ġshows Ġsimilar Ġtranslation Ġquality Ġdegradation Ġon Ġall Ġthree Ġlanguage - p airs , Ġwhich Ġdeclines Ġto Ġaround Ġthe Ġhalf Ġof Ġthe Ġoriginal ĠB LE U Ġscore Ġwith Ġfive Ġoperations . </s> Exper iment Ġ: :: ĠComparison Ġwith ĠSuper vised ĠEr asure </s> There Ġexists Ġanother Ġstraightforward Ġmethod , ĠEr asure ĠB IB REF 7 , ĠB IB REF 22 , ĠB IB REF 25 , Ġwhich Ġdirectly Ġevaluates Ġthe Ġword Ġimportance Ġby Ġmeasuring Ġthe Ġtranslation Ġperformance Ġdegradation Ġof Ġeach Ġword . ĠSpecifically , Ġit Ġer ases Ġ( i . e ., ĠMask ) Ġone Ġword Ġfrom Ġthe Ġinput Ġsentence Ġeach Ġtime Ġand Ġuses Ġthe ĠB LE U Ġscore Ġchanges Ġto Ġdenote Ġthe Ġword Ġimportance Ġ( after Ġnormal ization ). </s> In ĠFigure ĠFIG REF 31 , Ġwe Ġcompare ĠEr asure Ġmethod Ġwith ĠAttribution Ġmethod Ġunder Ġthe ĠMask Ġpert urb ation . ĠThe Ġresults Ġshow Ġthat ĠAttribution Ġmethod Ġis Ġless Ġeffective Ġthan ĠEr asure Ġmethod Ġwhen Ġonly Ġone Ġword Ġis Ġpert urbed . ĠBut Ġit Ġoutper forms Ġthe ĠEr asure Ġmethod Ġwhen Ġpert ur bing Ġ2 Ġor Ġmore Ġwords . ĠThe Ġresults Ġreveal Ġthat Ġthe Ġimportance Ġcalculated Ġby Ġer asing Ġonly Ġone Ġword Ġcannot Ġbe Ġgeneralized Ġto Ġmultiple - words Ġscenarios Ġvery Ġwell . ĠBesides , Ġthe ĠEr asure Ġmethod Ġis Ġa Ġsupervised Ġmethod Ġwhich Ġrequires Ġground - truth Ġreferences , Ġand Ġfinding Ġa Ġbetter Ġwords Ġcombination Ġis Ġcomputation Ġinf eas ible Ġwhen Ġer asing Ġmultiple Ġwords . </s> We Ġclose Ġthis Ġsection Ġby Ġpointing Ġout Ġthat Ġour Ġgradient - based Ġmethod Ġconsistently Ġoutper forms Ġits Ġblack - box Ġcounterparts Ġin Ġvarious Ġsettings , Ġdemonstrating Ġthe Ġeffectiveness Ġand Ġunivers ality Ġof Ġexploiting Ġgrad ients Ġfor Ġestimating Ġword Ġimportance . ĠIn Ġaddition , Ġour Ġapproach Ġis Ġon Ġpar Ġwith Ġor Ġeven Ġoutper forms Ġthe Ġsupervised Ġer asure Ġmethod Ġ( on Ġmultiple - word Ġpert urb ations ). ĠThis Ġis Ġencouraging Ġsince Ġour Ġapproach Ġdoes Ġnot Ġrequire Ġany Ġexternal Ġresource Ġand Ġis Ġfully Ġun super vised . </s> Analysis </s> In Ġthis Ġsection , Ġwe Ġconduct Ġanalyses Ġon Ġtwo Ġpotential Ġus ages Ġof Ġword Ġimportance , Ġwhich Ġcan Ġhelp Ġdebug ĠN MT Ġmodels Ġ( Section ĠSEC REF 33 ) Ġand Ġdesign Ġbetter Ġarchitectures Ġfor Ġspecific Ġlanguages Ġ( Section ĠSEC REF 37 ). ĠDue Ġto Ġthe Ġspace Ġlimitation , Ġwe Ġonly Ġanalyze Ġthe Ġresults Ġof ĠChinese $ \ Right arrow Ġ$ English , ĠEnglish $ \ Right arrow Ġ$ French , Ġand ĠEnglish $ \ Right arrow Ġ$ Japanese . ĠWe Ġlist Ġthe Ġresults Ġon Ġthe Ġreverse Ġdirections Ġin ĠAppendix , Ġin Ġwhich Ġthe Ġgeneral Ġconclusions Ġalso Ġhold . </s> Analysis Ġ: :: ĠEffect Ġon ĠDetect ing ĠTranslation ĠErrors </s> In Ġthis Ġexperiment , Ġwe Ġpropose Ġto Ġuse Ġthe Ġestimated Ġword Ġimportance Ġto Ġdetect Ġthe Ġunder - trans lated Ġwords Ġby ĠN MT Ġmodels . ĠInt uitive ly , Ġunder - trans lated Ġinput Ġwords Ġshould Ġcontribute Ġlittle Ġto Ġthe ĠN MT Ġoutputs , Ġyielding Ġmuch Ġsmaller Ġword Ġimportance . ĠGiven Ġ500 ĠChinese $ \ Right arrow Ġ$ English Ġsentence Ġpairs Ġtranslated Ġby Ġthe ĠTrans former Ġmodel Ġ( BLE U Ġ23 . 57 ), Ġwe Ġask Ġten Ġhuman Ġannot ators Ġto Ġmanually Ġlabel Ġthe Ġunder - trans lated Ġinput Ġwords , Ġand Ġat Ġleast Ġtwo Ġannot ators Ġlabel Ġeach Ġinput - hyp ot hesis Ġpair . ĠThese Ġannot ators Ġhave Ġat Ġleast Ġsix Ġyears Ġof ĠEnglish Ġstudy Ġexperience , Ġwhose Ġnative Ġlanguage Ġis ĠChinese . ĠAmong Ġthese Ġsentences , Ġ178 Ġsentences Ġhave Ġunder - translation Ġerrors Ġwith Ġ5 53 Ġunder - trans lated Ġwords Ġin Ġtotal . </s> Table ĠT AB REF 32 Ġlists Ġthe Ġaccuracy Ġof Ġdetecting Ġunder - translation Ġerrors Ġby Ġcomparing Ġwords Ġof Ġleast Ġimportance Ġand Ġhuman - annot ated Ġunder - trans lated Ġwords . ĠAs Ġseen , Ġour ĠAttribution Ġmethod Ġconsistently Ġand Ġsignificantly Ġoutper forms Ġboth ĠEr asure Ġand ĠAttention Ġapproaches . ĠBy Ġexploiting Ġthe Ġword Ġimportance Ġcalculated Ġby ĠAttribution Ġmethod , Ġwe Ġcan Ġidentify Ġthe Ġunder - translation Ġerrors Ġautomatically Ġwithout Ġthe Ġinvolvement Ġof Ġhuman Ġinterpre ters . ĠAlthough Ġthe Ġaccuracy Ġis Ġnot Ġhigh , Ġit Ġis Ġworth Ġnoting Ġthat Ġour Ġunder - translation Ġmethod Ġis Ġvery Ġsimple Ġand Ġstraightforward . ĠThis Ġis Ġpotentially Ġuseful Ġfor Ġdebugging ĠN MT Ġmodels , Ġe . g ., Ġautomatic Ġpost - ed iting Ġwith Ġconstraint Ġdecoding ĠB IB REF 26 , ĠB IB REF 27 . </s> Analysis Ġ: :: ĠAnalysis Ġon ĠL ingu istic ĠProperties </s> In Ġthis Ġsection , Ġwe Ġanalyze Ġthe Ġlinguistic Ġcharacteristics Ġof Ġimportant Ġwords Ġidentified Ġby Ġthe Ġattribution - based Ġapproach . ĠSpecifically , Ġwe Ġinvestigate Ġseveral Ġrepresentative Ġsets Ġof Ġlinguistic Ġproperties , Ġincluding ĠPOS Ġtags , Ġand Ġfertility , Ġand Ġdepth Ġin Ġa Ġsynt actic Ġparse Ġtree . ĠIn Ġthese Ġanalyses , Ġwe Ġmultiply Ġthe Ġword Ġimportance Ġwith Ġthe Ġcorresponding Ġsentence Ġlength Ġfor Ġfair Ġcomparison . ĠWe Ġuse Ġa Ġdecision Ġtree Ġbased Ġregression Ġmodel Ġto Ġcalculate Ġthe Ġcorrelation Ġbetween Ġthe Ġimportance Ġand Ġlinguistic Ġproperties . </s> Table ĠT AB REF 34 Ġlists Ġthe Ġcorrelations , Ġwhere Ġa Ġhigher Ġvalue Ġindicates Ġa Ġstronger Ġcorrelation . ĠWe Ġfind Ġthat Ġthe Ġsynt actic Ġinformation Ġis Ġalmost Ġindependent Ġof Ġthe Ġword Ġimportance Ġvalue . ĠInstead , Ġthe Ġword Ġimportance Ġstrongly Ġcorrelates Ġwith Ġthe ĠPOS Ġtags Ġand Ġfertility Ġfeatures , Ġand Ġthese Ġfeatures Ġin Ġtotal Ġcontribute Ġover Ġ95 %. ĠTherefore , Ġin Ġthe Ġfollowing Ġanalyses , Ġwe Ġmainly Ġfocus Ġon Ġthe ĠPOS Ġtags Ġ( Table ĠT AB REF 35 ) Ġand Ġfertility Ġproperties Ġ( Table ĠT AB REF 36 ). ĠFor Ġbetter Ġillustration , Ġwe Ġcalculate Ġthe Ġdistribution Ġover Ġthe Ġlinguistic Ġproperty Ġbased Ġon Ġboth Ġthe ĠAttribution Ġimportance Ġ( âĢ ľ Att r . âĢ Ŀ ) Ġand Ġthe Ġword Ġfrequency Ġ( âĢ ľ Count âĢ Ŀ ) Ġinside Ġa Ġsentence . ĠThe Ġlarger Ġthe Ġrelative Ġincrease Ġbetween Ġthese Ġtwo Ġvalues , Ġthe Ġmore Ġimportant Ġthe Ġlinguistic Ġproperty Ġis . </s> Analysis Ġ: :: ĠAnalysis Ġon ĠL ingu istic ĠProperties Ġ: :: ĠCertain Ġsynt actic Ġcategories Ġhave Ġhigher Ġimportance Ġwhile Ġthe Ġcategories Ġvary Ġacross Ġlanguage Ġpairs . </s> As Ġshown Ġin ĠTable ĠT AB REF 35 , Ġcontent Ġwords Ġare Ġmore Ġimportant Ġon ĠChinese $ \ Right arrow Ġ$ English Ġbut Ġcontent - free Ġwords Ġare Ġmore Ġimportant Ġon ĠEnglish $ \ Right arrow Ġ$ Japanese . ĠOn ĠEnglish $ \ Right arrow Ġ$ French , Ġthere Ġis Ġno Ġnotable Ġincrease Ġor Ġdecrease Ġof Ġthe Ġdistribution Ġsince ĠEnglish Ġand ĠFrench Ġare Ġin Ġessence Ġvery Ġsimilar . ĠWe Ġalso Ġobtain Ġsome Ġspecific Ġfindings Ġof Ġgreat Ġinterest . ĠFor Ġexample , Ġwe Ġfind Ġthat Ġnoun Ġis Ġmore Ġimportant Ġon ĠChinese $ \ Right arrow Ġ$ English Ġtranslation , Ġwhile Ġpre position Ġis Ġmore Ġimportant Ġon ĠEnglish $ \ Right arrow Ġ$ French Ġtranslation . ĠMore Ġinterestingly , ĠEnglish $ \ Right arrow Ġ$ Japanese Ġtranslation Ġshows Ġa Ġsubstantial Ġdiscrepancy Ġin Ġcontrast Ġto Ġthe Ġother Ġtwo Ġlanguage Ġpairs . ĠThe Ġresults Ġreveal Ġthat Ġpre position Ġand Ġpunct uation Ġare Ġvery Ġimportant Ġin ĠEnglish $ \ Right arrow Ġ$ Japanese Ġtranslation , Ġwhich Ġis Ġcounter - intuitive . </s> P unct uation Ġin ĠN MT Ġis Ġunder stud ied Ġsince Ġit Ġcarries Ġlittle Ġinformation Ġand Ġoften Ġdoes Ġnot Ġaffect Ġthe Ġunderstanding Ġof Ġa Ġsentence . ĠHowever , Ġwe Ġfind Ġthat Ġpunct uation Ġis Ġimportant Ġon ĠEnglish $ \ Right arrow Ġ$ Japanese Ġtranslation , Ġwhose Ġproportion Ġincreases Ġdramatically . ĠWe Ġconjecture Ġthat Ġit Ġis Ġbecause Ġthe Ġpunct uation Ġcould Ġaffect Ġthe Ġsense Ġgroups Ġin Ġa Ġsentence , Ġwhich Ġfurther Ġbenefits Ġthe Ġsynt actic Ġre ordering Ġin ĠJapanese . </s> Analysis Ġ: :: ĠAnalysis Ġon ĠL ingu istic ĠProperties Ġ: :: ĠWords Ġof Ġhigh Ġfertility Ġare Ġalways Ġimportant . </s> We Ġfurther Ġcompare Ġthe Ġfertility Ġdistribution Ġbased Ġon Ġword Ġimportance Ġand Ġthe Ġword Ġfrequency Ġon Ġthree Ġlanguage Ġpairs . ĠWe Ġhypothes ize Ġthat Ġa Ġsource Ġword Ġthat Ġcorresponds Ġto Ġmultiple Ġtarget Ġwords Ġshould Ġbe Ġmore Ġimportant Ġsince Ġit Ġcontributes Ġmore Ġto Ġboth Ġsentence Ġlength Ġand ĠB LE U Ġscore . </s> Table ĠT AB REF 36 Ġlists Ġthe Ġresults . ĠOverall Ġspeaking , Ġone - to - many Ġfertility Ġis Ġconsistently Ġmore Ġimportant Ġon Ġall Ġthree Ġlanguage Ġpairs , Ġwhich Ġconfirms Ġour Ġhypothesis . ĠOn Ġthe Ġcontrary , Ġnull - aligned Ġwords Ġreceive Ġmuch Ġless Ġattention , Ġwhich Ġshows Ġa Ġpersist ently Ġdecrease Ġon Ġthree Ġlanguage Ġpairs . ĠIt Ġis Ġalso Ġreasonable Ġsince Ġnull - aligned Ġinput Ġwords Ġcontribute Ġalmost Ġnothing Ġto Ġthe Ġtranslation Ġoutputs . </s> Discussion Ġand ĠConclusion </s> We Ġapproach Ġunderstanding ĠN MT Ġby Ġinvestigating Ġthe Ġword Ġimportance Ġvia Ġa Ġgradient - based Ġmethod , Ġwhich Ġbridges Ġthe Ġgap Ġbetween Ġword Ġimportance Ġand Ġtranslation Ġperformance . ĠEm pir ical Ġresults Ġshow Ġthat Ġthe Ġgradient - based Ġmethod Ġis Ġsuperior Ġto Ġseveral Ġblack - box Ġmethods Ġin Ġestimating Ġthe Ġword Ġimportance . ĠFurther Ġanalyses Ġshow Ġthat Ġimportant Ġwords Ġare Ġof Ġdistinct Ġsynt actic Ġcategories Ġon Ġdifferent Ġlanguage Ġpairs , Ġwhich Ġmight Ġsupport Ġthe Ġviewpoint Ġthat Ġessential Ġinduct ive Ġbias Ġshould Ġbe Ġintroduced Ġinto Ġthe Ġmodel Ġdesign ĠB IB REF 28 . ĠOur Ġstudy Ġalso Ġsuggests Ġthe Ġpossibility Ġof Ġdetecting Ġthe Ġnotorious Ġunder - translation Ġproblem Ġvia Ġthe Ġgradient - based Ġmethod . </s> This Ġpaper Ġis Ġan Ġinitiating Ġstep Ġtowards Ġthe Ġgeneral Ġunderstanding Ġof ĠN MT Ġmodels , Ġwhich Ġmay Ġbring Ġsome Ġpotential Ġimprovements , Ġsuch Ġas </s> Inter active ĠMT Ġand ĠCon str aint ĠDec oding ĠB IB REF 29 , ĠB IB REF 26 : ĠThe Ġmodel Ġpays Ġmore Ġattention Ġto Ġthe Ġdetected Ġun important Ġwords , Ġwhich Ġare Ġpossibly Ġunder - trans lated ; </s> Adapt ive ĠInput ĠEmb edd ing ĠB IB REF 30 : ĠWe Ġcan Ġextend Ġthe Ġadaptive Ġsoft max ĠB IB REF 31 Ġto Ġthe Ġinput Ġembed ding Ġof Ġvariable Ġcapacity ĠâĢĵ Ġmore Ġimportant Ġwords Ġare Ġassigned Ġwith Ġmore Ġcapacity ; </s> N MT ĠArchitecture ĠDesign : ĠThe Ġlanguage - specific Ġinduct ive Ġbias Ġ( e . g ., Ġdifferent Ġbehaviors Ġon ĠPOS ) Ġshould Ġbe Ġincorporated Ġinto Ġthe Ġmodel Ġdesign . </s> We Ġcan Ġalso Ġexplore Ġother Ġapplications Ġof Ġword Ġimportance Ġto Ġimprove ĠN MT Ġmodels , Ġsuch Ġas Ġmore Ġtailored Ġtraining Ġmethods . ĠIn Ġgeneral , Ġmodel Ġinterpret ability Ġcan Ġbuild Ġtrust Ġin Ġmodel Ġpredictions , Ġhelp Ġerror Ġdiagnosis Ġand Ġfacilitate Ġmodel Ġrefinement . ĠWe Ġexpect Ġour Ġwork Ġcould Ġshed Ġlight Ġon Ġthe ĠN MT Ġmodel Ġunderstanding Ġand Ġbenefit Ġthe Ġmodel Ġimprovement . </s> There Ġare Ġmany Ġpossible Ġways Ġto Ġimplement Ġthe Ġgeneral Ġidea Ġof Ġexploiting Ġgrad ients Ġfor Ġmodel Ġinterpretation . ĠThe Ġaim Ġof Ġthis Ġpaper Ġis Ġnot Ġto Ġexplore Ġthis Ġwhole Ġspace Ġbut Ġsimply Ġto Ġshow Ġthat Ġsome Ġfairly Ġstraightforward Ġimplementations Ġwork Ġwell . ĠOur Ġapproach Ġcan Ġbenefit Ġfrom Ġadvanced Ġexploitation Ġof Ġthe Ġgrad ients Ġor Ġother Ġuseful Ġintermediate Ġinformation , Ġwhich Ġwe Ġleave Ġto Ġthe Ġfuture Ġwork . </s> Acknowled gement </s> Sh il in ĠHe Ġand ĠMichael ĠR . ĠLy u Ġwere Ġsupported Ġby Ġthe ĠResearch ĠGrants ĠCouncil Ġof Ġthe ĠHong ĠKong ĠSpecial ĠAdministrative ĠRegion , ĠChina Ġ( No . ĠCU HK Ġ142 107 17 Ġof Ġthe ĠGeneral ĠResearch ĠFund ), Ġand ĠMicrosoft ĠResearch ĠAsia Ġ( 2018 ĠMicrosoft ĠResearch ĠAsia ĠCollabor ative ĠResearch ĠAward ). ĠWe Ġthank Ġthe Ġanonymous Ġreviewers Ġfor Ġtheir Ġinsightful Ġcomments Ġand Ġsuggestions . </s> An alyses Ġon ĠReverse ĠDirections </s> 2 </s> We Ġanalyze Ġthe Ġdistribution Ġof Ġsynt actic Ġcategories Ġand Ġword Ġfertility Ġon Ġthe Ġsame Ġlanguage Ġpairs Ġwith Ġreverse Ġdirections , Ġi . e ., ĠEnglish $ \ Right arrow Ġ$ Chinese , ĠFrench $ \ Right arrow Ġ$ English , Ġand ĠJapanese $ \ Right arrow Ġ$ English . ĠThe Ġresults Ġare Ġshown Ġin ĠTable ĠT AB REF 43 Ġand ĠTable ĠT AB REF 44 Ġrespectively , Ġwhere Ġwe Ġobserve Ġsimilar Ġfindings Ġas Ġbefore . ĠWe Ġuse Ġthe ĠStanford ĠPOS Ġtag ger Ġto Ġparse Ġthe ĠEnglish Ġand ĠFrench Ġinput Ġsentences , Ġand Ġuse Ġthe ĠKy te a Ġto Ġparse Ġthe ĠJapanese Ġinput Ġsentences . </s> An alyses Ġon ĠReverse ĠDirections Ġ: :: ĠSynt actic ĠCategories </s> On ĠEnglish $ \ Right arrow Ġ$ Chinese , Ġcontent Ġwords Ġare Ġmore Ġimportant Ġthan Ġcontent - free Ġwords , Ġwhile Ġthe Ġsituation Ġis Ġreversed Ġon Ġboth ĠFrench $ \ Right arrow Ġ$ English Ġand ĠJapanese $ \ Right arrow Ġ$ English Ġtranslations . ĠSince Ġthere Ġis Ġno Ġclear Ġboundary Ġbetween ĠPre position / D etermin er Ġand Ġother Ġcategories Ġin ĠJapanese , Ġwe Ġset Ġboth Ġcategories Ġto Ġbe Ġnone . ĠSimilarly , ĠPun ct uation Ġis Ġmore Ġimportant Ġon ĠJapanese $ \ Right arrow Ġ$ English , Ġwhich Ġis Ġin Ġline Ġwith Ġthe Ġfinding Ġon ĠEnglish $ \ Right arrow Ġ$ Japanese . ĠOverall Ġspeaking , Ġit Ġmight Ġindicate Ġthat Ġthe ĠSynt actic Ġdistribution Ġwith Ġword Ġimportance Ġis Ġlanguage - pair Ġrelated Ġinstead Ġof Ġthe Ġdirection . </s> An alyses Ġon ĠReverse ĠDirections Ġ: :: ĠWord ĠF ertility </s> The Ġword Ġfertility Ġalso Ġshows Ġsimilar Ġtrend Ġas Ġthe Ġpreviously Ġreported Ġresults , Ġwhere Ġone - to - many Ġfertility Ġis Ġmore Ġimportant Ġand Ġnull - aligned Ġfertility Ġis Ġless Ġimportant . ĠInterestingly , Ġmany - to - one Ġfertility Ġshows Ġan Ġincreasing Ġtrend Ġon ĠJapanese $ \ Right arrow Ġ$ English Ġtranslation , Ġbut Ġthe Ġproportion Ġis Ġrelatively Ġsmall . </s> In Ġsummary , Ġthe Ġfindings Ġon Ġlanguage Ġpairs Ġwith Ġreverse Ġdirections Ġstill Ġagree Ġwith Ġthe Ġfindings Ġin Ġthe Ġpaper , Ġwhich Ġfurther Ġconfirms Ġthe Ġgener ality Ġof Ġour Ġexperimental Ġfindings .