Yes
BOOLEAN
The multilingual model uses word alignments between sentences in a parallel corpus to exploit role correspondences across languages. We make copies of the monolingual model for each language and add additional crosslingual latent variables (CLVs) to couple the monolingual models, capturing crosslingual semantic role patterns. Concretely, when training on parallel sentences, whenever the head words of the arguments are aligned, we add a CLV as a parent of the two corresponding role variables. Figure FIGREF16 illustrates this model. The generative process, as explained below, remains the same as the monolingual model for the most part, with the exception of aligned roles which are now generated by both the monolingual process as well as the CLV.
1603.01514
Introduction </s> Sem antic ĠRole ĠLab eling Ġ( S RL ) Ġhas Ġemerged Ġas Ġan Ġimportant Ġtask Ġin ĠNatural ĠLanguage ĠProcessing Ġ( N LP ) Ġdue Ġto Ġits Ġapplic ability Ġin Ġinformation Ġextraction , Ġquestion Ġanswering , Ġand Ġother ĠN LP Ġtasks . ĠS RL Ġis Ġthe Ġproblem Ġof Ġfinding Ġpredicate - argument Ġstructure Ġin Ġa Ġsentence , Ġas Ġillustrated Ġbelow : </s> IN LINE FORM 0 </s> Here , Ġthe Ġpredicate ĠWRITE Ġhas Ġtwo Ġarguments : Ġ` Mike ' Ġas ĠA 0 Ġor Ġthe Ġwriter , Ġand Ġ` a Ġbook ' Ġas ĠA 1 Ġor Ġthe Ġthing Ġwritten . ĠThe Ġlabels ĠA 0 Ġand ĠA 1 Ġcorrespond Ġto Ġthe ĠProp Bank Ġannotations ĠB IB REF 0 Ġ. </s> As Ġthe Ġneed Ġfor ĠS RL Ġarises Ġin Ġdifferent Ġdomains Ġand Ġlanguages , Ġthe Ġexisting Ġmanually Ġannot ated Ġcorpor a Ġbecome Ġinsufficient Ġto Ġbuild Ġsupervised Ġsystems . ĠThis Ġhas Ġmotivated Ġwork Ġon Ġun super vised ĠS RL ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ. ĠPrevious Ġwork Ġhas Ġindicated Ġthat Ġun super vised Ġsystems Ġcould Ġbenefit Ġfrom Ġthe Ġword Ġalignment Ġinformation Ġin Ġparallel Ġtext Ġin Ġtwo Ġor Ġmore Ġlanguages ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ. ĠFor Ġexample , Ġconsider Ġthe ĠGerman Ġtranslation Ġof Ġsentence ĠIN LINE FORM 0 Ġ: </s> IN LINE FORM 0 </s> If Ġsentences ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġhave Ġthe Ġword Ġalign ments : ĠMike - Mike , Ġwritten - ges ch ri eb en , Ġand Ġbook - B uch , Ġthe Ġsystem Ġmight Ġbe Ġable Ġto Ġpredict ĠA 1 Ġfor ĠBuch , Ġeven Ġif Ġthere Ġis Ġinsufficient Ġinformation Ġin Ġthe Ġmon oling ual ĠGerman Ġdata Ġto Ġlearn Ġthis Ġassignment . ĠThus , Ġin Ġlanguages Ġwhere Ġthe Ġresources Ġare Ġsparse Ġor Ġnot Ġgood Ġenough , Ġor Ġthe Ġdistributions Ġare Ġnot Ġinformative , ĠS RL Ġsystems Ġcould Ġbe Ġmade Ġmore Ġaccurate Ġby Ġusing Ġparallel Ġdata Ġwith Ġresource Ġrich Ġor Ġmore Ġam enable Ġlanguages . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġjoint ĠBay esian Ġmodel Ġfor Ġun super vised Ġsemantic Ġrole Ġinduction Ġin Ġmultiple Ġlanguages . ĠThe Ġmodel Ġconsists Ġof Ġindividual ĠBay esian Ġmodels Ġfor Ġeach Ġlanguage ĠB IB REF 3 Ġ, Ġand Ġcross ling ual Ġlatent Ġvariables Ġto Ġincorporate Ġsoft Ġrole Ġagreement Ġbetween Ġaligned Ġconstituents . ĠThis Ġlatent Ġvariable Ġapproach Ġhas Ġbeen Ġdemonstrated Ġto Ġincrease Ġthe Ġperformance Ġin Ġa Ġmult ilingual Ġun super vised Ġpart - of - speech Ġtagging Ġmodel Ġbased Ġon ĠHM Ms ĠB IB REF 4 Ġ. ĠWe Ġinvestigate Ġthe Ġapplication Ġof Ġthis Ġapproach Ġto Ġun super vised ĠS RL , Ġpresenting Ġthe Ġperformance Ġimprovements Ġobtained Ġin Ġdifferent Ġsettings Ġinvolving Ġlabeled Ġand Ġunl abel ed Ġdata , Ġand Ġanalyzing Ġthe Ġannotation Ġeffort Ġrequired Ġto Ġobtain Ġsimilar Ġgains Ġusing Ġlabeled Ġdata . </s> We Ġbegin Ġby Ġbriefly Ġdescribing Ġthe Ġun super vised ĠS RL Ġpipeline Ġand Ġthe Ġmon oling ual Ġsemantic Ġrole Ġinduction Ġmodel Ġwe Ġuse , Ġand Ġthen Ġdescribe Ġour Ġmult ilingual Ġmodel . </s> Un super vised ĠS RL ĠPipeline </s> As Ġestablished Ġin Ġprevious Ġwork ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ, Ġwe Ġuse Ġa Ġstandard Ġun super vised ĠS RL Ġsetup , Ġconsisting Ġof Ġthe Ġfollowing Ġsteps : </s> The Ġtask Ġwe Ġmodel , Ġun super vised Ġsemantic Ġrole Ġinduction , Ġis Ġthe Ġstep Ġ4 Ġof Ġthis Ġpipeline . </s> Mon oling ual ĠModel </s> We Ġuse Ġthe ĠBay esian Ġmodel Ġof Ġgarg 2012 un super vised Ġas Ġour Ġbase Ġmon oling ual Ġmodel . ĠThe Ġsemantic Ġroles Ġare Ġpredicate - specific . ĠTo Ġmodel Ġthe Ġrole Ġordering Ġand Ġrepetition Ġpreferences , Ġthe Ġrole Ġinventory Ġfor Ġeach Ġpredicate Ġis Ġdivided Ġinto ĠPrimary Ġand ĠSecondary Ġroles Ġas Ġfollows : </s> For Ġexample , Ġthe Ġcomplete Ġrole Ġsequence Ġin Ġa Ġframe Ġcould Ġbe : ĠIN LINE FORM 0 ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġ, ĠIN LINE FORM 3 Ġ, ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġ, ĠIN LINE FORM 6 Ġ, ĠIN LINE FORM 7 Ġ, ĠIN LINE FORM 8 ĠIN LINE FORM 9 Ġ. ĠThe Ġordering Ġis Ġdefined Ġas Ġthe Ġsequence Ġof ĠPR s , ĠIN LINE FORM 10 ĠIN LINE FORM 11 Ġ, ĠIN LINE FORM 12 Ġ, ĠIN LINE FORM 13 Ġ, ĠIN LINE FORM 14 Ġ, ĠIN LINE FORM 15 ĠIN LINE FORM 16 Ġ. ĠEach Ġpair Ġof Ġconsecutive ĠPR s Ġin Ġan Ġordering Ġis Ġcalled Ġan Ġinterval . ĠThus , ĠIN LINE FORM 17 Ġis Ġan Ġinterval Ġthat Ġcontains Ġtwo ĠSR s , ĠIN LINE FORM 18 Ġand ĠIN LINE FORM 19 Ġ. ĠAn Ġinterval Ġcould Ġalso Ġbe Ġempty , Ġfor Ġinstance ĠIN LINE FORM 20 Ġcontains Ġno ĠSR s . ĠWhen Ġwe Ġevaluate , Ġthese Ġroles Ġget Ġmapped Ġto Ġgold Ġroles . ĠFor Ġinstance , Ġthe ĠPR ĠIN LINE FORM 21 Ġcould Ġget Ġmapped Ġto Ġa Ġcore Ġrole Ġlike ĠIN LINE FORM 22 Ġ, ĠIN LINE FORM 23 Ġ, Ġetc . Ġor Ġto Ġa Ġmodifier Ġrole Ġlike ĠIN LINE FORM 24 Ġ, ĠIN LINE FORM 25 Ġ, Ġetc . Ġgarg 2012 un super vised Ġreported Ġthat , Ġin Ġpractice , ĠPR s Ġmostly Ġget Ġmapped Ġto Ġcore Ġroles Ġand ĠSR s Ġto Ġmodifier Ġroles , Ġwhich Ġconform s Ġto Ġthe Ġlinguistic Ġmotivations Ġfor Ġthis Ġdistinction . </s> Figure ĠFIG REF 16 Ġillustrates Ġtwo Ġcopies Ġof Ġthe Ġmon oling ual Ġmodel , Ġon Ġeither Ġside Ġof Ġthe Ġcross ling ual Ġlatent Ġvariables . ĠThe Ġgener ative Ġprocess Ġis Ġas Ġfollows : </s> All Ġthe Ġmult in omial Ġand Ġbin omial Ġdistributions Ġhave Ġsymm etric ĠDir ich let Ġand Ġbeta Ġpri ors Ġrespectively . ĠFigure ĠFIG REF 7 Ġgives Ġthe Ġprobability Ġequations Ġfor Ġthe Ġmon oling ual Ġmodel . ĠThis Ġformulation Ġmodels Ġthe Ġglobal Ġrole Ġordering Ġand Ġrepetition Ġpreferences Ġusing ĠPR s , Ġand Ġlimited Ġcontext Ġfor ĠSR s Ġusing Ġintervals . ĠOrder ing Ġand Ġrepetition Ġinformation Ġwas Ġfound Ġto Ġbe Ġhelpful Ġin Ġsupervised ĠS RL Ġas Ġwell ĠB IB REF 9 Ġ, ĠB IB REF 8 Ġ, ĠB IB REF 10 Ġ. ĠMore Ġdetails , Ġincluding Ġthe Ġmotivations Ġbehind Ġthis Ġmodel , Ġare Ġin ĠB IB REF 3 Ġ. </s> Mult ilingual ĠModel </s> The Ġmult ilingual Ġmodel Ġuses Ġword Ġalign ments Ġbetween Ġsentences Ġin Ġa Ġparallel Ġcorpus Ġto Ġexploit Ġrole Ġcorrespond ences Ġacross Ġlanguages . ĠWe Ġmake Ġcopies Ġof Ġthe Ġmon oling ual Ġmodel Ġfor Ġeach Ġlanguage Ġand Ġadd Ġadditional Ġcross ling ual Ġlatent Ġvariables Ġ( CL Vs ) Ġto Ġcouple Ġthe Ġmon oling ual Ġmodels , Ġcapturing Ġcross ling ual Ġsemantic Ġrole Ġpatterns . ĠCon crete ly , Ġwhen Ġtraining Ġon Ġparallel Ġsentences , Ġwhenever Ġthe Ġhead Ġwords Ġof Ġthe Ġarguments Ġare Ġaligned , Ġwe Ġadd Ġa ĠCL V Ġas Ġa Ġparent Ġof Ġthe Ġtwo Ġcorresponding Ġrole Ġvariables . ĠFigure ĠFIG REF 16 Ġillustrates Ġthis Ġmodel . ĠThe Ġgener ative Ġprocess , Ġas Ġexplained Ġbelow , Ġremains Ġthe Ġsame Ġas Ġthe Ġmon oling ual Ġmodel Ġfor Ġthe Ġmost Ġpart , Ġwith Ġthe Ġexception Ġof Ġaligned Ġroles Ġwhich Ġare Ġnow Ġgenerated Ġby Ġboth Ġthe Ġmon oling ual Ġprocess Ġas Ġwell Ġas Ġthe ĠCL V . </s> Every Ġpredicate - t uple Ġhas Ġits Ġown Ġinventory Ġof ĠCL Vs Ġspecific Ġto Ġthat Ġtuple . ĠEach ĠCL V ĠIN LINE FORM 0 Ġis Ġa Ġmulti - valued Ġvariable Ġwhere Ġeach Ġvalue Ġdefines Ġa Ġdistribution Ġover Ġrole Ġlabels Ġfor Ġeach Ġlanguage Ġ( den oted Ġby ĠIN LINE FORM 1 Ġabove ). ĠThese Ġdistributions Ġover Ġlabels Ġare Ġtrained Ġto Ġbe Ġpeak y , Ġso Ġthat Ġeach Ġvalue ĠIN LINE FORM 2 Ġfor Ġa ĠCL V Ġrepresents Ġa Ġcorrelation Ġbetween Ġthe Ġlabels Ġthat ĠIN LINE FORM 3 Ġpredicts Ġin Ġthe Ġtwo Ġlanguages . ĠFor Ġexample , Ġa Ġvalue ĠIN LINE FORM 4 Ġfor Ġthe ĠCL V ĠIN LINE FORM 5 Ġmight Ġgive Ġhigh Ġprobabilities Ġto ĠIN LINE FORM 6 Ġand ĠIN LINE FORM 7 Ġin Ġlanguage Ġ1 , Ġand Ġto ĠIN LINE FORM 8 Ġin Ġlanguage Ġ2 . ĠIf ĠIN LINE FORM 9 Ġis Ġthe Ġonly Ġvalue Ġfor ĠIN LINE FORM 10 Ġthat Ġgives Ġhigh Ġprobability Ġto ĠIN LINE FORM 11 Ġin Ġlanguage Ġ1 , Ġand Ġthe Ġmon oling ual Ġmodel Ġin Ġlanguage Ġ1 Ġdecides Ġto Ġassign ĠIN LINE FORM 12 Ġto Ġthe Ġrole Ġfor ĠIN LINE FORM 13 Ġ, Ġthen ĠIN LINE FORM 14 Ġwill Ġpredict ĠIN LINE FORM 15 Ġin Ġlanguage Ġ2 , Ġwith Ġhigh Ġprobability . ĠWe Ġgenerate Ġthe ĠCL Vs Ġvia Ġa ĠChinese ĠRestaurant ĠProcess ĠB IB REF 11 Ġ, Ġa Ġnon - param etric ĠBay esian Ġmodel , Ġwhich Ġallows Ġus Ġto Ġinduce Ġthe Ġnumber Ġof ĠCL Vs Ġfor Ġevery Ġpredicate - t uple Ġfrom Ġthe Ġdata . ĠWe Ġcontinue Ġto Ġtrain Ġon Ġthe Ġnon - par allel Ġsentences Ġusing Ġthe Ġrespective Ġmon oling ual Ġmodels . </s> The Ġmult ilingual Ġmodel Ġis Ġdeficient , Ġsince Ġthe Ġaligned Ġroles Ġare Ġbeing Ġgenerated Ġtwice . ĠIdeally , Ġwe Ġwould Ġlike Ġto Ġadd Ġthe ĠCL V Ġas Ġadditional Ġconditioning Ġvariables Ġin Ġthe Ġmon oling ual Ġmodels . ĠThe Ġnew Ġjoint Ġprobability Ġcan Ġbe Ġwritten Ġas Ġequation ĠUID 11 Ġ( Figure ĠFIG REF 7 Ġ), Ġwhich Ġcan Ġbe Ġfurther Ġdecomp osed Ġfollowing Ġthe Ġdecom position Ġof Ġthe Ġmon oling ual Ġmodel Ġin ĠFigure ĠFIG REF 7 Ġ. ĠHowever , Ġhaving Ġthis Ġadditional Ġconditioning Ġvariable Ġbreaks Ġthe ĠDir ich let - mult in omial Ġconj ug acy , Ġwhich Ġmakes Ġit Ġint ract able Ġto Ġmarginal ize Ġout Ġthe Ġparameters Ġduring Ġinference . ĠHence , Ġwe Ġuse Ġan Ġapproximation Ġwhere Ġwe Ġtreat Ġeach Ġof Ġthe Ġaligned Ġroles Ġas Ġbeing Ġgenerated Ġtwice , Ġonce Ġby Ġthe Ġmon oling ual Ġmodel Ġand Ġonce Ġby Ġthe Ġcorresponding ĠCL V Ġ( equ ation Ġ). </s> This Ġis Ġthe Ġfirst Ġwork Ġto Ġincorporate Ġthe Ġcoupling Ġof Ġaligned Ġarguments Ġdirectly Ġin Ġa ĠBay esian ĠS RL Ġmodel . ĠThis Ġmakes Ġit Ġeasier Ġto Ġsee Ġhow Ġto Ġextend Ġthis Ġmodel Ġin Ġa Ġprincipled Ġway Ġto Ġincorporate Ġadditional Ġsources Ġof Ġinformation . ĠFirst , Ġthe Ġmodel Ġscales Ġgrace fully Ġto Ġmore Ġthan Ġtwo Ġlanguages . ĠIf Ġthere Ġare Ġa Ġtotal Ġof ĠIN LINE FORM 0 Ġlanguages , Ġand Ġthere Ġis Ġan Ġaligned Ġargument Ġin ĠIN LINE FORM 1 Ġof Ġthem , Ġthe Ġmult ilingual Ġlatent Ġvariable Ġis Ġconnected Ġto Ġonly Ġthose ĠIN LINE FORM 2 Ġaligned Ġarguments . </s> Second , Ġhaving Ġone Ġjoint ĠBay esian Ġmodel Ġallows Ġus Ġto Ġuse Ġthe Ġsame Ġmodel Ġin Ġvarious Ġsemi - super vised Ġlearning Ġsettings , Ġjust Ġby Ġfixing Ġthe Ġannot ated Ġvariables Ġduring Ġtraining . ĠSection ĠSEC REF 29 Ġevaluates Ġa Ġsetting Ġwhere Ġwe Ġhave Ġsome Ġlabeled Ġdata Ġin Ġone Ġlanguage Ġ( called Ġsource ), Ġwhile Ġno Ġlabeled Ġdata Ġin Ġthe Ġsecond Ġlanguage Ġ( called Ġtarget ). ĠNote Ġthat Ġthis Ġis Ġdifferent Ġfrom Ġa Ġclassic Ġannotation Ġprojection Ġsetting Ġ( e . g . ĠB IB REF 12 Ġ), Ġwhere Ġthe Ġrole Ġlabels Ġare Ġmapped Ġfrom Ġsource Ġconstituents Ġto Ġaligned Ġtarget Ġconstituents . </s> In ference Ġand ĠTraining </s> The Ġinference Ġproblem Ġconsists Ġof Ġpredicting Ġthe Ġrole Ġlabels Ġand ĠCL Vs Ġ( the Ġhidden Ġvariables ) Ġgiven Ġthe Ġpredicate , Ġits Ġvoice , Ġand Ġsynt actic Ġfeatures Ġof Ġall Ġthe Ġidentified Ġarguments Ġ( the Ġvisible Ġvariables ). ĠWe Ġuse Ġa Ġcollapsed ĠGibbs - sam pling Ġbased Ġapproach Ġto Ġgenerate Ġsamples Ġfor Ġthe Ġhidden Ġvariables Ġ( model Ġparameters Ġare Ġintegrated Ġout ). ĠThe Ġsample Ġcounts Ġand Ġthe Ġpri ors Ġare Ġthen Ġused Ġto Ġcalculate Ġthe ĠMAP Ġestimate Ġof Ġthe Ġmodel Ġparameters . </s> For Ġthe Ġmon oling ual Ġmodel , Ġthe Ġrole Ġat Ġa Ġgiven Ġposition Ġis Ġsampled Ġas : </s> DIS PLAY FORM 0 </s> where Ġthe Ġsubscript ĠIN LINE FORM 0 Ġrefers Ġto Ġall Ġthe Ġvariables Ġexcept Ġat Ġposition ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġrefers Ġto Ġthe Ġvariables Ġin Ġall Ġthe Ġtraining Ġinstances Ġexcept Ġthe Ġcurrent Ġone , Ġand ĠIN LINE FORM 3 Ġrefers Ġto Ġall Ġthe Ġmodel Ġparameters . ĠThe Ġabove Ġintegral Ġhas Ġa Ġclosed Ġform Ġsolution Ġdue Ġto ĠDir ich let - mult in omial Ġconj ug acy . </s> For Ġsampling Ġroles Ġin Ġthe Ġmult ilingual Ġmodel , Ġwe Ġalso Ġneed Ġto Ġconsider Ġthe Ġprobabilities Ġof Ġroles Ġbeing Ġgenerated Ġby Ġthe ĠCL Vs : </s> DIS PLAY FORM 0 </s> For Ġsampling ĠCL Vs , Ġwe Ġneed Ġto Ġconsider Ġthree Ġfactors : Ġtwo Ġcorresponding Ġto Ġprobabilities Ġof Ġgenerating Ġthe Ġaligned Ġroles , Ġand Ġthe Ġthird Ġone Ġcorresponding Ġto Ġselecting Ġthe ĠCL V Ġaccording Ġto ĠCR P . </s> DIS PLAY FORM 0 </s> where Ġthe Ġaligned Ġroles ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġare Ġconnected Ġto ĠIN LINE FORM 2 Ġ, Ġand ĠIN LINE FORM 3 Ġrefers Ġto Ġall Ġthe Ġvariables Ġexcept ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġ, Ġand ĠIN LINE FORM 6 Ġ. </s> We Ġuse Ġthe Ġtrained Ġparameters Ġto Ġparse Ġthe Ġmon oling ual Ġdata Ġusing Ġthe Ġmon oling ual Ġmodel . ĠThe Ġcross ling ual Ġparameters Ġare Ġignored Ġeven Ġif Ġthey Ġwere Ġused Ġduring Ġtraining . ĠThus , Ġthe Ġinformation Ġcoming Ġfrom Ġthe ĠCL Vs Ġacts Ġas Ġa Ġregular izer Ġfor Ġthe Ġmon oling ual Ġmodels . </s> E val uation </s> Following Ġthe Ġsetting Ġof Ġtit ov cross ling ual , Ġwe Ġevaluate Ġonly Ġon Ġthe Ġarguments Ġthat Ġwere Ġcorrectly Ġidentified , Ġas Ġthe Ġincorrectly Ġidentified Ġarguments Ġdo Ġnot Ġhave Ġany Ġgold Ġsemantic Ġlabels . ĠEvaluation Ġis Ġdone Ġusing Ġthe Ġmetric Ġproposed Ġby Ġlang 2011 un super vised , Ġwhich Ġhas Ġ3 Ġcomponents : Ġ( i ) ĠP urity Ġ( PU ) Ġmeasures Ġhow Ġwell Ġan Ġinduced Ġcluster Ġcorresponds Ġto Ġa Ġsingle Ġgold Ġrole , Ġ( ii ) ĠColl ocation Ġ( CO ) Ġmeasures Ġhow Ġwell Ġa Ġgold Ġrole Ġcorresponds Ġto Ġa Ġsingle Ġinduced Ġcluster , Ġand Ġ( iii ) ĠF 1 Ġis Ġthe Ġharmonic Ġmean Ġof ĠPU Ġand ĠCO . ĠFor Ġeach Ġpredicate , Ġlet ĠIN LINE FORM 0 Ġdenote Ġthe Ġtotal Ġnumber Ġof Ġargument Ġinstances , ĠIN LINE FORM 1 Ġthe Ġinstances Ġin Ġthe Ġinduced Ġcluster ĠIN LINE FORM 2 Ġ, Ġand ĠIN LINE FORM 3 Ġthe Ġinstances Ġhaving Ġlabel ĠIN LINE FORM 4 Ġin Ġgold Ġannotations . ĠIN LINE FORM 5 Ġ, ĠIN LINE FORM 6 Ġ, Ġand ĠIN LINE FORM 7 Ġ. ĠThe Ġscore Ġfor Ġeach Ġpredicate Ġis Ġweighted Ġby Ġthe Ġnumber Ġof Ġits Ġargument Ġinstances , Ġand Ġa Ġweighted Ġaverage Ġis Ġcomputed Ġover Ġall Ġthe Ġpred icates . </s> Bas eline </s> We Ġuse Ġthe Ġsame Ġbaseline Ġas Ġused Ġby Ġlang 2011 un super vised Ġwhich Ġhas Ġbeen Ġshown Ġto Ġbe Ġdifficult Ġto Ġoutper form . ĠThis Ġbaseline Ġassigns Ġa Ġsemantic Ġrole Ġto Ġa Ġconstituent Ġbased Ġon Ġits Ġsynt actic Ġfunction , Ġi . e . Ġthe Ġdependency Ġrelation Ġto Ġits Ġhead . ĠIf Ġthere Ġis Ġa Ġtotal Ġof ĠIN LINE FORM 0 Ġclusters , ĠIN LINE FORM 1 Ġmost Ġfrequent Ġsynt actic Ġfunctions Ġget Ġa Ġcluster Ġeach , Ġand Ġthe Ġrest Ġare Ġassigned Ġto Ġthe ĠIN LINE FORM 2 Ġth Ġcluster . </s> Cl os est ĠPrevious ĠWork </s> This Ġwork Ġis Ġclosely Ġrelated Ġto Ġthe Ġcross - ling ual Ġun super vised ĠS RL Ġwork Ġof Ġtit ov cross ling ual . ĠTheir Ġmodel Ġhas Ġseparate Ġmon oling ual Ġmodels Ġfor Ġeach Ġlanguage Ġand Ġan Ġextra Ġpenalty Ġterm Ġwhich Ġtries Ġto Ġmaximize ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġi . e . Ġfor Ġall Ġthe Ġaligned Ġarguments Ġwith Ġrole Ġlabel ĠIN LINE FORM 2 Ġin Ġlanguage Ġ1 , Ġit Ġtries Ġto Ġfind Ġa Ġrole Ġlabel ĠIN LINE FORM 3 Ġin Ġlanguage Ġ2 Ġsuch Ġthat Ġthe Ġgiven Ġproportion Ġis Ġmaxim ized Ġand Ġvice Ġverse . ĠHowever , Ġthere Ġis Ġno Ġefficient Ġway Ġto Ġoptimize Ġthe Ġobjective Ġwith Ġthis Ġpenalty Ġterm Ġand Ġthe Ġauthors Ġused Ġan Ġinference Ġmethod Ġsimilar Ġto Ġannotation Ġprojection . ĠFurther , Ġthe Ġmethod Ġdoes Ġnot Ġscale Ġnaturally Ġto Ġmore Ġthan Ġtwo Ġlanguages . ĠTheir Ġalgorithm Ġfirst Ġdoes Ġmon oling ual Ġinference Ġin Ġone Ġlanguage Ġignoring Ġthe Ġpenalty Ġand Ġthen Ġdoes Ġthe Ġinference Ġin Ġthe Ġsecond Ġlanguage Ġtaking Ġinto Ġaccount Ġthe Ġpenalty Ġterm . ĠIn Ġcontrast , Ġour Ġmodel Ġadds Ġthe Ġlatent Ġvariables Ġas Ġa Ġpart Ġof Ġthe Ġmodel Ġitself , Ġand Ġnot Ġan Ġexternal Ġpenalty , Ġwhich Ġenables Ġus Ġto Ġuse Ġthe Ġstandard ĠBay esian Ġlearning Ġmethods Ġsuch Ġas Ġsampling . </s> The Ġmon oling ual Ġmodel Ġwe Ġuse ĠB IB REF 3 Ġalso Ġhas Ġtwo Ġmain Ġadvantages Ġover Ġtit ov cross ling ual . ĠFirst , Ġthe Ġformer Ġincorporates Ġa Ġglobal Ġrole Ġordering Ġprobability Ġthat Ġis Ġmissing Ġin Ġthe Ġlatter . ĠSecondly , Ġthe Ġlatter Ġdefines Ġargument - keys Ġas Ġa Ġtuple Ġof Ġfour Ġsynt actic Ġfeatures Ġand Ġall Ġthe Ġarguments Ġhaving Ġthe Ġsame Ġargument - keys Ġare Ġassigned Ġthe Ġsame Ġrole . ĠThis Ġkind Ġof Ġhard Ġclust ering Ġis Ġavoided Ġin Ġthe Ġformer Ġmodel Ġwhere Ġtwo Ġconstituents Ġhaving Ġthe Ġsame Ġset Ġof Ġfeatures Ġmight Ġget Ġassigned Ġdifferent Ġroles Ġif Ġthey Ġappear Ġin Ġdifferent Ġcontexts . </s> Data </s> Following Ġtit ov cross ling ual , Ġwe Ġrun Ġour Ġexperiments Ġon Ġthe ĠEnglish Ġ( EN ) Ġand ĠGerman Ġ( DE ) Ġsections Ġof Ġthe ĠCo N LL Ġ2009 Ġcorpus ĠB IB REF 13 Ġ, Ġand ĠEN - DE Ġsection Ġof Ġthe ĠEuro par l Ġcorpus ĠB IB REF 14 Ġ. ĠWe Ġget Ġabout Ġ40 k ĠEN Ġand Ġ36 k ĠDE Ġsentences Ġfrom Ġthe ĠCo N LL Ġ2009 Ġtraining Ġset , Ġand Ġabout Ġ1 . 5 M Ġparallel ĠEN - DE Ġsentences Ġfrom ĠEuro par l . ĠFor Ġappropriate Ġcomparison , Ġwe Ġkeep Ġthe Ġsame Ġsetting Ġas Ġin ĠB IB REF 6 Ġfor Ġautomatic Ġpars es Ġand Ġargument Ġidentification , Ġwhich Ġwe Ġbriefly Ġdescribe Ġhere . ĠThe ĠEN Ġsentences Ġare Ġparsed Ġsynt act ically Ġusing ĠMalt Parser ĠB IB REF 15 Ġand ĠDE Ġusing ĠL TH Ġparser ĠB IB REF 16 Ġ. ĠAll Ġthe Ġnon - aux iliary Ġverbs Ġare Ġselected Ġas Ġpred icates . ĠIn ĠCo N LL Ġdata , Ġthis Ġgives Ġus Ġabout Ġ3 k ĠEN Ġand Ġ500 ĠDE Ġpred icates . ĠThe Ġtotal Ġnumber Ġof Ġpredicate Ġinstances Ġare Ġ3 . 4 M Ġin ĠEN Ġ( 89 k ĠCo N LL Ġ+ Ġ3 . 3 M ĠEuro par l ) Ġand Ġ2 . 62 M Ġin ĠDE Ġ( 17 k ĠCo N LL Ġ+ Ġ2 . 6 M ĠEuro par l ). ĠThe Ġarguments Ġfor ĠEN Ġare Ġidentified Ġusing Ġthe Ġhe ur istics Ġproposed Ġby Ġlang 2011 un super vised . ĠHowever , Ġwe Ġget Ġan ĠF 1 Ġscore Ġof Ġ85 . 1 % Ġfor Ġargument Ġidentification Ġon ĠCo N LL Ġ2009 ĠEN Ġdata Ġas Ġopposed Ġto Ġ80 . 7 % Ġreported Ġby Ġtit ov cross ling ual . ĠThis Ġcould Ġbe Ġdue Ġto Ġimplementation Ġdifferences , Ġwhich Ġunfortunately Ġmakes Ġour ĠEN Ġresults Ġincom parable . ĠFor ĠDE , Ġthe Ġarguments Ġare Ġidentified Ġusing Ġthe ĠL TH Ġsystem ĠB IB REF 16 Ġ, Ġwhich Ġgives Ġan ĠF 1 Ġscore Ġof Ġ86 . 5 % Ġon Ġthe ĠCo N LL Ġ2009 ĠDE Ġdata . ĠThe Ġword Ġalign ments Ġfor Ġthe ĠEN - DE Ġparallel ĠEuro par l Ġcorpus Ġare Ġcomputed Ġusing ĠG IZ A ++ ĠB IB REF 17 Ġ. ĠFor Ġhigh - pre cision , Ġonly Ġthe Ġintersect ing Ġalign ments Ġin Ġthe Ġtwo Ġdirections Ġare Ġkept . ĠWe Ġdefine Ġtwo Ġsemantic Ġarguments Ġas Ġaligned Ġif Ġtheir Ġhead - words Ġare Ġaligned . ĠIn Ġtotal Ġwe Ġget Ġ9 . 3 M Ġarguments Ġfor ĠEN Ġ( 240 k ĠCo N LL Ġ+ Ġ9 . 1 M ĠEuro par l ) Ġand Ġ4 . 43 M Ġfor ĠDE Ġ( 32 k ĠCo N LL Ġ+ Ġ4 . 4 M ĠEuro par l ). ĠOut Ġof Ġthese , Ġ0 . 76 M Ġarguments Ġare Ġaligned . </s> Main ĠResults </s> Since Ġthe ĠCo N LL Ġannotations Ġhave Ġ21 Ġsemantic Ġroles Ġin Ġtotal , Ġwe Ġuse Ġ21 Ġroles Ġin Ġour Ġmodel Ġas Ġwell Ġas Ġthe Ġbaseline . ĠFollowing Ġgarg 2012 un super vised , Ġwe Ġset Ġthe Ġnumber Ġof ĠPR s Ġto Ġ2 Ġ( excluding ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ), Ġand ĠSR s Ġto Ġ21 - 2 = 19 . ĠTable ĠT AB REF 27 Ġshows Ġthe Ġresults . </s> In Ġthe Ġfirst Ġsetting Ġ( Line Ġ1 ), Ġwe Ġtrain Ġand Ġtest Ġthe Ġmon oling ual Ġmodel Ġon Ġthe ĠCo N LL Ġdata . ĠWe Ġobserve Ġsignificant Ġimprovements Ġin ĠF 1 Ġscore Ġover Ġthe ĠBas eline Ġ( Line Ġ0 ) Ġin Ġboth Ġlanguages . ĠUsing Ġthe ĠCo N LL Ġ2009 Ġdataset Ġalone , Ġtit ov cross ling ual Ġreport Ġan ĠF 1 Ġscore Ġof Ġ80 . 9 % Ġ( PU = 86 . 8 %, ĠCO = 75 . 7 %) Ġfor ĠGerman . ĠThus , Ġour Ġmon oling ual Ġmodel Ġoutper forms Ġtheir Ġmon oling ual Ġmodel Ġin ĠGerman . ĠFor ĠEnglish , Ġthey Ġreport Ġan ĠF 1 Ġscore Ġof Ġ83 . 6 % Ġ( PU = 87 . 5 %, ĠCO = 80 . 1 %), Ġbut Ġnote Ġthat Ġour ĠEnglish Ġresults Ġare Ġnot Ġdirectly Ġcomparable Ġto Ġtheirs Ġdue Ġto Ġdifferences Ġargument Ġidentification , Ġas Ġdiscussed Ġin Ġsection ĠSEC REF 25 Ġ. ĠAs Ġtheir Ġargument Ġidentification Ġscore Ġis Ġlower , Ġperhaps Ġtheir Ġsystem Ġis Ġdisc arding ĠâĢ ľ diff icult âĢ Ŀ Ġarguments Ġwhich Ġleads Ġto Ġa Ġhigher Ġclust ering Ġscore . </s> In Ġthe Ġsecond Ġsetting Ġ( Line Ġ2 ), Ġwe Ġuse Ġthe Ġadditional Ġmon oling ual ĠEuro par l Ġ( EP ) Ġdata Ġfor Ġtraining . ĠWe Ġget Ġequivalent Ġresults Ġin ĠEnglish Ġand Ġa Ġsignificant Ġimprovement Ġin ĠGerman Ġcompared Ġto Ġour Ġprevious Ġsetting Ġ( Line Ġ1 ). ĠThe ĠGerman Ġdataset Ġin ĠCo N LL Ġis Ġquite Ġsmall Ġand Ġbenefits Ġfrom Ġthe Ġadditional ĠEP Ġtraining Ġdata . ĠIn Ġcontrast , Ġthe ĠEnglish Ġmodel Ġis Ġalready Ġquite Ġgood Ġdue Ġto Ġa Ġrelatively Ġbig Ġdataset Ġfrom ĠCo N LL , Ġand Ġgood Ġaccuracy Ġsynt actic Ġpars ers . ĠUnfortunately , Ġtit ov cross ling ual Ġdo Ġnot Ġreport Ġresults Ġwith Ġthis Ġsetting . </s> The Ġthird Ġsetting Ġ( Line Ġ3 ) Ġgives Ġthe Ġresults Ġof Ġour Ġmult ilingual Ġmodel , Ġwhich Ġadds Ġthe Ġword Ġalign ments Ġin Ġthe ĠEP Ġdata . ĠComp aring Ġwith ĠLine Ġ2 , Ġwe Ġget Ġnon - significant Ġimprovements Ġin Ġboth Ġlanguages . Ġtit ov cross ling ual Ġobtain Ġan ĠF 1 Ġscore Ġof Ġ82 . 7 % Ġ( PU = 85 . 0 %, ĠCO = 80 . 6 %) Ġfor ĠGerman , Ġand Ġ83 . 7 % Ġ( PU = 86 . 8 %, ĠCO = 80 . 7 %) Ġfor ĠEnglish . ĠThus , Ġfor ĠGerman , Ġour Ġmult ilingual ĠBay esian Ġmodel Ġis Ġable Ġto Ġcapture Ġthe Ġcross - ling ual Ġpatterns Ġat Ġleast Ġas Ġwell Ġas Ġthe Ġexternal Ġpenalty Ġterm Ġin ĠB IB REF 6 Ġ. ĠWe Ġcannot Ġcompare Ġthe ĠEnglish Ġresults Ġunfortunately Ġdue Ġto Ġdifferences Ġin Ġargument Ġidentification . </s> We Ġalso Ġcompared Ġmon oling ual Ġand Ġbilingual Ġtraining Ġdata Ġusing Ġa Ġsetting Ġthat Ġem ulates Ġthe Ġstandard Ġsupervised Ġsetup Ġof Ġseparate Ġtraining Ġand Ġtest Ġdata Ġsets . ĠWe Ġtrain Ġonly Ġon Ġthe ĠEP Ġdataset Ġand Ġtest Ġon Ġthe ĠCo N LL Ġdataset . ĠLines Ġ4 Ġand Ġ5 Ġof ĠTable ĠT AB REF 27 Ġgive Ġthe Ġresults . ĠThe Ġmult ilingual Ġmodel Ġob tains Ġsmall Ġimprovements Ġin Ġboth Ġlanguages , Ġwhich Ġconfirms Ġthe Ġresults Ġfrom Ġthe Ġstandard Ġun super vised Ġsetup , Ġcomparing Ġlines Ġ2 Ġto Ġ3 . </s> These Ġresults Ġindicate Ġthat Ġlittle Ġinformation Ġcan Ġbe Ġlearned Ġabout Ġsemantic Ġroles Ġfrom Ġthis Ġparallel Ġdata Ġsetup . ĠOne Ġpossible Ġexplanation Ġfor Ġthis Ġresult Ġis Ġthat Ġthe Ġsetup Ġitself Ġis Ġinadequate . ĠGiven Ġthe Ġdefinition Ġof Ġaligned Ġarguments , Ġonly Ġ8 % Ġof ĠEnglish Ġarguments Ġand Ġ17 % Ġof ĠGerman Ġarguments Ġare Ġaligned . ĠThis Ġplus Ġour Ġexperiments Ġsuggest Ġthat Ġimproving Ġthe Ġalignment Ġmodel Ġis Ġa Ġnecessary Ġstep Ġto Ġmaking Ġeffective Ġuse Ġof Ġparallel Ġdata Ġin Ġmult ilingual ĠS RI , Ġfor Ġexample Ġby Ġjoint Ġmodeling Ġwith ĠS RI . ĠWe Ġleave Ġthis Ġexploration Ġto Ġfuture Ġwork . </s> Mult ilingual ĠTraining Ġwith ĠLab eled ĠData Ġfor ĠOne ĠLanguage </s> Another Ġmotivation Ġfor Ġjointly Ġmodeling ĠS RL Ġin Ġmultiple Ġlanguages Ġis Ġthe Ġtransfer Ġof Ġinformation Ġfrom Ġa Ġresource Ġrich Ġlanguage Ġto Ġa Ġresource Ġpoor Ġlanguage . ĠWe Ġevaluated Ġour Ġmodel Ġin Ġa Ġvery Ġgeneral Ġannotation Ġtransfer Ġscenario , Ġwhere Ġwe Ġhave Ġa Ġsmall Ġlabeled Ġdataset Ġfor Ġone Ġlanguage Ġ( source ), Ġand Ġa Ġlarge Ġparallel Ġunl abel ed Ġdataset Ġfor Ġthe Ġsource Ġand Ġanother Ġ( target ) Ġlanguage . ĠWe Ġinvestigate Ġwhether Ġthis Ġsetting Ġimproves Ġthe Ġparameter Ġestimates Ġfor Ġthe Ġtarget Ġlanguage . ĠTo Ġthis Ġend , Ġwe Ġclamp Ġthe Ġrole Ġannotations Ġof Ġthe Ġsource Ġlanguage Ġin Ġthe ĠCo N LL Ġdataset Ġusing Ġa Ġpred efined Ġmapping , Ġand Ġdo Ġnot Ġsample Ġthem Ġduring Ġtraining . ĠThis Ġdata Ġgives Ġus Ġgood Ġparameters Ġfor Ġthe Ġsource Ġlanguage , Ġwhich Ġare Ġused Ġto Ġsample Ġthe Ġroles Ġof Ġthe Ġsource Ġlanguage Ġin Ġthe Ġunl abel ed ĠEuro par l Ġdata . ĠThe ĠCL Vs Ġaim Ġto Ġcapture Ġthis Ġimprovement Ġand Ġthereby Ġimprove Ġsampling Ġand Ġparameter Ġestimates Ġfor Ġthe Ġtarget Ġlanguage . ĠTable ĠT AB REF 28 Ġshows Ġthe Ġresults Ġof Ġthis Ġexperiment . ĠWe Ġobtain Ġsmall Ġimprovements Ġin Ġthe Ġtarget Ġlanguages . ĠAs Ġin Ġthe Ġun super vised Ġsetting , Ġthe Ġsmall Ġpercentage Ġof Ġaligned Ġroles Ġprobably Ġlimits Ġthe Ġimpact Ġof Ġthe Ġcross - ling ual Ġinformation . </s> Label ed ĠData Ġin ĠMon oling ual ĠModel </s> We Ġexplored Ġthe Ġimprovement Ġin Ġthe Ġmon oling ual Ġmodel Ġin Ġa Ġsemi - super vised Ġsetting . ĠTo Ġthis Ġend , Ġwe Ġrandomly Ġselected ĠIN LINE FORM 0 Ġof Ġthe Ġsentences Ġin Ġthe ĠCo N LL Ġdataset Ġas ĠâĢ ľ super vised Ġsentences âĢ Ŀ Ġand Ġthe Ġrest ĠIN LINE FORM 1 Ġwere Ġkept Ġun super vised . ĠNext , Ġwe Ġcl amped Ġthe Ġrole Ġlabels Ġof Ġthe Ġsupervised Ġsentences Ġusing Ġthe Ġpred efined Ġmapping Ġfrom ĠSection ĠSEC REF 29 Ġ. ĠSam pling Ġwas Ġdone Ġon Ġthe Ġun super vised Ġsentences Ġas Ġusual . ĠWe Ġthen Ġmeasured Ġthe Ġclust ering Ġperformance Ġusing Ġthe Ġtrained Ġparameters . </s> To Ġaccess Ġthe Ġcontribution Ġof Ġpartial Ġsupervision Ġbetter , Ġwe Ġconstructed Ġa ĠâĢ ľ super vised Ġbaseline âĢ Ŀ Ġas Ġfollows . ĠFor Ġpred icates Ġseen Ġin Ġthe Ġsupervised Ġsentences , Ġa ĠMAP Ġestimate Ġof Ġthe Ġparameters Ġwas Ġcalculated Ġusing Ġthe Ġpred efined Ġmapping . ĠFor Ġthe Ġunseen Ġpred icates , Ġthe Ġstandard Ġbaseline Ġwas Ġused . </s> Fig ures ĠFIG REF 33 Ġand ĠFIG REF 33 Ġshow Ġthe Ġperformance Ġvariation Ġwith ĠIN LINE FORM 0 Ġ. ĠWe Ġmake Ġthe Ġfollowing Ġobservations : </s> [ left margin = * ] </s> In Ġboth Ġlanguages , Ġat Ġaround ĠIN LINE FORM 0 Ġ, Ġthe Ġsupervised Ġbaseline Ġstarts Ġoutper forming Ġthe Ġsemi - super vised Ġmodel , Ġwhich Ġsuggests Ġthat Ġmanually Ġlabeling Ġabout Ġ10 % Ġof Ġthe Ġsentences Ġis Ġa Ġgood Ġenough Ġalternative Ġto Ġour Ġtraining Ġprocedure . ĠNote Ġthat Ġ10 % Ġamounts Ġto Ġabout Ġ3 . 6 k Ġsentences Ġin ĠGerman Ġand Ġ4 k Ġin ĠEnglish . ĠWe Ġnoticed Ġthat Ġthe Ġproportion Ġof Ġseen Ġpred icates Ġincreases Ġdramatically Ġas Ġwe Ġincrease Ġthe Ġproportion Ġof Ġsupervised Ġsentences . ĠAt Ġ10 % Ġsupervised Ġsentences , Ġthe Ġmodel Ġhas Ġalready Ġseen Ġ63 % Ġof Ġpred icates Ġin ĠGerman Ġand Ġ44 % Ġin ĠEnglish . ĠThis Ġexplains Ġto Ġsome Ġextent Ġwhy Ġonly Ġ10 % Ġlabeled Ġsentences Ġare Ġenough . </s> For ĠGerman , Ġit Ġtakes Ġabout Ġ3 . 5 % Ġor Ġ12 60 Ġsupervised Ġsentences Ġto Ġhave Ġthe Ġsame Ġperformance Ġincrease Ġas Ġ1 . 5 M Ġunl abel ed Ġsentences Ġ( Line Ġ1 Ġto ĠLine Ġ2 Ġin ĠTable ĠT AB REF 27 Ġ). ĠAdding Ġabout Ġ180 Ġmore Ġsupervised Ġsentences Ġalso Ġcovers Ġthe Ġbenefit Ġobtained Ġby Ġalign ments Ġin Ġthe Ġmult ilingual Ġmodel Ġ( Line Ġ2 Ġto ĠLine Ġ3 Ġin ĠTable ĠT AB REF 27 Ġ). ĠThere Ġis Ġno Ġnoticeable Ġperformance Ġdifference Ġin ĠEnglish . </s> We Ġalso Ġevaluated Ġthe Ġperformance Ġvariation Ġon Ġa Ġcompletely Ġunseen ĠCo N LL Ġtest Ġset . ĠSince Ġthe Ġtest Ġset Ġis Ġvery Ġsmall Ġcompared Ġto Ġthe Ġtraining Ġset , Ġthe Ġclust ering Ġevaluation Ġis Ġnot Ġas Ġreliable . ĠNonetheless , Ġwe Ġbroadly Ġobtained Ġthe Ġsame Ġpattern . </s> Related ĠWork </s> As Ġdiscussed Ġin Ġsection ĠSEC REF 24 Ġ, Ġour Ġwork Ġis Ġclosely Ġrelated Ġto Ġthe Ġcross ling ual Ġun super vised ĠS RL Ġwork Ġof Ġtit ov cross ling ual . ĠThe Ġidea Ġof Ġusing Ġsuper ling ual Ġlatent Ġvariables Ġto Ġcapture Ġcross - ling ual Ġinformation Ġwas Ġproposed Ġfor ĠPOS Ġtagging Ġby Ġn ase em 2009 mult ilingual , Ġwhich Ġwe Ġuse Ġhere Ġfor ĠS RL . ĠIn Ġa Ġsemi - super vised Ġsetting , Ġp ado 2009 cross Ġused Ġa Ġgraph Ġbased Ġapproach Ġto Ġtransfer Ġsemantic Ġrole Ġannotations Ġfrom ĠEnglish Ġto ĠGerman . Ġfur sten au 2009 graph Ġused Ġa Ġgraph Ġalignment Ġmethod Ġto Ġmeasure Ġthe Ġsemantic Ġand Ġsynt actic Ġsimilarity Ġbetween Ġdependency Ġtree Ġarguments Ġof Ġknown Ġand Ġunknown Ġverbs . </s> For Ġmon oling ual Ġun super vised ĠS RL , Ġsw ier 2004 un super vised Ġpresented Ġthe Ġfirst Ġwork Ġon Ġa Ġdomain - general Ġcorpus , Ġthe ĠBritish ĠNational ĠCorpus , Ġusing Ġ54 Ġverbs Ġtaken Ġfrom ĠVerb Net . Ġgarg 2012 un super vised Ġproposed Ġa ĠBay esian Ġmodel Ġfor Ġthis Ġproblem Ġthat Ġwe Ġuse Ġhere . Ġtit ov 2012 bay esian Ġalso Ġproposed Ġa Ġclosely Ġrelated ĠBay esian Ġmodel . Ġgren ager 2006 un super vised Ġproposed Ġa Ġgener ative Ġmodel Ġbut Ġtheir Ġparameter Ġspace Ġconsisted Ġof Ġall Ġpossible Ġlink ings Ġof Ġsynt actic Ġconstituents Ġand Ġsemantic Ġroles , Ġwhich Ġmade Ġun super vised Ġlearning Ġdifficult Ġand Ġa Ġseparate Ġlanguage - specific Ġrule Ġbased Ġmethod Ġhad Ġto Ġbe Ġused Ġto Ġconst rain Ġthis Ġspace . ĠOther Ġproposed Ġmodels Ġinclude Ġan Ġiter ative Ġsplit - mer ge Ġalgorithm ĠB IB REF 18 Ġand Ġa Ġgraph - part ition ing Ġbased Ġapproach ĠB IB REF 1 Ġ. Ġmar quez 2008 sem antic Ġprovide Ġa Ġgood Ġoverview Ġof Ġthe Ġsupervised ĠS RL Ġsystems . </s> Con clusions </s> We Ġpropose Ġa ĠBay esian Ġmodel Ġof Ġsemantic Ġrole Ġinduction Ġ( S RI ) Ġthat Ġuses Ġcross ling ual Ġlatent Ġvariables Ġto Ġcapture Ġrole Ġalign ments Ġin Ġparallel Ġcorpor a . ĠThe Ġcross ling ual Ġlatent Ġvariables Ġcapture Ġcorrelations Ġbetween Ġroles Ġin Ġdifferent Ġlanguages , Ġand Ġregular ize Ġthe Ġparameter Ġestimates Ġof Ġthe Ġmon oling ual Ġmodels . ĠBecause Ġthis Ġis Ġa Ġjoint ĠBay esian Ġmodel Ġof Ġmult ilingual ĠS RI , Ġwe Ġcan Ġapply Ġthe Ġsame Ġmodel Ġto Ġa Ġvariety Ġof Ġtraining Ġscenarios Ġjust Ġby Ġchanging Ġthe Ġinference Ġprocedure Ġappropriately . ĠWe Ġevaluate Ġmon oling ual ĠS RI Ġwith Ġa Ġlarge Ġunl abel ed Ġdataset , Ġbilingual ĠS RI Ġwith Ġa Ġparallel Ġcorpus , Ġbilingual ĠS RI Ġwith Ġannotations Ġavailable Ġfor Ġthe Ġsource Ġlanguage , Ġand Ġmon oling ual ĠS RI Ġwith Ġa Ġsmall Ġlabeled Ġdataset . ĠIncreasing Ġthe Ġamount Ġof Ġmon oling ual Ġunl abel ed Ġdata Ġsignificantly Ġimproves ĠS RI Ġin ĠGerman Ġbut Ġnot Ġin ĠEnglish . ĠAdding Ġword Ġalign ments Ġin Ġparallel Ġsentences Ġresults Ġin Ġsmall , Ġnon Ġsignificant Ġimprovements , Ġeven Ġif Ġthere Ġis Ġsome Ġlabeled Ġdata Ġavailable Ġin Ġthe Ġsource Ġlanguage . ĠThis Ġdifficulty Ġin Ġshowing Ġthe Ġusefulness Ġof Ġparallel Ġcorpor a Ġfor ĠS RI Ġmay Ġbe Ġdue Ġto Ġthe Ġcurrent Ġassumptions Ġabout Ġrole Ġalign ments , Ġwhich Ġmean Ġthat Ġonly Ġa Ġsmall Ġpercentage Ġof Ġroles Ġare Ġaligned . ĠFurther Ġanalyses Ġreveals Ġthat Ġannot ating Ġsmall Ġamounts Ġof Ġdata Ġcan Ġeasily Ġoutper form Ġthe Ġperformance Ġgains Ġobtained Ġby Ġadding Ġlarge Ġunl abel ed Ġdataset Ġas Ġwell Ġas Ġadding Ġparallel Ġcorpor a . </s> Future Ġwork Ġincludes Ġtraining Ġon Ġdifferent Ġlanguage Ġpairs , Ġon Ġmore Ġthan Ġtwo Ġlanguages , Ġand Ġwith Ġmore Ġinclusive Ġmodels Ġof Ġrole Ġalignment . </s> Acknowled gments </s> This Ġwork Ġwas Ġfunded Ġby Ġthe ĠSwiss ĠNS F Ġgrant Ġ2000 21 _ 125 137 Ġand ĠEC ĠFP 7 Ġgrant ĠPAR L ANCE .
Introduction Semantic Role Labeling (SRL) has emerged as an important task in Natural Language Processing (NLP) due to its applicability in information extraction, question answering, and other NLP tasks. SRL is the problem of finding predicate-argument structure in a sentence, as illustrated below: INLINEFORM0 Here, the predicate WRITE has two arguments: `Mike' as A0 or the writer, and `a book' as A1 or the thing written. The labels A0 and A1 correspond to the PropBank annotations BIBREF0 . As the need for SRL arises in different domains and languages, the existing manually annotated corpora become insufficient to build supervised systems. This has motivated work on unsupervised SRL BIBREF1 , BIBREF2 , BIBREF3 . Previous work has indicated that unsupervised systems could benefit from the word alignment information in parallel text in two or more languages BIBREF4 , BIBREF5 , BIBREF6 . For example, consider the German translation of sentence INLINEFORM0 : INLINEFORM0 If sentences INLINEFORM0 and INLINEFORM1 have the word alignments: Mike-Mike, written-geschrieben, and book-Buch, the system might be able to predict A1 for Buch, even if there is insufficient information in the monolingual German data to learn this assignment. Thus, in languages where the resources are sparse or not good enough, or the distributions are not informative, SRL systems could be made more accurate by using parallel data with resource rich or more amenable languages. In this paper, we propose a joint Bayesian model for unsupervised semantic role induction in multiple languages. The model consists of individual Bayesian models for each language BIBREF3 , and crosslingual latent variables to incorporate soft role agreement between aligned constituents. This latent variable approach has been demonstrated to increase the performance in a multilingual unsupervised part-of-speech tagging model based on HMMs BIBREF4 . We investigate the application of this approach to unsupervised SRL, presenting the performance improvements obtained in different settings involving labeled and unlabeled data, and analyzing the annotation effort required to obtain similar gains using labeled data. We begin by briefly describing the unsupervised SRL pipeline and the monolingual semantic role induction model we use, and then describe our multilingual model. Unsupervised SRL Pipeline As established in previous work BIBREF7 , BIBREF8 , we use a standard unsupervised SRL setup, consisting of the following steps: The task we model, unsupervised semantic role induction, is the step 4 of this pipeline. Monolingual Model We use the Bayesian model of garg2012unsupervised as our base monolingual model. The semantic roles are predicate-specific. To model the role ordering and repetition preferences, the role inventory for each predicate is divided into Primary and Secondary roles as follows: For example, the complete role sequence in a frame could be: INLINEFORM0 INLINEFORM1 , INLINEFORM2 , INLINEFORM3 , INLINEFORM4 , INLINEFORM5 , INLINEFORM6 , INLINEFORM7 , INLINEFORM8 INLINEFORM9 . The ordering is defined as the sequence of PRs, INLINEFORM10 INLINEFORM11 , INLINEFORM12 , INLINEFORM13 , INLINEFORM14 , INLINEFORM15 INLINEFORM16 . Each pair of consecutive PRs in an ordering is called an interval. Thus, INLINEFORM17 is an interval that contains two SRs, INLINEFORM18 and INLINEFORM19 . An interval could also be empty, for instance INLINEFORM20 contains no SRs. When we evaluate, these roles get mapped to gold roles. For instance, the PR INLINEFORM21 could get mapped to a core role like INLINEFORM22 , INLINEFORM23 , etc. or to a modifier role like INLINEFORM24 , INLINEFORM25 , etc. garg2012unsupervised reported that, in practice, PRs mostly get mapped to core roles and SRs to modifier roles, which conforms to the linguistic motivations for this distinction. Figure FIGREF16 illustrates two copies of the monolingual model, on either side of the crosslingual latent variables. The generative process is as follows: All the multinomial and binomial distributions have symmetric Dirichlet and beta priors respectively. Figure FIGREF7 gives the probability equations for the monolingual model. This formulation models the global role ordering and repetition preferences using PRs, and limited context for SRs using intervals. Ordering and repetition information was found to be helpful in supervised SRL as well BIBREF9 , BIBREF8 , BIBREF10 . More details, including the motivations behind this model, are in BIBREF3 . Multilingual Model The multilingual model uses word alignments between sentences in a parallel corpus to exploit role correspondences across languages. We make copies of the monolingual model for each language and add additional crosslingual latent variables (CLVs) to couple the monolingual models, capturing crosslingual semantic role patterns. Concretely, when training on parallel sentences, whenever the head words of the arguments are aligned, we add a CLV as a parent of the two corresponding role variables. Figure FIGREF16 illustrates this model. The generative process, as explained below, remains the same as the monolingual model for the most part, with the exception of aligned roles which are now generated by both the monolingual process as well as the CLV. Every predicate-tuple has its own inventory of CLVs specific to that tuple. Each CLV INLINEFORM0 is a multi-valued variable where each value defines a distribution over role labels for each language (denoted by INLINEFORM1 above). These distributions over labels are trained to be peaky, so that each value INLINEFORM2 for a CLV represents a correlation between the labels that INLINEFORM3 predicts in the two languages. For example, a value INLINEFORM4 for the CLV INLINEFORM5 might give high probabilities to INLINEFORM6 and INLINEFORM7 in language 1, and to INLINEFORM8 in language 2. If INLINEFORM9 is the only value for INLINEFORM10 that gives high probability to INLINEFORM11 in language 1, and the monolingual model in language 1 decides to assign INLINEFORM12 to the role for INLINEFORM13 , then INLINEFORM14 will predict INLINEFORM15 in language 2, with high probability. We generate the CLVs via a Chinese Restaurant Process BIBREF11 , a non-parametric Bayesian model, which allows us to induce the number of CLVs for every predicate-tuple from the data. We continue to train on the non-parallel sentences using the respective monolingual models. The multilingual model is deficient, since the aligned roles are being generated twice. Ideally, we would like to add the CLV as additional conditioning variables in the monolingual models. The new joint probability can be written as equation UID11 (Figure FIGREF7 ), which can be further decomposed following the decomposition of the monolingual model in Figure FIGREF7 . However, having this additional conditioning variable breaks the Dirichlet-multinomial conjugacy, which makes it intractable to marginalize out the parameters during inference. Hence, we use an approximation where we treat each of the aligned roles as being generated twice, once by the monolingual model and once by the corresponding CLV (equation ). This is the first work to incorporate the coupling of aligned arguments directly in a Bayesian SRL model. This makes it easier to see how to extend this model in a principled way to incorporate additional sources of information. First, the model scales gracefully to more than two languages. If there are a total of INLINEFORM0 languages, and there is an aligned argument in INLINEFORM1 of them, the multilingual latent variable is connected to only those INLINEFORM2 aligned arguments. Second, having one joint Bayesian model allows us to use the same model in various semi-supervised learning settings, just by fixing the annotated variables during training. Section SECREF29 evaluates a setting where we have some labeled data in one language (called source), while no labeled data in the second language (called target). Note that this is different from a classic annotation projection setting (e.g. BIBREF12 ), where the role labels are mapped from source constituents to aligned target constituents. Inference and Training The inference problem consists of predicting the role labels and CLVs (the hidden variables) given the predicate, its voice, and syntactic features of all the identified arguments (the visible variables). We use a collapsed Gibbs-sampling based approach to generate samples for the hidden variables (model parameters are integrated out). The sample counts and the priors are then used to calculate the MAP estimate of the model parameters. For the monolingual model, the role at a given position is sampled as: DISPLAYFORM0 where the subscript INLINEFORM0 refers to all the variables except at position INLINEFORM1 , INLINEFORM2 refers to the variables in all the training instances except the current one, and INLINEFORM3 refers to all the model parameters. The above integral has a closed form solution due to Dirichlet-multinomial conjugacy. For sampling roles in the multilingual model, we also need to consider the probabilities of roles being generated by the CLVs: DISPLAYFORM0 For sampling CLVs, we need to consider three factors: two corresponding to probabilities of generating the aligned roles, and the third one corresponding to selecting the CLV according to CRP. DISPLAYFORM0 where the aligned roles INLINEFORM0 and INLINEFORM1 are connected to INLINEFORM2 , and INLINEFORM3 refers to all the variables except INLINEFORM4 , INLINEFORM5 , and INLINEFORM6 . We use the trained parameters to parse the monolingual data using the monolingual model. The crosslingual parameters are ignored even if they were used during training. Thus, the information coming from the CLVs acts as a regularizer for the monolingual models. Evaluation Following the setting of titovcrosslingual, we evaluate only on the arguments that were correctly identified, as the incorrectly identified arguments do not have any gold semantic labels. Evaluation is done using the metric proposed by lang2011unsupervised, which has 3 components: (i) Purity (PU) measures how well an induced cluster corresponds to a single gold role, (ii) Collocation (CO) measures how well a gold role corresponds to a single induced cluster, and (iii) F1 is the harmonic mean of PU and CO. For each predicate, let INLINEFORM0 denote the total number of argument instances, INLINEFORM1 the instances in the induced cluster INLINEFORM2 , and INLINEFORM3 the instances having label INLINEFORM4 in gold annotations. INLINEFORM5 , INLINEFORM6 , and INLINEFORM7 . The score for each predicate is weighted by the number of its argument instances, and a weighted average is computed over all the predicates. Baseline We use the same baseline as used by lang2011unsupervised which has been shown to be difficult to outperform. This baseline assigns a semantic role to a constituent based on its syntactic function, i.e. the dependency relation to its head. If there is a total of INLINEFORM0 clusters, INLINEFORM1 most frequent syntactic functions get a cluster each, and the rest are assigned to the INLINEFORM2 th cluster. Closest Previous Work This work is closely related to the cross-lingual unsupervised SRL work of titovcrosslingual. Their model has separate monolingual models for each language and an extra penalty term which tries to maximize INLINEFORM0 and INLINEFORM1 i.e. for all the aligned arguments with role label INLINEFORM2 in language 1, it tries to find a role label INLINEFORM3 in language 2 such that the given proportion is maximized and vice verse. However, there is no efficient way to optimize the objective with this penalty term and the authors used an inference method similar to annotation projection. Further, the method does not scale naturally to more than two languages. Their algorithm first does monolingual inference in one language ignoring the penalty and then does the inference in the second language taking into account the penalty term. In contrast, our model adds the latent variables as a part of the model itself, and not an external penalty, which enables us to use the standard Bayesian learning methods such as sampling. The monolingual model we use BIBREF3 also has two main advantages over titovcrosslingual. First, the former incorporates a global role ordering probability that is missing in the latter. Secondly, the latter defines argument-keys as a tuple of four syntactic features and all the arguments having the same argument-keys are assigned the same role. This kind of hard clustering is avoided in the former model where two constituents having the same set of features might get assigned different roles if they appear in different contexts. Data Following titovcrosslingual, we run our experiments on the English (EN) and German (DE) sections of the CoNLL 2009 corpus BIBREF13 , and EN-DE section of the Europarl corpus BIBREF14 . We get about 40k EN and 36k DE sentences from the CoNLL 2009 training set, and about 1.5M parallel EN-DE sentences from Europarl. For appropriate comparison, we keep the same setting as in BIBREF6 for automatic parses and argument identification, which we briefly describe here. The EN sentences are parsed syntactically using MaltParser BIBREF15 and DE using LTH parser BIBREF16 . All the non-auxiliary verbs are selected as predicates. In CoNLL data, this gives us about 3k EN and 500 DE predicates. The total number of predicate instances are 3.4M in EN (89k CoNLL + 3.3M Europarl) and 2.62M in DE (17k CoNLL + 2.6M Europarl). The arguments for EN are identified using the heuristics proposed by lang2011unsupervised. However, we get an F1 score of 85.1% for argument identification on CoNLL 2009 EN data as opposed to 80.7% reported by titovcrosslingual. This could be due to implementation differences, which unfortunately makes our EN results incomparable. For DE, the arguments are identified using the LTH system BIBREF16 , which gives an F1 score of 86.5% on the CoNLL 2009 DE data. The word alignments for the EN-DE parallel Europarl corpus are computed using GIZA++ BIBREF17 . For high-precision, only the intersecting alignments in the two directions are kept. We define two semantic arguments as aligned if their head-words are aligned. In total we get 9.3M arguments for EN (240k CoNLL + 9.1M Europarl) and 4.43M for DE (32k CoNLL + 4.4M Europarl). Out of these, 0.76M arguments are aligned. Main Results Since the CoNLL annotations have 21 semantic roles in total, we use 21 roles in our model as well as the baseline. Following garg2012unsupervised, we set the number of PRs to 2 (excluding INLINEFORM0 , INLINEFORM1 and INLINEFORM2 ), and SRs to 21-2=19. Table TABREF27 shows the results. In the first setting (Line 1), we train and test the monolingual model on the CoNLL data. We observe significant improvements in F1 score over the Baseline (Line 0) in both languages. Using the CoNLL 2009 dataset alone, titovcrosslingual report an F1 score of 80.9% (PU=86.8%, CO=75.7%) for German. Thus, our monolingual model outperforms their monolingual model in German. For English, they report an F1 score of 83.6% (PU=87.5%, CO=80.1%), but note that our English results are not directly comparable to theirs due to differences argument identification, as discussed in section SECREF25 . As their argument identification score is lower, perhaps their system is discarding “difficult” arguments which leads to a higher clustering score. In the second setting (Line 2), we use the additional monolingual Europarl (EP) data for training. We get equivalent results in English and a significant improvement in German compared to our previous setting (Line 1). The German dataset in CoNLL is quite small and benefits from the additional EP training data. In contrast, the English model is already quite good due to a relatively big dataset from CoNLL, and good accuracy syntactic parsers. Unfortunately, titovcrosslingual do not report results with this setting. The third setting (Line 3) gives the results of our multilingual model, which adds the word alignments in the EP data. Comparing with Line 2, we get non-significant improvements in both languages. titovcrosslingual obtain an F1 score of 82.7% (PU=85.0%, CO=80.6%) for German, and 83.7% (PU=86.8%, CO=80.7%) for English. Thus, for German, our multilingual Bayesian model is able to capture the cross-lingual patterns at least as well as the external penalty term in BIBREF6 . We cannot compare the English results unfortunately due to differences in argument identification. We also compared monolingual and bilingual training data using a setting that emulates the standard supervised setup of separate training and test data sets. We train only on the EP dataset and test on the CoNLL dataset. Lines 4 and 5 of Table TABREF27 give the results. The multilingual model obtains small improvements in both languages, which confirms the results from the standard unsupervised setup, comparing lines 2 to 3. These results indicate that little information can be learned about semantic roles from this parallel data setup. One possible explanation for this result is that the setup itself is inadequate. Given the definition of aligned arguments, only 8% of English arguments and 17% of German arguments are aligned. This plus our experiments suggest that improving the alignment model is a necessary step to making effective use of parallel data in multilingual SRI, for example by joint modeling with SRI. We leave this exploration to future work. Multilingual Training with Labeled Data for One Language Another motivation for jointly modeling SRL in multiple languages is the transfer of information from a resource rich language to a resource poor language. We evaluated our model in a very general annotation transfer scenario, where we have a small labeled dataset for one language (source), and a large parallel unlabeled dataset for the source and another (target) language. We investigate whether this setting improves the parameter estimates for the target language. To this end, we clamp the role annotations of the source language in the CoNLL dataset using a predefined mapping, and do not sample them during training. This data gives us good parameters for the source language, which are used to sample the roles of the source language in the unlabeled Europarl data. The CLVs aim to capture this improvement and thereby improve sampling and parameter estimates for the target language. Table TABREF28 shows the results of this experiment. We obtain small improvements in the target languages. As in the unsupervised setting, the small percentage of aligned roles probably limits the impact of the cross-lingual information. Labeled Data in Monolingual Model We explored the improvement in the monolingual model in a semi-supervised setting. To this end, we randomly selected INLINEFORM0 of the sentences in the CoNLL dataset as “supervised sentences” and the rest INLINEFORM1 were kept unsupervised. Next, we clamped the role labels of the supervised sentences using the predefined mapping from Section SECREF29 . Sampling was done on the unsupervised sentences as usual. We then measured the clustering performance using the trained parameters. To access the contribution of partial supervision better, we constructed a “supervised baseline” as follows. For predicates seen in the supervised sentences, a MAP estimate of the parameters was calculated using the predefined mapping. For the unseen predicates, the standard baseline was used. Figures FIGREF33 and FIGREF33 show the performance variation with INLINEFORM0 . We make the following observations: [leftmargin=*] In both languages, at around INLINEFORM0 , the supervised baseline starts outperforming the semi-supervised model, which suggests that manually labeling about 10% of the sentences is a good enough alternative to our training procedure. Note that 10% amounts to about 3.6k sentences in German and 4k in English. We noticed that the proportion of seen predicates increases dramatically as we increase the proportion of supervised sentences. At 10% supervised sentences, the model has already seen 63% of predicates in German and 44% in English. This explains to some extent why only 10% labeled sentences are enough. For German, it takes about 3.5% or 1260 supervised sentences to have the same performance increase as 1.5M unlabeled sentences (Line 1 to Line 2 in Table TABREF27 ). Adding about 180 more supervised sentences also covers the benefit obtained by alignments in the multilingual model (Line 2 to Line 3 in Table TABREF27 ). There is no noticeable performance difference in English. We also evaluated the performance variation on a completely unseen CoNLL test set. Since the test set is very small compared to the training set, the clustering evaluation is not as reliable. Nonetheless, we broadly obtained the same pattern. Related Work As discussed in section SECREF24 , our work is closely related to the crosslingual unsupervised SRL work of titovcrosslingual. The idea of using superlingual latent variables to capture cross-lingual information was proposed for POS tagging by naseem2009multilingual, which we use here for SRL. In a semi-supervised setting, pado2009cross used a graph based approach to transfer semantic role annotations from English to German. furstenau2009graph used a graph alignment method to measure the semantic and syntactic similarity between dependency tree arguments of known and unknown verbs. For monolingual unsupervised SRL, swier2004unsupervised presented the first work on a domain-general corpus, the British National Corpus, using 54 verbs taken from VerbNet. garg2012unsupervised proposed a Bayesian model for this problem that we use here. titov2012bayesian also proposed a closely related Bayesian model. grenager2006unsupervised proposed a generative model but their parameter space consisted of all possible linkings of syntactic constituents and semantic roles, which made unsupervised learning difficult and a separate language-specific rule based method had to be used to constrain this space. Other proposed models include an iterative split-merge algorithm BIBREF18 and a graph-partitioning based approach BIBREF1 . marquez2008semantic provide a good overview of the supervised SRL systems. Conclusions We propose a Bayesian model of semantic role induction (SRI) that uses crosslingual latent variables to capture role alignments in parallel corpora. The crosslingual latent variables capture correlations between roles in different languages, and regularize the parameter estimates of the monolingual models. Because this is a joint Bayesian model of multilingual SRI, we can apply the same model to a variety of training scenarios just by changing the inference procedure appropriately. We evaluate monolingual SRI with a large unlabeled dataset, bilingual SRI with a parallel corpus, bilingual SRI with annotations available for the source language, and monolingual SRI with a small labeled dataset. Increasing the amount of monolingual unlabeled data significantly improves SRI in German but not in English. Adding word alignments in parallel sentences results in small, non significant improvements, even if there is some labeled data available in the source language. This difficulty in showing the usefulness of parallel corpora for SRI may be due to the current assumptions about role alignments, which mean that only a small percentage of roles are aligned. Further analyses reveals that annotating small amounts of data can easily outperform the performance gains obtained by adding large unlabeled dataset as well as adding parallel corpora. Future work includes training on different language pairs, on more than two languages, and with more inclusive models of role alignment. Acknowledgments This work was funded by the Swiss NSF grant 200021_125137 and EC FP7 grant PARLANCE.
Do they add one latent variable for each language pair in their Bayesian model?
6568a31241167f618ef5ede939053feaa2fb0d7e
Do Ġthey Ġadd Ġone Ġlatent Ġvariable Ġfor Ġeach Ġlanguage Ġpair Ġin Ġtheir ĠBay esian Ġmodel ?
Yes
<s> Do Ġthey Ġadd Ġone Ġlatent Ġvariable Ġfor Ġeach Ġlanguage Ġpair Ġin Ġtheir ĠBay esian Ġmodel ? </s> Introduction </s> Sem antic ĠRole ĠLab eling Ġ( S RL ) Ġhas Ġemerged Ġas Ġan Ġimportant Ġtask Ġin ĠNatural ĠLanguage ĠProcessing Ġ( N LP ) Ġdue Ġto Ġits Ġapplic ability Ġin Ġinformation Ġextraction , Ġquestion Ġanswering , Ġand Ġother ĠN LP Ġtasks . ĠS RL Ġis Ġthe Ġproblem Ġof Ġfinding Ġpredicate - argument Ġstructure Ġin Ġa Ġsentence , Ġas Ġillustrated Ġbelow : </s> IN LINE FORM 0 </s> Here , Ġthe Ġpredicate ĠWRITE Ġhas Ġtwo Ġarguments : Ġ` Mike ' Ġas ĠA 0 Ġor Ġthe Ġwriter , Ġand Ġ` a Ġbook ' Ġas ĠA 1 Ġor Ġthe Ġthing Ġwritten . ĠThe Ġlabels ĠA 0 Ġand ĠA 1 Ġcorrespond Ġto Ġthe ĠProp Bank Ġannotations ĠB IB REF 0 Ġ. </s> As Ġthe Ġneed Ġfor ĠS RL Ġarises Ġin Ġdifferent Ġdomains Ġand Ġlanguages , Ġthe Ġexisting Ġmanually Ġannot ated Ġcorpor a Ġbecome Ġinsufficient Ġto Ġbuild Ġsupervised Ġsystems . ĠThis Ġhas Ġmotivated Ġwork Ġon Ġun super vised ĠS RL ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ. ĠPrevious Ġwork Ġhas Ġindicated Ġthat Ġun super vised Ġsystems Ġcould Ġbenefit Ġfrom Ġthe Ġword Ġalignment Ġinformation Ġin Ġparallel Ġtext Ġin Ġtwo Ġor Ġmore Ġlanguages ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ. ĠFor Ġexample , Ġconsider Ġthe ĠGerman Ġtranslation Ġof Ġsentence ĠIN LINE FORM 0 Ġ: </s> IN LINE FORM 0 </s> If Ġsentences ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġhave Ġthe Ġword Ġalign ments : ĠMike - Mike , Ġwritten - ges ch ri eb en , Ġand Ġbook - B uch , Ġthe Ġsystem Ġmight Ġbe Ġable Ġto Ġpredict ĠA 1 Ġfor ĠBuch , Ġeven Ġif Ġthere Ġis Ġinsufficient Ġinformation Ġin Ġthe Ġmon oling ual ĠGerman Ġdata Ġto Ġlearn Ġthis Ġassignment . ĠThus , Ġin Ġlanguages Ġwhere Ġthe Ġresources Ġare Ġsparse Ġor Ġnot Ġgood Ġenough , Ġor Ġthe Ġdistributions Ġare Ġnot Ġinformative , ĠS RL Ġsystems Ġcould Ġbe Ġmade Ġmore Ġaccurate Ġby Ġusing Ġparallel Ġdata Ġwith Ġresource Ġrich Ġor Ġmore Ġam enable Ġlanguages . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġjoint ĠBay esian Ġmodel Ġfor Ġun super vised Ġsemantic Ġrole Ġinduction Ġin Ġmultiple Ġlanguages . ĠThe Ġmodel Ġconsists Ġof Ġindividual ĠBay esian Ġmodels Ġfor Ġeach Ġlanguage ĠB IB REF 3 Ġ, Ġand Ġcross ling ual Ġlatent Ġvariables Ġto Ġincorporate Ġsoft Ġrole Ġagreement Ġbetween Ġaligned Ġconstituents . ĠThis Ġlatent Ġvariable Ġapproach Ġhas Ġbeen Ġdemonstrated Ġto Ġincrease Ġthe Ġperformance Ġin Ġa Ġmult ilingual Ġun super vised Ġpart - of - speech Ġtagging Ġmodel Ġbased Ġon ĠHM Ms ĠB IB REF 4 Ġ. ĠWe Ġinvestigate Ġthe Ġapplication Ġof Ġthis Ġapproach Ġto Ġun super vised ĠS RL , Ġpresenting Ġthe Ġperformance Ġimprovements Ġobtained Ġin Ġdifferent Ġsettings Ġinvolving Ġlabeled Ġand Ġunl abel ed Ġdata , Ġand Ġanalyzing Ġthe Ġannotation Ġeffort Ġrequired Ġto Ġobtain Ġsimilar Ġgains Ġusing Ġlabeled Ġdata . </s> We Ġbegin Ġby Ġbriefly Ġdescribing Ġthe Ġun super vised ĠS RL Ġpipeline Ġand Ġthe Ġmon oling ual Ġsemantic Ġrole Ġinduction Ġmodel Ġwe Ġuse , Ġand Ġthen Ġdescribe Ġour Ġmult ilingual Ġmodel . </s> Un super vised ĠS RL ĠPipeline </s> As Ġestablished Ġin Ġprevious Ġwork ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ, Ġwe Ġuse Ġa Ġstandard Ġun super vised ĠS RL Ġsetup , Ġconsisting Ġof Ġthe Ġfollowing Ġsteps : </s> The Ġtask Ġwe Ġmodel , Ġun super vised Ġsemantic Ġrole Ġinduction , Ġis Ġthe Ġstep Ġ4 Ġof Ġthis Ġpipeline . </s> Mon oling ual ĠModel </s> We Ġuse Ġthe ĠBay esian Ġmodel Ġof Ġgarg 2012 un super vised Ġas Ġour Ġbase Ġmon oling ual Ġmodel . ĠThe Ġsemantic Ġroles Ġare Ġpredicate - specific . ĠTo Ġmodel Ġthe Ġrole Ġordering Ġand Ġrepetition Ġpreferences , Ġthe Ġrole Ġinventory Ġfor Ġeach Ġpredicate Ġis Ġdivided Ġinto ĠPrimary Ġand ĠSecondary Ġroles Ġas Ġfollows : </s> For Ġexample , Ġthe Ġcomplete Ġrole Ġsequence Ġin Ġa Ġframe Ġcould Ġbe : ĠIN LINE FORM 0 ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġ, ĠIN LINE FORM 3 Ġ, ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġ, ĠIN LINE FORM 6 Ġ, ĠIN LINE FORM 7 Ġ, ĠIN LINE FORM 8 ĠIN LINE FORM 9 Ġ. ĠThe Ġordering Ġis Ġdefined Ġas Ġthe Ġsequence Ġof ĠPR s , ĠIN LINE FORM 10 ĠIN LINE FORM 11 Ġ, ĠIN LINE FORM 12 Ġ, ĠIN LINE FORM 13 Ġ, ĠIN LINE FORM 14 Ġ, ĠIN LINE FORM 15 ĠIN LINE FORM 16 Ġ. ĠEach Ġpair Ġof Ġconsecutive ĠPR s Ġin Ġan Ġordering Ġis Ġcalled Ġan Ġinterval . ĠThus , ĠIN LINE FORM 17 Ġis Ġan Ġinterval Ġthat Ġcontains Ġtwo ĠSR s , ĠIN LINE FORM 18 Ġand ĠIN LINE FORM 19 Ġ. ĠAn Ġinterval Ġcould Ġalso Ġbe Ġempty , Ġfor Ġinstance ĠIN LINE FORM 20 Ġcontains Ġno ĠSR s . ĠWhen Ġwe Ġevaluate , Ġthese Ġroles Ġget Ġmapped Ġto Ġgold Ġroles . ĠFor Ġinstance , Ġthe ĠPR ĠIN LINE FORM 21 Ġcould Ġget Ġmapped Ġto Ġa Ġcore Ġrole Ġlike ĠIN LINE FORM 22 Ġ, ĠIN LINE FORM 23 Ġ, Ġetc . Ġor Ġto Ġa Ġmodifier Ġrole Ġlike ĠIN LINE FORM 24 Ġ, ĠIN LINE FORM 25 Ġ, Ġetc . Ġgarg 2012 un super vised Ġreported Ġthat , Ġin Ġpractice , ĠPR s Ġmostly Ġget Ġmapped Ġto Ġcore Ġroles Ġand ĠSR s Ġto Ġmodifier Ġroles , Ġwhich Ġconform s Ġto Ġthe Ġlinguistic Ġmotivations Ġfor Ġthis Ġdistinction . </s> Figure ĠFIG REF 16 Ġillustrates Ġtwo Ġcopies Ġof Ġthe Ġmon oling ual Ġmodel , Ġon Ġeither Ġside Ġof Ġthe Ġcross ling ual Ġlatent Ġvariables . ĠThe Ġgener ative Ġprocess Ġis Ġas Ġfollows : </s> All Ġthe Ġmult in omial Ġand Ġbin omial Ġdistributions Ġhave Ġsymm etric ĠDir ich let Ġand Ġbeta Ġpri ors Ġrespectively . ĠFigure ĠFIG REF 7 Ġgives Ġthe Ġprobability Ġequations Ġfor Ġthe Ġmon oling ual Ġmodel . ĠThis Ġformulation Ġmodels Ġthe Ġglobal Ġrole Ġordering Ġand Ġrepetition Ġpreferences Ġusing ĠPR s , Ġand Ġlimited Ġcontext Ġfor ĠSR s Ġusing Ġintervals . ĠOrder ing Ġand Ġrepetition Ġinformation Ġwas Ġfound Ġto Ġbe Ġhelpful Ġin Ġsupervised ĠS RL Ġas Ġwell ĠB IB REF 9 Ġ, ĠB IB REF 8 Ġ, ĠB IB REF 10 Ġ. ĠMore Ġdetails , Ġincluding Ġthe Ġmotivations Ġbehind Ġthis Ġmodel , Ġare Ġin ĠB IB REF 3 Ġ. </s> Mult ilingual ĠModel </s> The Ġmult ilingual Ġmodel Ġuses Ġword Ġalign ments Ġbetween Ġsentences Ġin Ġa Ġparallel Ġcorpus Ġto Ġexploit Ġrole Ġcorrespond ences Ġacross Ġlanguages . ĠWe Ġmake Ġcopies Ġof Ġthe Ġmon oling ual Ġmodel Ġfor Ġeach Ġlanguage Ġand Ġadd Ġadditional Ġcross ling ual Ġlatent Ġvariables Ġ( CL Vs ) Ġto Ġcouple Ġthe Ġmon oling ual Ġmodels , Ġcapturing Ġcross ling ual Ġsemantic Ġrole Ġpatterns . ĠCon crete ly , Ġwhen Ġtraining Ġon Ġparallel Ġsentences , Ġwhenever Ġthe Ġhead Ġwords Ġof Ġthe Ġarguments Ġare Ġaligned , Ġwe Ġadd Ġa ĠCL V Ġas Ġa Ġparent Ġof Ġthe Ġtwo Ġcorresponding Ġrole Ġvariables . ĠFigure ĠFIG REF 16 Ġillustrates Ġthis Ġmodel . ĠThe Ġgener ative Ġprocess , Ġas Ġexplained Ġbelow , Ġremains Ġthe Ġsame Ġas Ġthe Ġmon oling ual Ġmodel Ġfor Ġthe Ġmost Ġpart , Ġwith Ġthe Ġexception Ġof Ġaligned Ġroles Ġwhich Ġare Ġnow Ġgenerated Ġby Ġboth Ġthe Ġmon oling ual Ġprocess Ġas Ġwell Ġas Ġthe ĠCL V . </s> Every Ġpredicate - t uple Ġhas Ġits Ġown Ġinventory Ġof ĠCL Vs Ġspecific Ġto Ġthat Ġtuple . ĠEach ĠCL V ĠIN LINE FORM 0 Ġis Ġa Ġmulti - valued Ġvariable Ġwhere Ġeach Ġvalue Ġdefines Ġa Ġdistribution Ġover Ġrole Ġlabels Ġfor Ġeach Ġlanguage Ġ( den oted Ġby ĠIN LINE FORM 1 Ġabove ). ĠThese Ġdistributions Ġover Ġlabels Ġare Ġtrained Ġto Ġbe Ġpeak y , Ġso Ġthat Ġeach Ġvalue ĠIN LINE FORM 2 Ġfor Ġa ĠCL V Ġrepresents Ġa Ġcorrelation Ġbetween Ġthe Ġlabels Ġthat ĠIN LINE FORM 3 Ġpredicts Ġin Ġthe Ġtwo Ġlanguages . ĠFor Ġexample , Ġa Ġvalue ĠIN LINE FORM 4 Ġfor Ġthe ĠCL V ĠIN LINE FORM 5 Ġmight Ġgive Ġhigh Ġprobabilities Ġto ĠIN LINE FORM 6 Ġand ĠIN LINE FORM 7 Ġin Ġlanguage Ġ1 , Ġand Ġto ĠIN LINE FORM 8 Ġin Ġlanguage Ġ2 . ĠIf ĠIN LINE FORM 9 Ġis Ġthe Ġonly Ġvalue Ġfor ĠIN LINE FORM 10 Ġthat Ġgives Ġhigh Ġprobability Ġto ĠIN LINE FORM 11 Ġin Ġlanguage Ġ1 , Ġand Ġthe Ġmon oling ual Ġmodel Ġin Ġlanguage Ġ1 Ġdecides Ġto Ġassign ĠIN LINE FORM 12 Ġto Ġthe Ġrole Ġfor ĠIN LINE FORM 13 Ġ, Ġthen ĠIN LINE FORM 14 Ġwill Ġpredict ĠIN LINE FORM 15 Ġin Ġlanguage Ġ2 , Ġwith Ġhigh Ġprobability . ĠWe Ġgenerate Ġthe ĠCL Vs Ġvia Ġa ĠChinese ĠRestaurant ĠProcess ĠB IB REF 11 Ġ, Ġa Ġnon - param etric ĠBay esian Ġmodel , Ġwhich Ġallows Ġus Ġto Ġinduce Ġthe Ġnumber Ġof ĠCL Vs Ġfor Ġevery Ġpredicate - t uple Ġfrom Ġthe Ġdata . ĠWe Ġcontinue Ġto Ġtrain Ġon Ġthe Ġnon - par allel Ġsentences Ġusing Ġthe Ġrespective Ġmon oling ual Ġmodels . </s> The Ġmult ilingual Ġmodel Ġis Ġdeficient , Ġsince Ġthe Ġaligned Ġroles Ġare Ġbeing Ġgenerated Ġtwice . ĠIdeally , Ġwe Ġwould Ġlike Ġto Ġadd Ġthe ĠCL V Ġas Ġadditional Ġconditioning Ġvariables Ġin Ġthe Ġmon oling ual Ġmodels . ĠThe Ġnew Ġjoint Ġprobability Ġcan Ġbe Ġwritten Ġas Ġequation ĠUID 11 Ġ( Figure ĠFIG REF 7 Ġ), Ġwhich Ġcan Ġbe Ġfurther Ġdecomp osed Ġfollowing Ġthe Ġdecom position Ġof Ġthe Ġmon oling ual Ġmodel Ġin ĠFigure ĠFIG REF 7 Ġ. ĠHowever , Ġhaving Ġthis Ġadditional Ġconditioning Ġvariable Ġbreaks Ġthe ĠDir ich let - mult in omial Ġconj ug acy , Ġwhich Ġmakes Ġit Ġint ract able Ġto Ġmarginal ize Ġout Ġthe Ġparameters Ġduring Ġinference . ĠHence , Ġwe Ġuse Ġan Ġapproximation Ġwhere Ġwe Ġtreat Ġeach Ġof Ġthe Ġaligned Ġroles Ġas Ġbeing Ġgenerated Ġtwice , Ġonce Ġby Ġthe Ġmon oling ual Ġmodel Ġand Ġonce Ġby Ġthe Ġcorresponding ĠCL V Ġ( equ ation Ġ). </s> This Ġis Ġthe Ġfirst Ġwork Ġto Ġincorporate Ġthe Ġcoupling Ġof Ġaligned Ġarguments Ġdirectly Ġin Ġa ĠBay esian ĠS RL Ġmodel . ĠThis Ġmakes Ġit Ġeasier Ġto Ġsee Ġhow Ġto Ġextend Ġthis Ġmodel Ġin Ġa Ġprincipled Ġway Ġto Ġincorporate Ġadditional Ġsources Ġof Ġinformation . ĠFirst , Ġthe Ġmodel Ġscales Ġgrace fully Ġto Ġmore Ġthan Ġtwo Ġlanguages . ĠIf Ġthere Ġare Ġa Ġtotal Ġof ĠIN LINE FORM 0 Ġlanguages , Ġand Ġthere Ġis Ġan Ġaligned Ġargument Ġin ĠIN LINE FORM 1 Ġof Ġthem , Ġthe Ġmult ilingual Ġlatent Ġvariable Ġis Ġconnected Ġto Ġonly Ġthose ĠIN LINE FORM 2 Ġaligned Ġarguments . </s> Second , Ġhaving Ġone Ġjoint ĠBay esian Ġmodel Ġallows Ġus Ġto Ġuse Ġthe Ġsame Ġmodel Ġin Ġvarious Ġsemi - super vised Ġlearning Ġsettings , Ġjust Ġby Ġfixing Ġthe Ġannot ated Ġvariables Ġduring Ġtraining . ĠSection ĠSEC REF 29 Ġevaluates Ġa Ġsetting Ġwhere Ġwe Ġhave Ġsome Ġlabeled Ġdata Ġin Ġone Ġlanguage Ġ( called Ġsource ), Ġwhile Ġno Ġlabeled Ġdata Ġin Ġthe Ġsecond Ġlanguage Ġ( called Ġtarget ). ĠNote Ġthat Ġthis Ġis Ġdifferent Ġfrom Ġa Ġclassic Ġannotation Ġprojection Ġsetting Ġ( e . g . ĠB IB REF 12 Ġ), Ġwhere Ġthe Ġrole Ġlabels Ġare Ġmapped Ġfrom Ġsource Ġconstituents Ġto Ġaligned Ġtarget Ġconstituents . </s> In ference Ġand ĠTraining </s> The Ġinference Ġproblem Ġconsists Ġof Ġpredicting Ġthe Ġrole Ġlabels Ġand ĠCL Vs Ġ( the Ġhidden Ġvariables ) Ġgiven Ġthe Ġpredicate , Ġits Ġvoice , Ġand Ġsynt actic Ġfeatures Ġof Ġall Ġthe Ġidentified Ġarguments Ġ( the Ġvisible Ġvariables ). ĠWe Ġuse Ġa Ġcollapsed ĠGibbs - sam pling Ġbased Ġapproach Ġto Ġgenerate Ġsamples Ġfor Ġthe Ġhidden Ġvariables Ġ( model Ġparameters Ġare Ġintegrated Ġout ). ĠThe Ġsample Ġcounts Ġand Ġthe Ġpri ors Ġare Ġthen Ġused Ġto Ġcalculate Ġthe ĠMAP Ġestimate Ġof Ġthe Ġmodel Ġparameters . </s> For Ġthe Ġmon oling ual Ġmodel , Ġthe Ġrole Ġat Ġa Ġgiven Ġposition Ġis Ġsampled Ġas : </s> DIS PLAY FORM 0 </s> where Ġthe Ġsubscript ĠIN LINE FORM 0 Ġrefers Ġto Ġall Ġthe Ġvariables Ġexcept Ġat Ġposition ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġrefers Ġto Ġthe Ġvariables Ġin Ġall Ġthe Ġtraining Ġinstances Ġexcept Ġthe Ġcurrent Ġone , Ġand ĠIN LINE FORM 3 Ġrefers Ġto Ġall Ġthe Ġmodel Ġparameters . ĠThe Ġabove Ġintegral Ġhas Ġa Ġclosed Ġform Ġsolution Ġdue Ġto ĠDir ich let - mult in omial Ġconj ug acy . </s> For Ġsampling Ġroles Ġin Ġthe Ġmult ilingual Ġmodel , Ġwe Ġalso Ġneed Ġto Ġconsider Ġthe Ġprobabilities Ġof Ġroles Ġbeing Ġgenerated Ġby Ġthe ĠCL Vs : </s> DIS PLAY FORM 0 </s> For Ġsampling ĠCL Vs , Ġwe Ġneed Ġto Ġconsider Ġthree Ġfactors : Ġtwo Ġcorresponding Ġto Ġprobabilities Ġof Ġgenerating Ġthe Ġaligned Ġroles , Ġand Ġthe Ġthird Ġone Ġcorresponding Ġto Ġselecting Ġthe ĠCL V Ġaccording Ġto ĠCR P . </s> DIS PLAY FORM 0 </s> where Ġthe Ġaligned Ġroles ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġare Ġconnected Ġto ĠIN LINE FORM 2 Ġ, Ġand ĠIN LINE FORM 3 Ġrefers Ġto Ġall Ġthe Ġvariables Ġexcept ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġ, Ġand ĠIN LINE FORM 6 Ġ. </s> We Ġuse Ġthe Ġtrained Ġparameters Ġto Ġparse Ġthe Ġmon oling ual Ġdata Ġusing Ġthe Ġmon oling ual Ġmodel . ĠThe Ġcross ling ual Ġparameters Ġare Ġignored Ġeven Ġif Ġthey Ġwere Ġused Ġduring Ġtraining . ĠThus , Ġthe Ġinformation Ġcoming Ġfrom Ġthe ĠCL Vs Ġacts Ġas Ġa Ġregular izer Ġfor Ġthe Ġmon oling ual Ġmodels . </s> E val uation </s> Following Ġthe Ġsetting Ġof Ġtit ov cross ling ual , Ġwe Ġevaluate Ġonly Ġon Ġthe Ġarguments Ġthat Ġwere Ġcorrectly Ġidentified , Ġas Ġthe Ġincorrectly Ġidentified Ġarguments Ġdo Ġnot Ġhave Ġany Ġgold Ġsemantic Ġlabels . ĠEvaluation Ġis Ġdone Ġusing Ġthe Ġmetric Ġproposed Ġby Ġlang 2011 un super vised , Ġwhich Ġhas Ġ3 Ġcomponents : Ġ( i ) ĠP urity Ġ( PU ) Ġmeasures Ġhow Ġwell Ġan Ġinduced Ġcluster Ġcorresponds Ġto Ġa Ġsingle Ġgold Ġrole , Ġ( ii ) ĠColl ocation Ġ( CO ) Ġmeasures Ġhow Ġwell Ġa Ġgold Ġrole Ġcorresponds Ġto Ġa Ġsingle Ġinduced Ġcluster , Ġand Ġ( iii ) ĠF 1 Ġis Ġthe Ġharmonic Ġmean Ġof ĠPU Ġand ĠCO . ĠFor Ġeach Ġpredicate , Ġlet ĠIN LINE FORM 0 Ġdenote Ġthe Ġtotal Ġnumber Ġof Ġargument Ġinstances , ĠIN LINE FORM 1 Ġthe Ġinstances Ġin Ġthe Ġinduced Ġcluster ĠIN LINE FORM 2 Ġ, Ġand ĠIN LINE FORM 3 Ġthe Ġinstances Ġhaving Ġlabel ĠIN LINE FORM 4 Ġin Ġgold Ġannotations . ĠIN LINE FORM 5 Ġ, ĠIN LINE FORM 6 Ġ, Ġand ĠIN LINE FORM 7 Ġ. ĠThe Ġscore Ġfor Ġeach Ġpredicate Ġis Ġweighted Ġby Ġthe Ġnumber Ġof Ġits Ġargument Ġinstances , Ġand Ġa Ġweighted Ġaverage Ġis Ġcomputed Ġover Ġall Ġthe Ġpred icates . </s> Bas eline </s> We Ġuse Ġthe Ġsame Ġbaseline Ġas Ġused Ġby Ġlang 2011 un super vised Ġwhich Ġhas Ġbeen Ġshown Ġto Ġbe Ġdifficult Ġto Ġoutper form . ĠThis Ġbaseline Ġassigns Ġa Ġsemantic Ġrole Ġto Ġa Ġconstituent Ġbased Ġon Ġits Ġsynt actic Ġfunction , Ġi . e . Ġthe Ġdependency Ġrelation Ġto Ġits Ġhead . ĠIf Ġthere Ġis Ġa Ġtotal Ġof ĠIN LINE FORM 0 Ġclusters , ĠIN LINE FORM 1 Ġmost Ġfrequent Ġsynt actic Ġfunctions Ġget Ġa Ġcluster Ġeach , Ġand Ġthe Ġrest Ġare Ġassigned Ġto Ġthe ĠIN LINE FORM 2 Ġth Ġcluster . </s> Cl os est ĠPrevious ĠWork </s> This Ġwork Ġis Ġclosely Ġrelated Ġto Ġthe Ġcross - ling ual Ġun super vised ĠS RL Ġwork Ġof Ġtit ov cross ling ual . ĠTheir Ġmodel Ġhas Ġseparate Ġmon oling ual Ġmodels Ġfor Ġeach Ġlanguage Ġand Ġan Ġextra Ġpenalty Ġterm Ġwhich Ġtries Ġto Ġmaximize ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġi . e . Ġfor Ġall Ġthe Ġaligned Ġarguments Ġwith Ġrole Ġlabel ĠIN LINE FORM 2 Ġin Ġlanguage Ġ1 , Ġit Ġtries Ġto Ġfind Ġa Ġrole Ġlabel ĠIN LINE FORM 3 Ġin Ġlanguage Ġ2 Ġsuch Ġthat Ġthe Ġgiven Ġproportion Ġis Ġmaxim ized Ġand Ġvice Ġverse . ĠHowever , Ġthere Ġis Ġno Ġefficient Ġway Ġto Ġoptimize Ġthe Ġobjective Ġwith Ġthis Ġpenalty Ġterm Ġand Ġthe Ġauthors Ġused Ġan Ġinference Ġmethod Ġsimilar Ġto Ġannotation Ġprojection . ĠFurther , Ġthe Ġmethod Ġdoes Ġnot Ġscale Ġnaturally Ġto Ġmore Ġthan Ġtwo Ġlanguages . ĠTheir Ġalgorithm Ġfirst Ġdoes Ġmon oling ual Ġinference Ġin Ġone Ġlanguage Ġignoring Ġthe Ġpenalty Ġand Ġthen Ġdoes Ġthe Ġinference Ġin Ġthe Ġsecond Ġlanguage Ġtaking Ġinto Ġaccount Ġthe Ġpenalty Ġterm . ĠIn Ġcontrast , Ġour Ġmodel Ġadds Ġthe Ġlatent Ġvariables Ġas Ġa Ġpart Ġof Ġthe Ġmodel Ġitself , Ġand Ġnot Ġan Ġexternal Ġpenalty , Ġwhich Ġenables Ġus Ġto Ġuse Ġthe Ġstandard ĠBay esian Ġlearning Ġmethods Ġsuch Ġas Ġsampling . </s> The Ġmon oling ual Ġmodel Ġwe Ġuse ĠB IB REF 3 Ġalso Ġhas Ġtwo Ġmain Ġadvantages Ġover Ġtit ov cross ling ual . ĠFirst , Ġthe Ġformer Ġincorporates Ġa Ġglobal Ġrole Ġordering Ġprobability Ġthat Ġis Ġmissing Ġin Ġthe Ġlatter . ĠSecondly , Ġthe Ġlatter Ġdefines Ġargument - keys Ġas Ġa Ġtuple Ġof Ġfour Ġsynt actic Ġfeatures Ġand Ġall Ġthe Ġarguments Ġhaving Ġthe Ġsame Ġargument - keys Ġare Ġassigned Ġthe Ġsame Ġrole . ĠThis Ġkind Ġof Ġhard Ġclust ering Ġis Ġavoided Ġin Ġthe Ġformer Ġmodel Ġwhere Ġtwo Ġconstituents Ġhaving Ġthe Ġsame Ġset Ġof Ġfeatures Ġmight Ġget Ġassigned Ġdifferent Ġroles Ġif Ġthey Ġappear Ġin Ġdifferent Ġcontexts . </s> Data </s> Following Ġtit ov cross ling ual , Ġwe Ġrun Ġour Ġexperiments Ġon Ġthe ĠEnglish Ġ( EN ) Ġand ĠGerman Ġ( DE ) Ġsections Ġof Ġthe ĠCo N LL Ġ2009 Ġcorpus ĠB IB REF 13 Ġ, Ġand ĠEN - DE Ġsection Ġof Ġthe ĠEuro par l Ġcorpus ĠB IB REF 14 Ġ. ĠWe Ġget Ġabout Ġ40 k ĠEN Ġand Ġ36 k ĠDE Ġsentences Ġfrom Ġthe ĠCo N LL Ġ2009 Ġtraining Ġset , Ġand Ġabout Ġ1 . 5 M Ġparallel ĠEN - DE Ġsentences Ġfrom ĠEuro par l . ĠFor Ġappropriate Ġcomparison , Ġwe Ġkeep Ġthe Ġsame Ġsetting Ġas Ġin ĠB IB REF 6 Ġfor Ġautomatic Ġpars es Ġand Ġargument Ġidentification , Ġwhich Ġwe Ġbriefly Ġdescribe Ġhere . ĠThe ĠEN Ġsentences Ġare Ġparsed Ġsynt act ically Ġusing ĠMalt Parser ĠB IB REF 15 Ġand ĠDE Ġusing ĠL TH Ġparser ĠB IB REF 16 Ġ. ĠAll Ġthe Ġnon - aux iliary Ġverbs Ġare Ġselected Ġas Ġpred icates . ĠIn ĠCo N LL Ġdata , Ġthis Ġgives Ġus Ġabout Ġ3 k ĠEN Ġand Ġ500 ĠDE Ġpred icates . ĠThe Ġtotal Ġnumber Ġof Ġpredicate Ġinstances Ġare Ġ3 . 4 M Ġin ĠEN Ġ( 89 k ĠCo N LL Ġ+ Ġ3 . 3 M ĠEuro par l ) Ġand Ġ2 . 62 M Ġin ĠDE Ġ( 17 k ĠCo N LL Ġ+ Ġ2 . 6 M ĠEuro par l ). ĠThe Ġarguments Ġfor ĠEN Ġare Ġidentified Ġusing Ġthe Ġhe ur istics Ġproposed Ġby Ġlang 2011 un super vised . ĠHowever , Ġwe Ġget Ġan ĠF 1 Ġscore Ġof Ġ85 . 1 % Ġfor Ġargument Ġidentification Ġon ĠCo N LL Ġ2009 ĠEN Ġdata Ġas Ġopposed Ġto Ġ80 . 7 % Ġreported Ġby Ġtit ov cross ling ual . ĠThis Ġcould Ġbe Ġdue Ġto Ġimplementation Ġdifferences , Ġwhich Ġunfortunately Ġmakes Ġour ĠEN Ġresults Ġincom parable . ĠFor ĠDE , Ġthe Ġarguments Ġare Ġidentified Ġusing Ġthe ĠL TH Ġsystem ĠB IB REF 16 Ġ, Ġwhich Ġgives Ġan ĠF 1 Ġscore Ġof Ġ86 . 5 % Ġon Ġthe ĠCo N LL Ġ2009 ĠDE Ġdata . ĠThe Ġword Ġalign ments Ġfor Ġthe ĠEN - DE Ġparallel ĠEuro par l Ġcorpus Ġare Ġcomputed Ġusing ĠG IZ A ++ ĠB IB REF 17 Ġ. ĠFor Ġhigh - pre cision , Ġonly Ġthe Ġintersect ing Ġalign ments Ġin Ġthe Ġtwo Ġdirections Ġare Ġkept . ĠWe Ġdefine Ġtwo Ġsemantic Ġarguments Ġas Ġaligned Ġif Ġtheir Ġhead - words Ġare Ġaligned . ĠIn Ġtotal Ġwe Ġget Ġ9 . 3 M Ġarguments Ġfor ĠEN Ġ( 240 k ĠCo N LL Ġ+ Ġ9 . 1 M ĠEuro par l ) Ġand Ġ4 . 43 M Ġfor ĠDE Ġ( 32 k ĠCo N LL Ġ+ Ġ4 . 4 M ĠEuro par l ). ĠOut Ġof Ġthese , Ġ0 . 76 M Ġarguments Ġare Ġaligned . </s> Main ĠResults </s> Since Ġthe ĠCo N LL Ġannotations Ġhave Ġ21 Ġsemantic Ġroles Ġin Ġtotal , Ġwe Ġuse Ġ21 Ġroles Ġin Ġour Ġmodel Ġas Ġwell Ġas Ġthe Ġbaseline . ĠFollowing Ġgarg 2012 un super vised , Ġwe Ġset Ġthe Ġnumber Ġof ĠPR s Ġto Ġ2 Ġ( excluding ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ), Ġand ĠSR s Ġto Ġ21 - 2 = 19 . ĠTable ĠT AB REF 27 Ġshows Ġthe Ġresults . </s> In Ġthe Ġfirst Ġsetting Ġ( Line Ġ1 ), Ġwe Ġtrain Ġand Ġtest Ġthe Ġmon oling ual Ġmodel Ġon Ġthe ĠCo N LL Ġdata . ĠWe Ġobserve Ġsignificant Ġimprovements Ġin ĠF 1 Ġscore Ġover Ġthe ĠBas eline Ġ( Line Ġ0 ) Ġin Ġboth Ġlanguages . ĠUsing Ġthe ĠCo N LL Ġ2009 Ġdataset Ġalone , Ġtit ov cross ling ual Ġreport Ġan ĠF 1 Ġscore Ġof Ġ80 . 9 % Ġ( PU = 86 . 8 %, ĠCO = 75 . 7 %) Ġfor ĠGerman . ĠThus , Ġour Ġmon oling ual Ġmodel Ġoutper forms Ġtheir Ġmon oling ual Ġmodel Ġin ĠGerman . ĠFor ĠEnglish , Ġthey Ġreport Ġan ĠF 1 Ġscore Ġof Ġ83 . 6 % Ġ( PU = 87 . 5 %, ĠCO = 80 . 1 %), Ġbut Ġnote Ġthat Ġour ĠEnglish Ġresults Ġare Ġnot Ġdirectly Ġcomparable Ġto Ġtheirs Ġdue Ġto Ġdifferences Ġargument Ġidentification , Ġas Ġdiscussed Ġin Ġsection ĠSEC REF 25 Ġ. ĠAs Ġtheir Ġargument Ġidentification Ġscore Ġis Ġlower , Ġperhaps Ġtheir Ġsystem Ġis Ġdisc arding ĠâĢ ľ diff icult âĢ Ŀ Ġarguments Ġwhich Ġleads Ġto Ġa Ġhigher Ġclust ering Ġscore . </s> In Ġthe Ġsecond Ġsetting Ġ( Line Ġ2 ), Ġwe Ġuse Ġthe Ġadditional Ġmon oling ual ĠEuro par l Ġ( EP ) Ġdata Ġfor Ġtraining . ĠWe Ġget Ġequivalent Ġresults Ġin ĠEnglish Ġand Ġa Ġsignificant Ġimprovement Ġin ĠGerman Ġcompared Ġto Ġour Ġprevious Ġsetting Ġ( Line Ġ1 ). ĠThe ĠGerman Ġdataset Ġin ĠCo N LL Ġis Ġquite Ġsmall Ġand Ġbenefits Ġfrom Ġthe Ġadditional ĠEP Ġtraining Ġdata . ĠIn Ġcontrast , Ġthe ĠEnglish Ġmodel Ġis Ġalready Ġquite Ġgood Ġdue Ġto Ġa Ġrelatively Ġbig Ġdataset Ġfrom ĠCo N LL , Ġand Ġgood Ġaccuracy Ġsynt actic Ġpars ers . ĠUnfortunately , Ġtit ov cross ling ual Ġdo Ġnot Ġreport Ġresults Ġwith Ġthis Ġsetting . </s> The Ġthird Ġsetting Ġ( Line Ġ3 ) Ġgives Ġthe Ġresults Ġof Ġour Ġmult ilingual Ġmodel , Ġwhich Ġadds Ġthe Ġword Ġalign ments Ġin Ġthe ĠEP Ġdata . ĠComp aring Ġwith ĠLine Ġ2 , Ġwe Ġget Ġnon - significant Ġimprovements Ġin Ġboth Ġlanguages . Ġtit ov cross ling ual Ġobtain Ġan ĠF 1 Ġscore Ġof Ġ82 . 7 % Ġ( PU = 85 . 0 %, ĠCO = 80 . 6 %) Ġfor ĠGerman , Ġand Ġ83 . 7 % Ġ( PU = 86 . 8 %, ĠCO = 80 . 7 %) Ġfor ĠEnglish . ĠThus , Ġfor ĠGerman , Ġour Ġmult ilingual ĠBay esian Ġmodel Ġis Ġable Ġto Ġcapture Ġthe Ġcross - ling ual Ġpatterns Ġat Ġleast Ġas Ġwell Ġas Ġthe Ġexternal Ġpenalty Ġterm Ġin ĠB IB REF 6 Ġ. ĠWe Ġcannot Ġcompare Ġthe ĠEnglish Ġresults Ġunfortunately Ġdue Ġto Ġdifferences Ġin Ġargument Ġidentification . </s> We Ġalso Ġcompared Ġmon oling ual Ġand Ġbilingual Ġtraining Ġdata Ġusing Ġa Ġsetting Ġthat Ġem ulates Ġthe Ġstandard Ġsupervised Ġsetup Ġof Ġseparate Ġtraining Ġand Ġtest Ġdata Ġsets . ĠWe Ġtrain Ġonly Ġon Ġthe ĠEP Ġdataset Ġand Ġtest Ġon Ġthe ĠCo N LL Ġdataset . ĠLines Ġ4 Ġand Ġ5 Ġof ĠTable ĠT AB REF 27 Ġgive Ġthe Ġresults . ĠThe Ġmult ilingual Ġmodel Ġob tains Ġsmall Ġimprovements Ġin Ġboth Ġlanguages , Ġwhich Ġconfirms Ġthe Ġresults Ġfrom Ġthe Ġstandard Ġun super vised Ġsetup , Ġcomparing Ġlines Ġ2 Ġto Ġ3 . </s> These Ġresults Ġindicate Ġthat Ġlittle Ġinformation Ġcan Ġbe Ġlearned Ġabout Ġsemantic Ġroles Ġfrom Ġthis Ġparallel Ġdata Ġsetup . ĠOne Ġpossible Ġexplanation Ġfor Ġthis Ġresult Ġis Ġthat Ġthe Ġsetup Ġitself Ġis Ġinadequate . ĠGiven Ġthe Ġdefinition Ġof Ġaligned Ġarguments , Ġonly Ġ8 % Ġof ĠEnglish Ġarguments Ġand Ġ17 % Ġof ĠGerman Ġarguments Ġare Ġaligned . ĠThis Ġplus Ġour Ġexperiments Ġsuggest Ġthat Ġimproving Ġthe Ġalignment Ġmodel Ġis Ġa Ġnecessary Ġstep Ġto Ġmaking Ġeffective Ġuse Ġof Ġparallel Ġdata Ġin Ġmult ilingual ĠS RI , Ġfor Ġexample Ġby Ġjoint Ġmodeling Ġwith ĠS RI . ĠWe Ġleave Ġthis Ġexploration Ġto Ġfuture Ġwork . </s> Mult ilingual ĠTraining Ġwith ĠLab eled ĠData Ġfor ĠOne ĠLanguage </s> Another Ġmotivation Ġfor Ġjointly Ġmodeling ĠS RL Ġin Ġmultiple Ġlanguages Ġis Ġthe Ġtransfer Ġof Ġinformation Ġfrom Ġa Ġresource Ġrich Ġlanguage Ġto Ġa Ġresource Ġpoor Ġlanguage . ĠWe Ġevaluated Ġour Ġmodel Ġin Ġa Ġvery Ġgeneral Ġannotation Ġtransfer Ġscenario , Ġwhere Ġwe Ġhave Ġa Ġsmall Ġlabeled Ġdataset Ġfor Ġone Ġlanguage Ġ( source ), Ġand Ġa Ġlarge Ġparallel Ġunl abel ed Ġdataset Ġfor Ġthe Ġsource Ġand Ġanother Ġ( target ) Ġlanguage . ĠWe Ġinvestigate Ġwhether Ġthis Ġsetting Ġimproves Ġthe Ġparameter Ġestimates Ġfor Ġthe Ġtarget Ġlanguage . ĠTo Ġthis Ġend , Ġwe Ġclamp Ġthe Ġrole Ġannotations Ġof Ġthe Ġsource Ġlanguage Ġin Ġthe ĠCo N LL Ġdataset Ġusing Ġa Ġpred efined Ġmapping , Ġand Ġdo Ġnot Ġsample Ġthem Ġduring Ġtraining . ĠThis Ġdata Ġgives Ġus Ġgood Ġparameters Ġfor Ġthe Ġsource Ġlanguage , Ġwhich Ġare Ġused Ġto Ġsample Ġthe Ġroles Ġof Ġthe Ġsource Ġlanguage Ġin Ġthe Ġunl abel ed ĠEuro par l Ġdata . ĠThe ĠCL Vs Ġaim Ġto Ġcapture Ġthis Ġimprovement Ġand Ġthereby Ġimprove Ġsampling Ġand Ġparameter Ġestimates Ġfor Ġthe Ġtarget Ġlanguage . ĠTable ĠT AB REF 28 Ġshows Ġthe Ġresults Ġof Ġthis Ġexperiment . ĠWe Ġobtain Ġsmall Ġimprovements Ġin Ġthe Ġtarget Ġlanguages . ĠAs Ġin Ġthe Ġun super vised Ġsetting , Ġthe Ġsmall Ġpercentage Ġof Ġaligned Ġroles Ġprobably Ġlimits Ġthe Ġimpact Ġof Ġthe Ġcross - ling ual Ġinformation . </s> Label ed ĠData Ġin ĠMon oling ual ĠModel </s> We Ġexplored Ġthe Ġimprovement Ġin Ġthe Ġmon oling ual Ġmodel Ġin Ġa Ġsemi - super vised Ġsetting . ĠTo Ġthis Ġend , Ġwe Ġrandomly Ġselected ĠIN LINE FORM 0 Ġof Ġthe Ġsentences Ġin Ġthe ĠCo N LL Ġdataset Ġas ĠâĢ ľ super vised Ġsentences âĢ Ŀ Ġand Ġthe Ġrest ĠIN LINE FORM 1 Ġwere Ġkept Ġun super vised . ĠNext , Ġwe Ġcl amped Ġthe Ġrole Ġlabels Ġof Ġthe Ġsupervised Ġsentences Ġusing Ġthe Ġpred efined Ġmapping Ġfrom ĠSection ĠSEC REF 29 Ġ. ĠSam pling Ġwas Ġdone Ġon Ġthe Ġun super vised Ġsentences Ġas Ġusual . ĠWe Ġthen Ġmeasured Ġthe Ġclust ering Ġperformance Ġusing Ġthe Ġtrained Ġparameters . </s> To Ġaccess Ġthe Ġcontribution Ġof Ġpartial Ġsupervision Ġbetter , Ġwe Ġconstructed Ġa ĠâĢ ľ super vised Ġbaseline âĢ Ŀ Ġas Ġfollows . ĠFor Ġpred icates Ġseen Ġin Ġthe Ġsupervised Ġsentences , Ġa ĠMAP Ġestimate Ġof Ġthe Ġparameters Ġwas Ġcalculated Ġusing Ġthe Ġpred efined Ġmapping . ĠFor Ġthe Ġunseen Ġpred icates , Ġthe Ġstandard Ġbaseline Ġwas Ġused . </s> Fig ures ĠFIG REF 33 Ġand ĠFIG REF 33 Ġshow Ġthe Ġperformance Ġvariation Ġwith ĠIN LINE FORM 0 Ġ. ĠWe Ġmake Ġthe Ġfollowing Ġobservations : </s> [ left margin = * ] </s> In Ġboth Ġlanguages , Ġat Ġaround ĠIN LINE FORM 0 Ġ, Ġthe Ġsupervised Ġbaseline Ġstarts Ġoutper forming Ġthe Ġsemi - super vised Ġmodel , Ġwhich Ġsuggests Ġthat Ġmanually Ġlabeling Ġabout Ġ10 % Ġof Ġthe Ġsentences Ġis Ġa Ġgood Ġenough Ġalternative Ġto Ġour Ġtraining Ġprocedure . ĠNote Ġthat Ġ10 % Ġamounts Ġto Ġabout Ġ3 . 6 k Ġsentences Ġin ĠGerman Ġand Ġ4 k Ġin ĠEnglish . ĠWe Ġnoticed Ġthat Ġthe Ġproportion Ġof Ġseen Ġpred icates Ġincreases Ġdramatically Ġas Ġwe Ġincrease Ġthe Ġproportion Ġof Ġsupervised Ġsentences . ĠAt Ġ10 % Ġsupervised Ġsentences , Ġthe Ġmodel Ġhas Ġalready Ġseen Ġ63 % Ġof Ġpred icates Ġin ĠGerman Ġand Ġ44 % Ġin ĠEnglish . ĠThis Ġexplains Ġto Ġsome Ġextent Ġwhy Ġonly Ġ10 % Ġlabeled Ġsentences Ġare Ġenough . </s> For ĠGerman , Ġit Ġtakes Ġabout Ġ3 . 5 % Ġor Ġ12 60 Ġsupervised Ġsentences Ġto Ġhave Ġthe Ġsame Ġperformance Ġincrease Ġas Ġ1 . 5 M Ġunl abel ed Ġsentences Ġ( Line Ġ1 Ġto ĠLine Ġ2 Ġin ĠTable ĠT AB REF 27 Ġ). ĠAdding Ġabout Ġ180 Ġmore Ġsupervised Ġsentences Ġalso Ġcovers Ġthe Ġbenefit Ġobtained Ġby Ġalign ments Ġin Ġthe Ġmult ilingual Ġmodel Ġ( Line Ġ2 Ġto ĠLine Ġ3 Ġin ĠTable ĠT AB REF 27 Ġ). ĠThere Ġis Ġno Ġnoticeable Ġperformance Ġdifference Ġin ĠEnglish . </s> We Ġalso Ġevaluated Ġthe Ġperformance Ġvariation Ġon Ġa Ġcompletely Ġunseen ĠCo N LL Ġtest Ġset . ĠSince Ġthe Ġtest Ġset Ġis Ġvery Ġsmall Ġcompared Ġto Ġthe Ġtraining Ġset , Ġthe Ġclust ering Ġevaluation Ġis Ġnot Ġas Ġreliable . ĠNonetheless , Ġwe Ġbroadly Ġobtained Ġthe Ġsame Ġpattern . </s> Related ĠWork </s> As Ġdiscussed Ġin Ġsection ĠSEC REF 24 Ġ, Ġour Ġwork Ġis Ġclosely Ġrelated Ġto Ġthe Ġcross ling ual Ġun super vised ĠS RL Ġwork Ġof Ġtit ov cross ling ual . ĠThe Ġidea Ġof Ġusing Ġsuper ling ual Ġlatent Ġvariables Ġto Ġcapture Ġcross - ling ual Ġinformation Ġwas Ġproposed Ġfor ĠPOS Ġtagging Ġby Ġn ase em 2009 mult ilingual , Ġwhich Ġwe Ġuse Ġhere Ġfor ĠS RL . ĠIn Ġa Ġsemi - super vised Ġsetting , Ġp ado 2009 cross Ġused Ġa Ġgraph Ġbased Ġapproach Ġto Ġtransfer Ġsemantic Ġrole Ġannotations Ġfrom ĠEnglish Ġto ĠGerman . Ġfur sten au 2009 graph Ġused Ġa Ġgraph Ġalignment Ġmethod Ġto Ġmeasure Ġthe Ġsemantic Ġand Ġsynt actic Ġsimilarity Ġbetween Ġdependency Ġtree Ġarguments Ġof Ġknown Ġand Ġunknown Ġverbs . </s> For Ġmon oling ual Ġun super vised ĠS RL , Ġsw ier 2004 un super vised Ġpresented Ġthe Ġfirst Ġwork Ġon Ġa Ġdomain - general Ġcorpus , Ġthe ĠBritish ĠNational ĠCorpus , Ġusing Ġ54 Ġverbs Ġtaken Ġfrom ĠVerb Net . Ġgarg 2012 un super vised Ġproposed Ġa ĠBay esian Ġmodel Ġfor Ġthis Ġproblem Ġthat Ġwe Ġuse Ġhere . Ġtit ov 2012 bay esian Ġalso Ġproposed Ġa Ġclosely Ġrelated ĠBay esian Ġmodel . Ġgren ager 2006 un super vised Ġproposed Ġa Ġgener ative Ġmodel Ġbut Ġtheir Ġparameter Ġspace Ġconsisted Ġof Ġall Ġpossible Ġlink ings Ġof Ġsynt actic Ġconstituents Ġand Ġsemantic Ġroles , Ġwhich Ġmade Ġun super vised Ġlearning Ġdifficult Ġand Ġa Ġseparate Ġlanguage - specific Ġrule Ġbased Ġmethod Ġhad Ġto Ġbe Ġused Ġto Ġconst rain Ġthis Ġspace . ĠOther Ġproposed Ġmodels Ġinclude Ġan Ġiter ative Ġsplit - mer ge Ġalgorithm ĠB IB REF 18 Ġand Ġa Ġgraph - part ition ing Ġbased Ġapproach ĠB IB REF 1 Ġ. Ġmar quez 2008 sem antic Ġprovide Ġa Ġgood Ġoverview Ġof Ġthe Ġsupervised ĠS RL Ġsystems . </s> Con clusions </s> We Ġpropose Ġa ĠBay esian Ġmodel Ġof Ġsemantic Ġrole Ġinduction Ġ( S RI ) Ġthat Ġuses Ġcross ling ual Ġlatent Ġvariables Ġto Ġcapture Ġrole Ġalign ments Ġin Ġparallel Ġcorpor a . ĠThe Ġcross ling ual Ġlatent Ġvariables Ġcapture Ġcorrelations Ġbetween Ġroles Ġin Ġdifferent Ġlanguages , Ġand Ġregular ize Ġthe Ġparameter Ġestimates Ġof Ġthe Ġmon oling ual Ġmodels . ĠBecause Ġthis Ġis Ġa Ġjoint ĠBay esian Ġmodel Ġof Ġmult ilingual ĠS RI , Ġwe Ġcan Ġapply Ġthe Ġsame Ġmodel Ġto Ġa Ġvariety Ġof Ġtraining Ġscenarios Ġjust Ġby Ġchanging Ġthe Ġinference Ġprocedure Ġappropriately . ĠWe Ġevaluate Ġmon oling ual ĠS RI Ġwith Ġa Ġlarge Ġunl abel ed Ġdataset , Ġbilingual ĠS RI Ġwith Ġa Ġparallel Ġcorpus , Ġbilingual ĠS RI Ġwith Ġannotations Ġavailable Ġfor Ġthe Ġsource Ġlanguage , Ġand Ġmon oling ual ĠS RI Ġwith Ġa Ġsmall Ġlabeled Ġdataset . ĠIncreasing Ġthe Ġamount Ġof Ġmon oling ual Ġunl abel ed Ġdata Ġsignificantly Ġimproves ĠS RI Ġin ĠGerman Ġbut Ġnot Ġin ĠEnglish . ĠAdding Ġword Ġalign ments Ġin Ġparallel Ġsentences Ġresults Ġin Ġsmall , Ġnon Ġsignificant Ġimprovements , Ġeven Ġif Ġthere Ġis Ġsome Ġlabeled Ġdata Ġavailable Ġin Ġthe Ġsource Ġlanguage . ĠThis Ġdifficulty Ġin Ġshowing Ġthe Ġusefulness Ġof Ġparallel Ġcorpor a Ġfor ĠS RI Ġmay Ġbe Ġdue Ġto Ġthe Ġcurrent Ġassumptions Ġabout Ġrole Ġalign ments , Ġwhich Ġmean Ġthat Ġonly Ġa Ġsmall Ġpercentage Ġof Ġroles Ġare Ġaligned . ĠFurther Ġanalyses Ġreveals Ġthat Ġannot ating Ġsmall Ġamounts Ġof Ġdata Ġcan Ġeasily Ġoutper form Ġthe Ġperformance Ġgains Ġobtained Ġby Ġadding Ġlarge Ġunl abel ed Ġdataset Ġas Ġwell Ġas Ġadding Ġparallel Ġcorpor a . </s> Future Ġwork Ġincludes Ġtraining Ġon Ġdifferent Ġlanguage Ġpairs , Ġon Ġmore Ġthan Ġtwo Ġlanguages , Ġand Ġwith Ġmore Ġinclusive Ġmodels Ġof Ġrole Ġalignment . </s> Acknowled gments </s> This Ġwork Ġwas Ġfunded Ġby Ġthe ĠSwiss ĠNS F Ġgrant Ġ2000 21 _ 125 137 Ġand ĠEC ĠFP 7 Ġgrant ĠPAR L ANCE .