LSTM model
EXTRACTIVE
BIBREF15, BIBREF19, BIBREF20 
EXTRACTIVE
Inspired by BIBREF12, we integrate in this paper a boundary assembling step into the state-of-the-art LSTM model for Chinese word segmentation, and feed the output into a CRF model for NER, resulting in a 2% absolute improvement on the overall F1 score over current state-of-the-art methods.
Our best model performance with its Precision, Recall, and F1 scores on named entity and nominal mention are shown in Table TABREF5. This best model performance is achieved with a dropout rate of 0.1, and a learning rate of 0.05. Our results are compared with state-of-the-art models BIBREF15, BIBREF19, BIBREF20 on the same Sina Weibo training and test datasets. Our model shows an absolute improvement of 2% for the overall F1 score.
2002.11910
Introduction </s> N amed Ġentity Ġrecognition Ġ( NER ) Ġis Ġa Ġchallenging Ġproblem Ġin ĠNatural ĠLanguage ĠProcessing , Ġand Ġoften Ġserves Ġas Ġan Ġimportant Ġstep Ġfor Ġmany Ġpopular Ġapplications , Ġsuch Ġas Ġinformation Ġextraction Ġand Ġquestion Ġanswering . ĠN ER Ġrequires Ġphrases Ġreferring Ġto Ġentities Ġin Ġtext Ġbe Ġidentified Ġand Ġassigned Ġto Ġparticular Ġentity Ġtypes , Ġthus Ġcan Ġbe Ġnaturally Ġmodeled Ġas Ġa Ġsequence Ġlabeling Ġtask . ĠIn Ġrecent Ġyears , Ġa Ġlot Ġof Ġprogress Ġhas Ġbeen Ġmade Ġon ĠN ER Ġby Ġapplying Ġsequential Ġmodels Ġsuch Ġas Ġconditional Ġrandom Ġfield Ġ( CR F ) Ġor Ġneural Ġnetwork Ġmodels Ġsuch Ġas Ġlong Ġshort - term Ġmemory Ġ( L ST M ) Ġ( e . g ., ĠB IB REF 0 , ĠB IB REF 1 , ĠB IB REF 2 , ĠB IB REF 3 ). ĠYet Ġthis Ġtask Ġstill Ġremains Ġa Ġchallenging Ġone , Ġespecially Ġin Ġsocial Ġmedia Ġdomain Ġsuch Ġas Ġtweets , Ġpartially Ġbecause Ġof Ġinform ality Ġand Ġnoise Ġof Ġsuch Ġtext Ġand Ġlow Ġfrequencies Ġof Ġdistinctive Ġnamed Ġentities ĠB IB REF 4 . </s> Chinese Ġis Ġa Ġlanguage Ġthat Ġconsists Ġof Ġsequential ĠChinese Ġcharacters Ġwithout Ġcapital ization Ġinformation Ġand Ġdelim itation Ġbetween Ġwords . ĠRather Ġthan Ġwords Ġas Ġin ĠEnglish Ġor ĠRomance Ġlanguages , Ġthe Ġequival ence Ġof Ġan ĠEnglish Ġword Ġin ĠChinese Ġmay Ġcontain Ġone Ġor Ġseveral ĠChinese Ġcharacters . ĠThus ĠChinese Ġword Ġsegment ation Ġis Ġneeded Ġas Ġa Ġfirst Ġstep Ġbefore Ġentity Ġmentions Ġcan Ġbe Ġrecognized . ĠThe Ġoutputs Ġof ĠChinese Ġword Ġsegment ation Ġare Ġoften Ġused Ġas Ġfeatures Ġto Ġsupport Ġnamed Ġentity Ġrecognition . ĠIn Ġthe Ġneural Ġnetwork Ġbased Ġmodels , Ġthe Ġboundary Ġinformation Ġcan Ġbe Ġextracted Ġfrom Ġhidden Ġlayers Ġof Ġa ĠChinese Ġword Ġsegment ation Ġmodel Ġ( e . g ., ĠB IB REF 5 , ĠB IB REF 6 ). </s> Re lying Ġon Ġoutputs Ġof ĠChinese Ġword Ġsegment ation Ġhas Ġits Ġown Ġchallenge , Ġbecause ĠChinese Ġword Ġsegment ation Ġis Ġoften Ġinfluenced Ġby Ġthe Ġfollowing Ġfour Ġfactors . ĠFirst , Ġmodels Ġtrained Ġfrom Ġcorpor a Ġin Ġother Ġlanguages , Ġeven Ġif Ġnot Ġlanguage - specific Ġsuch Ġas Ġin ĠB IB REF 3 , Ġcannot Ġbe Ġdirectly Ġapplied Ġto ĠChinese Ġtext . ĠSome Ġefforts Ġhave Ġbeen Ġmade Ġon ĠChinese Ġword Ġsegment ation Ġrecently ĠB IB REF 7 , ĠB IB REF 8 , ĠB IB REF 9 , ĠB IB REF 10 . ĠSecond , Ġdifferences Ġin Ġsegment ation Ġcriteria Ġexist Ġamong Ġlabeled Ġcorpor a . ĠFor Ġinstance , Ġin Ġthe ĠPK U âĢ Ļ s ĠPeople âĢ Ļ s ĠDaily Ġcorpus , Ġa Ġperson 's Ġfamily Ġname Ġand Ġgiven Ġname Ġis Ġsegment ed Ġinto Ġtwo Ġtokens , Ġwhile Ġin Ġthe ĠPenn ĠChinese ĠTree bank Ġcorpus Ġthey Ġare Ġsegment ed Ġinto Ġone ĠB IB REF 11 . ĠThird , Ġfor Ġentity Ġmentions Ġthat Ġare Ġcompound Ġwords , Ġmodels Ġmay Ġseparate Ġthem Ġinto Ġfragmented Ġwords ĠB IB REF 12 . ĠMany ĠChinese Ġwords Ġare Ġalso Ġmorp hem es , Ġand Ġseparated Ġentity Ġmention Ġfragments Ġmay Ġbe Ġfalsely Ġcombined Ġwith Ġadjacent Ġwords . ĠFourth , Ġcurrent Ġsequential Ġmodels Ġperform Ġpoorly Ġin Ġidentifying Ġnamed Ġentities Ġwith Ġlong Ġdependency . ĠYet Ġmany Ġnamed Ġentities Ġexpand Ġover Ġa Ġlong Ġrange , Ġespecially Ġorganization Ġnames Ġand Ġlocation Ġnames . </s> Er rors Ġcaused Ġby ĠChinese Ġword Ġsegment ation Ġcan Ġgenerate Ġa Ġcasc ading Ġfailure , Ġwhich Ġhurt Ġthe Ġperformance Ġof Ġdownstream Ġtasks Ġsuch Ġas ĠN ER . ĠLatest Ġdevelopments Ġin ĠChinese ĠN ER Ġ( e . g ., ĠB IB REF 13 , ĠB IB REF 14 , ĠB IB REF 5 , ĠB IB REF 15 , ĠB IB REF 16 , ĠB IB REF 17 ) Ġhave Ġyet Ġshown Ġlittle Ġfocus Ġon Ġthis Ġissue . ĠB IB REF 12 Ġfound Ġthat Ġby Ġassembling Ġthese Ġsingle Ġwords Ġback Ġtogether , Ġinformation Ġgroups Ġand Ġsentence Ġstructure Ġare Ġbetter Ġreserved , Ġwhich Ġcould Ġbenefit Ġdownstream Ġtasks Ġsuch Ġas ĠN ER . </s> Insp ired Ġby ĠB IB REF 12 , Ġwe Ġintegrate Ġin Ġthis Ġpaper Ġa Ġboundary Ġassembling Ġstep Ġinto Ġthe Ġstate - of - the - art ĠL ST M Ġmodel Ġfor ĠChinese Ġword Ġsegment ation , Ġand Ġfeed Ġthe Ġoutput Ġinto Ġa ĠCR F Ġmodel Ġfor ĠN ER , Ġresulting Ġin Ġa Ġ2 % Ġabsolute Ġimprovement Ġon Ġthe Ġoverall ĠF 1 Ġscore Ġover Ġcurrent Ġstate - of - the - art Ġmethods . </s> This Ġpaper Ġis Ġorganized Ġas Ġfollows . ĠIn ĠSection ĠSEC REF 2 Ġwe Ġdiscuss Ġour Ġmodel , Ġwhich Ġconsists Ġof Ġan ĠL ST M Ġmodule Ġfor ĠChinese Ġword Ġsegment ation , Ġa Ġboundary Ġassembling Ġstep Ġfor Ġmore Ġaccurate Ġword Ġsegment ation , Ġand Ġa ĠCR F Ġmodule Ġfor ĠN ER . ĠWe Ġshow Ġthe Ġexperiment Ġresults Ġand Ġdiscuss Ġthe Ġmodel Ġperformance Ġin ĠSection ĠSEC REF 3 . ĠWe Ġconclude Ġin ĠSection ĠSEC REF 4 . </s> Model </s> Our Ġmodel Ġconsists Ġof Ġthree Ġmodules . ĠA Ġdiagram Ġof Ġthe Ġmodel Ġis Ġshown Ġin ĠFigure ĠFIG REF 1 . ĠCharacters Ġin Ġthe Ġinput Ġtext Ġfor ĠChinese Ġword Ġsegment ation Ġare Ġconverted Ġto Ġvectors Ġthat Ġare Ġused Ġto Ġtrain Ġthe ĠL ST M Ġmodule . ĠOutput Ġof Ġthe ĠL ST M Ġmodule Ġare Ġtransformed Ġby Ġa Ġbiased - linear Ġtransformation Ġto Ġget Ġlikelihood Ġscores Ġof Ġsegment ation Ġlabeling , Ġthen Ġpassed Ġthrough Ġthe Ġboundary Ġassembling Ġmodule . ĠThe Ġupdated Ġboundary Ġinformation Ġis Ġused Ġas Ġfeature Ġinput Ġinto Ġthe ĠCR F Ġfor ĠChinese Ġword Ġsegment ation Ġ( C WS ), Ġtogether Ġwith Ġcharacter - vector Ġsequences . ĠIn Ġeach Ġtraining Ġepoch , ĠCR F Ġfor ĠC WS Ġprovides Ġfeedback Ġinto Ġthe ĠL ST M Ġhidden Ġlayer Ġand Ġthe Ġbiased - linear Ġtransformation Ġto Ġupdate Ġthe Ġhyper - param eters . ĠAnother Ġcorpus Ġfor ĠN ER Ġis Ġthen Ġused Ġto Ġtrain Ġthe ĠL ST M Ġagain , Ġthe Ġhidden Ġvector Ġof Ġwhich Ġ( now Ġcontains Ġsegment ation Ġinformation Ġupdated Ġby Ġthe Ġboundary Ġassembling Ġmethod ) Ġis Ġtaken Ġas Ġfeature Ġinput Ġto ĠCR F Ġfor ĠN ER . ĠLex ical Ġfeatures Ġextracted Ġfrom Ġthe Ġinput Ġtext Ġfor ĠN ER , Ġas Ġwell Ġas Ġthe Ġword Ġembed ding Ġsequence , Ġare Ġalso Ġtaken Ġby Ġthe ĠCR F Ġmodule Ġas Ġinput Ġto Ġgenerate ĠN ER Ġlabels . ĠThis Ġsection Ġprovides Ġdescriptions Ġfor Ġeach Ġmodule . </s> Model Ġ: :: ĠL ST M Ġfor ĠWord ĠSe gment ation </s> We Ġchoose Ġan ĠL ST M Ġmodule Ġfor Ġthe ĠC WS Ġtask . ĠRaw Ġinput ĠChinese Ġtext Ġis Ġconverted Ġfrom Ġcharacters Ġto Ġvectors Ġwith Ġcharacter - pos itional Ġinput Ġembed d ings Ġpre - trained Ġby ĠB IB REF 5 Ġover Ġ112 , 9 71 , 7 34 ĠWe ibo Ġmessages Ġusing Ġword 2 vec ĠB IB REF 18 . ĠDet ailed Ġparameter Ġsettings Ġcan Ġbe Ġfound Ġin ĠB IB REF 13 . ĠThe Ġembed d ings Ġcontain Ġ52 , 057 Ġunique Ġcharacters Ġin Ġa Ġ100 - dimension Ġspace . </s> The ĠL ST M Ġmodule Ġtakes Ġthese Ġvectors Ġinto Ġa Ġsingle Ġlayer Ġthat Ġcontains Ġ150 Ġnodes , Ġand Ġmod ifies Ġthem Ġinto Ġlikelihood Ġscores Ġfor Ġeach Ġsegment ation Ġlabel . ĠA Ġbiased - linear Ġtransformation Ġis Ġcarried Ġout Ġon Ġthese Ġlikelihood Ġscores , Ġgenerating Ġpredicted Ġlabels Ġfor Ġsegment ation . ĠThese Ġlabels Ġare Ġthen Ġmodified Ġby Ġthe ĠBound ary ĠAs semb ling Ġmodule , Ġwhich Ġwe Ġwill Ġdiscuss Ġin Ġdetail Ġin Ġthe Ġnext Ġsection . ĠBecause Ġlabels Ġare Ġin Ġsequence , Ġand Ġdependency Ġmay Ġexist Ġamong Ġadjacent Ġlabels , Ġa Ġtransition Ġprobability Ġmatrix Ġis Ġintroduced . ĠThe Ġlikelihood Ġscore Ġtogether Ġwith Ġthe Ġtransition Ġscore Ġare Ġtaken Ġby Ġa ĠCR F Ġmodule Ġwith Ġa Ġmaximum - like lihood Ġtraining Ġobjective . ĠFeed backs Ġbased Ġon Ġthe Ġloss Ġfunction Ġof Ġthe ĠCR F Ġare Ġthen Ġgiven Ġto Ġthe ĠL ST M 's Ġhidden Ġlayer Ġand Ġthe Ġbiased - linear Ġtransformation 's Ġparameters Ġfor Ġupdate . </s> Model Ġ: :: ĠBound ary ĠAs semb ling ĠMethod </s> In Ġeach Ġsentence , ĠChinese Ġcharacters Ġare Ġlabeled Ġas Ġeither ĠBegin , ĠInside , ĠEnd , Ġor ĠSing leton Ġ( BI ES Ġlabeling ). ĠThe Ġlikelihood Ġof Ġindividual ĠChinese Ġcharacters Ġbeing Ġlabeled Ġas Ġeach Ġtype Ġis Ġcalculated Ġby Ġthe ĠL ST M Ġmodule Ġdescribed Ġin Ġthe Ġprevious Ġsection . ĠB IB REF 12 Ġfound Ġin Ġa ĠChinese Ġcorpus Ġthat Ġthe Ġword Ġlabel Ġ" End " Ġhas Ġa Ġbetter Ġperformance Ġthan Ġ" Begin ". ĠThis Ġmotiv ates Ġus Ġto Ġcarry Ġout Ġa Ġbackward Ġgreedy Ġsearch Ġover Ġeach Ġsentence 's Ġlabel Ġsequence Ġto Ġidentify Ġword Ġboundaries . ĠIf Ġtwo Ġwords Ġsegment ed Ġin Ġa Ġsentence Ġare Ġidentified Ġas Ġnoun s , Ġand Ġone Ġword Ġis Ġimmediately Ġbefore Ġthe Ġother , Ġwe Ġassemble Ġtheir Ġboundaries , Ġcreating Ġa Ġnew Ġword Ġcandidate Ġfor Ġentity Ġrecognition . ĠThis Ġstrategy Ġhas Ġthe Ġadvantage Ġto Ġfind Ġnamed Ġentities Ġwith Ġlong Ġword Ġlength . ĠIt Ġalso Ġreduces Ġthe Ġinfluence Ġcaused Ġby Ġdifferent Ġsegment ation Ġcriteria . </s> Model Ġ: :: ĠCR F Ġfor ĠNamed ĠEntity ĠRecogn ition </s> A Ġlog - bil in ear ĠCR F Ġmodule Ġis Ġused Ġfor Ġthe ĠN ER Ġtask , Ġand Ġtakes Ġthree Ġinputs . ĠThe Ġfirst Ġis Ġthe Ġsequential Ġcharacter - pos itional Ġembed d ings Ġmentioned Ġabove . ĠThe Ġsecond Ġis Ġthe Ġhidden Ġvector Ġfrom ĠL ST M Ġas Ġdynamic Ġfeature Ġinputs . ĠThe Ġthird Ġis Ġlex ical Ġfeatures Ġextracted Ġfrom Ġthe Ġinput Ġtext . ĠThese Ġlex ical Ġfeatures Ġare Ġthe Ġlikelihood Ġof Ġa Ġcharacter Ġbeing Ġat Ġa Ġspecific Ġposition Ġof Ġa Ġnoun Ġ( first Ġcharacter Ġof Ġa Ġnoun , Ġsecond Ġcharacter Ġof Ġa Ġnoun , Ġetc .), Ġand Ġis Ġachieved Ġby Ġcomparing Ġthe Ġcharacter Ġwith Ġa Ġpre - defined Ġdictionary Ġtrained Ġby ĠB IB REF 13 . </s> Exper iments Ġ: :: ĠDat as ets </s> Dat as ets Ġused Ġin Ġthis Ġstudy Ġfor Ġtraining , Ġvalidation , Ġand Ġtest Ġare Ġthe Ġsame Ġas Ġused Ġin ĠPeng Ġet Ġal . Ġp eng 2016 impro ving Ġfor Ġboth Ġword Ġsegment ation Ġand Ġnamed Ġentity Ġrecognition . ĠSpecifically , Ġdataset Ġfor Ġword Ġsegment ation Ġis Ġtaken Ġfrom Ġthe ĠS IGH AN Ġ2005 Ġbake off ĠPK U Ġcorpus ĠB IB REF 21 , Ġwhich Ġincludes Ġ123 , 530 Ġsentences Ġfor Ġtraining Ġand Ġ11 , 697 Ġsentences Ġfor Ġtesting . ĠDat as et Ġfor Ġnamed Ġentity Ġrecognition Ġis Ġa Ġcorpus Ġcomposed Ġof Ġ1 , 8 90 ĠSina ĠWe ibo Ġ( a ĠChinese Ġsocial Ġmedia Ġwebsite ) Ġmessages , Ġwith Ġ1 , 350 Ġmessages Ġsplit Ġinto Ġtraining Ġset , Ġ270 Ġinto Ġvalidation Ġset , Ġand Ġ270 Ġinto Ġtest Ġset ĠB IB REF 5 . ĠBoth Ġnamed Ġentity Ġand Ġnominal Ġmention Ġare Ġannot ated , Ġeach Ġwith Ġfour Ġentity Ġtypes : Ġperson , Ġorganization , Ġlocation , Ġand Ġgeo - political Ġentity . ĠA Ġmajor Ġcleanup Ġand Ġrevision Ġof Ġannotations Ġof Ġthis Ġcorpus Ġhas Ġbeen Ġperformed Ġby ĠHe Ġand ĠSun ĠHe S 16 . ĠIn Ġthis Ġstudy , Ġall Ġresults Ġfor Ġcomparisons Ġare Ġbased Ġon Ġthis Ġupdated Ġcorpus . </s> Exper iments Ġ: :: ĠTraining ĠSettings </s> The Ġweights Ġand Ġhyper - param eters Ġin Ġthe ĠL ST M Ġmodule Ġare Ġadjusted Ġin Ġeach Ġiteration Ġusing Ġsto ch astic Ġgradient Ġdescent Ġfor Ġtwo Ġstages Ġin Ġtandem . ĠThe Ġfirst Ġstage Ġis Ġbased Ġon Ġthe ĠC WS Ġinput Ġtext , Ġand Ġthe Ġsecond Ġstage Ġon ĠN ER Ġinput Ġtext . ĠSince Ġthe Ġcorpus Ġfor ĠC WS Ġis Ġmuch Ġlarger Ġthan Ġthe Ġcorpus Ġfor ĠN ER , Ġthe Ġformer Ġcorpus Ġis Ġrandomly Ġsub - sam pled Ġduring Ġeach Ġtraining Ġepoch , Ġwith Ġeach Ġsample Ġcontaining Ġ13 , 500 Ġsentences Ġfor Ġtraining Ġstep Ġone , Ġand Ġ1 , 350 Ġsentences Ġfor Ġtraining Ġstep Ġtwo . ĠModels Ġare Ġtrained Ġuntil Ġthe ĠF 1 Ġscore Ġof ĠN ER Ġmodel Ġconver ges Ġon Ġthe Ġvalidation Ġdataset , Ġor Ġup Ġto Ġ30 Ġepoch s . </s> Mod els Ġare Ġtrained Ġon Ġa ĠWindows ĠPC Ġwith Ġa Ġ2 . 7 ĠGHz ĠIntel ĠCore Ġi 7 ĠCPU . ĠFor Ġthe Ġbest Ġperformed Ġmodel , Ġthe Ġaverage Ġtraining Ġtime Ġfor Ġthe ĠL ST M Ġmodule Ġis Ġ1897 . 6 Ġseconds Ġper Ġiteration . ĠTime Ġfor Ġdata Ġloading , Ġpre - processing , Ġor Ġmodel Ġevaluation Ġis Ġnot Ġincluded . </s> Exper iments Ġ: :: ĠResults Ġand ĠDiscussion </s> Our Ġbest Ġmodel Ġperformance Ġwith Ġits ĠPrecision , ĠRecall , Ġand ĠF 1 Ġscores Ġon Ġnamed Ġentity Ġand Ġnominal Ġmention Ġare Ġshown Ġin ĠTable ĠT AB REF 5 . ĠThis Ġbest Ġmodel Ġperformance Ġis Ġachieved Ġwith Ġa Ġdrop out Ġrate Ġof Ġ0 . 1 , Ġand Ġa Ġlearning Ġrate Ġof Ġ0 . 05 . ĠOur Ġresults Ġare Ġcompared Ġwith Ġstate - of - the - art Ġmodels ĠB IB REF 15 , ĠB IB REF 19 , ĠB IB REF 20 Ġon Ġthe Ġsame ĠSina ĠWe ibo Ġtraining Ġand Ġtest Ġdatasets . ĠOur Ġmodel Ġshows Ġan Ġabsolute Ġimprovement Ġof Ġ2 % Ġfor Ġthe Ġoverall ĠF 1 Ġscore . </s> This Ġsignificant Ġimprovement Ġvalid ates Ġour Ġmethod Ġof Ġapplying Ġboundary Ġassembling Ġto Ġthe Ġsegment ed Ġsentences , Ġwhich Ġresults Ġin Ġmore Ġaccurate Ġword Ġsegment ation Ġand Ġbetter Ġsemantic Ġunderstanding . ĠSent ences Ġin Ġthe ĠPK U Ġcorpus Ġare Ġoften Ġsegment ed Ġinto Ġthe Ġsmallest Ġword Ġunits . ĠThis Ġresults Ġin Ġtoo Ġfragmented Ġinformation Ġand Ġincorrect Ġlex ical Ġunits Ġwhen Ġinput Ġinto Ġa Ġnamed Ġentity Ġrecognition Ġmodel , Ġalthough Ġthis Ġmay Ġbenefit Ġsome Ġother Ġnatural Ġlanguage Ġprocess Ġtasks . ĠBy Ġapplying Ġthe Ġboundary Ġassembling Ġmethod , Ġsense Ġgroup Ġin Ġa Ġsentence Ġis Ġbetter Ġpreserved . ĠThe Ġdownstream ĠN ER Ġmodel Ġcan Ġthen Ġtake Ġadvantage Ġof Ġthis , Ġimproving Ġits Ġresult . </s> The ĠPK U Ġcorpus Ġused Ġfor ĠC WS Ġmodule Ġconsists Ġof Ġmainly Ġnews Ġarticles , Ġwhich Ġare Ġquite Ġdifferent Ġfrom Ġthe Ġsocial Ġmedia ĠWe ibo Ġcorpus . ĠPerformance Ġof Ġan ĠN LP Ġtask Ġoften Ġdrops Ġwhen Ġtested Ġon Ġa Ġdifferent Ġdomain Ġor Ġa Ġcorpus Ġof Ġdifferent Ġcharacteristics . ĠOur Ġimprovement Ġindicates Ġthat Ġthe Ġboundary Ġassembling Ġmethod Ġis Ġnot Ġsensitive Ġto Ġthe Ġspecific Ġdomain , Ġand Ġis Ġa Ġrobust Ġmethod Ġfor Ġcross - domain Ġscenarios . </s> The Ġidentification Ġof Ġtwo Ġnoun s Ġthat Ġare Ġnext Ġto Ġeach Ġother Ġdepends Ġon Ġthe Ġpre - trained Ġlex ical Ġfeatures . ĠIf Ġour Ġmodel Ġis Ġtested Ġover Ġout - of - voc abulary Ġdataset , Ġit Ġmay Ġnot Ġperform Ġwell Ġdue Ġto Ġthe Ġlack Ġof Ġthis Ġlex ical Ġinformation . </s> Conclusion </s> In Ġthis Ġpaper Ġwe Ġintegrate Ġa Ġboundary Ġassembling Ġstep Ġwith Ġan ĠL ST M Ġmodule Ġand Ġa ĠCR F Ġmodule Ġfor ĠNamed ĠEntity ĠRecogn ition Ġin ĠChinese Ġsocial Ġmedia Ġtext . ĠWith Ġthe Ġabundance Ġof Ġsocial Ġmedia Ġinformation , Ġour Ġwork Ġis Ġtimely Ġand Ġdesirable . ĠThe Ġimprovement Ġin Ġexperiment Ġresults Ġover Ġexisting Ġmethods Ġclearly Ġshows Ġthe Ġeffectiveness Ġof Ġour Ġapproach . </s> Acknowled gments </s> This Ġresearch Ġwas Ġpartially Ġfunded Ġby Ġthe ĠEngineering ĠDirectorate Ġof Ġthe ĠNational ĠScience ĠFoundation Ġ( 18 201 18 ).
Introduction Named entity recognition (NER) is a challenging problem in Natural Language Processing, and often serves as an important step for many popular applications, such as information extraction and question answering. NER requires phrases referring to entities in text be identified and assigned to particular entity types, thus can be naturally modeled as a sequence labeling task. In recent years, a lot of progress has been made on NER by applying sequential models such as conditional random field (CRF) or neural network models such as long short-term memory (LSTM) (e.g., BIBREF0, BIBREF1, BIBREF2, BIBREF3). Yet this task still remains a challenging one, especially in social media domain such as tweets, partially because of informality and noise of such text and low frequencies of distinctive named entities BIBREF4. Chinese is a language that consists of sequential Chinese characters without capitalization information and delimitation between words. Rather than words as in English or Romance languages, the equivalence of an English word in Chinese may contain one or several Chinese characters. Thus Chinese word segmentation is needed as a first step before entity mentions can be recognized. The outputs of Chinese word segmentation are often used as features to support named entity recognition. In the neural network based models, the boundary information can be extracted from hidden layers of a Chinese word segmentation model (e.g., BIBREF5, BIBREF6). Relying on outputs of Chinese word segmentation has its own challenge, because Chinese word segmentation is often influenced by the following four factors. First, models trained from corpora in other languages, even if not language-specific such as in BIBREF3, cannot be directly applied to Chinese text. Some efforts have been made on Chinese word segmentation recently BIBREF7, BIBREF8, BIBREF9, BIBREF10. Second, differences in segmentation criteria exist among labeled corpora. For instance, in the PKU’s People’s Daily corpus, a person's family name and given name is segmented into two tokens, while in the Penn Chinese Treebank corpus they are segmented into one BIBREF11. Third, for entity mentions that are compound words, models may separate them into fragmented words BIBREF12. Many Chinese words are also morphemes, and separated entity mention fragments may be falsely combined with adjacent words. Fourth, current sequential models perform poorly in identifying named entities with long dependency. Yet many named entities expand over a long range, especially organization names and location names. Errors caused by Chinese word segmentation can generate a cascading failure, which hurt the performance of downstream tasks such as NER. Latest developments in Chinese NER (e.g., BIBREF13, BIBREF14, BIBREF5, BIBREF15, BIBREF16, BIBREF17) have yet shown little focus on this issue. BIBREF12 found that by assembling these single words back together, information groups and sentence structure are better reserved, which could benefit downstream tasks such as NER. Inspired by BIBREF12, we integrate in this paper a boundary assembling step into the state-of-the-art LSTM model for Chinese word segmentation, and feed the output into a CRF model for NER, resulting in a 2% absolute improvement on the overall F1 score over current state-of-the-art methods. This paper is organized as follows. In Section SECREF2 we discuss our model, which consists of an LSTM module for Chinese word segmentation, a boundary assembling step for more accurate word segmentation, and a CRF module for NER. We show the experiment results and discuss the model performance in Section SECREF3. We conclude in Section SECREF4. Model Our model consists of three modules. A diagram of the model is shown in Figure FIGREF1. Characters in the input text for Chinese word segmentation are converted to vectors that are used to train the LSTM module. Output of the LSTM module are transformed by a biased-linear transformation to get likelihood scores of segmentation labeling, then passed through the boundary assembling module. The updated boundary information is used as feature input into the CRF for Chinese word segmentation (CWS), together with character-vector sequences. In each training epoch, CRF for CWS provides feedback into the LSTM hidden layer and the biased-linear transformation to update the hyper-parameters. Another corpus for NER is then used to train the LSTM again, the hidden vector of which (now contains segmentation information updated by the boundary assembling method) is taken as feature input to CRF for NER. Lexical features extracted from the input text for NER, as well as the word embedding sequence, are also taken by the CRF module as input to generate NER labels. This section provides descriptions for each module. Model ::: LSTM for Word Segmentation We choose an LSTM module for the CWS task. Raw input Chinese text is converted from characters to vectors with character-positional input embeddings pre-trained by BIBREF5 over 112,971,734 Weibo messages using word2vec BIBREF18. Detailed parameter settings can be found in BIBREF13. The embeddings contain 52,057 unique characters in a 100-dimension space. The LSTM module takes these vectors into a single layer that contains 150 nodes, and modifies them into likelihood scores for each segmentation label. A biased-linear transformation is carried out on these likelihood scores, generating predicted labels for segmentation. These labels are then modified by the Boundary Assembling module, which we will discuss in detail in the next section. Because labels are in sequence, and dependency may exist among adjacent labels, a transition probability matrix is introduced. The likelihood score together with the transition score are taken by a CRF module with a maximum-likelihood training objective. Feedbacks based on the loss function of the CRF are then given to the LSTM's hidden layer and the biased-linear transformation's parameters for update. Model ::: Boundary Assembling Method In each sentence, Chinese characters are labeled as either Begin, Inside, End, or Singleton (BIES labeling). The likelihood of individual Chinese characters being labeled as each type is calculated by the LSTM module described in the previous section. BIBREF12 found in a Chinese corpus that the word label "End" has a better performance than "Begin". This motivates us to carry out a backward greedy search over each sentence's label sequence to identify word boundaries. If two words segmented in a sentence are identified as nouns, and one word is immediately before the other, we assemble their boundaries, creating a new word candidate for entity recognition. This strategy has the advantage to find named entities with long word length. It also reduces the influence caused by different segmentation criteria. Model ::: CRF for Named Entity Recognition A log-bilinear CRF module is used for the NER task, and takes three inputs. The first is the sequential character-positional embeddings mentioned above. The second is the hidden vector from LSTM as dynamic feature inputs. The third is lexical features extracted from the input text. These lexical features are the likelihood of a character being at a specific position of a noun (first character of a noun, second character of a noun, etc.), and is achieved by comparing the character with a pre-defined dictionary trained by BIBREF13. Experiments ::: Datasets Datasets used in this study for training, validation, and test are the same as used in Peng et al. peng2016improving for both word segmentation and named entity recognition. Specifically, dataset for word segmentation is taken from the SIGHAN 2005 bakeoff PKU corpus BIBREF21, which includes 123,530 sentences for training and 11,697 sentences for testing. Dataset for named entity recognition is a corpus composed of 1,890 Sina Weibo (a Chinese social media website) messages, with 1,350 messages split into training set, 270 into validation set, and 270 into test set BIBREF5. Both named entity and nominal mention are annotated, each with four entity types: person, organization, location, and geo-political entity. A major cleanup and revision of annotations of this corpus has been performed by He and Sun HeS16. In this study, all results for comparisons are based on this updated corpus. Experiments ::: Training Settings The weights and hyper-parameters in the LSTM module are adjusted in each iteration using stochastic gradient descent for two stages in tandem. The first stage is based on the CWS input text, and the second stage on NER input text. Since the corpus for CWS is much larger than the corpus for NER, the former corpus is randomly sub-sampled during each training epoch, with each sample containing 13,500 sentences for training step one, and 1,350 sentences for training step two. Models are trained until the F1 score of NER model converges on the validation dataset, or up to 30 epochs. Models are trained on a Windows PC with a 2.7 GHz Intel Core i7 CPU. For the best performed model, the average training time for the LSTM module is 1897.6 seconds per iteration. Time for data loading, pre-processing, or model evaluation is not included. Experiments ::: Results and Discussion Our best model performance with its Precision, Recall, and F1 scores on named entity and nominal mention are shown in Table TABREF5. This best model performance is achieved with a dropout rate of 0.1, and a learning rate of 0.05. Our results are compared with state-of-the-art models BIBREF15, BIBREF19, BIBREF20 on the same Sina Weibo training and test datasets. Our model shows an absolute improvement of 2% for the overall F1 score. This significant improvement validates our method of applying boundary assembling to the segmented sentences, which results in more accurate word segmentation and better semantic understanding. Sentences in the PKU corpus are often segmented into the smallest word units. This results in too fragmented information and incorrect lexical units when input into a named entity recognition model, although this may benefit some other natural language process tasks. By applying the boundary assembling method, sense group in a sentence is better preserved. The downstream NER model can then take advantage of this, improving its result. The PKU corpus used for CWS module consists of mainly news articles, which are quite different from the social media Weibo corpus. Performance of an NLP task often drops when tested on a different domain or a corpus of different characteristics. Our improvement indicates that the boundary assembling method is not sensitive to the specific domain, and is a robust method for cross-domain scenarios. The identification of two nouns that are next to each other depends on the pre-trained lexical features. If our model is tested over out-of-vocabulary dataset, it may not perform well due to the lack of this lexical information. Conclusion In this paper we integrate a boundary assembling step with an LSTM module and a CRF module for Named Entity Recognition in Chinese social media text. With the abundance of social media information, our work is timely and desirable. The improvement in experiment results over existing methods clearly shows the effectiveness of our approach. Acknowledgments This research was partially funded by the Engineering Directorate of the National Science Foundation (1820118).
What state-of-the-art deep neural network is used?
23e16c1173b7def2c5cb56053b57047c9971e3bb
What Ġstate - of - the - art Ġdeep Ġneural Ġnetwork Ġis Ġused ?
L ST M Ġmodel
<s> What Ġstate - of - the - art Ġdeep Ġneural Ġnetwork Ġis Ġused ? </s> Introduction </s> N amed Ġentity Ġrecognition Ġ( NER ) Ġis Ġa Ġchallenging Ġproblem Ġin ĠNatural ĠLanguage ĠProcessing , Ġand Ġoften Ġserves Ġas Ġan Ġimportant Ġstep Ġfor Ġmany Ġpopular Ġapplications , Ġsuch Ġas Ġinformation Ġextraction Ġand Ġquestion Ġanswering . ĠN ER Ġrequires Ġphrases Ġreferring Ġto Ġentities Ġin Ġtext Ġbe Ġidentified Ġand Ġassigned Ġto Ġparticular Ġentity Ġtypes , Ġthus Ġcan Ġbe Ġnaturally Ġmodeled Ġas Ġa Ġsequence Ġlabeling Ġtask . ĠIn Ġrecent Ġyears , Ġa Ġlot Ġof Ġprogress Ġhas Ġbeen Ġmade Ġon ĠN ER Ġby Ġapplying Ġsequential Ġmodels Ġsuch Ġas Ġconditional Ġrandom Ġfield Ġ( CR F ) Ġor Ġneural Ġnetwork Ġmodels Ġsuch Ġas Ġlong Ġshort - term Ġmemory Ġ( L ST M ) Ġ( e . g ., ĠB IB REF 0 , ĠB IB REF 1 , ĠB IB REF 2 , ĠB IB REF 3 ). ĠYet Ġthis Ġtask Ġstill Ġremains Ġa Ġchallenging Ġone , Ġespecially Ġin Ġsocial Ġmedia Ġdomain Ġsuch Ġas Ġtweets , Ġpartially Ġbecause Ġof Ġinform ality Ġand Ġnoise Ġof Ġsuch Ġtext Ġand Ġlow Ġfrequencies Ġof Ġdistinctive Ġnamed Ġentities ĠB IB REF 4 . </s> Chinese Ġis Ġa Ġlanguage Ġthat Ġconsists Ġof Ġsequential ĠChinese Ġcharacters Ġwithout Ġcapital ization Ġinformation Ġand Ġdelim itation Ġbetween Ġwords . ĠRather Ġthan Ġwords Ġas Ġin ĠEnglish Ġor ĠRomance Ġlanguages , Ġthe Ġequival ence Ġof Ġan ĠEnglish Ġword Ġin ĠChinese Ġmay Ġcontain Ġone Ġor Ġseveral ĠChinese Ġcharacters . ĠThus ĠChinese Ġword Ġsegment ation Ġis Ġneeded Ġas Ġa Ġfirst Ġstep Ġbefore Ġentity Ġmentions Ġcan Ġbe Ġrecognized . ĠThe Ġoutputs Ġof ĠChinese Ġword Ġsegment ation Ġare Ġoften Ġused Ġas Ġfeatures Ġto Ġsupport Ġnamed Ġentity Ġrecognition . ĠIn Ġthe Ġneural Ġnetwork Ġbased Ġmodels , Ġthe Ġboundary Ġinformation Ġcan Ġbe Ġextracted Ġfrom Ġhidden Ġlayers Ġof Ġa ĠChinese Ġword Ġsegment ation Ġmodel Ġ( e . g ., ĠB IB REF 5 , ĠB IB REF 6 ). </s> Re lying Ġon Ġoutputs Ġof ĠChinese Ġword Ġsegment ation Ġhas Ġits Ġown Ġchallenge , Ġbecause ĠChinese Ġword Ġsegment ation Ġis Ġoften Ġinfluenced Ġby Ġthe Ġfollowing Ġfour Ġfactors . ĠFirst , Ġmodels Ġtrained Ġfrom Ġcorpor a Ġin Ġother Ġlanguages , Ġeven Ġif Ġnot Ġlanguage - specific Ġsuch Ġas Ġin ĠB IB REF 3 , Ġcannot Ġbe Ġdirectly Ġapplied Ġto ĠChinese Ġtext . ĠSome Ġefforts Ġhave Ġbeen Ġmade Ġon ĠChinese Ġword Ġsegment ation Ġrecently ĠB IB REF 7 , ĠB IB REF 8 , ĠB IB REF 9 , ĠB IB REF 10 . ĠSecond , Ġdifferences Ġin Ġsegment ation Ġcriteria Ġexist Ġamong Ġlabeled Ġcorpor a . ĠFor Ġinstance , Ġin Ġthe ĠPK U âĢ Ļ s ĠPeople âĢ Ļ s ĠDaily Ġcorpus , Ġa Ġperson 's Ġfamily Ġname Ġand Ġgiven Ġname Ġis Ġsegment ed Ġinto Ġtwo Ġtokens , Ġwhile Ġin Ġthe ĠPenn ĠChinese ĠTree bank Ġcorpus Ġthey Ġare Ġsegment ed Ġinto Ġone ĠB IB REF 11 . ĠThird , Ġfor Ġentity Ġmentions Ġthat Ġare Ġcompound Ġwords , Ġmodels Ġmay Ġseparate Ġthem Ġinto Ġfragmented Ġwords ĠB IB REF 12 . ĠMany ĠChinese Ġwords Ġare Ġalso Ġmorp hem es , Ġand Ġseparated Ġentity Ġmention Ġfragments Ġmay Ġbe Ġfalsely Ġcombined Ġwith Ġadjacent Ġwords . ĠFourth , Ġcurrent Ġsequential Ġmodels Ġperform Ġpoorly Ġin Ġidentifying Ġnamed Ġentities Ġwith Ġlong Ġdependency . ĠYet Ġmany Ġnamed Ġentities Ġexpand Ġover Ġa Ġlong Ġrange , Ġespecially Ġorganization Ġnames Ġand Ġlocation Ġnames . </s> Er rors Ġcaused Ġby ĠChinese Ġword Ġsegment ation Ġcan Ġgenerate Ġa Ġcasc ading Ġfailure , Ġwhich Ġhurt Ġthe Ġperformance Ġof Ġdownstream Ġtasks Ġsuch Ġas ĠN ER . ĠLatest Ġdevelopments Ġin ĠChinese ĠN ER Ġ( e . g ., ĠB IB REF 13 , ĠB IB REF 14 , ĠB IB REF 5 , ĠB IB REF 15 , ĠB IB REF 16 , ĠB IB REF 17 ) Ġhave Ġyet Ġshown Ġlittle Ġfocus Ġon Ġthis Ġissue . ĠB IB REF 12 Ġfound Ġthat Ġby Ġassembling Ġthese Ġsingle Ġwords Ġback Ġtogether , Ġinformation Ġgroups Ġand Ġsentence Ġstructure Ġare Ġbetter Ġreserved , Ġwhich Ġcould Ġbenefit Ġdownstream Ġtasks Ġsuch Ġas ĠN ER . </s> Insp ired Ġby ĠB IB REF 12 , Ġwe Ġintegrate Ġin Ġthis Ġpaper Ġa Ġboundary Ġassembling Ġstep Ġinto Ġthe Ġstate - of - the - art ĠL ST M Ġmodel Ġfor ĠChinese Ġword Ġsegment ation , Ġand Ġfeed Ġthe Ġoutput Ġinto Ġa ĠCR F Ġmodel Ġfor ĠN ER , Ġresulting Ġin Ġa Ġ2 % Ġabsolute Ġimprovement Ġon Ġthe Ġoverall ĠF 1 Ġscore Ġover Ġcurrent Ġstate - of - the - art Ġmethods . </s> This Ġpaper Ġis Ġorganized Ġas Ġfollows . ĠIn ĠSection ĠSEC REF 2 Ġwe Ġdiscuss Ġour Ġmodel , Ġwhich Ġconsists Ġof Ġan ĠL ST M Ġmodule Ġfor ĠChinese Ġword Ġsegment ation , Ġa Ġboundary Ġassembling Ġstep Ġfor Ġmore Ġaccurate Ġword Ġsegment ation , Ġand Ġa ĠCR F Ġmodule Ġfor ĠN ER . ĠWe Ġshow Ġthe Ġexperiment Ġresults Ġand Ġdiscuss Ġthe Ġmodel Ġperformance Ġin ĠSection ĠSEC REF 3 . ĠWe Ġconclude Ġin ĠSection ĠSEC REF 4 . </s> Model </s> Our Ġmodel Ġconsists Ġof Ġthree Ġmodules . ĠA Ġdiagram Ġof Ġthe Ġmodel Ġis Ġshown Ġin ĠFigure ĠFIG REF 1 . ĠCharacters Ġin Ġthe Ġinput Ġtext Ġfor ĠChinese Ġword Ġsegment ation Ġare Ġconverted Ġto Ġvectors Ġthat Ġare Ġused Ġto Ġtrain Ġthe ĠL ST M Ġmodule . ĠOutput Ġof Ġthe ĠL ST M Ġmodule Ġare Ġtransformed Ġby Ġa Ġbiased - linear Ġtransformation Ġto Ġget Ġlikelihood Ġscores Ġof Ġsegment ation Ġlabeling , Ġthen Ġpassed Ġthrough Ġthe Ġboundary Ġassembling Ġmodule . ĠThe Ġupdated Ġboundary Ġinformation Ġis Ġused Ġas Ġfeature Ġinput Ġinto Ġthe ĠCR F Ġfor ĠChinese Ġword Ġsegment ation Ġ( C WS ), Ġtogether Ġwith Ġcharacter - vector Ġsequences . ĠIn Ġeach Ġtraining Ġepoch , ĠCR F Ġfor ĠC WS Ġprovides Ġfeedback Ġinto Ġthe ĠL ST M Ġhidden Ġlayer Ġand Ġthe Ġbiased - linear Ġtransformation Ġto Ġupdate Ġthe Ġhyper - param eters . ĠAnother Ġcorpus Ġfor ĠN ER Ġis Ġthen Ġused Ġto Ġtrain Ġthe ĠL ST M Ġagain , Ġthe Ġhidden Ġvector Ġof Ġwhich Ġ( now Ġcontains Ġsegment ation Ġinformation Ġupdated Ġby Ġthe Ġboundary Ġassembling Ġmethod ) Ġis Ġtaken Ġas Ġfeature Ġinput Ġto ĠCR F Ġfor ĠN ER . ĠLex ical Ġfeatures Ġextracted Ġfrom Ġthe Ġinput Ġtext Ġfor ĠN ER , Ġas Ġwell Ġas Ġthe Ġword Ġembed ding Ġsequence , Ġare Ġalso Ġtaken Ġby Ġthe ĠCR F Ġmodule Ġas Ġinput Ġto Ġgenerate ĠN ER Ġlabels . ĠThis Ġsection Ġprovides Ġdescriptions Ġfor Ġeach Ġmodule . </s> Model Ġ: :: ĠL ST M Ġfor ĠWord ĠSe gment ation </s> We Ġchoose Ġan ĠL ST M Ġmodule Ġfor Ġthe ĠC WS Ġtask . ĠRaw Ġinput ĠChinese Ġtext Ġis Ġconverted Ġfrom Ġcharacters Ġto Ġvectors Ġwith Ġcharacter - pos itional Ġinput Ġembed d ings Ġpre - trained Ġby ĠB IB REF 5 Ġover Ġ112 , 9 71 , 7 34 ĠWe ibo Ġmessages Ġusing Ġword 2 vec ĠB IB REF 18 . ĠDet ailed Ġparameter Ġsettings Ġcan Ġbe Ġfound Ġin ĠB IB REF 13 . ĠThe Ġembed d ings Ġcontain Ġ52 , 057 Ġunique Ġcharacters Ġin Ġa Ġ100 - dimension Ġspace . </s> The ĠL ST M Ġmodule Ġtakes Ġthese Ġvectors Ġinto Ġa Ġsingle Ġlayer Ġthat Ġcontains Ġ150 Ġnodes , Ġand Ġmod ifies Ġthem Ġinto Ġlikelihood Ġscores Ġfor Ġeach Ġsegment ation Ġlabel . ĠA Ġbiased - linear Ġtransformation Ġis Ġcarried Ġout Ġon Ġthese Ġlikelihood Ġscores , Ġgenerating Ġpredicted Ġlabels Ġfor Ġsegment ation . ĠThese Ġlabels Ġare Ġthen Ġmodified Ġby Ġthe ĠBound ary ĠAs semb ling Ġmodule , Ġwhich Ġwe Ġwill Ġdiscuss Ġin Ġdetail Ġin Ġthe Ġnext Ġsection . ĠBecause Ġlabels Ġare Ġin Ġsequence , Ġand Ġdependency Ġmay Ġexist Ġamong Ġadjacent Ġlabels , Ġa Ġtransition Ġprobability Ġmatrix Ġis Ġintroduced . ĠThe Ġlikelihood Ġscore Ġtogether Ġwith Ġthe Ġtransition Ġscore Ġare Ġtaken Ġby Ġa ĠCR F Ġmodule Ġwith Ġa Ġmaximum - like lihood Ġtraining Ġobjective . ĠFeed backs Ġbased Ġon Ġthe Ġloss Ġfunction Ġof Ġthe ĠCR F Ġare Ġthen Ġgiven Ġto Ġthe ĠL ST M 's Ġhidden Ġlayer Ġand Ġthe Ġbiased - linear Ġtransformation 's Ġparameters Ġfor Ġupdate . </s> Model Ġ: :: ĠBound ary ĠAs semb ling ĠMethod </s> In Ġeach Ġsentence , ĠChinese Ġcharacters Ġare Ġlabeled Ġas Ġeither ĠBegin , ĠInside , ĠEnd , Ġor ĠSing leton Ġ( BI ES Ġlabeling ). ĠThe Ġlikelihood Ġof Ġindividual ĠChinese Ġcharacters Ġbeing Ġlabeled Ġas Ġeach Ġtype Ġis Ġcalculated Ġby Ġthe ĠL ST M Ġmodule Ġdescribed Ġin Ġthe Ġprevious Ġsection . ĠB IB REF 12 Ġfound Ġin Ġa ĠChinese Ġcorpus Ġthat Ġthe Ġword Ġlabel Ġ" End " Ġhas Ġa Ġbetter Ġperformance Ġthan Ġ" Begin ". ĠThis Ġmotiv ates Ġus Ġto Ġcarry Ġout Ġa Ġbackward Ġgreedy Ġsearch Ġover Ġeach Ġsentence 's Ġlabel Ġsequence Ġto Ġidentify Ġword Ġboundaries . ĠIf Ġtwo Ġwords Ġsegment ed Ġin Ġa Ġsentence Ġare Ġidentified Ġas Ġnoun s , Ġand Ġone Ġword Ġis Ġimmediately Ġbefore Ġthe Ġother , Ġwe Ġassemble Ġtheir Ġboundaries , Ġcreating Ġa Ġnew Ġword Ġcandidate Ġfor Ġentity Ġrecognition . ĠThis Ġstrategy Ġhas Ġthe Ġadvantage Ġto Ġfind Ġnamed Ġentities Ġwith Ġlong Ġword Ġlength . ĠIt Ġalso Ġreduces Ġthe Ġinfluence Ġcaused Ġby Ġdifferent Ġsegment ation Ġcriteria . </s> Model Ġ: :: ĠCR F Ġfor ĠNamed ĠEntity ĠRecogn ition </s> A Ġlog - bil in ear ĠCR F Ġmodule Ġis Ġused Ġfor Ġthe ĠN ER Ġtask , Ġand Ġtakes Ġthree Ġinputs . ĠThe Ġfirst Ġis Ġthe Ġsequential Ġcharacter - pos itional Ġembed d ings Ġmentioned Ġabove . ĠThe Ġsecond Ġis Ġthe Ġhidden Ġvector Ġfrom ĠL ST M Ġas Ġdynamic Ġfeature Ġinputs . ĠThe Ġthird Ġis Ġlex ical Ġfeatures Ġextracted Ġfrom Ġthe Ġinput Ġtext . ĠThese Ġlex ical Ġfeatures Ġare Ġthe Ġlikelihood Ġof Ġa Ġcharacter Ġbeing Ġat Ġa Ġspecific Ġposition Ġof Ġa Ġnoun Ġ( first Ġcharacter Ġof Ġa Ġnoun , Ġsecond Ġcharacter Ġof Ġa Ġnoun , Ġetc .), Ġand Ġis Ġachieved Ġby Ġcomparing Ġthe Ġcharacter Ġwith Ġa Ġpre - defined Ġdictionary Ġtrained Ġby ĠB IB REF 13 . </s> Exper iments Ġ: :: ĠDat as ets </s> Dat as ets Ġused Ġin Ġthis Ġstudy Ġfor Ġtraining , Ġvalidation , Ġand Ġtest Ġare Ġthe Ġsame Ġas Ġused Ġin ĠPeng Ġet Ġal . Ġp eng 2016 impro ving Ġfor Ġboth Ġword Ġsegment ation Ġand Ġnamed Ġentity Ġrecognition . ĠSpecifically , Ġdataset Ġfor Ġword Ġsegment ation Ġis Ġtaken Ġfrom Ġthe ĠS IGH AN Ġ2005 Ġbake off ĠPK U Ġcorpus ĠB IB REF 21 , Ġwhich Ġincludes Ġ123 , 530 Ġsentences Ġfor Ġtraining Ġand Ġ11 , 697 Ġsentences Ġfor Ġtesting . ĠDat as et Ġfor Ġnamed Ġentity Ġrecognition Ġis Ġa Ġcorpus Ġcomposed Ġof Ġ1 , 8 90 ĠSina ĠWe ibo Ġ( a ĠChinese Ġsocial Ġmedia Ġwebsite ) Ġmessages , Ġwith Ġ1 , 350 Ġmessages Ġsplit Ġinto Ġtraining Ġset , Ġ270 Ġinto Ġvalidation Ġset , Ġand Ġ270 Ġinto Ġtest Ġset ĠB IB REF 5 . ĠBoth Ġnamed Ġentity Ġand Ġnominal Ġmention Ġare Ġannot ated , Ġeach Ġwith Ġfour Ġentity Ġtypes : Ġperson , Ġorganization , Ġlocation , Ġand Ġgeo - political Ġentity . ĠA Ġmajor Ġcleanup Ġand Ġrevision Ġof Ġannotations Ġof Ġthis Ġcorpus Ġhas Ġbeen Ġperformed Ġby ĠHe Ġand ĠSun ĠHe S 16 . ĠIn Ġthis Ġstudy , Ġall Ġresults Ġfor Ġcomparisons Ġare Ġbased Ġon Ġthis Ġupdated Ġcorpus . </s> Exper iments Ġ: :: ĠTraining ĠSettings </s> The Ġweights Ġand Ġhyper - param eters Ġin Ġthe ĠL ST M Ġmodule Ġare Ġadjusted Ġin Ġeach Ġiteration Ġusing Ġsto ch astic Ġgradient Ġdescent Ġfor Ġtwo Ġstages Ġin Ġtandem . ĠThe Ġfirst Ġstage Ġis Ġbased Ġon Ġthe ĠC WS Ġinput Ġtext , Ġand Ġthe Ġsecond Ġstage Ġon ĠN ER Ġinput Ġtext . ĠSince Ġthe Ġcorpus Ġfor ĠC WS Ġis Ġmuch Ġlarger Ġthan Ġthe Ġcorpus Ġfor ĠN ER , Ġthe Ġformer Ġcorpus Ġis Ġrandomly Ġsub - sam pled Ġduring Ġeach Ġtraining Ġepoch , Ġwith Ġeach Ġsample Ġcontaining Ġ13 , 500 Ġsentences Ġfor Ġtraining Ġstep Ġone , Ġand Ġ1 , 350 Ġsentences Ġfor Ġtraining Ġstep Ġtwo . ĠModels Ġare Ġtrained Ġuntil Ġthe ĠF 1 Ġscore Ġof ĠN ER Ġmodel Ġconver ges Ġon Ġthe Ġvalidation Ġdataset , Ġor Ġup Ġto Ġ30 Ġepoch s . </s> Mod els Ġare Ġtrained Ġon Ġa ĠWindows ĠPC Ġwith Ġa Ġ2 . 7 ĠGHz ĠIntel ĠCore Ġi 7 ĠCPU . ĠFor Ġthe Ġbest Ġperformed Ġmodel , Ġthe Ġaverage Ġtraining Ġtime Ġfor Ġthe ĠL ST M Ġmodule Ġis Ġ1897 . 6 Ġseconds Ġper Ġiteration . ĠTime Ġfor Ġdata Ġloading , Ġpre - processing , Ġor Ġmodel Ġevaluation Ġis Ġnot Ġincluded . </s> Exper iments Ġ: :: ĠResults Ġand ĠDiscussion </s> Our Ġbest Ġmodel Ġperformance Ġwith Ġits ĠPrecision , ĠRecall , Ġand ĠF 1 Ġscores Ġon Ġnamed Ġentity Ġand Ġnominal Ġmention Ġare Ġshown Ġin ĠTable ĠT AB REF 5 . ĠThis Ġbest Ġmodel Ġperformance Ġis Ġachieved Ġwith Ġa Ġdrop out Ġrate Ġof Ġ0 . 1 , Ġand Ġa Ġlearning Ġrate Ġof Ġ0 . 05 . ĠOur Ġresults Ġare Ġcompared Ġwith Ġstate - of - the - art Ġmodels ĠB IB REF 15 , ĠB IB REF 19 , ĠB IB REF 20 Ġon Ġthe Ġsame ĠSina ĠWe ibo Ġtraining Ġand Ġtest Ġdatasets . ĠOur Ġmodel Ġshows Ġan Ġabsolute Ġimprovement Ġof Ġ2 % Ġfor Ġthe Ġoverall ĠF 1 Ġscore . </s> This Ġsignificant Ġimprovement Ġvalid ates Ġour Ġmethod Ġof Ġapplying Ġboundary Ġassembling Ġto Ġthe Ġsegment ed Ġsentences , Ġwhich Ġresults Ġin Ġmore Ġaccurate Ġword Ġsegment ation Ġand Ġbetter Ġsemantic Ġunderstanding . ĠSent ences Ġin Ġthe ĠPK U Ġcorpus Ġare Ġoften Ġsegment ed Ġinto Ġthe Ġsmallest Ġword Ġunits . ĠThis Ġresults Ġin Ġtoo Ġfragmented Ġinformation Ġand Ġincorrect Ġlex ical Ġunits Ġwhen Ġinput Ġinto Ġa Ġnamed Ġentity Ġrecognition Ġmodel , Ġalthough Ġthis Ġmay Ġbenefit Ġsome Ġother Ġnatural Ġlanguage Ġprocess Ġtasks . ĠBy Ġapplying Ġthe Ġboundary Ġassembling Ġmethod , Ġsense Ġgroup Ġin Ġa Ġsentence Ġis Ġbetter Ġpreserved . ĠThe Ġdownstream ĠN ER Ġmodel Ġcan Ġthen Ġtake Ġadvantage Ġof Ġthis , Ġimproving Ġits Ġresult . </s> The ĠPK U Ġcorpus Ġused Ġfor ĠC WS Ġmodule Ġconsists Ġof Ġmainly Ġnews Ġarticles , Ġwhich Ġare Ġquite Ġdifferent Ġfrom Ġthe Ġsocial Ġmedia ĠWe ibo Ġcorpus . ĠPerformance Ġof Ġan ĠN LP Ġtask Ġoften Ġdrops Ġwhen Ġtested Ġon Ġa Ġdifferent Ġdomain Ġor Ġa Ġcorpus Ġof Ġdifferent Ġcharacteristics . ĠOur Ġimprovement Ġindicates Ġthat Ġthe Ġboundary Ġassembling Ġmethod Ġis Ġnot Ġsensitive Ġto Ġthe Ġspecific Ġdomain , Ġand Ġis Ġa Ġrobust Ġmethod Ġfor Ġcross - domain Ġscenarios . </s> The Ġidentification Ġof Ġtwo Ġnoun s Ġthat Ġare Ġnext Ġto Ġeach Ġother Ġdepends Ġon Ġthe Ġpre - trained Ġlex ical Ġfeatures . ĠIf Ġour Ġmodel Ġis Ġtested Ġover Ġout - of - voc abulary Ġdataset , Ġit Ġmay Ġnot Ġperform Ġwell Ġdue Ġto Ġthe Ġlack Ġof Ġthis Ġlex ical Ġinformation . </s> Conclusion </s> In Ġthis Ġpaper Ġwe Ġintegrate Ġa Ġboundary Ġassembling Ġstep Ġwith Ġan ĠL ST M Ġmodule Ġand Ġa ĠCR F Ġmodule Ġfor ĠNamed ĠEntity ĠRecogn ition Ġin ĠChinese Ġsocial Ġmedia Ġtext . ĠWith Ġthe Ġabundance Ġof Ġsocial Ġmedia Ġinformation , Ġour Ġwork Ġis Ġtimely Ġand Ġdesirable . ĠThe Ġimprovement Ġin Ġexperiment Ġresults Ġover Ġexisting Ġmethods Ġclearly Ġshows Ġthe Ġeffectiveness Ġof Ġour Ġapproach . </s> Acknowled gments </s> This Ġresearch Ġwas Ġpartially Ġfunded Ġby Ġthe ĠEngineering ĠDirectorate Ġof Ġthe ĠNational ĠScience ĠFoundation Ġ( 18 201 18 ).