Adaptive Multi-task Learning
, Margin Ranking (MR) Loss
, Pairwise Neural Ranking Model

EXTRACTIVE
Multi-task Pairwise Neural Ranking We propose a multi-task pairwise neural ranking approach to better incorporate and distinguish the relative order between the candidate segmentations of a given hashtag. Our model adapts to address single- and multi-token hashtags differently via a multi-task learning strategy without requiring additional annotations. In this section, we describe the task setup and three variants of pairwise neural ranking models (Figure FIGREF11 ). Pairwise Neural Ranking Model Margin Ranking (MR) Loss Adaptive Multi-task Learning
1906.00790
Introduction </s> A Ġhashtag Ġis Ġa Ġkey phrase Ġrepresented Ġas Ġa Ġsequence Ġof Ġal phan umeric Ġcharacters Ġplus Ġunderscore , Ġpreceded Ġby Ġthe Ġ# Ġsymbol . ĠHash tags Ġplay Ġa Ġcentral Ġrole Ġin Ġonline Ġcommunication Ġby Ġproviding Ġa Ġtool Ġto Ġcategor ize Ġthe Ġmillions Ġof Ġposts Ġgenerated Ġdaily Ġon ĠTwitter , ĠInstagram , Ġetc . ĠThey Ġare Ġuseful Ġin Ġsearch , Ġtracking Ġcontent Ġabout Ġa Ġcertain Ġtopic ĠB IB REF 0 Ġ, ĠB IB REF 1 Ġ, Ġor Ġdiscovering Ġemerging Ġtrends ĠB IB REF 2 Ġ. </s> Hash tags Ġoften Ġcarry Ġvery Ġimportant Ġinformation , Ġsuch Ġas Ġemotion ĠB IB REF 3 Ġ, Ġsentiment ĠB IB REF 4 Ġ, Ġsarc asm ĠB IB REF 5 Ġ, Ġand Ġnamed Ġentities ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ. ĠHowever , Ġinfer ring Ġthe Ġsemantics Ġof Ġhasht ags Ġis Ġnon - t riv ial Ġsince Ġmany Ġhasht ags Ġcontain Ġmultiple Ġtokens Ġjoined Ġtogether , Ġwhich Ġfrequently Ġleads Ġto Ġmultiple Ġpotential Ġinterpretations Ġ( e . g ., Ġlion Ġhead Ġvs . Ġlion head ). ĠTable ĠT AB REF 3 Ġshows Ġseveral Ġexamples Ġof Ġsingle - Ġand Ġmulti - token Ġhasht ags . ĠWhile Ġmost Ġhasht ags Ġrepresent Ġa Ġmix Ġof Ġstandard Ġtokens , Ġnamed Ġentities Ġand Ġevent Ġnames Ġare Ġprevalent Ġand Ġpose Ġchallenges Ġto Ġboth Ġhuman Ġand Ġautomatic Ġcomprehension , Ġas Ġthese Ġare Ġmore Ġlikely Ġto Ġbe Ġrare Ġtokens . ĠHash tags Ġalso Ġtend Ġto Ġbe Ġshorter Ġto Ġallow Ġfast Ġtyping , Ġto Ġattract Ġattention Ġor Ġto Ġsatisfy Ġlength Ġlimitations Ġimposed Ġby Ġsome Ġsocial Ġmedia Ġplatforms . ĠThus , Ġthey Ġtend Ġto Ġcontain Ġa Ġlarge Ġnumber Ġof Ġabbrevi ations Ġor Ġnon - standard Ġspelling Ġvariations Ġ( e . g ., Ġ# il ove u 4 eva ) ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġ, Ġwhich Ġhind ers Ġtheir Ġunderstanding . </s> The Ġgoal Ġof Ġour Ġstudy Ġis Ġto Ġbuild Ġefficient Ġmethods Ġfor Ġautomatically Ġsplitting Ġa Ġhashtag Ġinto Ġa Ġmeaningful Ġword Ġsequence . ĠOur Ġcontributions Ġare : </s> Our Ġnew Ġdataset Ġincludes Ġsegment ation Ġfor Ġ12 , 594 Ġunique Ġhasht ags Ġand Ġtheir Ġassociated Ġtweets Ġannot ated Ġin Ġa Ġmulti - step Ġprocess Ġfor Ġhigher Ġquality Ġthan Ġthe Ġprevious Ġdataset Ġof Ġ1 , 108 Ġhasht ags ĠB IB REF 10 Ġ. ĠWe Ġframe Ġthe Ġsegment ation Ġtask Ġas Ġa Ġpair wise Ġranking Ġproblem , Ġgiven Ġa Ġset Ġof Ġcandidate Ġsegment ations . ĠWe Ġbuild Ġseveral Ġneural Ġarchitectures Ġusing Ġthis Ġproblem Ġformulation Ġwhich Ġuse Ġcorpus - based , Ġlinguistic Ġand Ġthe s aurus Ġbased Ġfeatures . ĠWe Ġfurther Ġpropose Ġa Ġmulti - task Ġlearning Ġapproach Ġwhich Ġjointly Ġlearns Ġsegment Ġranking Ġand Ġsingle - Ġvs . Ġmulti - token Ġhashtag Ġclassification . ĠThe Ġlatter Ġleads Ġto Ġan Ġerror Ġreduction Ġof Ġ24 . 6 % Ġover Ġthe Ġcurrent Ġstate - of - the - art . ĠFinally , Ġwe Ġdemonstrate Ġthe Ġutility Ġof Ġour Ġmethod Ġby Ġusing Ġhashtag Ġsegment ation Ġin Ġthe Ġdownstream Ġtask Ġof Ġsentiment Ġanalysis . ĠFeed ing Ġthe Ġautomatically Ġsegment ed Ġhasht ags Ġto Ġa Ġstate - of - the - art Ġsentiment Ġanalysis Ġmethod Ġon Ġthe ĠSem E val Ġ2017 Ġbenchmark Ġdataset Ġresults Ġin Ġa Ġ2 . 6 % Ġincrease Ġin Ġthe Ġofficial Ġmetric Ġfor Ġthe Ġtask . </s> Background Ġand ĠPrel im in aries </s> Current Ġapproaches Ġfor Ġhashtag Ġsegment ation Ġcan Ġbe Ġbroadly Ġdivided Ġinto Ġthree Ġcategories : Ġ( a ) Ġg az ete er Ġand Ġrule Ġbased ĠB IB REF 11 Ġ, ĠB IB REF 12 Ġ, ĠB IB REF 13 Ġ, Ġ( b ) Ġword Ġboundary Ġdetection ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, Ġand Ġ( c ) Ġranking Ġwith Ġlanguage Ġmodel Ġand Ġother Ġfeatures ĠB IB REF 16 Ġ, ĠB IB REF 10 Ġ, ĠB IB REF 0 Ġ, ĠB IB REF 17 Ġ, ĠB IB REF 18 Ġ. ĠHash tag Ġsegment ation Ġapproaches Ġdraw Ġupon Ġwork Ġon Ġcompound Ġsplitting Ġfor Ġlanguages Ġsuch Ġas ĠGerman Ġor ĠFinnish ĠB IB REF 19 Ġand Ġword Ġsegment ation ĠB IB REF 20 Ġfor Ġlanguages Ġwith Ġno Ġspaces Ġbetween Ġwords Ġsuch Ġas ĠChinese ĠB IB REF 21 Ġ, ĠB IB REF 22 Ġ. ĠSimilar Ġto Ġour Ġwork , ĠB IB REF 10 ĠBans al B V 15 Ġextract Ġan Ġinitial Ġset Ġof Ġcandidate Ġsegment ations Ġusing Ġa Ġsliding Ġwindow , Ġthen Ġre rank Ġthem Ġusing Ġa Ġlinear Ġregression Ġmodel Ġtrained Ġon Ġlex ical , Ġbig ram Ġand Ġother Ġcorpus - based Ġfeatures . ĠThe Ġcurrent Ġstate - of - the - art Ġapproach ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġuses Ġmaximum Ġentropy Ġand ĠCR F Ġmodels Ġwith Ġa Ġcombination Ġof Ġlanguage Ġmodel Ġand Ġhand - crafted Ġfeatures Ġto Ġpredict Ġif Ġeach Ġcharacter Ġin Ġthe Ġhashtag Ġis Ġthe Ġbeginning Ġof Ġa Ġnew Ġword . </s> Gener ating ĠCandidate ĠSe gment ations . ĠMicrosoft ĠWord ĠBreaker ĠB IB REF 16 Ġis , Ġamong Ġthe Ġexisting Ġmethods , Ġa Ġstrong Ġbaseline Ġfor Ġhashtag Ġsegment ation , Ġas Ġreported Ġin ĠB IB REF 14 Ġand ĠB IB REF 10 Ġ. ĠIt Ġemploys Ġa Ġbeam Ġsearch Ġalgorithm Ġto Ġextract ĠIN LINE FORM 0 Ġbest Ġsegment ations Ġas Ġranked Ġby Ġthe Ġn - gram Ġlanguage Ġmodel Ġprobability : ĠIN LINE FORM 1 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġword Ġsequence Ġof Ġsegment ation ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġis Ġthe Ġwindow Ġsize . ĠMore Ġsophisticated Ġranking Ġstrategies , Ġsuch Ġas ĠBin omial Ġand Ġword Ġlength Ġdistribution Ġbased Ġranking , Ġdid Ġnot Ġlead Ġto Ġa Ġfurther Ġimprovement Ġin Ġperformance ĠB IB REF 16 Ġ. ĠThe Ġoriginal ĠWord ĠBreaker Ġwas Ġdesigned Ġfor Ġsegment ing ĠURLs Ġusing Ġlanguage Ġmodels Ġtrained Ġon Ġweb Ġdata . ĠIn Ġthis Ġpaper , Ġwe Ġreim ple mented Ġand Ġtailored Ġthis Ġapproach Ġto Ġsegment ing Ġhasht ags Ġby Ġusing Ġa Ġlanguage Ġmodel Ġspecifically Ġtrained Ġon ĠTwitter Ġdata Ġ( im plementation Ġdetails Ġin ĠÂ§ ĠSEC REF 26 Ġ). ĠThe Ġperformance Ġof Ġthis Ġmethod Ġitself Ġis Ġcompetitive Ġwith Ġstate - of - the - art Ġmethods Ġ( eval uation Ġresults Ġin ĠÂ§ ĠSEC REF 46 Ġ). ĠOur Ġproposed Ġpair wise Ġranking Ġmethod Ġwill Ġeffectively Ġtake Ġthe Ġtop ĠIN LINE FORM 3 Ġsegment ations Ġgenerated Ġby Ġthis Ġbaseline Ġas Ġcandidates Ġfor Ġre ranking . </s> However , Ġin Ġprior Ġwork , Ġthe Ġranking Ġscores Ġof Ġeach Ġsegment ation Ġwere Ġcalculated Ġindependently , Ġignoring Ġthe Ġrelative Ġorder Ġamong Ġthe Ġtop ĠIN LINE FORM 0 Ġcandidate Ġsegment ations . ĠTo Ġaddress Ġthis Ġlimitation , Ġwe Ġutilize Ġa Ġpair wise Ġranking Ġstrategy Ġfor Ġthe Ġfirst Ġtime Ġfor Ġthis Ġtask Ġand Ġpropose Ġneural Ġarchitectures Ġto Ġmodel Ġthis . </s> Multi - task ĠPair wise ĠNeural ĠRanking </s> We Ġpropose Ġa Ġmulti - task Ġpair wise Ġneural Ġranking Ġapproach Ġto Ġbetter Ġincorporate Ġand Ġdistinguish Ġthe Ġrelative Ġorder Ġbetween Ġthe Ġcandidate Ġsegment ations Ġof Ġa Ġgiven Ġhashtag . ĠOur Ġmodel Ġadapt s Ġto Ġaddress Ġsingle - Ġand Ġmulti - token Ġhasht ags Ġdifferently Ġvia Ġa Ġmulti - task Ġlearning Ġstrategy Ġwithout Ġrequiring Ġadditional Ġannotations . ĠIn Ġthis Ġsection , Ġwe Ġdescribe Ġthe Ġtask Ġsetup Ġand Ġthree Ġvariants Ġof Ġpair wise Ġneural Ġranking Ġmodels Ġ( Figure ĠFIG REF 11 Ġ). </s> Seg mentation Ġas ĠPair wise ĠRanking </s> The Ġgoal Ġof Ġhashtag Ġsegment ation Ġis Ġto Ġdivide Ġa Ġgiven Ġhashtag ĠIN LINE FORM 0 Ġinto Ġa Ġsequence Ġof Ġmeaningful Ġwords ĠIN LINE FORM 1 Ġ. ĠFor Ġa Ġhashtag Ġof ĠIN LINE FORM 2 Ġcharacters , Ġthere Ġare Ġa Ġtotal Ġof ĠIN LINE FORM 3 Ġpossible Ġsegment ations Ġbut Ġonly Ġone , Ġor Ġoccasionally Ġtwo , Ġof Ġthem Ġ( ĠIN LINE FORM 4 Ġ) Ġare Ġconsidered Ġcorrect Ġ( Table ĠT AB REF 9 Ġ). </s> We Ġtransform Ġthis Ġtask Ġinto Ġa Ġpair wise Ġranking Ġproblem : Ġgiven ĠIN LINE FORM 0 Ġcandidate Ġsegment ations Ġ{ ĠIN LINE FORM 1 Ġ}, Ġwe Ġrank Ġthem Ġby Ġcomparing Ġeach Ġwith Ġthe Ġrest Ġin Ġa Ġpair wise Ġmanner . ĠMore Ġspecifically , Ġwe Ġtrain Ġa Ġmodel Ġto Ġpredict Ġa Ġreal Ġnumber ĠIN LINE FORM 2 Ġfor Ġany Ġtwo Ġcandidate Ġsegment ations ĠIN LINE FORM 3 Ġand ĠIN LINE FORM 4 Ġof Ġhashtag ĠIN LINE FORM 5 Ġ, Ġwhich Ġindicates ĠIN LINE FORM 6 Ġis Ġa Ġbetter Ġsegment ation Ġthan ĠIN LINE FORM 7 Ġif Ġpositive , Ġand Ġvice Ġversa . ĠTo Ġquantify Ġthe Ġquality Ġof Ġa Ġsegment ation Ġin Ġtraining , Ġwe Ġdefine Ġa Ġgold Ġscoring Ġfunction ĠIN LINE FORM 8 Ġbased Ġon Ġthe Ġsimilarities Ġwith Ġthe Ġground - truth Ġsegment ation ĠIN LINE FORM 9 Ġ: ĠIN LINE FORM 10 </s> We Ġuse Ġthe ĠLe ven sh tein Ġdistance Ġ( minimum Ġnumber Ġof Ġsingle - character Ġedits ) Ġin Ġthis Ġpaper , Ġalthough Ġit Ġis Ġpossible Ġto Ġuse Ġother Ġsimilarity Ġmeasurements Ġas Ġalternatives . ĠWe Ġuse Ġthe Ġtop ĠIN LINE FORM 0 Ġsegment ations Ġgenerated Ġby ĠMicrosoft ĠWord ĠBreaker Ġ( Â§ ĠSEC REF 2 Ġ) Ġas Ġinitial Ġcandidates . </s> P air wise ĠNeural ĠRanking ĠModel </s> For Ġan Ġinput Ġcandidate Ġsegment ation Ġpair ĠIN LINE FORM 0 Ġ, Ġwe Ġconc aten ate Ġtheir Ġfeature Ġvectors ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ, Ġand Ġfeed Ġthem Ġinto Ġa Ġfeed forward Ġnetwork Ġwhich Ġemits Ġa Ġcomparison Ġscore ĠIN LINE FORM 3 Ġ. ĠThe Ġfeature Ġvector ĠIN LINE FORM 4 Ġor ĠIN LINE FORM 5 Ġconsists Ġof Ġlanguage Ġmodel Ġprobabilities Ġusing ĠGood - T uring ĠB IB REF 23 Ġand Ġmodified ĠK nes er - N ey Ġsmoot hing ĠB IB REF 24 Ġ, ĠB IB REF 25 Ġ, Ġlex ical Ġand Ġlinguistic Ġfeatures Ġ( more Ġdetails Ġin ĠÂ§ ĠSEC REF 23 Ġ). ĠFor Ġtraining , Ġwe Ġuse Ġall Ġthe Ġpossible Ġpairs ĠIN LINE FORM 6 Ġof Ġthe ĠIN LINE FORM 7 Ġcandidates Ġas Ġthe Ġinput Ġand Ġtheir Ġgold Ġscores ĠIN LINE FORM 8 Ġas Ġthe Ġtarget . ĠThe Ġtraining Ġobjective Ġis Ġto Ġminimize Ġthe ĠMean ĠSqu ared ĠError Ġ( M SE ): ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġnumber Ġof Ġtraining Ġexamples . </s> To Ġaggregate Ġthe Ġpair wise Ġcomparisons , Ġwe Ġfollow Ġa Ġgreedy Ġalgorithm Ġproposed Ġby ĠB IB REF 26 Ġco hen 1998 learning Ġand Ġused Ġfor Ġpreference Ġranking ĠB IB REF 27 Ġ. ĠFor Ġeach Ġsegment ation ĠIN LINE FORM 0 Ġin Ġthe Ġcandidate Ġset ĠIN LINE FORM 1 Ġ, Ġwe Ġcalculate Ġa Ġsingle Ġscore ĠIN LINE FORM 2 Ġ, Ġand Ġfind Ġthe Ġsegment ation ĠIN LINE FORM 3 Ġcorresponding Ġto Ġthe Ġhighest Ġscore . ĠWe Ġrepeat Ġthe Ġsame Ġprocedure Ġafter Ġremoving ĠIN LINE FORM 4 Ġfrom ĠIN LINE FORM 5 Ġ, Ġand Ġcontinue Ġuntil ĠIN LINE FORM 6 Ġreduces Ġto Ġan Ġempty Ġset . ĠFigure ĠFIG REF 11 Ġ( a ) Ġshows Ġthe Ġarchitecture Ġof Ġthis Ġmodel . </s> Marg in ĠRanking Ġ( MR ) ĠLoss </s> As Ġan Ġalternative Ġto Ġthe Ġpair wise Ġrank er Ġ( Â§ ĠSEC REF 15 Ġ), Ġwe Ġpropose Ġa Ġpair wise Ġmodel Ġwhich Ġlearns Ġfrom Ġcandidate Ġpairs ĠIN LINE FORM 0 Ġbut Ġranks Ġeach Ġindividual Ġcandidate Ġdirectly Ġrather Ġthan Ġrelatively . ĠWe Ġdefine Ġa Ġnew Ġscoring Ġfunction ĠIN LINE FORM 1 Ġwhich Ġassigns Ġa Ġhigher Ġscore Ġto Ġthe Ġbetter Ġcandidate , Ġi . e ., ĠIN LINE FORM 2 Ġ, Ġif ĠIN LINE FORM 3 Ġis Ġa Ġbetter Ġcandidate Ġthan ĠIN LINE FORM 4 Ġand Ġvice - vers a . ĠInstead Ġof Ġconc aten ating Ġthe Ġfeatures Ġvectors ĠIN LINE FORM 5 Ġand ĠIN LINE FORM 6 Ġ, Ġwe Ġfeed Ġthem Ġseparately Ġinto Ġtwo Ġidentical Ġfeed forward Ġnetworks Ġwith Ġshared Ġparameters . ĠDuring Ġtesting , Ġwe Ġuse Ġonly Ġone Ġof Ġthe Ġnetworks Ġto Ġrank Ġthe Ġcandidates Ġbased Ġon Ġthe ĠIN LINE FORM 7 Ġscores . ĠFor Ġtraining , Ġwe Ġadd Ġa Ġranking Ġlayer Ġon Ġtop Ġof Ġthe Ġnetworks Ġto Ġmeasure Ġthe Ġviolations Ġin Ġthe Ġranking Ġorder Ġand Ġminimize Ġthe ĠMarg in ĠRanking ĠLoss Ġ( MR ): ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġnumber Ġof Ġtraining Ġsamples . ĠThe Ġarchitecture Ġof Ġthis Ġmodel Ġis Ġpresented Ġin ĠFigure ĠFIG REF 11 Ġ( b ). </s> Adapt ive ĠMulti - task ĠLearning </s> Both Ġmodels Ġin ĠÂ§ ĠSEC REF 15 Ġand ĠÂ§ ĠSEC REF 17 Ġtreat Ġall Ġthe Ġhasht ags Ġuniformly . ĠHowever , Ġdifferent Ġfeatures Ġaddress Ġdifferent Ġtypes Ġof Ġhasht ags . ĠBy Ġdesign , Ġthe Ġlinguistic Ġfeatures Ġcapture Ġnamed Ġentities Ġand Ġmulti - word Ġhasht ags Ġthat Ġexhibit Ġword Ġshape Ġpatterns , Ġsuch Ġas Ġcamel Ġcase . ĠThe Ġn gram Ġprobabilities Ġwith ĠGood - T uring Ġsmoot hing Ġgrav itate Ġtowards Ġmulti - word Ġsegment ations Ġwith Ġknown Ġwords , Ġas Ġits Ġestimate Ġfor Ġunseen Ġn gram s Ġdepends Ġon Ġthe Ġfraction Ġof Ġn gram s Ġseen Ġonce Ġwhich Ġcan Ġbe Ġvery Ġlow ĠB IB REF 28 Ġ. ĠThe Ġmodified ĠK nes er - N ey Ġsmoot hing Ġis Ġmore Ġlikely Ġto Ġfavor Ġsegment ations Ġthat Ġcontain Ġrare Ġwords , Ġand Ġsingle - word Ġsegment ations Ġin Ġparticular . ĠPlease Ġrefer Ġto ĠÂ§ ĠSEC REF 46 Ġfor Ġa Ġmore Ġdetailed Ġquantitative Ġand Ġqualitative Ġanalysis . </s> To Ġleverage Ġthis Ġintuition , Ġwe Ġintroduce Ġa Ġbinary Ġclassification Ġtask Ġto Ġhelp Ġthe Ġmodel Ġdifferentiate Ġsingle - word Ġfrom Ġmulti - word Ġhasht ags . ĠThe Ġbinary Ġclass ifier Ġtakes Ġhashtag Ġfeatures ĠIN LINE FORM 0 Ġas Ġthe Ġinput Ġand Ġoutputs ĠIN LINE FORM 1 Ġ, Ġwhich Ġrepresents Ġthe Ġprobability Ġof ĠIN LINE FORM 2 Ġbeing Ġa Ġmulti - word Ġhashtag . ĠIN LINE FORM 3 Ġis Ġused Ġas Ġan Ġadaptive Ġg ating Ġvalue Ġin Ġour Ġmulti - task Ġlearning Ġsetup . ĠThe Ġgold Ġlabels Ġfor Ġthis Ġtask Ġare Ġobtained Ġat Ġno Ġextra Ġcost Ġby Ġsimply Ġverifying Ġwhether Ġthe Ġground - truth Ġsegment ation Ġhas Ġmultiple Ġwords . ĠWe Ġtrain Ġthe Ġpair wise Ġsegment ation Ġrank er Ġand Ġthe Ġbinary Ġsingle - Ġvs . Ġmulti - token Ġhashtag Ġclass ifier Ġjointly , Ġby Ġminimizing ĠIN LINE FORM 4 Ġfor Ġthe Ġpair wise Ġrank er Ġand Ġthe ĠBinary ĠCross ĠEnt ropy ĠError Ġ( ĠIN LINE FORM 5 Ġ) Ġfor Ġthe Ġclass ifier : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġadaptive Ġg ating Ġvalue , ĠIN LINE FORM 1 Ġindicates Ġif ĠIN LINE FORM 2 Ġis Ġactually Ġa Ġmulti - word Ġhashtag Ġand ĠIN LINE FORM 3 Ġis Ġthe Ġnumber Ġof Ġtraining Ġexamples . ĠIN LINE FORM 4 Ġand ĠIN LINE FORM 5 Ġare Ġthe Ġweights Ġfor Ġeach Ġloss . ĠFor Ġour Ġexperiments , Ġwe Ġapply Ġequal Ġweights . </s> More Ġspecifically , Ġwe Ġdivide Ġthe Ġsegment ation Ġfeature Ġvector ĠIN LINE FORM 0 Ġinto Ġtwo Ġsubs ets : Ġ( a ) ĠIN LINE FORM 1 Ġwith Ġmodified ĠK nes er - N ey Ġsmoot hing Ġfeatures , Ġand Ġ( b ) ĠIN LINE FORM 2 Ġwith ĠGood - T uring Ġsmoot hing Ġand Ġlinguistic Ġfeatures . ĠFor Ġan Ġinput Ġcandidate Ġsegment ation Ġpair ĠIN LINE FORM 3 Ġ, Ġwe Ġconstruct Ġtwo Ġpair wise Ġvectors ĠIN LINE FORM 4 Ġand ĠIN LINE FORM 5 Ġby Ġconc aten ation , Ġthen Ġcombine Ġthem Ġbased Ġon Ġthe Ġadaptive Ġg ating Ġvalue ĠIN LINE FORM 6 Ġbefore Ġfeeding Ġthem Ġinto Ġthe Ġfeed forward Ġnetwork ĠIN LINE FORM 7 Ġfor Ġpair wise Ġranking : ĠDIS PLAY FORM 0 </s> We Ġuse Ġsumm ation Ġwith Ġpadding , Ġas Ġwe Ġfind Ġthis Ġsimple Ġensemble Ġmethod Ġachieves Ġsimilar Ġperformance Ġin Ġour Ġexperiments Ġas Ġthe Ġmore Ġcomplex Ġmulti - column Ġnetworks ĠB IB REF 29 Ġ. ĠFigure ĠFIG REF 11 Ġ( c ) Ġshows Ġthe Ġarchitecture Ġof Ġthis Ġmodel . ĠAn Ġanalogue Ġmulti - task Ġformulation Ġcan Ġalso Ġbe Ġused Ġfor Ġthe ĠMarg in ĠRanking Ġloss Ġas : ĠDIS PLAY FORM 0 </s> Features </s> We Ġuse Ġa Ġcombination Ġof Ġcorpus - based Ġand Ġlinguistic Ġfeatures Ġto Ġrank Ġthe Ġsegment ations . ĠFor Ġa Ġcandidate Ġsegment ation ĠIN LINE FORM 0 Ġ, Ġits Ġfeature Ġvector ĠIN LINE FORM 1 Ġincludes Ġthe Ġnumber Ġof Ġwords Ġin Ġthe Ġcandidate , Ġthe Ġlength Ġof Ġeach Ġword , Ġthe Ġproportion Ġof Ġwords Ġin Ġan ĠEnglish Ġdictionary Ġor ĠUrban ĠDictionary ĠB IB REF 30 Ġ, Ġn gram Ġcounts Ġfrom ĠGoogle ĠWeb Ġ1 TB Ġcorpus ĠB IB REF 31 Ġ, Ġand Ġn gram Ġprobabilities Ġfrom Ġtrig ram Ġlanguage Ġmodels Ġtrained Ġon Ġthe ĠGig aw ord Ġcorpus ĠB IB REF 32 Ġand Ġ1 . 1 Ġbillion ĠEnglish Ġtweets Ġfrom Ġ2010 , Ġrespectively . ĠWe Ġtrain Ġtwo Ġlanguage Ġmodels Ġon Ġeach Ġcorpus : Ġone Ġwith ĠGood - T uring Ġsmoot hing Ġusing ĠSR IL M ĠB IB REF 33 Ġand Ġthe Ġother Ġwith Ġmodified ĠK nes er - N ey Ġsmoot hing Ġusing ĠKen LM ĠB IB REF 34 Ġ. ĠWe Ġalso Ġadd Ġboolean Ġfeatures , Ġsuch Ġas Ġif Ġthe Ġcandidate Ġis Ġa Ġnamed - entity Ġpresent Ġin Ġthe Ġlist Ġof ĠWikipedia Ġtitles , Ġand Ġif Ġthe Ġcandidate Ġsegment ation ĠIN LINE FORM 2 Ġand Ġits Ġcorresponding Ġhashtag ĠIN LINE FORM 3 Ġsatisfy Ġcertain Ġword - sh apes Ġ( more Ġdetails Ġin Ġappendix ĠSEC REF 61 Ġ). </s> Similarly , Ġfor Ġhashtag ĠIN LINE FORM 0 Ġ, Ġwe Ġextract Ġthe Ġfeature Ġvector ĠIN LINE FORM 1 Ġconsisting Ġof Ġhashtag Ġlength , Ġn gram Ġcount Ġof Ġthe Ġhashtag Ġin ĠGoogle Ġ1 TB Ġcorpus ĠB IB REF 31 Ġ, Ġand Ġboolean Ġfeatures Ġindicating Ġif Ġthe Ġhashtag Ġis Ġin Ġan ĠEnglish Ġdictionary Ġor ĠUrban ĠDictionary , Ġis Ġa Ġnamed - entity , Ġis Ġin Ġcamel Ġcase , Ġends Ġwith Ġa Ġnumber , Ġand Ġhas Ġall Ġthe Ġletters Ġas Ġconson ants . ĠWe Ġalso Ġinclude Ġfeatures Ġof Ġthe Ġbest - ranked Ġcandidate Ġby Ġthe ĠWord ĠBreaker Ġmodel . </s> Im plementation ĠDetails </s> We Ġuse Ġthe ĠPy Tor ch Ġframework Ġto Ġimplement Ġour Ġmulti - task Ġpair wise Ġranking Ġmodel . ĠThe Ġpair wise Ġrank er Ġconsists Ġof Ġan Ġinput Ġlayer , Ġthree Ġhidden Ġlayers Ġwith Ġeight Ġnodes Ġin Ġeach Ġlayer Ġand Ġhyper b olic Ġtang ent Ġ( ĠIN LINE FORM 0 Ġ) Ġactivation , Ġand Ġa Ġsingle Ġlinear Ġoutput Ġnode . ĠThe Ġauxiliary Ġclass ifier Ġconsists Ġof Ġan Ġinput Ġlayer , Ġone Ġhidden Ġlayer Ġwith Ġeight Ġnodes Ġand Ġone Ġoutput Ġnode Ġwith Ġs igm oid Ġactivation . ĠWe Ġuse Ġthe ĠAdam Ġalgorithm ĠB IB REF 35 Ġfor Ġoptimization Ġand Ġapply Ġa Ġdrop out Ġof Ġ0 . 5 Ġto Ġprevent Ġover fitting . ĠWe Ġset Ġthe Ġlearning Ġrate Ġto Ġ0 . 01 Ġand Ġ0 . 05 Ġfor Ġthe Ġpair wise Ġrank er Ġand Ġauxiliary Ġclass ifier Ġrespectively . ĠFor Ġeach Ġexperiment , Ġwe Ġreport Ġresults Ġobtained Ġafter Ġ100 Ġepoch s . </s> For Ġthe Ġbaseline Ġmodel Ġused Ġto Ġextract Ġthe ĠIN LINE FORM 0 Ġinitial Ġcandidates , Ġwe Ġreim plement ated Ġthe ĠWord ĠBreaker ĠB IB REF 16 Ġas Ġdescribed Ġin ĠÂ§ ĠSEC REF 2 Ġand Ġadapted Ġit Ġto Ġuse Ġa Ġlanguage Ġmodel Ġtrained Ġon Ġ1 . 1 Ġbillion Ġtweets Ġwith ĠGood - T uring Ġsmoot hing Ġusing ĠSR IL M ĠB IB REF 33 Ġto Ġgive Ġa Ġbetter Ġperformance Ġin Ġsegment ing Ġhasht ags Ġ( Â§ ĠSEC REF 46 Ġ). ĠFor Ġall Ġour Ġexperiments , Ġwe Ġset ĠIN LINE FORM 1 Ġ. </s> Hash tag ĠSe gment ation ĠData </s> We Ġuse Ġtwo Ġdatasets Ġfor Ġexperiments Ġ( Table ĠT AB REF 29 Ġ): Ġ( a ) ĠST AN ĠIN LINE FORM 0 Ġ, Ġcreated Ġby ĠB IB REF 10 ĠBans al B V 15 , Ġwhich Ġconsists Ġof Ġ1 , 108 Ġunique ĠEnglish Ġhasht ags Ġfrom Ġ1 , 268 Ġrandomly Ġselected Ġtweets Ġin Ġthe ĠStanford ĠSent iment ĠAnalysis ĠDat as et ĠB IB REF 36 Ġalong Ġwith Ġtheir Ġcrowds ourced Ġsegment ations Ġand Ġour Ġadditional Ġcorrections ; Ġand Ġ( b ) ĠST AN ĠIN LINE FORM 1 Ġ, Ġour Ġnew Ġexpert Ġcurated Ġdataset , Ġwhich Ġincludes Ġall Ġ12 , 594 Ġunique ĠEnglish Ġhasht ags Ġand Ġtheir Ġassociated Ġtweets Ġfrom Ġthe Ġsame ĠStanford Ġdataset . </s> Exper iments </s> In Ġthis Ġsection , Ġwe Ġpresent Ġexperimental Ġresults Ġthat Ġcompare Ġour Ġproposed Ġmethod Ġwith Ġthe Ġother Ġstate - of - the - art Ġapproaches Ġon Ġhashtag Ġsegment ation Ġdatasets . ĠThe Ġnext Ġsection Ġwill Ġshow Ġexperiments Ġof Ġapplying Ġhashtag Ġsegment ation Ġto Ġthe Ġpopular Ġtask Ġof Ġsentiment Ġanalysis . </s> Ex isting ĠMethods </s> We Ġcompare Ġour Ġpair wise Ġneural Ġrank er Ġwith Ġthe Ġfollowing Ġbaseline Ġand Ġstate - of - the - art Ġapproaches : </s> The Ġoriginal Ġhashtag Ġas Ġa Ġsingle Ġtoken ; </s> A Ġrule - based Ġsegment er , Ġwhich Ġemploys Ġa Ġset Ġof Ġword - shape Ġrules Ġwith Ġan ĠEnglish Ġdictionary ĠB IB REF 13 Ġ; </s> A ĠV iter bi Ġmodel Ġwhich Ġuses Ġword Ġfrequencies Ġfrom Ġa Ġbook Ġcorpus ĠB IB REF 0 Ġ; </s> The Ġspecially Ġdeveloped ĠG ATE ĠHash tag ĠToken izer Ġfrom Ġthe Ġopen Ġsource Ġtool kit , Ġwhich Ġcombines Ġdiction aries Ġand Ġg az ette ers Ġin Ġa ĠV iter bi - like Ġalgorithm ĠB IB REF 11 Ġ; </s> A Ġmaximum Ġentropy Ġclass ifier Ġ( Max Ent ) Ġtrained Ġon Ġthe ĠST AN ĠIN LINE FORM 0 Ġtraining Ġdataset . ĠIt Ġpredicts Ġwhether Ġa Ġspace Ġshould Ġbe Ġinserted Ġat Ġeach Ġposition Ġin Ġthe Ġhashtag Ġand Ġis Ġthe Ġcurrent Ġstate - of - the - art ĠB IB REF 14 Ġ; </s> Our Ġreim plementation Ġof Ġthe ĠWord ĠBreaker Ġalgorithm Ġwhich Ġuses Ġbeam Ġsearch Ġand Ġa ĠTwitter Ġn gram Ġlanguage Ġmodel ĠB IB REF 16 Ġ; </s> A Ġpair wise Ġlinear Ġrank er Ġwhich Ġwe Ġimplemented Ġfor Ġcomparison Ġpurposes Ġwith Ġthe Ġsame Ġfeatures Ġas Ġour Ġneural Ġmodel , Ġbut Ġusing Ġpercept ron Ġas Ġthe Ġunderlying Ġclass ifier ĠB IB REF 38 Ġand Ġminimizing Ġthe Ġhinge Ġloss Ġbetween ĠIN LINE FORM 0 Ġand Ġa Ġscoring Ġfunction Ġsimilar Ġto ĠIN LINE FORM 1 Ġ. ĠIt Ġis Ġtrained Ġon Ġthe ĠST AN ĠIN LINE FORM 2 Ġdataset . </s> E val uation ĠMet rics </s> We Ġevaluate Ġthe Ġperformance Ġby Ġthe Ġtop ĠIN LINE FORM 0 Ġ( ĠIN LINE FORM 1 Ġ) Ġaccuracy Ġ( A @ 1 , ĠA @ 2 ), Ġaverage Ġtoken - level ĠF ĠIN LINE FORM 2 Ġscore Ġ( F ĠIN LINE FORM 3 Ġ@ 1 ), Ġand Ġmean Ġreciprocal Ġrank Ġ( MR R ). ĠIn Ġparticular , Ġthe Ġaccuracy Ġand ĠMR R Ġare Ġcalculated Ġat Ġthe Ġsegment ation - level , Ġwhich Ġmeans Ġthat Ġan Ġoutput Ġsegment ation Ġis Ġconsidered Ġcorrect Ġif Ġand Ġonly Ġif Ġit Ġfully Ġmatches Ġthe Ġhuman Ġsegment ation . ĠAverage Ġtoken - level ĠF ĠIN LINE FORM 4 Ġscore Ġaccounts Ġfor Ġpartially Ġcorrect Ġsegment ation Ġin Ġthe Ġmulti - token Ġhashtag Ġcases . </s> Results </s> T ables ĠT AB REF 32 Ġand ĠT AB REF 33 Ġshow Ġthe Ġresults Ġon Ġthe ĠST AN ĠIN LINE FORM 0 Ġand ĠST AN ĠIN LINE FORM 1 Ġdatasets , Ġrespectively . ĠAll Ġof Ġour Ġpair wise Ġneural Ġrank ers Ġare Ġtrained Ġon Ġthe Ġ2 , 518 Ġmanually Ġsegment ed Ġhasht ags Ġin Ġthe Ġtraining Ġset Ġof ĠST AN ĠIN LINE FORM 2 Ġand Ġperform Ġfavorably Ġagainst Ġother Ġstate - of - the - art Ġapproaches . ĠOur Ġbest Ġmodel Ġ( M SE + mult it ask ) Ġthat Ġutilizes Ġdifferent Ġfeatures Ġadapt ively Ġvia Ġa Ġmulti - task Ġlearning Ġprocedure Ġis Ġshown Ġto Ġperform Ġbetter Ġthan Ġsimply Ġcombining Ġall Ġthe Ġfeatures Ġtogether Ġ( MR Ġand ĠM SE ). ĠWe Ġhighlight Ġthe Ġ24 . 6 % Ġerror Ġreduction Ġon ĠST AN ĠIN LINE FORM 3 Ġand Ġ16 . 5 % Ġon ĠST AN ĠIN LINE FORM 4 Ġof Ġour Ġapproach Ġover Ġthe Ġprevious ĠS OTA ĠB IB REF 14 Ġon Ġthe ĠMulti - token Ġhasht ags , Ġand Ġthe Ġimportance Ġof Ġhaving Ġa Ġseparate Ġevaluation Ġof Ġmulti - word Ġcases Ġas Ġit Ġis Ġtrivial Ġto Ġobtain Ġ100 % Ġaccuracy Ġfor ĠSingle - token Ġhasht ags . ĠWhile Ġour Ġhashtag Ġsegment ation Ġmodel Ġis Ġachieving Ġa Ġvery Ġhigh Ġaccuracy @ 2 , Ġto Ġbe Ġpractically Ġuseful , Ġit Ġremains Ġa Ġchallenge Ġto Ġget Ġthe Ġtop Ġone Ġpred ication Ġexactly Ġcorrect . ĠSome Ġhasht ags Ġare Ġvery Ġdifficult Ġto Ġinterpret , Ġe . g ., Ġ# B TV brown S MB Ġrefers Ġto Ġthe ĠSocial ĠMedia ĠBreakfast Ġ( S MB ) Ġin ĠBurlington , ĠVermont Ġ( B TV ). </s> The Ġimproved ĠWord ĠBreaker Ġwith Ġour Ġaddition Ġof Ġa ĠTwitter - specific Ġlanguage Ġmodel Ġis Ġa Ġvery Ġstrong Ġbaseline , Ġwhich Ġe ch os Ġthe Ġfindings Ġof Ġthe Ġoriginal ĠWord ĠBreaker Ġpaper ĠB IB REF 16 Ġthat Ġhaving Ġa Ġlarge Ġin - domain Ġlanguage Ġmodel Ġis Ġextremely Ġhelpful Ġfor Ġword Ġsegment ation Ġtasks . ĠIt Ġis Ġworth Ġnoting Ġthat Ġthe Ġother Ġstate - of - the - art Ġsystem ĠB IB REF 14 Ġalso Ġutilized Ġa Ġ4 - gram Ġlanguage Ġmodel Ġtrained Ġon Ġ4 76 Ġmillion Ġtweets Ġfrom Ġ2009 . </s> Analysis Ġand ĠDiscussion </s> To Ġempir ically Ġillustrate Ġthe Ġeffectiveness Ġof Ġdifferent Ġfeatures Ġon Ġdifferent Ġtypes Ġof Ġhasht ags , Ġwe Ġshow Ġthe Ġresults Ġfor Ġmodels Ġusing Ġindividual Ġfeature Ġsets Ġin Ġpair wise Ġranking Ġmodels Ġ( M SE ) Ġin ĠTable ĠT AB REF 45 Ġ. ĠLanguage Ġmodels Ġwith Ġmodified ĠK nes er - N ey Ġsmoot hing Ġperform Ġbest Ġon Ġsingle - token Ġhasht ags , Ġwhile ĠGood - T uring Ġand ĠL ingu istic Ġfeatures Ġwork Ġbest Ġon Ġmulti - token Ġhasht ags , Ġconfirming Ġour Ġintuition Ġabout Ġtheir Ġusefulness Ġin Ġa Ġmulti - task Ġlearning Ġapproach . ĠTable ĠT AB REF 47 Ġshows Ġa Ġqualitative Ġanalysis Ġwith Ġthe Ġfirst Ġcolumn Ġ( ĠIN LINE FORM 0 ĠIN LINE FORM 1 ĠIN LINE FORM 2 Ġ) Ġindicating Ġwhich Ġfeatures Ġlead Ġto Ġcorrect Ġor Ġwrong Ġsegment ations , Ġtheir Ġcount Ġin Ġour Ġdata Ġand Ġillust rative Ġexamples Ġwith Ġhuman Ġsegment ation . </s> As Ġexpected , Ġlonger Ġhasht ags Ġwith Ġmore Ġthan Ġthree Ġtokens Ġpose Ġgreater Ġchallenges Ġand Ġthe Ġsegment ation - level Ġaccuracy Ġof Ġour Ġbest Ġmodel Ġ( M SE + mult it ask ) Ġdrops Ġto Ġ82 . 1 %. ĠFor Ġmany Ġerror Ġcases , Ġour Ġmodel Ġpredicts Ġa Ġclose - to - correct Ġsegment ation , Ġe . g ., Ġ# you brown know you brown upt to ob rown early , Ġ# ise eb rown l ondon ise eb rown f rance , Ġwhich Ġis Ġalso Ġreflected Ġby Ġthe Ġhigher Ġtoken - level ĠF ĠIN LINE FORM 0 Ġscores Ġacross Ġhasht ags Ġwith Ġdifferent Ġlengths Ġ( Figure ĠFIG REF 51 Ġ). </s> Since Ġour Ġapproach Ġheavily Ġrelies Ġon Ġbuilding Ġa ĠTwitter Ġlanguage Ġmodel , Ġwe Ġexperimented Ġwith Ġits Ġsizes Ġand Ġshow Ġthe Ġresults Ġin ĠFigure ĠFIG REF 52 Ġ. ĠOur Ġapproach Ġcan Ġperform Ġwell Ġeven Ġwith Ġaccess Ġto Ġa Ġsmaller Ġamount Ġof Ġtweets . ĠThe Ġdrop Ġin ĠF ĠIN LINE FORM 0 Ġscore Ġfor Ġour Ġpair wise Ġneural Ġrank er Ġis Ġonly Ġ1 . 4 % Ġand Ġ3 . 9 % Ġwhen Ġusing Ġthe Ġlanguage Ġmodels Ġtrained Ġon Ġ10 % Ġand Ġ1 % Ġof Ġthe Ġtotal Ġ1 . 1 Ġbillion Ġtweets , Ġrespectively . </s> Language Ġuse Ġin ĠTwitter Ġchanges Ġwith Ġtime ĠB IB REF 9 Ġ. ĠOur Ġpair wise Ġrank er Ġuses Ġlanguage Ġmodels Ġtrained Ġon Ġthe Ġtweets Ġfrom Ġthe Ġyear Ġ2010 . ĠWe Ġtested Ġour Ġapproach Ġon Ġa Ġset Ġof Ġ500 Ġrandom ĠEnglish Ġhasht ags Ġposted Ġin Ġtweets Ġfrom Ġthe Ġyear Ġ2019 Ġand Ġshow Ġthe Ġresults Ġin ĠTable ĠT AB REF 55 Ġ. ĠWith Ġa Ġsegment ation - level Ġaccuracy Ġof Ġ94 . 6 % Ġand Ġaverage Ġtoken - level ĠF ĠIN LINE FORM 0 Ġscore Ġof Ġ95 . 6 %, Ġour Ġapproach Ġperforms Ġfavorably Ġon Ġ2019 Ġhasht ags . </s> Ext r ins ic ĠEvaluation : ĠTwitter ĠSent iment ĠAnalysis </s> We Ġattempt Ġto Ġdemonstrate Ġthe Ġeffectiveness Ġof Ġour Ġhashtag Ġsegment ation Ġsystem Ġby Ġstudying Ġits Ġimpact Ġon Ġthe Ġtask Ġof Ġsentiment Ġanalysis Ġin ĠTwitter ĠB IB REF 39 Ġ, ĠB IB REF 40 Ġ, ĠB IB REF 41 Ġ. ĠWe Ġuse Ġour Ġbest Ġmodel Ġ( M SE + mult it ask ), Ġunder Ġthe Ġname ĠHash tag Master , Ġin Ġthe Ġfollowing Ġexperiments . </s> Exper imental ĠSetup </s> We Ġcompare Ġthe Ġperformance Ġof Ġthe ĠBi L ST M + Lex ĠB IB REF 42 Ġsentiment Ġanalysis Ġmodel Ġunder Ġthree Ġconfigurations : Ġ( a ) Ġtweets Ġwith Ġhasht ags Ġremoved , Ġ( b ) Ġtweets Ġwith Ġhasht ags Ġas Ġsingle Ġtokens Ġexcluding Ġthe Ġ# Ġsymbol , Ġand Ġ( c ) Ġtweets Ġwith Ġhasht ags Ġas Ġsegment ed Ġby Ġour Ġsystem , ĠHash tag Master . ĠBi L ST M + Lex Ġis Ġa Ġstate - of - the - art Ġopen Ġsource Ġsystem Ġfor Ġpredicting Ġtweet - level Ġsentiment ĠB IB REF 43 Ġ. ĠIt Ġlearns Ġa Ġcontext - sensitive Ġsentiment Ġintensity Ġscore Ġby Ġleveraging Ġa ĠTwitter - based Ġsentiment Ġlex icon ĠB IB REF 44 Ġ. ĠWe Ġuse Ġthe Ġsame Ġsettings Ġas Ġdescribed Ġby ĠB IB REF 42 Ġt eng - vo - zh ang : 2016 : EM N LP 2016 Ġto Ġtrain Ġthe Ġmodel . </s> We Ġuse Ġthe Ġdataset Ġfrom Ġthe ĠSent iment ĠAnalysis Ġin ĠTwitter Ġshared Ġtask Ġ( sub task ĠA ) Ġat ĠSem E val Ġ2017 ĠB IB REF 41 Ġ. ĠGiven Ġa Ġtweet , Ġthe Ġgoal Ġis Ġto Ġpredict Ġwhether Ġit Ġexpresses ĠPOS IT IVE , ĠN EG ATIVE Ġor ĠNE UT RAL Ġsentiment . ĠThe Ġtraining Ġand Ġdevelopment Ġsets Ġconsist Ġof Ġ49 , 669 Ġtweets Ġand Ġwe Ġuse Ġ40 , 000 Ġfor Ġtraining Ġand Ġthe Ġrest Ġfor Ġdevelopment . ĠThere Ġare Ġa Ġtotal Ġof Ġ12 , 284 Ġtweets Ġcontaining Ġ12 , 128 Ġhasht ags Ġin Ġthe ĠSem E val Ġ2017 Ġtest Ġset , Ġand Ġour Ġhashtag Ġsegment er Ġended Ġup Ġsplitting Ġ6 , 975 Ġof Ġthose Ġhasht ags Ġpresent Ġin Ġ3 , 384 Ġtweets . </s> Results Ġand ĠAnalysis </s> In ĠTable ĠT AB REF 59 Ġ, Ġwe Ġreport Ġthe Ġresults Ġbased Ġon Ġthe Ġ3 , 384 Ġtweets Ġwhere ĠHash tag Master Ġpredicted Ġa Ġsplit , Ġas Ġfor Ġthe Ġrest Ġof Ġtweets Ġin Ġthe Ġtest Ġset , Ġthe Ġhashtag Ġsegment er Ġwould Ġneither Ġimprove Ġnor Ġworsen Ġthe Ġsentiment Ġprediction . ĠOur Ġhashtag Ġsegment er Ġsuccessfully Ġimproved Ġthe Ġsentiment Ġanalysis Ġperformance Ġby Ġ2 % Ġon Ġaverage Ġrecall Ġand ĠF ĠIN LINE FORM 0 Ġcomparing Ġto Ġhaving Ġhasht ags Ġunse gment ed . ĠThis Ġimprovement Ġis Ġseemingly Ġsmall Ġbut Ġdecidedly Ġimportant Ġfor Ġtweets Ġwhere Ġsentiment - related Ġinformation Ġis Ġembedded Ġin Ġmulti - word Ġhasht ags Ġand Ġsentiment Ġprediction Ġwould Ġbe Ġincorrect Ġbased Ġonly Ġon Ġthe Ġtext Ġ( see ĠTable ĠT AB REF 60 Ġfor Ġexamples ). ĠIn Ġfact , Ġ2 , 605 Ġout Ġof Ġthe Ġ3 , 384 Ġtweets Ġhave Ġmulti - word Ġhasht ags Ġthat Ġcontain Ġwords Ġin Ġthe ĠTwitter - based Ġsentiment Ġlex icon ĠB IB REF 44 Ġand Ġ125 Ġtweets Ġcontain Ġsentiment Ġwords Ġonly Ġin Ġthe Ġhasht ags Ġbut Ġnot Ġin Ġthe Ġrest Ġof Ġthe Ġtweet . ĠOn Ġthe Ġentire Ġtest Ġset Ġof Ġ12 , 284 Ġtweets , Ġthe Ġincrease Ġin Ġthe Ġaverage Ġrecall Ġis Ġ0 . 5 %. </s> Other ĠRelated ĠWork </s> Aut omatic Ġhashtag Ġsegment ation Ġcan Ġimprove Ġthe Ġperformance Ġof Ġmany Ġapplications Ġbesides Ġsentiment Ġanalysis , Ġsuch Ġas Ġtext Ġclassification ĠB IB REF 13 Ġ, Ġnamed Ġentity Ġlinking ĠB IB REF 10 Ġand Ġmodeling Ġuser Ġinterests Ġfor Ġrecommendations ĠB IB REF 45 Ġ. ĠIt Ġcan Ġalso Ġhelp Ġin Ġcollecting Ġdata Ġof Ġhigher Ġvolume Ġand Ġquality Ġby Ġproviding Ġa Ġmore Ġnuanced Ġinterpretation Ġof Ġits Ġcontent , Ġas Ġshown Ġfor Ġemotion Ġanalysis ĠB IB REF 46 Ġ, Ġsarc asm Ġand Ġirony Ġdetection ĠB IB REF 11 Ġ, ĠB IB REF 47 Ġ. ĠBetter Ġsemantic Ġanalysis Ġof Ġhasht ags Ġcan Ġalso Ġpotentially Ġbe Ġapplied Ġto Ġhashtag Ġannotation ĠB IB REF 48 Ġ, Ġto Ġimprove Ġdistant Ġsupervision Ġlabels Ġin Ġtraining Ġclass ifiers Ġfor Ġtasks Ġsuch Ġas Ġsarc asm ĠB IB REF 5 Ġ, Ġsentiment ĠB IB REF 4 Ġ, Ġemotions ĠB IB REF 3 Ġ; Ġand , Ġmore Ġgenerally , Ġas Ġlabels Ġfor Ġpre - training Ġrepresentations Ġof Ġwords ĠB IB REF 49 Ġ, Ġsentences ĠB IB REF 50 Ġ, Ġand Ġimages ĠB IB REF 51 Ġ. </s> Conclusion </s> We Ġproposed Ġa Ġnew Ġpair wise Ġneural Ġranking Ġmodel Ġfor Ġhashtag Ġsegment ion Ġand Ġshowed Ġsignificant Ġperformance Ġimprovements Ġover Ġthe Ġstate - of - the - art . ĠWe Ġalso Ġconstructed Ġa Ġlarger Ġand Ġmore Ġcurated Ġdataset Ġfor Ġanalyzing Ġand Ġbenchmark ing Ġhashtag Ġsegment ation Ġmethods . ĠWe Ġdemonstrated Ġthat Ġhashtag Ġsegment ation Ġhelps Ġwith Ġdownstream Ġtasks Ġsuch Ġas Ġsentiment Ġanalysis . ĠAlthough Ġwe Ġfocused Ġon ĠEnglish Ġhasht ags , Ġour Ġpair wise Ġranking Ġapproach Ġis Ġlanguage - independent Ġand Ġwe Ġintend Ġto Ġextend Ġour Ġtool kit Ġto Ġlanguages Ġother Ġthan ĠEnglish Ġas Ġfuture Ġwork . </s> Acknowled gments </s> We Ġthank ĠOhio ĠSuper computer ĠCenter ĠB IB REF 52 Ġfor Ġcomputing Ġresources Ġand Ġthe ĠNVIDIA Ġfor Ġproviding ĠGPU Ġhardware . ĠWe Ġthank ĠAlan ĠR itter , ĠQuan ze ĠChen , ĠWang ĠLing , ĠP rav ar ĠMah aj an , Ġand ĠD ush y anta ĠD hy ani Ġfor Ġvaluable Ġdiscussions . ĠWe Ġalso Ġthank Ġthe Ġannot ators : ĠSarah ĠFlan agan , ĠKa ush ik ĠMan i , Ġand ĠAs w ath nar ayan ĠRad hak rish nan . ĠThis Ġmaterial Ġis Ġbased Ġin Ġpart Ġon Ġresearch Ġsponsored Ġby Ġthe ĠNS F Ġunder Ġgrants ĠI IS - 18 22 754 Ġand ĠI IS - 17 558 98 , ĠDAR PA Ġthrough Ġthe ĠAR O Ġunder Ġagreement Ġnumber ĠW 911 NF - 17 - C - 00 95 , Ġthrough Ġa ĠFigure - Eight Ġ( C rowd Fl ower ) ĠAI Ġfor ĠEveryone ĠAward Ġand Ġa ĠCr ite o ĠFaculty ĠResearch ĠAward Ġto ĠWei ĠXu . ĠThe Ġviews Ġand Ġconclusions Ġcontained Ġin Ġthis Ġpublication Ġare Ġthose Ġof Ġthe Ġauthors Ġand Ġshould Ġnot Ġbe Ġinterpreted Ġas Ġrepresenting Ġofficial Ġpolicies Ġor Ġendorsements Ġof Ġthe ĠU . S . ĠGovernment . </s> Word - shape Ġrules </s> Our Ġmodel Ġuses Ġthe Ġfollowing Ġword Ġshape Ġrules Ġas Ġboolean Ġfeatures . ĠIf Ġthe Ġcandidate Ġsegment ation ĠIN LINE FORM 0 Ġand Ġits Ġcorresponding Ġhashtag ĠIN LINE FORM 1 Ġsatisfies Ġa Ġword Ġshape Ġrule , Ġthen Ġthe Ġboolean Ġfeature Ġis Ġset Ġto ĠTrue .
Introduction A hashtag is a keyphrase represented as a sequence of alphanumeric characters plus underscore, preceded by the # symbol. Hashtags play a central role in online communication by providing a tool to categorize the millions of posts generated daily on Twitter, Instagram, etc. They are useful in search, tracking content about a certain topic BIBREF0 , BIBREF1 , or discovering emerging trends BIBREF2 . Hashtags often carry very important information, such as emotion BIBREF3 , sentiment BIBREF4 , sarcasm BIBREF5 , and named entities BIBREF6 , BIBREF7 . However, inferring the semantics of hashtags is non-trivial since many hashtags contain multiple tokens joined together, which frequently leads to multiple potential interpretations (e.g., lion head vs. lionhead). Table TABREF3 shows several examples of single- and multi-token hashtags. While most hashtags represent a mix of standard tokens, named entities and event names are prevalent and pose challenges to both human and automatic comprehension, as these are more likely to be rare tokens. Hashtags also tend to be shorter to allow fast typing, to attract attention or to satisfy length limitations imposed by some social media platforms. Thus, they tend to contain a large number of abbreviations or non-standard spelling variations (e.g., #iloveu4eva) BIBREF8 , BIBREF9 , which hinders their understanding. The goal of our study is to build efficient methods for automatically splitting a hashtag into a meaningful word sequence. Our contributions are: Our new dataset includes segmentation for 12,594 unique hashtags and their associated tweets annotated in a multi-step process for higher quality than the previous dataset of 1,108 hashtags BIBREF10 . We frame the segmentation task as a pairwise ranking problem, given a set of candidate segmentations. We build several neural architectures using this problem formulation which use corpus-based, linguistic and thesaurus based features. We further propose a multi-task learning approach which jointly learns segment ranking and single- vs. multi-token hashtag classification. The latter leads to an error reduction of 24.6% over the current state-of-the-art. Finally, we demonstrate the utility of our method by using hashtag segmentation in the downstream task of sentiment analysis. Feeding the automatically segmented hashtags to a state-of-the-art sentiment analysis method on the SemEval 2017 benchmark dataset results in a 2.6% increase in the official metric for the task. Background and Preliminaries Current approaches for hashtag segmentation can be broadly divided into three categories: (a) gazeteer and rule based BIBREF11 , BIBREF12 , BIBREF13 , (b) word boundary detection BIBREF14 , BIBREF15 , and (c) ranking with language model and other features BIBREF16 , BIBREF10 , BIBREF0 , BIBREF17 , BIBREF18 . Hashtag segmentation approaches draw upon work on compound splitting for languages such as German or Finnish BIBREF19 and word segmentation BIBREF20 for languages with no spaces between words such as Chinese BIBREF21 , BIBREF22 . Similar to our work, BIBREF10 BansalBV15 extract an initial set of candidate segmentations using a sliding window, then rerank them using a linear regression model trained on lexical, bigram and other corpus-based features. The current state-of-the-art approach BIBREF14 , BIBREF15 uses maximum entropy and CRF models with a combination of language model and hand-crafted features to predict if each character in the hashtag is the beginning of a new word. Generating Candidate Segmentations. Microsoft Word Breaker BIBREF16 is, among the existing methods, a strong baseline for hashtag segmentation, as reported in BIBREF14 and BIBREF10 . It employs a beam search algorithm to extract INLINEFORM0 best segmentations as ranked by the n-gram language model probability: INLINEFORM1 where INLINEFORM0 is the word sequence of segmentation INLINEFORM1 and INLINEFORM2 is the window size. More sophisticated ranking strategies, such as Binomial and word length distribution based ranking, did not lead to a further improvement in performance BIBREF16 . The original Word Breaker was designed for segmenting URLs using language models trained on web data. In this paper, we reimplemented and tailored this approach to segmenting hashtags by using a language model specifically trained on Twitter data (implementation details in § SECREF26 ). The performance of this method itself is competitive with state-of-the-art methods (evaluation results in § SECREF46 ). Our proposed pairwise ranking method will effectively take the top INLINEFORM3 segmentations generated by this baseline as candidates for reranking. However, in prior work, the ranking scores of each segmentation were calculated independently, ignoring the relative order among the top INLINEFORM0 candidate segmentations. To address this limitation, we utilize a pairwise ranking strategy for the first time for this task and propose neural architectures to model this. Multi-task Pairwise Neural Ranking We propose a multi-task pairwise neural ranking approach to better incorporate and distinguish the relative order between the candidate segmentations of a given hashtag. Our model adapts to address single- and multi-token hashtags differently via a multi-task learning strategy without requiring additional annotations. In this section, we describe the task setup and three variants of pairwise neural ranking models (Figure FIGREF11 ). Segmentation as Pairwise Ranking The goal of hashtag segmentation is to divide a given hashtag INLINEFORM0 into a sequence of meaningful words INLINEFORM1 . For a hashtag of INLINEFORM2 characters, there are a total of INLINEFORM3 possible segmentations but only one, or occasionally two, of them ( INLINEFORM4 ) are considered correct (Table TABREF9 ). We transform this task into a pairwise ranking problem: given INLINEFORM0 candidate segmentations { INLINEFORM1 }, we rank them by comparing each with the rest in a pairwise manner. More specifically, we train a model to predict a real number INLINEFORM2 for any two candidate segmentations INLINEFORM3 and INLINEFORM4 of hashtag INLINEFORM5 , which indicates INLINEFORM6 is a better segmentation than INLINEFORM7 if positive, and vice versa. To quantify the quality of a segmentation in training, we define a gold scoring function INLINEFORM8 based on the similarities with the ground-truth segmentation INLINEFORM9 : INLINEFORM10 We use the Levenshtein distance (minimum number of single-character edits) in this paper, although it is possible to use other similarity measurements as alternatives. We use the top INLINEFORM0 segmentations generated by Microsoft Word Breaker (§ SECREF2 ) as initial candidates. Pairwise Neural Ranking Model For an input candidate segmentation pair INLINEFORM0 , we concatenate their feature vectors INLINEFORM1 and INLINEFORM2 , and feed them into a feedforward network which emits a comparison score INLINEFORM3 . The feature vector INLINEFORM4 or INLINEFORM5 consists of language model probabilities using Good-Turing BIBREF23 and modified Kneser-Ney smoothing BIBREF24 , BIBREF25 , lexical and linguistic features (more details in § SECREF23 ). For training, we use all the possible pairs INLINEFORM6 of the INLINEFORM7 candidates as the input and their gold scores INLINEFORM8 as the target. The training objective is to minimize the Mean Squared Error (MSE): DISPLAYFORM0 where INLINEFORM0 is the number of training examples. To aggregate the pairwise comparisons, we follow a greedy algorithm proposed by BIBREF26 cohen1998learning and used for preference ranking BIBREF27 . For each segmentation INLINEFORM0 in the candidate set INLINEFORM1 , we calculate a single score INLINEFORM2 , and find the segmentation INLINEFORM3 corresponding to the highest score. We repeat the same procedure after removing INLINEFORM4 from INLINEFORM5 , and continue until INLINEFORM6 reduces to an empty set. Figure FIGREF11 (a) shows the architecture of this model. Margin Ranking (MR) Loss As an alternative to the pairwise ranker (§ SECREF15 ), we propose a pairwise model which learns from candidate pairs INLINEFORM0 but ranks each individual candidate directly rather than relatively. We define a new scoring function INLINEFORM1 which assigns a higher score to the better candidate, i.e., INLINEFORM2 , if INLINEFORM3 is a better candidate than INLINEFORM4 and vice-versa. Instead of concatenating the features vectors INLINEFORM5 and INLINEFORM6 , we feed them separately into two identical feedforward networks with shared parameters. During testing, we use only one of the networks to rank the candidates based on the INLINEFORM7 scores. For training, we add a ranking layer on top of the networks to measure the violations in the ranking order and minimize the Margin Ranking Loss (MR): DISPLAYFORM0 where INLINEFORM0 is the number of training samples. The architecture of this model is presented in Figure FIGREF11 (b). Adaptive Multi-task Learning Both models in § SECREF15 and § SECREF17 treat all the hashtags uniformly. However, different features address different types of hashtags. By design, the linguistic features capture named entities and multi-word hashtags that exhibit word shape patterns, such as camel case. The ngram probabilities with Good-Turing smoothing gravitate towards multi-word segmentations with known words, as its estimate for unseen ngrams depends on the fraction of ngrams seen once which can be very low BIBREF28 . The modified Kneser-Ney smoothing is more likely to favor segmentations that contain rare words, and single-word segmentations in particular. Please refer to § SECREF46 for a more detailed quantitative and qualitative analysis. To leverage this intuition, we introduce a binary classification task to help the model differentiate single-word from multi-word hashtags. The binary classifier takes hashtag features INLINEFORM0 as the input and outputs INLINEFORM1 , which represents the probability of INLINEFORM2 being a multi-word hashtag. INLINEFORM3 is used as an adaptive gating value in our multi-task learning setup. The gold labels for this task are obtained at no extra cost by simply verifying whether the ground-truth segmentation has multiple words. We train the pairwise segmentation ranker and the binary single- vs. multi-token hashtag classifier jointly, by minimizing INLINEFORM4 for the pairwise ranker and the Binary Cross Entropy Error ( INLINEFORM5 ) for the classifier: DISPLAYFORM0 where INLINEFORM0 is the adaptive gating value, INLINEFORM1 indicates if INLINEFORM2 is actually a multi-word hashtag and INLINEFORM3 is the number of training examples. INLINEFORM4 and INLINEFORM5 are the weights for each loss. For our experiments, we apply equal weights. More specifically, we divide the segmentation feature vector INLINEFORM0 into two subsets: (a) INLINEFORM1 with modified Kneser-Ney smoothing features, and (b) INLINEFORM2 with Good-Turing smoothing and linguistic features. For an input candidate segmentation pair INLINEFORM3 , we construct two pairwise vectors INLINEFORM4 and INLINEFORM5 by concatenation, then combine them based on the adaptive gating value INLINEFORM6 before feeding them into the feedforward network INLINEFORM7 for pairwise ranking: DISPLAYFORM0 We use summation with padding, as we find this simple ensemble method achieves similar performance in our experiments as the more complex multi-column networks BIBREF29 . Figure FIGREF11 (c) shows the architecture of this model. An analogue multi-task formulation can also be used for the Margin Ranking loss as: DISPLAYFORM0 Features We use a combination of corpus-based and linguistic features to rank the segmentations. For a candidate segmentation INLINEFORM0 , its feature vector INLINEFORM1 includes the number of words in the candidate, the length of each word, the proportion of words in an English dictionary or Urban Dictionary BIBREF30 , ngram counts from Google Web 1TB corpus BIBREF31 , and ngram probabilities from trigram language models trained on the Gigaword corpus BIBREF32 and 1.1 billion English tweets from 2010, respectively. We train two language models on each corpus: one with Good-Turing smoothing using SRILM BIBREF33 and the other with modified Kneser-Ney smoothing using KenLM BIBREF34 . We also add boolean features, such as if the candidate is a named-entity present in the list of Wikipedia titles, and if the candidate segmentation INLINEFORM2 and its corresponding hashtag INLINEFORM3 satisfy certain word-shapes (more details in appendix SECREF61 ). Similarly, for hashtag INLINEFORM0 , we extract the feature vector INLINEFORM1 consisting of hashtag length, ngram count of the hashtag in Google 1TB corpus BIBREF31 , and boolean features indicating if the hashtag is in an English dictionary or Urban Dictionary, is a named-entity, is in camel case, ends with a number, and has all the letters as consonants. We also include features of the best-ranked candidate by the Word Breaker model. Implementation Details We use the PyTorch framework to implement our multi-task pairwise ranking model. The pairwise ranker consists of an input layer, three hidden layers with eight nodes in each layer and hyperbolic tangent ( INLINEFORM0 ) activation, and a single linear output node. The auxiliary classifier consists of an input layer, one hidden layer with eight nodes and one output node with sigmoid activation. We use the Adam algorithm BIBREF35 for optimization and apply a dropout of 0.5 to prevent overfitting. We set the learning rate to 0.01 and 0.05 for the pairwise ranker and auxiliary classifier respectively. For each experiment, we report results obtained after 100 epochs. For the baseline model used to extract the INLINEFORM0 initial candidates, we reimplementated the Word Breaker BIBREF16 as described in § SECREF2 and adapted it to use a language model trained on 1.1 billion tweets with Good-Turing smoothing using SRILM BIBREF33 to give a better performance in segmenting hashtags (§ SECREF46 ). For all our experiments, we set INLINEFORM1 . Hashtag Segmentation Data We use two datasets for experiments (Table TABREF29 ): (a) STAN INLINEFORM0 , created by BIBREF10 BansalBV15, which consists of 1,108 unique English hashtags from 1,268 randomly selected tweets in the Stanford Sentiment Analysis Dataset BIBREF36 along with their crowdsourced segmentations and our additional corrections; and (b) STAN INLINEFORM1 , our new expert curated dataset, which includes all 12,594 unique English hashtags and their associated tweets from the same Stanford dataset. Experiments In this section, we present experimental results that compare our proposed method with the other state-of-the-art approaches on hashtag segmentation datasets. The next section will show experiments of applying hashtag segmentation to the popular task of sentiment analysis. Existing Methods We compare our pairwise neural ranker with the following baseline and state-of-the-art approaches: The original hashtag as a single token; A rule-based segmenter, which employs a set of word-shape rules with an English dictionary BIBREF13 ; A Viterbi model which uses word frequencies from a book corpus BIBREF0 ; The specially developed GATE Hashtag Tokenizer from the open source toolkit, which combines dictionaries and gazetteers in a Viterbi-like algorithm BIBREF11 ; A maximum entropy classifier (MaxEnt) trained on the STAN INLINEFORM0 training dataset. It predicts whether a space should be inserted at each position in the hashtag and is the current state-of-the-art BIBREF14 ; Our reimplementation of the Word Breaker algorithm which uses beam search and a Twitter ngram language model BIBREF16 ; A pairwise linear ranker which we implemented for comparison purposes with the same features as our neural model, but using perceptron as the underlying classifier BIBREF38 and minimizing the hinge loss between INLINEFORM0 and a scoring function similar to INLINEFORM1 . It is trained on the STAN INLINEFORM2 dataset. Evaluation Metrics We evaluate the performance by the top INLINEFORM0 ( INLINEFORM1 ) accuracy (A@1, A@2), average token-level F INLINEFORM2 score (F INLINEFORM3 @1), and mean reciprocal rank (MRR). In particular, the accuracy and MRR are calculated at the segmentation-level, which means that an output segmentation is considered correct if and only if it fully matches the human segmentation. Average token-level F INLINEFORM4 score accounts for partially correct segmentation in the multi-token hashtag cases. Results Tables TABREF32 and TABREF33 show the results on the STAN INLINEFORM0 and STAN INLINEFORM1 datasets, respectively. All of our pairwise neural rankers are trained on the 2,518 manually segmented hashtags in the training set of STAN INLINEFORM2 and perform favorably against other state-of-the-art approaches. Our best model (MSE+multitask) that utilizes different features adaptively via a multi-task learning procedure is shown to perform better than simply combining all the features together (MR and MSE). We highlight the 24.6% error reduction on STAN INLINEFORM3 and 16.5% on STAN INLINEFORM4 of our approach over the previous SOTA BIBREF14 on the Multi-token hashtags, and the importance of having a separate evaluation of multi-word cases as it is trivial to obtain 100% accuracy for Single-token hashtags. While our hashtag segmentation model is achieving a very high accuracy@2, to be practically useful, it remains a challenge to get the top one predication exactly correct. Some hashtags are very difficult to interpret, e.g., #BTVbrownSMB refers to the Social Media Breakfast (SMB) in Burlington, Vermont (BTV). The improved Word Breaker with our addition of a Twitter-specific language model is a very strong baseline, which echos the findings of the original Word Breaker paper BIBREF16 that having a large in-domain language model is extremely helpful for word segmentation tasks. It is worth noting that the other state-of-the-art system BIBREF14 also utilized a 4-gram language model trained on 476 million tweets from 2009. Analysis and Discussion To empirically illustrate the effectiveness of different features on different types of hashtags, we show the results for models using individual feature sets in pairwise ranking models (MSE) in Table TABREF45 . Language models with modified Kneser-Ney smoothing perform best on single-token hashtags, while Good-Turing and Linguistic features work best on multi-token hashtags, confirming our intuition about their usefulness in a multi-task learning approach. Table TABREF47 shows a qualitative analysis with the first column ( INLINEFORM0 INLINEFORM1 INLINEFORM2 ) indicating which features lead to correct or wrong segmentations, their count in our data and illustrative examples with human segmentation. As expected, longer hashtags with more than three tokens pose greater challenges and the segmentation-level accuracy of our best model (MSE+multitask) drops to 82.1%. For many error cases, our model predicts a close-to-correct segmentation, e.g., #youbrownknowyoubrownupttoobrownearly, #iseebrownlondoniseebrownfrance, which is also reflected by the higher token-level F INLINEFORM0 scores across hashtags with different lengths (Figure FIGREF51 ). Since our approach heavily relies on building a Twitter language model, we experimented with its sizes and show the results in Figure FIGREF52 . Our approach can perform well even with access to a smaller amount of tweets. The drop in F INLINEFORM0 score for our pairwise neural ranker is only 1.4% and 3.9% when using the language models trained on 10% and 1% of the total 1.1 billion tweets, respectively. Language use in Twitter changes with time BIBREF9 . Our pairwise ranker uses language models trained on the tweets from the year 2010. We tested our approach on a set of 500 random English hashtags posted in tweets from the year 2019 and show the results in Table TABREF55 . With a segmentation-level accuracy of 94.6% and average token-level F INLINEFORM0 score of 95.6%, our approach performs favorably on 2019 hashtags. Extrinsic Evaluation: Twitter Sentiment Analysis We attempt to demonstrate the effectiveness of our hashtag segmentation system by studying its impact on the task of sentiment analysis in Twitter BIBREF39 , BIBREF40 , BIBREF41 . We use our best model (MSE+multitask), under the name HashtagMaster, in the following experiments. Experimental Setup We compare the performance of the BiLSTM+Lex BIBREF42 sentiment analysis model under three configurations: (a) tweets with hashtags removed, (b) tweets with hashtags as single tokens excluding the # symbol, and (c) tweets with hashtags as segmented by our system, HashtagMaster. BiLSTM+Lex is a state-of-the-art open source system for predicting tweet-level sentiment BIBREF43 . It learns a context-sensitive sentiment intensity score by leveraging a Twitter-based sentiment lexicon BIBREF44 . We use the same settings as described by BIBREF42 teng-vo-zhang:2016:EMNLP2016 to train the model. We use the dataset from the Sentiment Analysis in Twitter shared task (subtask A) at SemEval 2017 BIBREF41 . Given a tweet, the goal is to predict whether it expresses POSITIVE, NEGATIVE or NEUTRAL sentiment. The training and development sets consist of 49,669 tweets and we use 40,000 for training and the rest for development. There are a total of 12,284 tweets containing 12,128 hashtags in the SemEval 2017 test set, and our hashtag segmenter ended up splitting 6,975 of those hashtags present in 3,384 tweets. Results and Analysis In Table TABREF59 , we report the results based on the 3,384 tweets where HashtagMaster predicted a split, as for the rest of tweets in the test set, the hashtag segmenter would neither improve nor worsen the sentiment prediction. Our hashtag segmenter successfully improved the sentiment analysis performance by 2% on average recall and F INLINEFORM0 comparing to having hashtags unsegmented. This improvement is seemingly small but decidedly important for tweets where sentiment-related information is embedded in multi-word hashtags and sentiment prediction would be incorrect based only on the text (see Table TABREF60 for examples). In fact, 2,605 out of the 3,384 tweets have multi-word hashtags that contain words in the Twitter-based sentiment lexicon BIBREF44 and 125 tweets contain sentiment words only in the hashtags but not in the rest of the tweet. On the entire test set of 12,284 tweets, the increase in the average recall is 0.5%. Other Related Work Automatic hashtag segmentation can improve the performance of many applications besides sentiment analysis, such as text classification BIBREF13 , named entity linking BIBREF10 and modeling user interests for recommendations BIBREF45 . It can also help in collecting data of higher volume and quality by providing a more nuanced interpretation of its content, as shown for emotion analysis BIBREF46 , sarcasm and irony detection BIBREF11 , BIBREF47 . Better semantic analysis of hashtags can also potentially be applied to hashtag annotation BIBREF48 , to improve distant supervision labels in training classifiers for tasks such as sarcasm BIBREF5 , sentiment BIBREF4 , emotions BIBREF3 ; and, more generally, as labels for pre-training representations of words BIBREF49 , sentences BIBREF50 , and images BIBREF51 . Conclusion We proposed a new pairwise neural ranking model for hashtag segmention and showed significant performance improvements over the state-of-the-art. We also constructed a larger and more curated dataset for analyzing and benchmarking hashtag segmentation methods. We demonstrated that hashtag segmentation helps with downstream tasks such as sentiment analysis. Although we focused on English hashtags, our pairwise ranking approach is language-independent and we intend to extend our toolkit to languages other than English as future work. Acknowledgments We thank Ohio Supercomputer Center BIBREF52 for computing resources and the NVIDIA for providing GPU hardware. We thank Alan Ritter, Quanze Chen, Wang Ling, Pravar Mahajan, and Dushyanta Dhyani for valuable discussions. We also thank the annotators: Sarah Flanagan, Kaushik Mani, and Aswathnarayan Radhakrishnan. This material is based in part on research sponsored by the NSF under grants IIS-1822754 and IIS-1755898, DARPA through the ARO under agreement number W911NF-17-C-0095, through a Figure-Eight (CrowdFlower) AI for Everyone Award and a Criteo Faculty Research Award to Wei Xu. The views and conclusions contained in this publication are those of the authors and should not be interpreted as representing official policies or endorsements of the U.S. Government. Word-shape rules Our model uses the following word shape rules as boolean features. If the candidate segmentation INLINEFORM0 and its corresponding hashtag INLINEFORM1 satisfies a word shape rule, then the boolean feature is set to True.
What set of approaches to hashtag segmentation are proposed?
4aa9b60c0ccd379c6fb089c84a6c7b872ee9ec4f
What Ġset Ġof Ġapproaches Ġto Ġhashtag Ġsegment ation Ġare Ġproposed ?
Adapt ive ĠMulti - task ĠLearning Ċ , ĠMarg in ĠRanking Ġ( MR ) ĠLoss Ċ , ĠPair wise ĠNeural ĠRanking ĠModel Ċ
<s> What Ġset Ġof Ġapproaches Ġto Ġhashtag Ġsegment ation Ġare Ġproposed ? </s> Introduction </s> A Ġhashtag Ġis Ġa Ġkey phrase Ġrepresented Ġas Ġa Ġsequence Ġof Ġal phan umeric Ġcharacters Ġplus Ġunderscore , Ġpreceded Ġby Ġthe Ġ# Ġsymbol . ĠHash tags Ġplay Ġa Ġcentral Ġrole Ġin Ġonline Ġcommunication Ġby Ġproviding Ġa Ġtool Ġto Ġcategor ize Ġthe Ġmillions Ġof Ġposts Ġgenerated Ġdaily Ġon ĠTwitter , ĠInstagram , Ġetc . ĠThey Ġare Ġuseful Ġin Ġsearch , Ġtracking Ġcontent Ġabout Ġa Ġcertain Ġtopic ĠB IB REF 0 Ġ, ĠB IB REF 1 Ġ, Ġor Ġdiscovering Ġemerging Ġtrends ĠB IB REF 2 Ġ. </s> Hash tags Ġoften Ġcarry Ġvery Ġimportant Ġinformation , Ġsuch Ġas Ġemotion ĠB IB REF 3 Ġ, Ġsentiment ĠB IB REF 4 Ġ, Ġsarc asm ĠB IB REF 5 Ġ, Ġand Ġnamed Ġentities ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ. ĠHowever , Ġinfer ring Ġthe Ġsemantics Ġof Ġhasht ags Ġis Ġnon - t riv ial Ġsince Ġmany Ġhasht ags Ġcontain Ġmultiple Ġtokens Ġjoined Ġtogether , Ġwhich Ġfrequently Ġleads Ġto Ġmultiple Ġpotential Ġinterpretations Ġ( e . g ., Ġlion Ġhead Ġvs . Ġlion head ). ĠTable ĠT AB REF 3 Ġshows Ġseveral Ġexamples Ġof Ġsingle - Ġand Ġmulti - token Ġhasht ags . ĠWhile Ġmost Ġhasht ags Ġrepresent Ġa Ġmix Ġof Ġstandard Ġtokens , Ġnamed Ġentities Ġand Ġevent Ġnames Ġare Ġprevalent Ġand Ġpose Ġchallenges Ġto Ġboth Ġhuman Ġand Ġautomatic Ġcomprehension , Ġas Ġthese Ġare Ġmore Ġlikely Ġto Ġbe Ġrare Ġtokens . ĠHash tags Ġalso Ġtend Ġto Ġbe Ġshorter Ġto Ġallow Ġfast Ġtyping , Ġto Ġattract Ġattention Ġor Ġto Ġsatisfy Ġlength Ġlimitations Ġimposed Ġby Ġsome Ġsocial Ġmedia Ġplatforms . ĠThus , Ġthey Ġtend Ġto Ġcontain Ġa Ġlarge Ġnumber Ġof Ġabbrevi ations Ġor Ġnon - standard Ġspelling Ġvariations Ġ( e . g ., Ġ# il ove u 4 eva ) ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġ, Ġwhich Ġhind ers Ġtheir Ġunderstanding . </s> The Ġgoal Ġof Ġour Ġstudy Ġis Ġto Ġbuild Ġefficient Ġmethods Ġfor Ġautomatically Ġsplitting Ġa Ġhashtag Ġinto Ġa Ġmeaningful Ġword Ġsequence . ĠOur Ġcontributions Ġare : </s> Our Ġnew Ġdataset Ġincludes Ġsegment ation Ġfor Ġ12 , 594 Ġunique Ġhasht ags Ġand Ġtheir Ġassociated Ġtweets Ġannot ated Ġin Ġa Ġmulti - step Ġprocess Ġfor Ġhigher Ġquality Ġthan Ġthe Ġprevious Ġdataset Ġof Ġ1 , 108 Ġhasht ags ĠB IB REF 10 Ġ. ĠWe Ġframe Ġthe Ġsegment ation Ġtask Ġas Ġa Ġpair wise Ġranking Ġproblem , Ġgiven Ġa Ġset Ġof Ġcandidate Ġsegment ations . ĠWe Ġbuild Ġseveral Ġneural Ġarchitectures Ġusing Ġthis Ġproblem Ġformulation Ġwhich Ġuse Ġcorpus - based , Ġlinguistic Ġand Ġthe s aurus Ġbased Ġfeatures . ĠWe Ġfurther Ġpropose Ġa Ġmulti - task Ġlearning Ġapproach Ġwhich Ġjointly Ġlearns Ġsegment Ġranking Ġand Ġsingle - Ġvs . Ġmulti - token Ġhashtag Ġclassification . ĠThe Ġlatter Ġleads Ġto Ġan Ġerror Ġreduction Ġof Ġ24 . 6 % Ġover Ġthe Ġcurrent Ġstate - of - the - art . ĠFinally , Ġwe Ġdemonstrate Ġthe Ġutility Ġof Ġour Ġmethod Ġby Ġusing Ġhashtag Ġsegment ation Ġin Ġthe Ġdownstream Ġtask Ġof Ġsentiment Ġanalysis . ĠFeed ing Ġthe Ġautomatically Ġsegment ed Ġhasht ags Ġto Ġa Ġstate - of - the - art Ġsentiment Ġanalysis Ġmethod Ġon Ġthe ĠSem E val Ġ2017 Ġbenchmark Ġdataset Ġresults Ġin Ġa Ġ2 . 6 % Ġincrease Ġin Ġthe Ġofficial Ġmetric Ġfor Ġthe Ġtask . </s> Background Ġand ĠPrel im in aries </s> Current Ġapproaches Ġfor Ġhashtag Ġsegment ation Ġcan Ġbe Ġbroadly Ġdivided Ġinto Ġthree Ġcategories : Ġ( a ) Ġg az ete er Ġand Ġrule Ġbased ĠB IB REF 11 Ġ, ĠB IB REF 12 Ġ, ĠB IB REF 13 Ġ, Ġ( b ) Ġword Ġboundary Ġdetection ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, Ġand Ġ( c ) Ġranking Ġwith Ġlanguage Ġmodel Ġand Ġother Ġfeatures ĠB IB REF 16 Ġ, ĠB IB REF 10 Ġ, ĠB IB REF 0 Ġ, ĠB IB REF 17 Ġ, ĠB IB REF 18 Ġ. ĠHash tag Ġsegment ation Ġapproaches Ġdraw Ġupon Ġwork Ġon Ġcompound Ġsplitting Ġfor Ġlanguages Ġsuch Ġas ĠGerman Ġor ĠFinnish ĠB IB REF 19 Ġand Ġword Ġsegment ation ĠB IB REF 20 Ġfor Ġlanguages Ġwith Ġno Ġspaces Ġbetween Ġwords Ġsuch Ġas ĠChinese ĠB IB REF 21 Ġ, ĠB IB REF 22 Ġ. ĠSimilar Ġto Ġour Ġwork , ĠB IB REF 10 ĠBans al B V 15 Ġextract Ġan Ġinitial Ġset Ġof Ġcandidate Ġsegment ations Ġusing Ġa Ġsliding Ġwindow , Ġthen Ġre rank Ġthem Ġusing Ġa Ġlinear Ġregression Ġmodel Ġtrained Ġon Ġlex ical , Ġbig ram Ġand Ġother Ġcorpus - based Ġfeatures . ĠThe Ġcurrent Ġstate - of - the - art Ġapproach ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġuses Ġmaximum Ġentropy Ġand ĠCR F Ġmodels Ġwith Ġa Ġcombination Ġof Ġlanguage Ġmodel Ġand Ġhand - crafted Ġfeatures Ġto Ġpredict Ġif Ġeach Ġcharacter Ġin Ġthe Ġhashtag Ġis Ġthe Ġbeginning Ġof Ġa Ġnew Ġword . </s> Gener ating ĠCandidate ĠSe gment ations . ĠMicrosoft ĠWord ĠBreaker ĠB IB REF 16 Ġis , Ġamong Ġthe Ġexisting Ġmethods , Ġa Ġstrong Ġbaseline Ġfor Ġhashtag Ġsegment ation , Ġas Ġreported Ġin ĠB IB REF 14 Ġand ĠB IB REF 10 Ġ. ĠIt Ġemploys Ġa Ġbeam Ġsearch Ġalgorithm Ġto Ġextract ĠIN LINE FORM 0 Ġbest Ġsegment ations Ġas Ġranked Ġby Ġthe Ġn - gram Ġlanguage Ġmodel Ġprobability : ĠIN LINE FORM 1 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġword Ġsequence Ġof Ġsegment ation ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġis Ġthe Ġwindow Ġsize . ĠMore Ġsophisticated Ġranking Ġstrategies , Ġsuch Ġas ĠBin omial Ġand Ġword Ġlength Ġdistribution Ġbased Ġranking , Ġdid Ġnot Ġlead Ġto Ġa Ġfurther Ġimprovement Ġin Ġperformance ĠB IB REF 16 Ġ. ĠThe Ġoriginal ĠWord ĠBreaker Ġwas Ġdesigned Ġfor Ġsegment ing ĠURLs Ġusing Ġlanguage Ġmodels Ġtrained Ġon Ġweb Ġdata . ĠIn Ġthis Ġpaper , Ġwe Ġreim ple mented Ġand Ġtailored Ġthis Ġapproach Ġto Ġsegment ing Ġhasht ags Ġby Ġusing Ġa Ġlanguage Ġmodel Ġspecifically Ġtrained Ġon ĠTwitter Ġdata Ġ( im plementation Ġdetails Ġin ĠÂ§ ĠSEC REF 26 Ġ). ĠThe Ġperformance Ġof Ġthis Ġmethod Ġitself Ġis Ġcompetitive Ġwith Ġstate - of - the - art Ġmethods Ġ( eval uation Ġresults Ġin ĠÂ§ ĠSEC REF 46 Ġ). ĠOur Ġproposed Ġpair wise Ġranking Ġmethod Ġwill Ġeffectively Ġtake Ġthe Ġtop ĠIN LINE FORM 3 Ġsegment ations Ġgenerated Ġby Ġthis Ġbaseline Ġas Ġcandidates Ġfor Ġre ranking . </s> However , Ġin Ġprior Ġwork , Ġthe Ġranking Ġscores Ġof Ġeach Ġsegment ation Ġwere Ġcalculated Ġindependently , Ġignoring Ġthe Ġrelative Ġorder Ġamong Ġthe Ġtop ĠIN LINE FORM 0 Ġcandidate Ġsegment ations . ĠTo Ġaddress Ġthis Ġlimitation , Ġwe Ġutilize Ġa Ġpair wise Ġranking Ġstrategy Ġfor Ġthe Ġfirst Ġtime Ġfor Ġthis Ġtask Ġand Ġpropose Ġneural Ġarchitectures Ġto Ġmodel Ġthis . </s> Multi - task ĠPair wise ĠNeural ĠRanking </s> We Ġpropose Ġa Ġmulti - task Ġpair wise Ġneural Ġranking Ġapproach Ġto Ġbetter Ġincorporate Ġand Ġdistinguish Ġthe Ġrelative Ġorder Ġbetween Ġthe Ġcandidate Ġsegment ations Ġof Ġa Ġgiven Ġhashtag . ĠOur Ġmodel Ġadapt s Ġto Ġaddress Ġsingle - Ġand Ġmulti - token Ġhasht ags Ġdifferently Ġvia Ġa Ġmulti - task Ġlearning Ġstrategy Ġwithout Ġrequiring Ġadditional Ġannotations . ĠIn Ġthis Ġsection , Ġwe Ġdescribe Ġthe Ġtask Ġsetup Ġand Ġthree Ġvariants Ġof Ġpair wise Ġneural Ġranking Ġmodels Ġ( Figure ĠFIG REF 11 Ġ). </s> Seg mentation Ġas ĠPair wise ĠRanking </s> The Ġgoal Ġof Ġhashtag Ġsegment ation Ġis Ġto Ġdivide Ġa Ġgiven Ġhashtag ĠIN LINE FORM 0 Ġinto Ġa Ġsequence Ġof Ġmeaningful Ġwords ĠIN LINE FORM 1 Ġ. ĠFor Ġa Ġhashtag Ġof ĠIN LINE FORM 2 Ġcharacters , Ġthere Ġare Ġa Ġtotal Ġof ĠIN LINE FORM 3 Ġpossible Ġsegment ations Ġbut Ġonly Ġone , Ġor Ġoccasionally Ġtwo , Ġof Ġthem Ġ( ĠIN LINE FORM 4 Ġ) Ġare Ġconsidered Ġcorrect Ġ( Table ĠT AB REF 9 Ġ). </s> We Ġtransform Ġthis Ġtask Ġinto Ġa Ġpair wise Ġranking Ġproblem : Ġgiven ĠIN LINE FORM 0 Ġcandidate Ġsegment ations Ġ{ ĠIN LINE FORM 1 Ġ}, Ġwe Ġrank Ġthem Ġby Ġcomparing Ġeach Ġwith Ġthe Ġrest Ġin Ġa Ġpair wise Ġmanner . ĠMore Ġspecifically , Ġwe Ġtrain Ġa Ġmodel Ġto Ġpredict Ġa Ġreal Ġnumber ĠIN LINE FORM 2 Ġfor Ġany Ġtwo Ġcandidate Ġsegment ations ĠIN LINE FORM 3 Ġand ĠIN LINE FORM 4 Ġof Ġhashtag ĠIN LINE FORM 5 Ġ, Ġwhich Ġindicates ĠIN LINE FORM 6 Ġis Ġa Ġbetter Ġsegment ation Ġthan ĠIN LINE FORM 7 Ġif Ġpositive , Ġand Ġvice Ġversa . ĠTo Ġquantify Ġthe Ġquality Ġof Ġa Ġsegment ation Ġin Ġtraining , Ġwe Ġdefine Ġa Ġgold Ġscoring Ġfunction ĠIN LINE FORM 8 Ġbased Ġon Ġthe Ġsimilarities Ġwith Ġthe Ġground - truth Ġsegment ation ĠIN LINE FORM 9 Ġ: ĠIN LINE FORM 10 </s> We Ġuse Ġthe ĠLe ven sh tein Ġdistance Ġ( minimum Ġnumber Ġof Ġsingle - character Ġedits ) Ġin Ġthis Ġpaper , Ġalthough Ġit Ġis Ġpossible Ġto Ġuse Ġother Ġsimilarity Ġmeasurements Ġas Ġalternatives . ĠWe Ġuse Ġthe Ġtop ĠIN LINE FORM 0 Ġsegment ations Ġgenerated Ġby ĠMicrosoft ĠWord ĠBreaker Ġ( Â§ ĠSEC REF 2 Ġ) Ġas Ġinitial Ġcandidates . </s> P air wise ĠNeural ĠRanking ĠModel </s> For Ġan Ġinput Ġcandidate Ġsegment ation Ġpair ĠIN LINE FORM 0 Ġ, Ġwe Ġconc aten ate Ġtheir Ġfeature Ġvectors ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ, Ġand Ġfeed Ġthem Ġinto Ġa Ġfeed forward Ġnetwork Ġwhich Ġemits Ġa Ġcomparison Ġscore ĠIN LINE FORM 3 Ġ. ĠThe Ġfeature Ġvector ĠIN LINE FORM 4 Ġor ĠIN LINE FORM 5 Ġconsists Ġof Ġlanguage Ġmodel Ġprobabilities Ġusing ĠGood - T uring ĠB IB REF 23 Ġand Ġmodified ĠK nes er - N ey Ġsmoot hing ĠB IB REF 24 Ġ, ĠB IB REF 25 Ġ, Ġlex ical Ġand Ġlinguistic Ġfeatures Ġ( more Ġdetails Ġin ĠÂ§ ĠSEC REF 23 Ġ). ĠFor Ġtraining , Ġwe Ġuse Ġall Ġthe Ġpossible Ġpairs ĠIN LINE FORM 6 Ġof Ġthe ĠIN LINE FORM 7 Ġcandidates Ġas Ġthe Ġinput Ġand Ġtheir Ġgold Ġscores ĠIN LINE FORM 8 Ġas Ġthe Ġtarget . ĠThe Ġtraining Ġobjective Ġis Ġto Ġminimize Ġthe ĠMean ĠSqu ared ĠError Ġ( M SE ): ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġnumber Ġof Ġtraining Ġexamples . </s> To Ġaggregate Ġthe Ġpair wise Ġcomparisons , Ġwe Ġfollow Ġa Ġgreedy Ġalgorithm Ġproposed Ġby ĠB IB REF 26 Ġco hen 1998 learning Ġand Ġused Ġfor Ġpreference Ġranking ĠB IB REF 27 Ġ. ĠFor Ġeach Ġsegment ation ĠIN LINE FORM 0 Ġin Ġthe Ġcandidate Ġset ĠIN LINE FORM 1 Ġ, Ġwe Ġcalculate Ġa Ġsingle Ġscore ĠIN LINE FORM 2 Ġ, Ġand Ġfind Ġthe Ġsegment ation ĠIN LINE FORM 3 Ġcorresponding Ġto Ġthe Ġhighest Ġscore . ĠWe Ġrepeat Ġthe Ġsame Ġprocedure Ġafter Ġremoving ĠIN LINE FORM 4 Ġfrom ĠIN LINE FORM 5 Ġ, Ġand Ġcontinue Ġuntil ĠIN LINE FORM 6 Ġreduces Ġto Ġan Ġempty Ġset . ĠFigure ĠFIG REF 11 Ġ( a ) Ġshows Ġthe Ġarchitecture Ġof Ġthis Ġmodel . </s> Marg in ĠRanking Ġ( MR ) ĠLoss </s> As Ġan Ġalternative Ġto Ġthe Ġpair wise Ġrank er Ġ( Â§ ĠSEC REF 15 Ġ), Ġwe Ġpropose Ġa Ġpair wise Ġmodel Ġwhich Ġlearns Ġfrom Ġcandidate Ġpairs ĠIN LINE FORM 0 Ġbut Ġranks Ġeach Ġindividual Ġcandidate Ġdirectly Ġrather Ġthan Ġrelatively . ĠWe Ġdefine Ġa Ġnew Ġscoring Ġfunction ĠIN LINE FORM 1 Ġwhich Ġassigns Ġa Ġhigher Ġscore Ġto Ġthe Ġbetter Ġcandidate , Ġi . e ., ĠIN LINE FORM 2 Ġ, Ġif ĠIN LINE FORM 3 Ġis Ġa Ġbetter Ġcandidate Ġthan ĠIN LINE FORM 4 Ġand Ġvice - vers a . ĠInstead Ġof Ġconc aten ating Ġthe Ġfeatures Ġvectors ĠIN LINE FORM 5 Ġand ĠIN LINE FORM 6 Ġ, Ġwe Ġfeed Ġthem Ġseparately Ġinto Ġtwo Ġidentical Ġfeed forward Ġnetworks Ġwith Ġshared Ġparameters . ĠDuring Ġtesting , Ġwe Ġuse Ġonly Ġone Ġof Ġthe Ġnetworks Ġto Ġrank Ġthe Ġcandidates Ġbased Ġon Ġthe ĠIN LINE FORM 7 Ġscores . ĠFor Ġtraining , Ġwe Ġadd Ġa Ġranking Ġlayer Ġon Ġtop Ġof Ġthe Ġnetworks Ġto Ġmeasure Ġthe Ġviolations Ġin Ġthe Ġranking Ġorder Ġand Ġminimize Ġthe ĠMarg in ĠRanking ĠLoss Ġ( MR ): ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġnumber Ġof Ġtraining Ġsamples . ĠThe Ġarchitecture Ġof Ġthis Ġmodel Ġis Ġpresented Ġin ĠFigure ĠFIG REF 11 Ġ( b ). </s> Adapt ive ĠMulti - task ĠLearning </s> Both Ġmodels Ġin ĠÂ§ ĠSEC REF 15 Ġand ĠÂ§ ĠSEC REF 17 Ġtreat Ġall Ġthe Ġhasht ags Ġuniformly . ĠHowever , Ġdifferent Ġfeatures Ġaddress Ġdifferent Ġtypes Ġof Ġhasht ags . ĠBy Ġdesign , Ġthe Ġlinguistic Ġfeatures Ġcapture Ġnamed Ġentities Ġand Ġmulti - word Ġhasht ags Ġthat Ġexhibit Ġword Ġshape Ġpatterns , Ġsuch Ġas Ġcamel Ġcase . ĠThe Ġn gram Ġprobabilities Ġwith ĠGood - T uring Ġsmoot hing Ġgrav itate Ġtowards Ġmulti - word Ġsegment ations Ġwith Ġknown Ġwords , Ġas Ġits Ġestimate Ġfor Ġunseen Ġn gram s Ġdepends Ġon Ġthe Ġfraction Ġof Ġn gram s Ġseen Ġonce Ġwhich Ġcan Ġbe Ġvery Ġlow ĠB IB REF 28 Ġ. ĠThe Ġmodified ĠK nes er - N ey Ġsmoot hing Ġis Ġmore Ġlikely Ġto Ġfavor Ġsegment ations Ġthat Ġcontain Ġrare Ġwords , Ġand Ġsingle - word Ġsegment ations Ġin Ġparticular . ĠPlease Ġrefer Ġto ĠÂ§ ĠSEC REF 46 Ġfor Ġa Ġmore Ġdetailed Ġquantitative Ġand Ġqualitative Ġanalysis . </s> To Ġleverage Ġthis Ġintuition , Ġwe Ġintroduce Ġa Ġbinary Ġclassification Ġtask Ġto Ġhelp Ġthe Ġmodel Ġdifferentiate Ġsingle - word Ġfrom Ġmulti - word Ġhasht ags . ĠThe Ġbinary Ġclass ifier Ġtakes Ġhashtag Ġfeatures ĠIN LINE FORM 0 Ġas Ġthe Ġinput Ġand Ġoutputs ĠIN LINE FORM 1 Ġ, Ġwhich Ġrepresents Ġthe Ġprobability Ġof ĠIN LINE FORM 2 Ġbeing Ġa Ġmulti - word Ġhashtag . ĠIN LINE FORM 3 Ġis Ġused Ġas Ġan Ġadaptive Ġg ating Ġvalue Ġin Ġour Ġmulti - task Ġlearning Ġsetup . ĠThe Ġgold Ġlabels Ġfor Ġthis Ġtask Ġare Ġobtained Ġat Ġno Ġextra Ġcost Ġby Ġsimply Ġverifying Ġwhether Ġthe Ġground - truth Ġsegment ation Ġhas Ġmultiple Ġwords . ĠWe Ġtrain Ġthe Ġpair wise Ġsegment ation Ġrank er Ġand Ġthe Ġbinary Ġsingle - Ġvs . Ġmulti - token Ġhashtag Ġclass ifier Ġjointly , Ġby Ġminimizing ĠIN LINE FORM 4 Ġfor Ġthe Ġpair wise Ġrank er Ġand Ġthe ĠBinary ĠCross ĠEnt ropy ĠError Ġ( ĠIN LINE FORM 5 Ġ) Ġfor Ġthe Ġclass ifier : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġadaptive Ġg ating Ġvalue , ĠIN LINE FORM 1 Ġindicates Ġif ĠIN LINE FORM 2 Ġis Ġactually Ġa Ġmulti - word Ġhashtag Ġand ĠIN LINE FORM 3 Ġis Ġthe Ġnumber Ġof Ġtraining Ġexamples . ĠIN LINE FORM 4 Ġand ĠIN LINE FORM 5 Ġare Ġthe Ġweights Ġfor Ġeach Ġloss . ĠFor Ġour Ġexperiments , Ġwe Ġapply Ġequal Ġweights . </s> More Ġspecifically , Ġwe Ġdivide Ġthe Ġsegment ation Ġfeature Ġvector ĠIN LINE FORM 0 Ġinto Ġtwo Ġsubs ets : Ġ( a ) ĠIN LINE FORM 1 Ġwith Ġmodified ĠK nes er - N ey Ġsmoot hing Ġfeatures , Ġand Ġ( b ) ĠIN LINE FORM 2 Ġwith ĠGood - T uring Ġsmoot hing Ġand Ġlinguistic Ġfeatures . ĠFor Ġan Ġinput Ġcandidate Ġsegment ation Ġpair ĠIN LINE FORM 3 Ġ, Ġwe Ġconstruct Ġtwo Ġpair wise Ġvectors ĠIN LINE FORM 4 Ġand ĠIN LINE FORM 5 Ġby Ġconc aten ation , Ġthen Ġcombine Ġthem Ġbased Ġon Ġthe Ġadaptive Ġg ating Ġvalue ĠIN LINE FORM 6 Ġbefore Ġfeeding Ġthem Ġinto Ġthe Ġfeed forward Ġnetwork ĠIN LINE FORM 7 Ġfor Ġpair wise Ġranking : ĠDIS PLAY FORM 0 </s> We Ġuse Ġsumm ation Ġwith Ġpadding , Ġas Ġwe Ġfind Ġthis Ġsimple Ġensemble Ġmethod Ġachieves Ġsimilar Ġperformance Ġin Ġour Ġexperiments Ġas Ġthe Ġmore Ġcomplex Ġmulti - column Ġnetworks ĠB IB REF 29 Ġ. ĠFigure ĠFIG REF 11 Ġ( c ) Ġshows Ġthe Ġarchitecture Ġof Ġthis Ġmodel . ĠAn Ġanalogue Ġmulti - task Ġformulation Ġcan Ġalso Ġbe Ġused Ġfor Ġthe ĠMarg in ĠRanking Ġloss Ġas : ĠDIS PLAY FORM 0 </s> Features </s> We Ġuse Ġa Ġcombination Ġof Ġcorpus - based Ġand Ġlinguistic Ġfeatures Ġto Ġrank Ġthe Ġsegment ations . ĠFor Ġa Ġcandidate Ġsegment ation ĠIN LINE FORM 0 Ġ, Ġits Ġfeature Ġvector ĠIN LINE FORM 1 Ġincludes Ġthe Ġnumber Ġof Ġwords Ġin Ġthe Ġcandidate , Ġthe Ġlength Ġof Ġeach Ġword , Ġthe Ġproportion Ġof Ġwords Ġin Ġan ĠEnglish Ġdictionary Ġor ĠUrban ĠDictionary ĠB IB REF 30 Ġ, Ġn gram Ġcounts Ġfrom ĠGoogle ĠWeb Ġ1 TB Ġcorpus ĠB IB REF 31 Ġ, Ġand Ġn gram Ġprobabilities Ġfrom Ġtrig ram Ġlanguage Ġmodels Ġtrained Ġon Ġthe ĠGig aw ord Ġcorpus ĠB IB REF 32 Ġand Ġ1 . 1 Ġbillion ĠEnglish Ġtweets Ġfrom Ġ2010 , Ġrespectively . ĠWe Ġtrain Ġtwo Ġlanguage Ġmodels Ġon Ġeach Ġcorpus : Ġone Ġwith ĠGood - T uring Ġsmoot hing Ġusing ĠSR IL M ĠB IB REF 33 Ġand Ġthe Ġother Ġwith Ġmodified ĠK nes er - N ey Ġsmoot hing Ġusing ĠKen LM ĠB IB REF 34 Ġ. ĠWe Ġalso Ġadd Ġboolean Ġfeatures , Ġsuch Ġas Ġif Ġthe Ġcandidate Ġis Ġa Ġnamed - entity Ġpresent Ġin Ġthe Ġlist Ġof ĠWikipedia Ġtitles , Ġand Ġif Ġthe Ġcandidate Ġsegment ation ĠIN LINE FORM 2 Ġand Ġits Ġcorresponding Ġhashtag ĠIN LINE FORM 3 Ġsatisfy Ġcertain Ġword - sh apes Ġ( more Ġdetails Ġin Ġappendix ĠSEC REF 61 Ġ). </s> Similarly , Ġfor Ġhashtag ĠIN LINE FORM 0 Ġ, Ġwe Ġextract Ġthe Ġfeature Ġvector ĠIN LINE FORM 1 Ġconsisting Ġof Ġhashtag Ġlength , Ġn gram Ġcount Ġof Ġthe Ġhashtag Ġin ĠGoogle Ġ1 TB Ġcorpus ĠB IB REF 31 Ġ, Ġand Ġboolean Ġfeatures Ġindicating Ġif Ġthe Ġhashtag Ġis Ġin Ġan ĠEnglish Ġdictionary Ġor ĠUrban ĠDictionary , Ġis Ġa Ġnamed - entity , Ġis Ġin Ġcamel Ġcase , Ġends Ġwith Ġa Ġnumber , Ġand Ġhas Ġall Ġthe Ġletters Ġas Ġconson ants . ĠWe Ġalso Ġinclude Ġfeatures Ġof Ġthe Ġbest - ranked Ġcandidate Ġby Ġthe ĠWord ĠBreaker Ġmodel . </s> Im plementation ĠDetails </s> We Ġuse Ġthe ĠPy Tor ch Ġframework Ġto Ġimplement Ġour Ġmulti - task Ġpair wise Ġranking Ġmodel . ĠThe Ġpair wise Ġrank er Ġconsists Ġof Ġan Ġinput Ġlayer , Ġthree Ġhidden Ġlayers Ġwith Ġeight Ġnodes Ġin Ġeach Ġlayer Ġand Ġhyper b olic Ġtang ent Ġ( ĠIN LINE FORM 0 Ġ) Ġactivation , Ġand Ġa Ġsingle Ġlinear Ġoutput Ġnode . ĠThe Ġauxiliary Ġclass ifier Ġconsists Ġof Ġan Ġinput Ġlayer , Ġone Ġhidden Ġlayer Ġwith Ġeight Ġnodes Ġand Ġone Ġoutput Ġnode Ġwith Ġs igm oid Ġactivation . ĠWe Ġuse Ġthe ĠAdam Ġalgorithm ĠB IB REF 35 Ġfor Ġoptimization Ġand Ġapply Ġa Ġdrop out Ġof Ġ0 . 5 Ġto Ġprevent Ġover fitting . ĠWe Ġset Ġthe Ġlearning Ġrate Ġto Ġ0 . 01 Ġand Ġ0 . 05 Ġfor Ġthe Ġpair wise Ġrank er Ġand Ġauxiliary Ġclass ifier Ġrespectively . ĠFor Ġeach Ġexperiment , Ġwe Ġreport Ġresults Ġobtained Ġafter Ġ100 Ġepoch s . </s> For Ġthe Ġbaseline Ġmodel Ġused Ġto Ġextract Ġthe ĠIN LINE FORM 0 Ġinitial Ġcandidates , Ġwe Ġreim plement ated Ġthe ĠWord ĠBreaker ĠB IB REF 16 Ġas Ġdescribed Ġin ĠÂ§ ĠSEC REF 2 Ġand Ġadapted Ġit Ġto Ġuse Ġa Ġlanguage Ġmodel Ġtrained Ġon Ġ1 . 1 Ġbillion Ġtweets Ġwith ĠGood - T uring Ġsmoot hing Ġusing ĠSR IL M ĠB IB REF 33 Ġto Ġgive Ġa Ġbetter Ġperformance Ġin Ġsegment ing Ġhasht ags Ġ( Â§ ĠSEC REF 46 Ġ). ĠFor Ġall Ġour Ġexperiments , Ġwe Ġset ĠIN LINE FORM 1 Ġ. </s> Hash tag ĠSe gment ation ĠData </s> We Ġuse Ġtwo Ġdatasets Ġfor Ġexperiments Ġ( Table ĠT AB REF 29 Ġ): Ġ( a ) ĠST AN ĠIN LINE FORM 0 Ġ, Ġcreated Ġby ĠB IB REF 10 ĠBans al B V 15 , Ġwhich Ġconsists Ġof Ġ1 , 108 Ġunique ĠEnglish Ġhasht ags Ġfrom Ġ1 , 268 Ġrandomly Ġselected Ġtweets Ġin Ġthe ĠStanford ĠSent iment ĠAnalysis ĠDat as et ĠB IB REF 36 Ġalong Ġwith Ġtheir Ġcrowds ourced Ġsegment ations Ġand Ġour Ġadditional Ġcorrections ; Ġand Ġ( b ) ĠST AN ĠIN LINE FORM 1 Ġ, Ġour Ġnew Ġexpert Ġcurated Ġdataset , Ġwhich Ġincludes Ġall Ġ12 , 594 Ġunique ĠEnglish Ġhasht ags Ġand Ġtheir Ġassociated Ġtweets Ġfrom Ġthe Ġsame ĠStanford Ġdataset . </s> Exper iments </s> In Ġthis Ġsection , Ġwe Ġpresent Ġexperimental Ġresults Ġthat Ġcompare Ġour Ġproposed Ġmethod Ġwith Ġthe Ġother Ġstate - of - the - art Ġapproaches Ġon Ġhashtag Ġsegment ation Ġdatasets . ĠThe Ġnext Ġsection Ġwill Ġshow Ġexperiments Ġof Ġapplying Ġhashtag Ġsegment ation Ġto Ġthe Ġpopular Ġtask Ġof Ġsentiment Ġanalysis . </s> Ex isting ĠMethods </s> We Ġcompare Ġour Ġpair wise Ġneural Ġrank er Ġwith Ġthe Ġfollowing Ġbaseline Ġand Ġstate - of - the - art Ġapproaches : </s> The Ġoriginal Ġhashtag Ġas Ġa Ġsingle Ġtoken ; </s> A Ġrule - based Ġsegment er , Ġwhich Ġemploys Ġa Ġset Ġof Ġword - shape Ġrules Ġwith Ġan ĠEnglish Ġdictionary ĠB IB REF 13 Ġ; </s> A ĠV iter bi Ġmodel Ġwhich Ġuses Ġword Ġfrequencies Ġfrom Ġa Ġbook Ġcorpus ĠB IB REF 0 Ġ; </s> The Ġspecially Ġdeveloped ĠG ATE ĠHash tag ĠToken izer Ġfrom Ġthe Ġopen Ġsource Ġtool kit , Ġwhich Ġcombines Ġdiction aries Ġand Ġg az ette ers Ġin Ġa ĠV iter bi - like Ġalgorithm ĠB IB REF 11 Ġ; </s> A Ġmaximum Ġentropy Ġclass ifier Ġ( Max Ent ) Ġtrained Ġon Ġthe ĠST AN ĠIN LINE FORM 0 Ġtraining Ġdataset . ĠIt Ġpredicts Ġwhether Ġa Ġspace Ġshould Ġbe Ġinserted Ġat Ġeach Ġposition Ġin Ġthe Ġhashtag Ġand Ġis Ġthe Ġcurrent Ġstate - of - the - art ĠB IB REF 14 Ġ; </s> Our Ġreim plementation Ġof Ġthe ĠWord ĠBreaker Ġalgorithm Ġwhich Ġuses Ġbeam Ġsearch Ġand Ġa ĠTwitter Ġn gram Ġlanguage Ġmodel ĠB IB REF 16 Ġ; </s> A Ġpair wise Ġlinear Ġrank er Ġwhich Ġwe Ġimplemented Ġfor Ġcomparison Ġpurposes Ġwith Ġthe Ġsame Ġfeatures Ġas Ġour Ġneural Ġmodel , Ġbut Ġusing Ġpercept ron Ġas Ġthe Ġunderlying Ġclass ifier ĠB IB REF 38 Ġand Ġminimizing Ġthe Ġhinge Ġloss Ġbetween ĠIN LINE FORM 0 Ġand Ġa Ġscoring Ġfunction Ġsimilar Ġto ĠIN LINE FORM 1 Ġ. ĠIt Ġis Ġtrained Ġon Ġthe ĠST AN ĠIN LINE FORM 2 Ġdataset . </s> E val uation ĠMet rics </s> We Ġevaluate Ġthe Ġperformance Ġby Ġthe Ġtop ĠIN LINE FORM 0 Ġ( ĠIN LINE FORM 1 Ġ) Ġaccuracy Ġ( A @ 1 , ĠA @ 2 ), Ġaverage Ġtoken - level ĠF ĠIN LINE FORM 2 Ġscore Ġ( F ĠIN LINE FORM 3 Ġ@ 1 ), Ġand Ġmean Ġreciprocal Ġrank Ġ( MR R ). ĠIn Ġparticular , Ġthe Ġaccuracy Ġand ĠMR R Ġare Ġcalculated Ġat Ġthe Ġsegment ation - level , Ġwhich Ġmeans Ġthat Ġan Ġoutput Ġsegment ation Ġis Ġconsidered Ġcorrect Ġif Ġand Ġonly Ġif Ġit Ġfully Ġmatches Ġthe Ġhuman Ġsegment ation . ĠAverage Ġtoken - level ĠF ĠIN LINE FORM 4 Ġscore Ġaccounts Ġfor Ġpartially Ġcorrect Ġsegment ation Ġin Ġthe Ġmulti - token Ġhashtag Ġcases . </s> Results </s> T ables ĠT AB REF 32 Ġand ĠT AB REF 33 Ġshow Ġthe Ġresults Ġon Ġthe ĠST AN ĠIN LINE FORM 0 Ġand ĠST AN ĠIN LINE FORM 1 Ġdatasets , Ġrespectively . ĠAll Ġof Ġour Ġpair wise Ġneural Ġrank ers Ġare Ġtrained Ġon Ġthe Ġ2 , 518 Ġmanually Ġsegment ed Ġhasht ags Ġin Ġthe Ġtraining Ġset Ġof ĠST AN ĠIN LINE FORM 2 Ġand Ġperform Ġfavorably Ġagainst Ġother Ġstate - of - the - art Ġapproaches . ĠOur Ġbest Ġmodel Ġ( M SE + mult it ask ) Ġthat Ġutilizes Ġdifferent Ġfeatures Ġadapt ively Ġvia Ġa Ġmulti - task Ġlearning Ġprocedure Ġis Ġshown Ġto Ġperform Ġbetter Ġthan Ġsimply Ġcombining Ġall Ġthe Ġfeatures Ġtogether Ġ( MR Ġand ĠM SE ). ĠWe Ġhighlight Ġthe Ġ24 . 6 % Ġerror Ġreduction Ġon ĠST AN ĠIN LINE FORM 3 Ġand Ġ16 . 5 % Ġon ĠST AN ĠIN LINE FORM 4 Ġof Ġour Ġapproach Ġover Ġthe Ġprevious ĠS OTA ĠB IB REF 14 Ġon Ġthe ĠMulti - token Ġhasht ags , Ġand Ġthe Ġimportance Ġof Ġhaving Ġa Ġseparate Ġevaluation Ġof Ġmulti - word Ġcases Ġas Ġit Ġis Ġtrivial Ġto Ġobtain Ġ100 % Ġaccuracy Ġfor ĠSingle - token Ġhasht ags . ĠWhile Ġour Ġhashtag Ġsegment ation Ġmodel Ġis Ġachieving Ġa Ġvery Ġhigh Ġaccuracy @ 2 , Ġto Ġbe Ġpractically Ġuseful , Ġit Ġremains Ġa Ġchallenge Ġto Ġget Ġthe Ġtop Ġone Ġpred ication Ġexactly Ġcorrect . ĠSome Ġhasht ags Ġare Ġvery Ġdifficult Ġto Ġinterpret , Ġe . g ., Ġ# B TV brown S MB Ġrefers Ġto Ġthe ĠSocial ĠMedia ĠBreakfast Ġ( S MB ) Ġin ĠBurlington , ĠVermont Ġ( B TV ). </s> The Ġimproved ĠWord ĠBreaker Ġwith Ġour Ġaddition Ġof Ġa ĠTwitter - specific Ġlanguage Ġmodel Ġis Ġa Ġvery Ġstrong Ġbaseline , Ġwhich Ġe ch os Ġthe Ġfindings Ġof Ġthe Ġoriginal ĠWord ĠBreaker Ġpaper ĠB IB REF 16 Ġthat Ġhaving Ġa Ġlarge Ġin - domain Ġlanguage Ġmodel Ġis Ġextremely Ġhelpful Ġfor Ġword Ġsegment ation Ġtasks . ĠIt Ġis Ġworth Ġnoting Ġthat Ġthe Ġother Ġstate - of - the - art Ġsystem ĠB IB REF 14 Ġalso Ġutilized Ġa Ġ4 - gram Ġlanguage Ġmodel Ġtrained Ġon Ġ4 76 Ġmillion Ġtweets Ġfrom Ġ2009 . </s> Analysis Ġand ĠDiscussion </s> To Ġempir ically Ġillustrate Ġthe Ġeffectiveness Ġof Ġdifferent Ġfeatures Ġon Ġdifferent Ġtypes Ġof Ġhasht ags , Ġwe Ġshow Ġthe Ġresults Ġfor Ġmodels Ġusing Ġindividual Ġfeature Ġsets Ġin Ġpair wise Ġranking Ġmodels Ġ( M SE ) Ġin ĠTable ĠT AB REF 45 Ġ. ĠLanguage Ġmodels Ġwith Ġmodified ĠK nes er - N ey Ġsmoot hing Ġperform Ġbest Ġon Ġsingle - token Ġhasht ags , Ġwhile ĠGood - T uring Ġand ĠL ingu istic Ġfeatures Ġwork Ġbest Ġon Ġmulti - token Ġhasht ags , Ġconfirming Ġour Ġintuition Ġabout Ġtheir Ġusefulness Ġin Ġa Ġmulti - task Ġlearning Ġapproach . ĠTable ĠT AB REF 47 Ġshows Ġa Ġqualitative Ġanalysis Ġwith Ġthe Ġfirst Ġcolumn Ġ( ĠIN LINE FORM 0 ĠIN LINE FORM 1 ĠIN LINE FORM 2 Ġ) Ġindicating Ġwhich Ġfeatures Ġlead Ġto Ġcorrect Ġor Ġwrong Ġsegment ations , Ġtheir Ġcount Ġin Ġour Ġdata Ġand Ġillust rative Ġexamples Ġwith Ġhuman Ġsegment ation . </s> As Ġexpected , Ġlonger Ġhasht ags Ġwith Ġmore Ġthan Ġthree Ġtokens Ġpose Ġgreater Ġchallenges Ġand Ġthe Ġsegment ation - level Ġaccuracy Ġof Ġour Ġbest Ġmodel Ġ( M SE + mult it ask ) Ġdrops Ġto Ġ82 . 1 %. ĠFor Ġmany Ġerror Ġcases , Ġour Ġmodel Ġpredicts Ġa Ġclose - to - correct Ġsegment ation , Ġe . g ., Ġ# you brown know you brown upt to ob rown early , Ġ# ise eb rown l ondon ise eb rown f rance , Ġwhich Ġis Ġalso Ġreflected Ġby Ġthe Ġhigher Ġtoken - level ĠF ĠIN LINE FORM 0 Ġscores Ġacross Ġhasht ags Ġwith Ġdifferent Ġlengths Ġ( Figure ĠFIG REF 51 Ġ). </s> Since Ġour Ġapproach Ġheavily Ġrelies Ġon Ġbuilding Ġa ĠTwitter Ġlanguage Ġmodel , Ġwe Ġexperimented Ġwith Ġits Ġsizes Ġand Ġshow Ġthe Ġresults Ġin ĠFigure ĠFIG REF 52 Ġ. ĠOur Ġapproach Ġcan Ġperform Ġwell Ġeven Ġwith Ġaccess Ġto Ġa Ġsmaller Ġamount Ġof Ġtweets . ĠThe Ġdrop Ġin ĠF ĠIN LINE FORM 0 Ġscore Ġfor Ġour Ġpair wise Ġneural Ġrank er Ġis Ġonly Ġ1 . 4 % Ġand Ġ3 . 9 % Ġwhen Ġusing Ġthe Ġlanguage Ġmodels Ġtrained Ġon Ġ10 % Ġand Ġ1 % Ġof Ġthe Ġtotal Ġ1 . 1 Ġbillion Ġtweets , Ġrespectively . </s> Language Ġuse Ġin ĠTwitter Ġchanges Ġwith Ġtime ĠB IB REF 9 Ġ. ĠOur Ġpair wise Ġrank er Ġuses Ġlanguage Ġmodels Ġtrained Ġon Ġthe Ġtweets Ġfrom Ġthe Ġyear Ġ2010 . ĠWe Ġtested Ġour Ġapproach Ġon Ġa Ġset Ġof Ġ500 Ġrandom ĠEnglish Ġhasht ags Ġposted Ġin Ġtweets Ġfrom Ġthe Ġyear Ġ2019 Ġand Ġshow Ġthe Ġresults Ġin ĠTable ĠT AB REF 55 Ġ. ĠWith Ġa Ġsegment ation - level Ġaccuracy Ġof Ġ94 . 6 % Ġand Ġaverage Ġtoken - level ĠF ĠIN LINE FORM 0 Ġscore Ġof Ġ95 . 6 %, Ġour Ġapproach Ġperforms Ġfavorably Ġon Ġ2019 Ġhasht ags . </s> Ext r ins ic ĠEvaluation : ĠTwitter ĠSent iment ĠAnalysis </s> We Ġattempt Ġto Ġdemonstrate Ġthe Ġeffectiveness Ġof Ġour Ġhashtag Ġsegment ation Ġsystem Ġby Ġstudying Ġits Ġimpact Ġon Ġthe Ġtask Ġof Ġsentiment Ġanalysis Ġin ĠTwitter ĠB IB REF 39 Ġ, ĠB IB REF 40 Ġ, ĠB IB REF 41 Ġ. ĠWe Ġuse Ġour Ġbest Ġmodel Ġ( M SE + mult it ask ), Ġunder Ġthe Ġname ĠHash tag Master , Ġin Ġthe Ġfollowing Ġexperiments . </s> Exper imental ĠSetup </s> We Ġcompare Ġthe Ġperformance Ġof Ġthe ĠBi L ST M + Lex ĠB IB REF 42 Ġsentiment Ġanalysis Ġmodel Ġunder Ġthree Ġconfigurations : Ġ( a ) Ġtweets Ġwith Ġhasht ags Ġremoved , Ġ( b ) Ġtweets Ġwith Ġhasht ags Ġas Ġsingle Ġtokens Ġexcluding Ġthe Ġ# Ġsymbol , Ġand Ġ( c ) Ġtweets Ġwith Ġhasht ags Ġas Ġsegment ed Ġby Ġour Ġsystem , ĠHash tag Master . ĠBi L ST M + Lex Ġis Ġa Ġstate - of - the - art Ġopen Ġsource Ġsystem Ġfor Ġpredicting Ġtweet - level Ġsentiment ĠB IB REF 43 Ġ. ĠIt Ġlearns Ġa Ġcontext - sensitive Ġsentiment Ġintensity Ġscore Ġby Ġleveraging Ġa ĠTwitter - based Ġsentiment Ġlex icon ĠB IB REF 44 Ġ. ĠWe Ġuse Ġthe Ġsame Ġsettings Ġas Ġdescribed Ġby ĠB IB REF 42 Ġt eng - vo - zh ang : 2016 : EM N LP 2016 Ġto Ġtrain Ġthe Ġmodel . </s> We Ġuse Ġthe Ġdataset Ġfrom Ġthe ĠSent iment ĠAnalysis Ġin ĠTwitter Ġshared Ġtask Ġ( sub task ĠA ) Ġat ĠSem E val Ġ2017 ĠB IB REF 41 Ġ. ĠGiven Ġa Ġtweet , Ġthe Ġgoal Ġis Ġto Ġpredict Ġwhether Ġit Ġexpresses ĠPOS IT IVE , ĠN EG ATIVE Ġor ĠNE UT RAL Ġsentiment . ĠThe Ġtraining Ġand Ġdevelopment Ġsets Ġconsist Ġof Ġ49 , 669 Ġtweets Ġand Ġwe Ġuse Ġ40 , 000 Ġfor Ġtraining Ġand Ġthe Ġrest Ġfor Ġdevelopment . ĠThere Ġare Ġa Ġtotal Ġof Ġ12 , 284 Ġtweets Ġcontaining Ġ12 , 128 Ġhasht ags Ġin Ġthe ĠSem E val Ġ2017 Ġtest Ġset , Ġand Ġour Ġhashtag Ġsegment er Ġended Ġup Ġsplitting Ġ6 , 975 Ġof Ġthose Ġhasht ags Ġpresent Ġin Ġ3 , 384 Ġtweets . </s> Results Ġand ĠAnalysis </s> In ĠTable ĠT AB REF 59 Ġ, Ġwe Ġreport Ġthe Ġresults Ġbased Ġon Ġthe Ġ3 , 384 Ġtweets Ġwhere ĠHash tag Master Ġpredicted Ġa Ġsplit , Ġas Ġfor Ġthe Ġrest Ġof Ġtweets Ġin Ġthe Ġtest Ġset , Ġthe Ġhashtag Ġsegment er Ġwould Ġneither Ġimprove Ġnor Ġworsen Ġthe Ġsentiment Ġprediction . ĠOur Ġhashtag Ġsegment er Ġsuccessfully Ġimproved Ġthe Ġsentiment Ġanalysis Ġperformance Ġby Ġ2 % Ġon Ġaverage Ġrecall Ġand ĠF ĠIN LINE FORM 0 Ġcomparing Ġto Ġhaving Ġhasht ags Ġunse gment ed . ĠThis Ġimprovement Ġis Ġseemingly Ġsmall Ġbut Ġdecidedly Ġimportant Ġfor Ġtweets Ġwhere Ġsentiment - related Ġinformation Ġis Ġembedded Ġin Ġmulti - word Ġhasht ags Ġand Ġsentiment Ġprediction Ġwould Ġbe Ġincorrect Ġbased Ġonly Ġon Ġthe Ġtext Ġ( see ĠTable ĠT AB REF 60 Ġfor Ġexamples ). ĠIn Ġfact , Ġ2 , 605 Ġout Ġof Ġthe Ġ3 , 384 Ġtweets Ġhave Ġmulti - word Ġhasht ags Ġthat Ġcontain Ġwords Ġin Ġthe ĠTwitter - based Ġsentiment Ġlex icon ĠB IB REF 44 Ġand Ġ125 Ġtweets Ġcontain Ġsentiment Ġwords Ġonly Ġin Ġthe Ġhasht ags Ġbut Ġnot Ġin Ġthe Ġrest Ġof Ġthe Ġtweet . ĠOn Ġthe Ġentire Ġtest Ġset Ġof Ġ12 , 284 Ġtweets , Ġthe Ġincrease Ġin Ġthe Ġaverage Ġrecall Ġis Ġ0 . 5 %. </s> Other ĠRelated ĠWork </s> Aut omatic Ġhashtag Ġsegment ation Ġcan Ġimprove Ġthe Ġperformance Ġof Ġmany Ġapplications Ġbesides Ġsentiment Ġanalysis , Ġsuch Ġas Ġtext Ġclassification ĠB IB REF 13 Ġ, Ġnamed Ġentity Ġlinking ĠB IB REF 10 Ġand Ġmodeling Ġuser Ġinterests Ġfor Ġrecommendations ĠB IB REF 45 Ġ. ĠIt Ġcan Ġalso Ġhelp Ġin Ġcollecting Ġdata Ġof Ġhigher Ġvolume Ġand Ġquality Ġby Ġproviding Ġa Ġmore Ġnuanced Ġinterpretation Ġof Ġits Ġcontent , Ġas Ġshown Ġfor Ġemotion Ġanalysis ĠB IB REF 46 Ġ, Ġsarc asm Ġand Ġirony Ġdetection ĠB IB REF 11 Ġ, ĠB IB REF 47 Ġ. ĠBetter Ġsemantic Ġanalysis Ġof Ġhasht ags Ġcan Ġalso Ġpotentially Ġbe Ġapplied Ġto Ġhashtag Ġannotation ĠB IB REF 48 Ġ, Ġto Ġimprove Ġdistant Ġsupervision Ġlabels Ġin Ġtraining Ġclass ifiers Ġfor Ġtasks Ġsuch Ġas Ġsarc asm ĠB IB REF 5 Ġ, Ġsentiment ĠB IB REF 4 Ġ, Ġemotions ĠB IB REF 3 Ġ; Ġand , Ġmore Ġgenerally , Ġas Ġlabels Ġfor Ġpre - training Ġrepresentations Ġof Ġwords ĠB IB REF 49 Ġ, Ġsentences ĠB IB REF 50 Ġ, Ġand Ġimages ĠB IB REF 51 Ġ. </s> Conclusion </s> We Ġproposed Ġa Ġnew Ġpair wise Ġneural Ġranking Ġmodel Ġfor Ġhashtag Ġsegment ion Ġand Ġshowed Ġsignificant Ġperformance Ġimprovements Ġover Ġthe Ġstate - of - the - art . ĠWe Ġalso Ġconstructed Ġa Ġlarger Ġand Ġmore Ġcurated Ġdataset Ġfor Ġanalyzing Ġand Ġbenchmark ing Ġhashtag Ġsegment ation Ġmethods . ĠWe Ġdemonstrated Ġthat Ġhashtag Ġsegment ation Ġhelps Ġwith Ġdownstream Ġtasks Ġsuch Ġas Ġsentiment Ġanalysis . ĠAlthough Ġwe Ġfocused Ġon ĠEnglish Ġhasht ags , Ġour Ġpair wise Ġranking Ġapproach Ġis Ġlanguage - independent Ġand Ġwe Ġintend Ġto Ġextend Ġour Ġtool kit Ġto Ġlanguages Ġother Ġthan ĠEnglish Ġas Ġfuture Ġwork . </s> Acknowled gments </s> We Ġthank ĠOhio ĠSuper computer ĠCenter ĠB IB REF 52 Ġfor Ġcomputing Ġresources Ġand Ġthe ĠNVIDIA Ġfor Ġproviding ĠGPU Ġhardware . ĠWe Ġthank ĠAlan ĠR itter , ĠQuan ze ĠChen , ĠWang ĠLing , ĠP rav ar ĠMah aj an , Ġand ĠD ush y anta ĠD hy ani Ġfor Ġvaluable Ġdiscussions . ĠWe Ġalso Ġthank Ġthe Ġannot ators : ĠSarah ĠFlan agan , ĠKa ush ik ĠMan i , Ġand ĠAs w ath nar ayan ĠRad hak rish nan . ĠThis Ġmaterial Ġis Ġbased Ġin Ġpart Ġon Ġresearch Ġsponsored Ġby Ġthe ĠNS F Ġunder Ġgrants ĠI IS - 18 22 754 Ġand ĠI IS - 17 558 98 , ĠDAR PA Ġthrough Ġthe ĠAR O Ġunder Ġagreement Ġnumber ĠW 911 NF - 17 - C - 00 95 , Ġthrough Ġa ĠFigure - Eight Ġ( C rowd Fl ower ) ĠAI Ġfor ĠEveryone ĠAward Ġand Ġa ĠCr ite o ĠFaculty ĠResearch ĠAward Ġto ĠWei ĠXu . ĠThe Ġviews Ġand Ġconclusions Ġcontained Ġin Ġthis Ġpublication Ġare Ġthose Ġof Ġthe Ġauthors Ġand Ġshould Ġnot Ġbe Ġinterpreted Ġas Ġrepresenting Ġofficial Ġpolicies Ġor Ġendorsements Ġof Ġthe ĠU . S . ĠGovernment . </s> Word - shape Ġrules </s> Our Ġmodel Ġuses Ġthe Ġfollowing Ġword Ġshape Ġrules Ġas Ġboolean Ġfeatures . ĠIf Ġthe Ġcandidate Ġsegment ation ĠIN LINE FORM 0 Ġand Ġits Ġcorresponding Ġhashtag ĠIN LINE FORM 1 Ġsatisfies Ġa Ġword Ġshape Ġrule , Ġthen Ġthe Ġboolean Ġfeature Ġis Ġset Ġto ĠTrue .