rupnik2016news
EXTRACTIVE
rupnik2016news, Deutsche Welle's news website
EXTRACTIVE
More recently, crosslingual linking of clusters has been discussed by rupnik2016news in the context of linking existing clusters from the Event Registry BIBREF7 in a batch fashion, and by steinberger2016mediagist who also present a batch clustering linking system. However, these are not “truly” online crosslingual clustering systems since they only decide on the linking of already-built monolingual clusters. In particular, rupnik2016news compute distances of document pairs across clusters using nearest neighbors, which might not scale well in an online setting. As detailed before, we adapted the cluster-linking dataset from rupnik2016news to evaluate our online crosslingual clustering approach. Preliminary work makes use of deep learning techniques BIBREF8 , BIBREF9 to cluster documents while learning their representations, but not in an online or multilingual fashion, and with a very small number of cluster labels (4, in the case of the text benchmark).
More recently, crosslingual linking of clusters has been discussed by rupnik2016news in the context of linking existing clusters from the Event Registry BIBREF7 in a batch fashion, and by steinberger2016mediagist who also present a batch clustering linking system. However, these are not “truly” online crosslingual clustering systems since they only decide on the linking of already-built monolingual clusters. In particular, rupnik2016news compute distances of document pairs across clusters using nearest neighbors, which might not scale well in an online setting. As detailed before, we adapted the cluster-linking dataset from rupnik2016news to evaluate our online crosslingual clustering approach. Preliminary work makes use of deep learning techniques BIBREF8 , BIBREF9 to cluster documents while learning their representations, but not in an online or multilingual fashion, and with a very small number of cluster labels (4, in the case of the text benchmark). Statistics about this dataset are given in Table TABREF30 . As described further, we tune the hyper-parameter INLINEFORM0 on the development set. As for the hyper-parameters related to the timestamp features, we fixed INLINEFORM1 and tuned INLINEFORM2 on the development set, yielding INLINEFORM3 . To compute IDF scores (which are global numbers computed across a corpus), we used a different and much larger dataset that we collected from Deutsche Welle's news website (http://www.dw.com/). The dataset consists of 77,268, 118,045 and 134,243 documents for Spanish, English and German, respectively.
1809.00540
Introduction </s> Following Ġdeveloping Ġnews Ġstories Ġis Ġimperative Ġto Ġmaking Ġreal - time Ġdecisions Ġon Ġimportant Ġpolitical Ġand Ġpublic Ġsafety Ġmatters . ĠGiven Ġthe Ġabundance Ġof Ġmedia Ġproviders Ġand Ġlanguages , Ġthis Ġendeavor Ġis Ġan Ġextremely Ġdifficult Ġtask . ĠAs Ġsuch , Ġthere Ġis Ġa Ġstrong Ġdemand Ġfor Ġautomatic Ġclust ering Ġof Ġnews Ġstreams , Ġso Ġthat Ġthey Ġcan Ġbe Ġorganized Ġinto Ġstories Ġor Ġthemes Ġfor Ġfurther Ġprocessing . ĠPer forming Ġthis Ġtask Ġin Ġan Ġonline Ġand Ġefficient Ġmanner Ġis Ġa Ġchallenging Ġproblem , Ġnot Ġonly Ġfor Ġnews wire , Ġbut Ġalso Ġfor Ġscientific Ġarticles , Ġonline Ġreviews , Ġforum Ġposts , Ġblogs , Ġand Ġmicro blogs . </s> A Ġkey Ġchallenge Ġin Ġhandling Ġdocument Ġstreams Ġis Ġthat Ġthe Ġstory Ġclusters Ġmust Ġbe Ġgenerated Ġon Ġthe Ġfly Ġin Ġan Ġonline Ġfashion : Ġthis Ġrequires Ġhandling Ġdocuments Ġone - by - one Ġas Ġthey Ġappear Ġin Ġthe Ġdocument Ġstream . ĠIn Ġthis Ġpaper , Ġwe Ġprovide Ġa Ġtreatment Ġto Ġthe Ġproblem Ġof Ġonline Ġdocument Ġclust ering , Ġi . e . Ġthe Ġtask Ġof Ġclust ering Ġa Ġstream Ġof Ġdocuments Ġinto Ġthemes . ĠFor Ġexample , Ġfor Ġnews Ġarticles , Ġwe Ġwould Ġwant Ġto Ġcluster Ġthem Ġinto Ġrelated Ġnews Ġstories . </s> To Ġthis Ġend , Ġwe Ġintroduce Ġa Ġsystem Ġwhich Ġaggreg ates Ġnews Ġarticles Ġinto Ġfine - gr ained Ġstory Ġclusters Ġacross Ġdifferent Ġlanguages Ġin Ġa Ġcompletely Ġonline Ġand Ġscalable Ġfashion Ġfrom Ġa Ġcontinuous Ġstream . ĠOur Ġclust ering Ġapproach Ġis Ġpart Ġof Ġa Ġlarger Ġmedia Ġmonitoring Ġproject Ġto Ġsolve Ġthe Ġproblem Ġof Ġmonitoring Ġmassive Ġtext Ġand ĠTV / Radio Ġstreams Ġ( speech - to - text ). ĠIn Ġparticular , Ġmedia Ġmonitors Ġwrite Ġintelligence Ġreports Ġabout Ġthe Ġmost Ġrelevant Ġevents , Ġand Ġbeing Ġable Ġto Ġsearch , Ġvisualize Ġand Ġexplore Ġnews Ġclusters Ġassists Ġin Ġgathering Ġmore Ġinsight Ġabout Ġa Ġparticular Ġstory . ĠSince Ġrelevant Ġevents Ġmay Ġbe Ġspawned Ġfrom Ġany Ġpart Ġof Ġthe Ġworld Ġ( and Ġfrom Ġmany Ġmult ilingual Ġsources ), Ġit Ġbecomes Ġimperative Ġto Ġcluster Ġnews Ġacross Ġdifferent Ġlanguages . </s> In Ġterms Ġof Ġgran ularity , Ġthe Ġtype Ġof Ġstory Ġclusters Ġwe Ġare Ġinterested Ġin Ġare Ġthe Ġgroup Ġof Ġarticles Ġwhich , Ġfor Ġexample Ġ: Ġ( i ) ĠNarr ate Ġrecent Ġair - st rikes Ġin ĠEastern ĠGh out a Ġ( Syria ); Ġ( ii ) ĠDesc ribe Ġthe Ġrecent Ġlaunch Ġof ĠSpace ĠX 's ĠFalcon ĠHeavy Ġrocket . </s> Problem ĠForm ulation </s> We Ġfocus Ġon Ġclust ering Ġof Ġa Ġstream Ġof Ġdocuments , Ġwhere Ġthe Ġnumber Ġof Ġclusters Ġis Ġnot Ġfixed Ġand Ġlearned Ġautomatically . ĠWe Ġdenote Ġby ĠIN LINE FORM 0 Ġa Ġ( pot entially Ġinfinite ) Ġspace Ġof Ġmult ilingual Ġdocuments . ĠEach Ġdocument ĠIN LINE FORM 1 Ġis Ġassociated Ġwith Ġa Ġlanguage Ġin Ġwhich Ġit Ġis Ġwritten Ġthrough Ġa Ġfunction ĠIN LINE FORM 2 Ġwhere ĠIN LINE FORM 3 Ġis Ġa Ġset Ġof Ġlanguages . ĠFor Ġexample , ĠIN LINE FORM 4 Ġcould Ġreturn ĠEnglish , ĠSpanish Ġor ĠGerman . Ġ( In Ġthe Ġrest Ġof Ġthe Ġpaper , Ġfor Ġan Ġinteger ĠIN LINE FORM 5 Ġ, Ġwe Ġdenote Ġby ĠIN LINE FORM 6 Ġthe Ġset ĠIN LINE FORM 7 Ġ.) </s> We Ġare Ġinterested Ġin Ġassoci ating Ġeach Ġdocument Ġwith Ġa Ġmon oling ual Ġcluster Ġvia Ġthe Ġfunction ĠIN LINE FORM 0 Ġ, Ġwhich Ġreturns Ġthe Ġcluster Ġlabel Ġgiven Ġa Ġdocument . ĠThis Ġis Ġdone Ġindependently Ġfor Ġeach Ġlanguage , Ġsuch Ġthat Ġthe Ġspace Ġof Ġindices Ġwe Ġuse Ġfor Ġeach Ġlanguage Ġis Ġseparate . </s> Furthermore , Ġwe Ġinter lace Ġthe Ġproblem Ġof Ġmon oling ual Ġclust ering Ġwith Ġcross ling ual Ġclust ering . ĠThis Ġmeans Ġthat Ġas Ġpart Ġof Ġour Ġproblem Ġformulation Ġwe Ġare Ġalso Ġinterested Ġin Ġa Ġfunction ĠIN LINE FORM 0 Ġthat Ġassociates Ġeach Ġmon oling ual Ġcluster Ġwith Ġa Ġcross ling ual Ġcluster , Ġsuch Ġthat Ġeach Ġcross ling ual Ġcluster Ġonly Ġgroups Ġone Ġmon oling ual Ġcluster Ġper Ġdifferent Ġlanguage , Ġat Ġa Ġgiven Ġtime . ĠThe Ġcross ling ual Ġcluster Ġfor Ġa Ġdocument ĠIN LINE FORM 1 Ġis ĠIN LINE FORM 2 Ġ. ĠAs Ġsuch , Ġa Ġcross ling ual Ġcluster Ġgroups Ġtogether Ġmon oling ual Ġclusters , Ġat Ġmost Ġone Ġfor Ġeach Ġdifferent Ġlanguage . </s> Int uitive ly , Ġbuilding Ġboth Ġmon oling ual Ġand Ġcross ling ual Ġclusters Ġallows Ġthe Ġsystem Ġto Ġleverage Ġhigh - pre cision Ġmon oling ual Ġfeatures Ġ( e . g ., Ġwords , Ġnamed Ġentities ) Ġto Ġcluster Ġdocuments Ġof Ġthe Ġsame Ġlanguage , Ġwhile Ġsimpl ifying Ġthe Ġtask Ġof Ġcross ling ual Ġclust ering Ġto Ġthe Ġcomputation Ġof Ġsimilarity Ġscores Ġacross Ġmon oling ual Ġclusters Ġ- Ġwhich Ġis Ġa Ġsmaller Ġproblem Ġspace , Ġsince Ġthere Ġare Ġ( by Ġdefinition ) Ġless Ġclusters Ġthan Ġarticles . ĠWe Ġvalidate Ġthis Ġchoice Ġin ĠÂ§ ĠSEC REF 5 Ġ. </s> The ĠCl ust ering ĠAl gorithm </s> Each Ġdocument ĠIN LINE FORM 0 Ġis Ġrepresented Ġby Ġtwo Ġvectors Ġin ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ. ĠThe Ġfirst Ġvector Ġexists Ġin Ġa ĠâĢ ľ mon oling ual Ġspace âĢ Ŀ Ġ( of Ġdimension ality ĠIN LINE FORM 3 Ġ) Ġand Ġis Ġbased Ġon Ġa Ġbag - of - words Ġrepresentation Ġof Ġthe Ġdocument . ĠThe Ġsecond Ġvector Ġexists Ġin Ġa ĠâĢ ľ cross ling ual Ġspace âĢ Ŀ Ġ( of Ġdimension ality ĠIN LINE FORM 4 Ġ) Ġwhich Ġis Ġcommon Ġto Ġall Ġlanguages . ĠMore Ġdetails Ġabout Ġthese Ġrepresentations Ġare Ġdiscussed Ġin ĠÂ§ ĠSEC REF 4 Ġ. </s> Document ĠRepresent ation </s> In Ġthis Ġsection , Ġwe Ġgive Ġmore Ġdetails Ġabout Ġthe Ġway Ġwe Ġconstruct Ġthe Ġdocument Ġrepresentations Ġin Ġthe Ġmon oling ual Ġand Ġcross ling ual Ġspaces . ĠIn Ġparticular , Ġwe Ġintroduce Ġthe Ġdefinition Ġof Ġthe Ġsimilarity Ġfunctions ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġthat Ġwere Ġreferred Ġin ĠÂ§ ĠSEC REF 3 Ġ. </s> Similar ity ĠMet rics </s> Our Ġsimilarity Ġmetric Ġcomp utes Ġweighted Ġcos ine Ġsimilarity Ġon Ġthe Ġdifferent Ġsub ve ctors , Ġboth Ġin Ġthe Ġcase Ġof Ġmon oling ual Ġclust ering Ġand Ġcross ling ual Ġclust ering . ĠForm ally , Ġfor Ġthe Ġmon oling ual Ġcase , Ġthe Ġsimilarity Ġis Ġgiven Ġby Ġa Ġfunction Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> and Ġis Ġcomputed Ġon Ġthe ĠTF - ID F Ġsub ve ctors Ġwhere ĠIN LINE FORM 0 Ġis Ġthe Ġnumber Ġof Ġsub ve ctors Ġfor Ġthe Ġrelevant Ġdocument Ġrepresentation . ĠFor Ġthe Ġcross ling ual Ġcase , Ġwe Ġdiscuss Ġbelow Ġthe Ġfunction ĠIN LINE FORM 1 Ġ, Ġwhich Ġhas Ġa Ġsimilar Ġstructure . </s> Here , ĠIN LINE FORM 0 Ġis Ġthe ĠIN LINE FORM 1 Ġth Ġdocument Ġin Ġthe Ġstream Ġand ĠIN LINE FORM 2 Ġis Ġa Ġmon oling ual Ġcluster . ĠThe Ġfunction ĠIN LINE FORM 3 Ġreturns Ġthe Ġcos ine Ġsimilarity Ġbetween Ġthe Ġdocument Ġrepresentation Ġof Ġthe ĠIN LINE FORM 4 Ġth Ġdocument Ġand Ġthe Ġcent roid Ġfor Ġcluster ĠIN LINE FORM 5 Ġ. ĠThe Ġvector ĠIN LINE FORM 6 Ġdenotes Ġthe Ġweights Ġthrough Ġwhich Ġeach Ġof Ġthe Ġcos ine Ġsimilarity Ġvalues Ġfor Ġeach Ġsub ve ctors Ġare Ġweighted , Ġwhereas ĠIN LINE FORM 7 Ġdenotes Ġthe Ġweights Ġfor Ġthe Ġtimestamp Ġfeatures , Ġas Ġdetailed Ġfurther . ĠDetails Ġon Ġlearning Ġthe Ġweights ĠIN LINE FORM 8 Ġand ĠIN LINE FORM 9 Ġare Ġdiscussed Ġin ĠÂ§ ĠSEC REF 26 Ġ. </s> The Ġfunction ĠIN LINE FORM 0 Ġthat Ġmaps Ġa Ġpair Ġof Ġdocument Ġand Ġcluster Ġto ĠIN LINE FORM 1 Ġis Ġdefined Ġas Ġfollows . ĠLet ĠDIS PLAY FORM 0 </s> for Ġa Ġgiven ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġ. ĠFor Ġeach Ġdocument ĠIN LINE FORM 2 Ġand Ġcluster ĠIN LINE FORM 3 Ġ, Ġwe Ġgenerate Ġthe Ġfollowing Ġthree - dimensional Ġvector ĠIN LINE FORM 4 Ġ: </s> IN LINE FORM 0 Ġwhere ĠIN LINE FORM 1 Ġis Ġthe Ġtimestamp Ġfor Ġdocument ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġis Ġthe Ġtimestamp Ġfor Ġthe Ġnewest Ġdocument Ġin Ġcluster ĠIN LINE FORM 4 Ġ. </s> IN LINE FORM 0 Ġwhere ĠIN LINE FORM 1 Ġis Ġthe Ġaverage Ġtimestamp Ġfor Ġall Ġdocuments Ġin Ġcluster ĠIN LINE FORM 2 Ġ. </s> IN LINE FORM 0 Ġwhere ĠIN LINE FORM 1 Ġis Ġthe Ġtimestamp Ġfor Ġthe Ġoldest Ġdocument Ġin Ġcluster ĠIN LINE FORM 2 Ġ. </s> These Ġthree Ġtimestamp Ġfeatures Ġmodel Ġthe Ġtime Ġaspect Ġof Ġthe Ġonline Ġstream Ġof Ġnews Ġdata Ġand Ġhelp Ġdis amb ig uate Ġclust ering Ġdecisions , Ġsince Ġtime Ġis Ġa Ġvaluable Ġindicator Ġthat Ġa Ġnews Ġstory Ġhas Ġchanged , Ġeven Ġif Ġa Ġcluster Ġrepresentation Ġhas Ġa Ġreasonable Ġmatch Ġin Ġthe Ġtextual Ġfeatures Ġwith Ġthe Ġincoming Ġdocument . ĠThe Ġsame Ġway Ġa Ġnews Ġstory Ġbecomes Ġpopular Ġand Ġfades Ġover Ġtime ĠB IB REF 2 Ġ, Ġwe Ġmodel Ġthe Ġprobability Ġof Ġa Ġdocument Ġbelonging Ġto Ġa Ġcluster Ġ( in Ġterms Ġof Ġtimestamp Ġdifference ) Ġwith Ġa Ġprobability Ġdistribution . </s> For Ġthe Ġcase Ġof Ġcross ling ual Ġclust ering , Ġwe Ġintroduce ĠIN LINE FORM 0 Ġ, Ġwhich Ġhas Ġa Ġsimilar Ġdefinition Ġto ĠIN LINE FORM 1 Ġ, Ġonly Ġinstead Ġof Ġpassing Ġdocument / cl uster Ġsimilarity Ġfeature Ġvectors , Ġwe Ġpass Ġcluster / cl uster Ġsimilarities , Ġacross Ġall Ġlanguage Ġpairs . ĠFurthermore , Ġthe Ġfeatures Ġare Ġthe Ġcross ling ual Ġembed ding Ġvectors Ġof Ġthe Ġsections Ġtitle , Ġbody Ġand Ġboth Ġcombined Ġ( similar ly Ġto Ġthe Ġmon oling ual Ġcase ) Ġand Ġthe Ġtimestamp Ġfeatures . ĠFor Ġden oting Ġthe Ġcluster Ġtimestamp , Ġwe Ġuse Ġthe Ġaverage Ġtim est amps Ġof Ġall Ġarticles Ġin Ġit . </s> Learning Ġto ĠRank ĠCand idates </s> In ĠÂ§ ĠSEC REF 19 Ġwe Ġintroduced ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġas Ġthe Ġweight Ġvectors Ġfor Ġthe Ġseveral Ġdocument Ġrepresentation Ġfeatures . ĠWe Ġexperiment Ġwith Ġboth Ġsetting Ġthese Ġweights Ġto Ġjust Ġ1 Ġ( ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġ) Ġand Ġalso Ġlearning Ġthese Ġweights Ġusing Ġsupport Ġvector Ġmachines Ġ( S V Ms ). ĠTo Ġgenerate Ġthe ĠS VM Ġtraining Ġdata , Ġwe Ġsimulate Ġthe Ġexecution Ġof Ġthe Ġalgorithm Ġon Ġa Ġtraining Ġdata Ġpartition Ġ( which Ġwe Ġdo Ġnot Ġget Ġevaluated Ġon ) Ġand Ġin Ġwhich Ġthe Ġgold Ġstandard Ġlabels Ġare Ġgiven . ĠWe Ġrun Ġthe Ġalgorithm Ġusing Ġonly Ġthe Ġfirst Ġsub vector ĠIN LINE FORM 4 Ġ, Ġwhich Ġis Ġthe ĠTF - ID F Ġvector Ġwith Ġthe Ġwords Ġof Ġthe Ġdocument Ġin Ġthe Ġbody Ġand Ġtitle . ĠFor Ġeach Ġincoming Ġdocument , Ġwe Ġcreate Ġa Ġcollection Ġof Ġpositive Ġexamples , Ġfor Ġthe Ġdocument Ġand Ġthe Ġclusters Ġwhich Ġshare Ġat Ġleast Ġone Ġdocument Ġin Ġthe Ġgold Ġlabeling . ĠWe Ġthen Ġgenerate Ġ20 Ġnegative Ġexamples Ġfor Ġthe Ġdocument Ġfrom Ġthe Ġ20 Ġbest - match ing Ġclusters Ġwhich Ġare Ġnot Ġcorrect . ĠTo Ġfind Ġout Ġthe Ġbest - match ing Ġclusters , Ġwe Ġrank Ġthem Ġaccording Ġto Ġtheir Ġsimilarity Ġto Ġthe Ġinput Ġdocument Ġusing Ġonly Ġthe Ġfirst Ġsub vector ĠIN LINE FORM 5 Ġ. </s> Using Ġthis Ġscheme Ġwe Ġgenerate Ġa Ġcollection Ġof Ġranking Ġexamples Ġ( one Ġfor Ġeach Ġdocument Ġin Ġthe Ġdataset , Ġwith Ġthe Ġranking Ġof Ġthe Ġbest Ġcluster Ġmatches ), Ġwhich Ġare Ġthen Ġtrained Ġusing Ġthe ĠSV MR ank Ġalgorithm ĠB IB REF 3 Ġ. ĠWe Ġrun Ġ5 - fold Ġcross - valid ation Ġon Ġthis Ġdata Ġto Ġselect Ġthe Ġbest Ġmodel , Ġand Ġtrain Ġboth Ġa Ġseparate Ġmodel Ġfor Ġeach Ġlanguage Ġaccording Ġto ĠIN LINE FORM 0 Ġand Ġa Ġcross ling ual Ġmodel Ġaccording Ġto ĠIN LINE FORM 1 Ġ. </s> Exper iments </s> Our Ġsystem Ġwas Ġdesigned Ġto Ġcluster Ġdocuments Ġfrom Ġa Ġ( pot entially Ġinfinite ) Ġreal - word Ġdata Ġstream . ĠThe Ġdatasets Ġtypically Ġused Ġin Ġthe Ġliterature Ġ( TD T , ĠReuters ) Ġhave Ġa Ġsmall Ġnumber Ġof Ġclusters Ġ( ĠIN LINE FORM 0 Ġ20 ) Ġwith Ġcoarse Ġtopics Ġ( econom y , Ġsociety , Ġetc .), Ġand Ġtherefore Ġare Ġnot Ġrelevant Ġto Ġthe Ġuse Ġcase Ġof Ġmedia Ġmonitoring Ġwe Ġtreat Ġ- Ġas Ġit Ġrequires Ġmuch Ġmore Ġfine - gr ained Ġstory Ġclusters Ġabout Ġparticular Ġevents . ĠTo Ġevaluate Ġour Ġapproach , Ġwe Ġadapted Ġa Ġdataset Ġconstructed Ġfor Ġthe Ġdifferent Ġpurpose Ġof Ġbinary Ġclassification Ġof Ġjoining Ġcluster Ġpairs . ĠWe Ġprocessed Ġit Ġto Ġbecome Ġa Ġcollection Ġof Ġarticles Ġannot ated Ġwith Ġmon oling ual Ġand Ġcross ling ual Ġcluster Ġlabels . </s> Statistics Ġabout Ġthis Ġdataset Ġare Ġgiven Ġin ĠTable ĠT AB REF 30 Ġ. ĠAs Ġdescribed Ġfurther , Ġwe Ġtune Ġthe Ġhyper - param eter ĠIN LINE FORM 0 Ġon Ġthe Ġdevelopment Ġset . ĠAs Ġfor Ġthe Ġhyper - param eters Ġrelated Ġto Ġthe Ġtimestamp Ġfeatures , Ġwe Ġfixed ĠIN LINE FORM 1 Ġand Ġtuned ĠIN LINE FORM 2 Ġon Ġthe Ġdevelopment Ġset , Ġyielding ĠIN LINE FORM 3 Ġ. ĠTo Ġcompute ĠIDF Ġscores Ġ( which Ġare Ġglobal Ġnumbers Ġcomputed Ġacross Ġa Ġcorpus ), Ġwe Ġused Ġa Ġdifferent Ġand Ġmuch Ġlarger Ġdataset Ġthat Ġwe Ġcollected Ġfrom ĠDeutsche ĠW elle 's Ġnews Ġwebsite Ġ( http :// www . d w . com / ). ĠThe Ġdataset Ġconsists Ġof Ġ77 , 268 , Ġ118 , 045 Ġand Ġ134 , 243 Ġdocuments Ġfor ĠSpanish , ĠEnglish Ġand ĠGerman , Ġrespectively . </s> The Ġconclusions Ġfrom Ġour Ġexperiments Ġare : Ġ( a ) Ġthe Ġweight ing Ġof Ġthe Ġsimilarity Ġmetric Ġfeatures Ġusing ĠS VM Ġsignificantly Ġoutper forms Ġun super vised Ġbas elines Ġsuch Ġas ĠCl u Stream Ġ( Table ĠT AB REF 35 Ġ); Ġ( b ) Ġthe ĠS VM Ġapproach Ġsignificantly Ġhelps Ġto Ġlearn Ġwhen Ġto Ġcreate Ġa Ġnew Ġcluster , Ġcompared Ġto Ġsimple Ġgrid Ġsearch Ġfor Ġthe Ġoptimal ĠIN LINE FORM 0 Ġ( Table ĠT AB REF 39 Ġ); Ġ( c ) Ġseparating Ġthe Ġfeature Ġspace Ġinto Ġone Ġfor Ġmon oling ual Ġclusters Ġin Ġthe Ġform Ġof Ġkeywords Ġand Ġthe Ġother Ġfor Ġcross ling ual Ġclusters Ġbased Ġon Ġcross ling ual Ġembed d ings Ġsignificantly Ġhelps Ġperformance . </s> Mon oling ual ĠResults </s> In Ġour Ġfirst Ġset Ġof Ġexperiments , Ġwe Ġreport Ġresults Ġon Ġmon oling ual Ġclust ering Ġfor Ġeach Ġlanguage Ġseparately . ĠMon oling ual Ġclust ering Ġof Ġa Ġstream Ġof Ġdocuments Ġis Ġan Ġimportant Ġproblem Ġthat Ġhas Ġbeen Ġinspected Ġby Ġothers , Ġsuch Ġas Ġby Ġah med 2011 un ified Ġand Ġby Ġagg ar wal 2006 framework . ĠWe Ġcompare Ġour Ġresults Ġto Ġour Ġown Ġimplementation Ġof Ġthe Ġonline Ġmicro - cl ust ering Ġroutine Ġpresented Ġby Ġagg ar wal 2006 framework , Ġwhich Ġshall Ġbe Ġreferred Ġto Ġas ĠCl u Stream . ĠWe Ġnote Ġthat ĠCl u Stream Ġof Ġagg ar wal 2006 framework Ġhas Ġbeen Ġa Ġwidely Ġused Ġstate - of - the - art Ġsystem Ġin Ġmedia Ġmonitoring Ġcompanies Ġas Ġwell Ġas Ġacademia , Ġand Ġserves Ġas Ġa Ġstrong Ġbaseline Ġto Ġthis Ġday . </s> In Ġour Ġpreliminary Ġexperiments , Ġwe Ġalso Ġevaluated Ġan Ġonline Ġlatent Ġsemantic Ġanalysis Ġmethod , Ġin Ġwhich Ġthe Ġcent ro ids Ġwe Ġkeep Ġfor Ġthe Ġfunction ĠIN LINE FORM 0 Ġ( see ĠÂ§ ĠSEC REF 3 Ġ) Ġare Ġthe Ġaverage Ġof Ġreduced Ġdimensional Ġvectors Ġof Ġthe Ġincoming Ġdocuments Ġas Ġgenerated Ġby Ġan Ġincremental Ġsingular Ġvalue Ġdecom position Ġ( S VD ) Ġof Ġa Ġdocument - term Ġmatrix Ġthat Ġis Ġupdated Ġafter Ġeach Ġincoming Ġdocument . ĠHowever , Ġwe Ġdiscovered Ġthat Ġonline ĠL SA Ġperforms Ġsignificantly Ġworse Ġthan Ġrepresenting Ġthe Ġdocuments Ġthe Ġway Ġis Ġdescribed Ġin ĠÂ§ ĠSEC REF 4 Ġ. ĠFurthermore , Ġit Ġwas Ġalso Ġsignificantly Ġslower Ġthan Ġour Ġalgorithm Ġdue Ġto Ġthe Ġtime Ġit Ġtook Ġto Ġperform Ġsingular Ġvalue Ġdecom position . </s> Table ĠT AB REF 35 Ġgives Ġthe Ġfinal Ġmon oling ual Ġresults Ġon Ġthe Ġthree Ġdatasets . ĠFor ĠEnglish , Ġwe Ġsee Ġthat Ġthe Ġsignificant Ġimprovement Ġwe Ġget Ġusing Ġour Ġalgorithm Ġover Ġthe Ġalgorithm Ġof Ġagg ar wal 2006 framework Ġis Ġdue Ġto Ġan Ġincreased Ġrecall Ġscore . ĠWe Ġalso Ġnote Ġthat Ġthe Ġtrained Ġmodels Ġsurpass Ġthe Ġbaseline Ġfor Ġall Ġlanguages , Ġand Ġthat Ġthe Ġtimestamp Ġfeature Ġ( den oted Ġby ĠTS ), Ġwhile Ġnot Ġrequired Ġto Ġbeat Ġthe Ġbaseline , Ġhas Ġa Ġvery Ġrelevant Ġcontribution Ġin Ġall Ġcases . ĠAlthough Ġthe Ġresults Ġfor Ġboth Ġthe Ġbaseline Ġand Ġour Ġmodels Ġseem Ġto Ġdiffer Ġacross Ġlanguages , Ġone Ġcan Ġverify Ġa Ġconsistent Ġimprovement Ġfrom Ġthe Ġlatter Ġto Ġthe Ġformer , Ġsuggesting Ġthat Ġthe Ġscore Ġdifferences Ġshould Ġbe Ġmostly Ġtied Ġto Ġthe Ġdifferent Ġdifficulty Ġfound Ġacross Ġthe Ġdatasets Ġfor Ġeach Ġlanguage . ĠThe Ġpresented Ġscores Ġshow Ġthat Ġour Ġlearning Ġframework Ġgeneral izes Ġwell Ġto Ġdifferent Ġlanguages Ġand Ġenables Ġhigh Ġquality Ġclust ering Ġresults . </s> To Ġinvestigate Ġthe Ġimpact Ġof Ġthe Ġtimestamp Ġfeatures , Ġwe Ġran Ġan Ġadditional Ġexperiment Ġusing Ġonly Ġthe Ġsame Ġthree Ġtimestamp Ġfeatures Ġas Ġused Ġin Ġthe Ġbest Ġmodel Ġon Ġthe ĠEnglish Ġdataset . ĠThis Ġexperiment Ġyielded Ġscores Ġof ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ, Ġwhich Ġlead Ġus Ġto Ġconclude Ġthat Ġwhile Ġthese Ġfeatures Ġare Ġnot Ġcompetitive Ġwhen Ġused Ġalone Ġ( hen ce Ġtemporal Ġinformation Ġby Ġitself Ġis Ġnot Ġsufficient Ġto Ġpredict Ġthe Ġclusters ), Ġthey Ġcontribute Ġsignificantly Ġto Ġrecall Ġwith Ġthe Ġfinal Ġfeature Ġensemble . </s> We Ġnote Ġthat Ġas Ġdescribed Ġin ĠÂ§ ĠSEC REF 3 Ġ, Ġthe Ġoptimization Ġof Ġthe ĠIN LINE FORM 0 Ġparameter Ġis Ġpart Ġof Ġthe Ġdevelopment Ġprocess . ĠThe Ġparameter ĠIN LINE FORM 1 Ġis Ġa Ġsimilarity Ġthreshold Ġused Ġto Ġdecide Ġwhen Ġan Ġincoming Ġdocument Ġshould Ġmerge Ġto Ġthe Ġbest Ġcluster Ġor Ġcreate Ġa Ġnew Ġone . ĠWe Ġtune ĠIN LINE FORM 2 Ġon Ġthe Ġdevelopment Ġset Ġfor Ġeach Ġlanguage , Ġand Ġthe Ġsensitivity Ġto Ġit Ġis Ġdemonstrated Ġin ĠFigure ĠFIG REF 36 Ġ( this Ġprocess Ġis Ġfurther Ġreferred Ġto Ġas ĠIN LINE FORM 3 Ġ). ĠAlthough Ġapplying Ġgrid - search Ġon Ġthis Ġparameter Ġis Ġthe Ġmost Ġimmediate Ġapproach Ġto Ġthis Ġproblem , Ġwe Ġexperimented Ġwith Ġa Ġdifferent Ġmethod Ġwhich Ġyielded Ġsuperior Ġresults : Ġas Ġdescribed Ġfurther , Ġwe Ġdiscuss Ġhow Ġto Ġdo Ġthis Ġprocess Ġwith Ġan Ġadditional Ġclass ifier Ġ( den oted ĠS VM - mer ge ), Ġwhich Ġcaptures Ġmore Ġinformation Ġabout Ġthe Ġincoming Ġdocuments Ġand Ġthe Ġexisting Ġclusters . </s> Additionally , Ġwe Ġalso Ġexperimented Ġwith Ġcomputing Ġthe Ġmon oling ual Ġclusters Ġwith Ġthe Ġsame Ġembed d ings Ġas Ġused Ġin Ġthe Ġcross ling ual Ġclust ering Ġphase , Ġwhich Ġyielded Ġpoor Ġresults . ĠIn Ġparticular , Ġthis Ġsystem Ġachieved ĠIN LINE FORM 0 Ġscore Ġof ĠIN LINE FORM 1 Ġfor ĠEnglish , Ġwhich Ġis Ġbelow Ġthe Ġbag - of - words Ġbaseline Ġpresented Ġin ĠTable ĠT AB REF 35 Ġ. ĠThis Ġresult Ġsupports Ġthe Ġapproach Ġwe Ġthen Ġfollowed Ġof Ġhaving Ġtwo Ġseparate Ġfeature Ġspaces Ġfor Ġthe Ġmon oling ual Ġand Ġcross ling ual Ġclust ering Ġsystems , Ġwhere Ġthe Ġmon oling ual Ġspace Ġis Ġdiscrete Ġand Ġthe Ġcross ling ual Ġspace Ġis Ġbased Ġon Ġembed d ings . </s> To Ġinvestigate Ġthe Ġimportance Ġof Ġeach Ġfeature , Ġwe Ġnow Ġconsider Ġin ĠTable ĠT AB REF 37 Ġthe Ġaccuracy Ġof Ġthe ĠS VM Ġrank er Ġfor ĠEnglish Ġas Ġdescribed Ġin ĠÂ§ ĠSEC REF 19 Ġ. ĠWe Ġnote Ġthat Ġadding Ġfeatures Ġincreases Ġthe Ġaccuracy Ġof Ġthe ĠS VM Ġrank er , Ġespecially Ġthe Ġtimestamp Ġfeatures . ĠHowever , Ġthe Ġtimestamp Ġfeature Ġactually Ġinter fe res Ġwith Ġour Ġoptimization Ġof ĠIN LINE FORM 0 Ġto Ġidentify Ġwhen Ġnew Ġclusters Ġare Ġneeded , Ġalthough Ġthey Ġimprove Ġthe ĠS VM Ġre ranking Ġaccuracy . ĠWe Ġspeculate Ġthis Ġis Ġtrue Ġbecause Ġhigh Ġaccuracy Ġin Ġthe Ġre ranking Ġproblem Ġdoes Ġnot Ġnecessarily Ġhelp Ġwith Ġidentifying Ġwhen Ġnew Ġclusters Ġneed Ġto Ġbe Ġopened . </s> To Ġinvestigate Ġthis Ġissue , Ġwe Ġexperimented Ġwith Ġa Ġdifferent Ġtechnique Ġto Ġlearn Ġwhen Ġto Ġcreate Ġa Ġnew Ġcluster . ĠTo Ġthis Ġend , Ġwe Ġtrained Ġanother ĠS VM Ġclass ifier Ġjust Ġto Ġlearn Ġthis Ġdecision , Ġthis Ġtime Ġa Ġbinary Ġclass ifier Ġusing ĠLI BL INE AR ĠB IB REF 4 Ġ, Ġby Ġpassing Ġthe Ġmax Ġof Ġthe Ġsimilarity Ġof Ġeach Ġfeature Ġbetween Ġthe Ġincoming Ġdocument Ġand Ġthe Ġcurrent Ġclust ering Ġpool Ġas Ġthe Ġinput Ġfeature Ġvector . ĠThis Ġway , Ġthe Ġclass ifier Ġlearns Ġwhen Ġthe Ġcurrent Ġclusters , Ġas Ġa Ġwhole , Ġare Ġof Ġa Ġdifferent Ġnews Ġstory Ġthan Ġthe Ġincoming Ġdocument . ĠAs Ġpresented Ġin ĠTable ĠT AB REF 39 Ġ, Ġthis Ġmethod , Ġwhich Ġwe Ġrefer Ġto Ġas ĠS VM - mer ge , Ġsolved Ġthe Ġissue Ġof Ġsearching Ġfor Ġthe Ġoptimal ĠIN LINE FORM 0 Ġparameter Ġfor Ġthe ĠS VM - rank Ġmodel Ġwith Ġtim est amps , Ġby Ġgreatly Ġimproving Ġthe ĠF ĠIN LINE FORM 1 Ġscore Ġin Ġrespect Ġto Ġthe Ġoriginal Ġgrid - search Ġapproach Ġ( ĠIN LINE FORM 2 Ġ). </s> Cross ling ual ĠResults </s> As Ġmentioned Ġin ĠÂ§ ĠSEC REF 3 Ġ, Ġcross ling ual Ġembed d ings Ġare Ġused Ġfor Ġcross ling ual Ġclust ering . ĠWe Ġexperimented Ġwith Ġthe Ġcross ling ual Ġembed d ings Ġof Ġgard ner 2015 translation Ġand Ġam mar 2016 mass ively . ĠIn Ġour Ġpreliminary Ġexperiments Ġwe Ġfound Ġthat Ġthe Ġformer Ġworked Ġbetter Ġfor Ġour Ġuse - case Ġthan Ġthe Ġlatter . </s> We Ġtest Ġtwo Ġdifferent Ġscenarios Ġfor Ġoptimizing Ġthe Ġsimilarity Ġthreshold ĠIN LINE FORM 0 Ġfor Ġthe Ġcross ling ual Ġcase . ĠTable ĠT AB REF 41 Ġshows Ġthe Ġresults Ġfor Ġthese Ġexperiments . ĠFirst , Ġwe Ġconsider Ġthe Ġsimpler Ġcase Ġof Ġadjusting Ġa Ġglobal ĠIN LINE FORM 1 Ġparameter Ġfor Ġthe Ġcross ling ual Ġdistances , Ġas Ġalso Ġdescribed Ġfor Ġthe Ġmon oling ual Ġcase . ĠAs Ġshown , Ġthis Ġmethod Ġworks Ġpoorly , Ġsince Ġthe ĠIN LINE FORM 2 Ġgrid - search Ġcould Ġnot Ġfind Ġa Ġreasonable ĠIN LINE FORM 3 Ġwhich Ġworked Ġwell Ġfor Ġevery Ġpossible Ġlanguage Ġpair . </s> Sub sequently , Ġwe Ġalso Ġconsider Ġthe Ġcase Ġof Ġusing ĠEnglish Ġas Ġa Ġpivot Ġlanguage Ġ( see ĠÂ§ ĠSEC REF 3 Ġ), Ġwhere Ġdistances Ġfor Ġevery Ġother Ġlanguage Ġare Ġonly Ġcompared Ġto ĠEnglish , Ġand Ġcross ling ual Ġclust ering Ġdecisions Ġare Ġmade Ġonly Ġbased Ġon Ġthis Ġdistance . ĠThis Ġyielded Ġour Ġbest Ġcross ling ual Ġscore Ġof ĠIN LINE FORM 0 Ġ, Ġconfirming Ġthat Ġcross ling ual Ġsimilarity Ġis Ġof Ġhigher Ġquality Ġbetween Ġeach Ġlanguage Ġand ĠEnglish , Ġfor Ġthe Ġembed d ings Ġwe Ġused . ĠThis Ġscore Ġrepresents Ġonly Ġa Ġsmall Ġdegradation Ġin Ġrespect Ġto Ġthe Ġmon oling ual Ġresults , Ġsince Ġclust ering Ġacross Ġdifferent Ġlanguages Ġis Ġa Ġharder Ġproblem . </s> Related ĠWork </s> Early Ġresearch Ġefforts , Ġsuch Ġas Ġthe ĠTD T Ġprogram ĠB IB REF 5 Ġ, Ġhave Ġstudied Ġnews Ġclust ering Ġfor Ġsome Ġtime . ĠThe Ġproblem Ġof Ġonline Ġmon oling ual Ġclust ering Ġalgorithms Ġ( for ĠEnglish ) Ġhas Ġalso Ġreceived Ġa Ġfair Ġamount Ġof Ġattention Ġin Ġthe Ġliterature . ĠOne Ġof Ġthe Ġearlier Ġpapers Ġby Ġagg ar wal 2006 framework Ġintroduced Ġa Ġtwo - step Ġclust ering Ġsystem Ġwith Ġboth Ġoffline Ġand Ġonline Ġcomponents , Ġwhere Ġthe Ġonline Ġmodel Ġis Ġbased Ġon Ġa Ġstreaming Ġimplementation Ġof ĠIN LINE FORM 0 Ġ- me ans Ġand Ġa Ġbag - of - words Ġdocument Ġrepresentation . ĠOther Ġauthors Ġhave Ġexperimented Ġwith Ġdistributed Ġrepresentations , Ġsuch Ġas Ġah med 2011 un ified , Ġwho Ġcluster Ġnews Ġinto Ġstorylines Ġusing ĠMark ov Ġchain ĠMonte ĠCarlo Ġmethods , Ġre h ure kl rec Ġwho Ġused Ġincremental ĠSing ular ĠValue ĠDec om position Ġ( S VD ) Ġto Ġfind Ġrelevant Ġtopics Ġfrom Ġstreaming Ġdata , Ġand Ġsat o 2017 dist ributed Ġwho Ġused Ġthe Ġparagraph Ġvector Ġmodel ĠB IB REF 6 Ġin Ġan Ġoffline Ġclust ering Ġsetting . </s> More Ġrecently , Ġcross ling ual Ġlinking Ġof Ġclusters Ġhas Ġbeen Ġdiscussed Ġby Ġr up nik 2016 news Ġin Ġthe Ġcontext Ġof Ġlinking Ġexisting Ġclusters Ġfrom Ġthe ĠEvent ĠRegistry ĠB IB REF 7 Ġin Ġa Ġbatch Ġfashion , Ġand Ġby Ġste in berger 2016 medi ag ist Ġwho Ġalso Ġpresent Ġa Ġbatch Ġclust ering Ġlinking Ġsystem . ĠHowever , Ġthese Ġare Ġnot ĠâĢ ľ t ruly âĢ Ŀ Ġonline Ġcross ling ual Ġclust ering Ġsystems Ġsince Ġthey Ġonly Ġdecide Ġon Ġthe Ġlinking Ġof Ġalready - built Ġmon oling ual Ġclusters . ĠIn Ġparticular , Ġr up nik 2016 news Ġcompute Ġdistances Ġof Ġdocument Ġpairs Ġacross Ġclusters Ġusing Ġnearest Ġneighbors , Ġwhich Ġmight Ġnot Ġscale Ġwell Ġin Ġan Ġonline Ġsetting . ĠAs Ġdetailed Ġbefore , Ġwe Ġadapted Ġthe Ġcluster - l inking Ġdataset Ġfrom Ġr up nik 2016 news Ġto Ġevaluate Ġour Ġonline Ġcross ling ual Ġclust ering Ġapproach . ĠPrel iminary Ġwork Ġmakes Ġuse Ġof Ġdeep Ġlearning Ġtechniques ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġto Ġcluster Ġdocuments Ġwhile Ġlearning Ġtheir Ġrepresentations , Ġbut Ġnot Ġin Ġan Ġonline Ġor Ġmult ilingual Ġfashion , Ġand Ġwith Ġa Ġvery Ġsmall Ġnumber Ġof Ġcluster Ġlabels Ġ( 4 , Ġin Ġthe Ġcase Ġof Ġthe Ġtext Ġbenchmark ). </s> In Ġour Ġwork , Ġwe Ġstudied Ġthe Ġproblem Ġof Ġmon oling ual Ġand Ġcross ling ual Ġclust ering , Ġhaving Ġexperimented Ġseveral Ġdirections Ġand Ġmethods Ġand Ġthe Ġimpact Ġthey Ġhave Ġon Ġthe Ġfinal Ġclust ering Ġquality . ĠWe Ġdescribed Ġthe Ġfirst Ġsystem Ġwhich Ġaggreg ates Ġnews Ġarticles Ġinto Ġfine - gr ained Ġstory Ġclusters Ġacross Ġdifferent Ġlanguages Ġin Ġa Ġcompletely Ġonline Ġand Ġscalable Ġfashion Ġfrom Ġa Ġcontinuous Ġstream . </s> Conclusion </s> We Ġdescribed Ġa Ġmethod Ġfor Ġmon oling ual Ġand Ġcross ling ual Ġclust ering Ġof Ġan Ġincoming Ġstream Ġof Ġdocuments . ĠThe Ġmethod Ġworks Ġby Ġmaintaining Ġcent ro ids Ġfor Ġthe Ġmon oling ual Ġand Ġcross ling ual Ġclusters , Ġwhere Ġa Ġmon oling ual Ġcluster Ġgroups Ġa Ġset Ġof Ġdocuments Ġand Ġa Ġcross ling ual Ġcluster Ġgroups Ġa Ġset Ġof Ġmon oling ual Ġclusters . ĠWe Ġpresented Ġan Ġonline Ġcross ling ual Ġclust ering Ġmethod Ġwhich Ġauto - correct s Ġpast Ġdecisions Ġin Ġan Ġefficient Ġway . ĠWe Ġshowed Ġthat Ġour Ġmethod Ġgives Ġstate - of - the - art Ġresults Ġon Ġa Ġmult ilingual Ġnews Ġarticle Ġdataset Ġfor ĠEnglish , ĠSpanish Ġand ĠGerman . ĠFinally , Ġwe Ġdiscussed Ġhow Ġto Ġleverage Ġdifferent ĠS VM Ġtraining Ġprocedures Ġfor Ġranking Ġand Ġclassification Ġto Ġimprove Ġmon oling ual Ġand Ġcross ling ual Ġclust ering Ġdecisions . ĠOur Ġsystem Ġis Ġintegrated Ġin Ġa Ġlarger Ġmedia Ġmonitoring Ġproject ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġand Ġsolving Ġthe Ġuse - cases Ġof Ġmonitors Ġand Ġjournalists , Ġhaving Ġbeen Ġvalidated Ġwith Ġqualitative Ġuser Ġtesting . </s> Acknowled gments </s> We Ġwould Ġlike Ġto Ġthank ĠEs ma ĠBalk Ä± r , ĠNik os ĠPap as arant opoulos , ĠAf onso ĠMend es , ĠSh ashi ĠNar ayan Ġand Ġthe Ġanonymous Ġreviewers Ġfor Ġtheir Ġfeedback . ĠThis Ġproject Ġwas Ġsupported Ġby Ġthe ĠEuropean ĠH 2020 Ġproject ĠSUM MA , Ġgrant Ġagreement Ġ6 88 139 Ġ( see Ġhttp :// www . sum ma - project . eu ) Ġand Ġby Ġa Ġgrant Ġfrom ĠBloomberg .
Introduction Following developing news stories is imperative to making real-time decisions on important political and public safety matters. Given the abundance of media providers and languages, this endeavor is an extremely difficult task. As such, there is a strong demand for automatic clustering of news streams, so that they can be organized into stories or themes for further processing. Performing this task in an online and efficient manner is a challenging problem, not only for newswire, but also for scientific articles, online reviews, forum posts, blogs, and microblogs. A key challenge in handling document streams is that the story clusters must be generated on the fly in an online fashion: this requires handling documents one-by-one as they appear in the document stream. In this paper, we provide a treatment to the problem of online document clustering, i.e. the task of clustering a stream of documents into themes. For example, for news articles, we would want to cluster them into related news stories. To this end, we introduce a system which aggregates news articles into fine-grained story clusters across different languages in a completely online and scalable fashion from a continuous stream. Our clustering approach is part of a larger media monitoring project to solve the problem of monitoring massive text and TV/Radio streams (speech-to-text). In particular, media monitors write intelligence reports about the most relevant events, and being able to search, visualize and explore news clusters assists in gathering more insight about a particular story. Since relevant events may be spawned from any part of the world (and from many multilingual sources), it becomes imperative to cluster news across different languages. In terms of granularity, the type of story clusters we are interested in are the group of articles which, for example : (i) Narrate recent air-strikes in Eastern Ghouta (Syria); (ii) Describe the recent launch of Space X's Falcon Heavy rocket. Problem Formulation We focus on clustering of a stream of documents, where the number of clusters is not fixed and learned automatically. We denote by INLINEFORM0 a (potentially infinite) space of multilingual documents. Each document INLINEFORM1 is associated with a language in which it is written through a function INLINEFORM2 where INLINEFORM3 is a set of languages. For example, INLINEFORM4 could return English, Spanish or German. (In the rest of the paper, for an integer INLINEFORM5 , we denote by INLINEFORM6 the set INLINEFORM7 .) We are interested in associating each document with a monolingual cluster via the function INLINEFORM0 , which returns the cluster label given a document. This is done independently for each language, such that the space of indices we use for each language is separate. Furthermore, we interlace the problem of monolingual clustering with crosslingual clustering. This means that as part of our problem formulation we are also interested in a function INLINEFORM0 that associates each monolingual cluster with a crosslingual cluster, such that each crosslingual cluster only groups one monolingual cluster per different language, at a given time. The crosslingual cluster for a document INLINEFORM1 is INLINEFORM2 . As such, a crosslingual cluster groups together monolingual clusters, at most one for each different language. Intuitively, building both monolingual and crosslingual clusters allows the system to leverage high-precision monolingual features (e.g., words, named entities) to cluster documents of the same language, while simplifying the task of crosslingual clustering to the computation of similarity scores across monolingual clusters - which is a smaller problem space, since there are (by definition) less clusters than articles. We validate this choice in § SECREF5 . The Clustering Algorithm Each document INLINEFORM0 is represented by two vectors in INLINEFORM1 and INLINEFORM2 . The first vector exists in a “monolingual space” (of dimensionality INLINEFORM3 ) and is based on a bag-of-words representation of the document. The second vector exists in a “crosslingual space” (of dimensionality INLINEFORM4 ) which is common to all languages. More details about these representations are discussed in § SECREF4 . Document Representation In this section, we give more details about the way we construct the document representations in the monolingual and crosslingual spaces. In particular, we introduce the definition of the similarity functions INLINEFORM0 and INLINEFORM1 that were referred in § SECREF3 . Similarity Metrics Our similarity metric computes weighted cosine similarity on the different subvectors, both in the case of monolingual clustering and crosslingual clustering. Formally, for the monolingual case, the similarity is given by a function defined as: DISPLAYFORM0 and is computed on the TF-IDF subvectors where INLINEFORM0 is the number of subvectors for the relevant document representation. For the crosslingual case, we discuss below the function INLINEFORM1 , which has a similar structure. Here, INLINEFORM0 is the INLINEFORM1 th document in the stream and INLINEFORM2 is a monolingual cluster. The function INLINEFORM3 returns the cosine similarity between the document representation of the INLINEFORM4 th document and the centroid for cluster INLINEFORM5 . The vector INLINEFORM6 denotes the weights through which each of the cosine similarity values for each subvectors are weighted, whereas INLINEFORM7 denotes the weights for the timestamp features, as detailed further. Details on learning the weights INLINEFORM8 and INLINEFORM9 are discussed in § SECREF26 . The function INLINEFORM0 that maps a pair of document and cluster to INLINEFORM1 is defined as follows. Let DISPLAYFORM0 for a given INLINEFORM0 and INLINEFORM1 . For each document INLINEFORM2 and cluster INLINEFORM3 , we generate the following three-dimensional vector INLINEFORM4 : INLINEFORM0 where INLINEFORM1 is the timestamp for document INLINEFORM2 and INLINEFORM3 is the timestamp for the newest document in cluster INLINEFORM4 . INLINEFORM0 where INLINEFORM1 is the average timestamp for all documents in cluster INLINEFORM2 . INLINEFORM0 where INLINEFORM1 is the timestamp for the oldest document in cluster INLINEFORM2 . These three timestamp features model the time aspect of the online stream of news data and help disambiguate clustering decisions, since time is a valuable indicator that a news story has changed, even if a cluster representation has a reasonable match in the textual features with the incoming document. The same way a news story becomes popular and fades over time BIBREF2 , we model the probability of a document belonging to a cluster (in terms of timestamp difference) with a probability distribution. For the case of crosslingual clustering, we introduce INLINEFORM0 , which has a similar definition to INLINEFORM1 , only instead of passing document/cluster similarity feature vectors, we pass cluster/cluster similarities, across all language pairs. Furthermore, the features are the crosslingual embedding vectors of the sections title, body and both combined (similarly to the monolingual case) and the timestamp features. For denoting the cluster timestamp, we use the average timestamps of all articles in it. Learning to Rank Candidates In § SECREF19 we introduced INLINEFORM0 and INLINEFORM1 as the weight vectors for the several document representation features. We experiment with both setting these weights to just 1 ( INLINEFORM2 and INLINEFORM3 ) and also learning these weights using support vector machines (SVMs). To generate the SVM training data, we simulate the execution of the algorithm on a training data partition (which we do not get evaluated on) and in which the gold standard labels are given. We run the algorithm using only the first subvector INLINEFORM4 , which is the TF-IDF vector with the words of the document in the body and title. For each incoming document, we create a collection of positive examples, for the document and the clusters which share at least one document in the gold labeling. We then generate 20 negative examples for the document from the 20 best-matching clusters which are not correct. To find out the best-matching clusters, we rank them according to their similarity to the input document using only the first subvector INLINEFORM5 . Using this scheme we generate a collection of ranking examples (one for each document in the dataset, with the ranking of the best cluster matches), which are then trained using the SVMRank algorithm BIBREF3 . We run 5-fold cross-validation on this data to select the best model, and train both a separate model for each language according to INLINEFORM0 and a crosslingual model according to INLINEFORM1 . Experiments Our system was designed to cluster documents from a (potentially infinite) real-word data stream. The datasets typically used in the literature (TDT, Reuters) have a small number of clusters ( INLINEFORM0 20) with coarse topics (economy, society, etc.), and therefore are not relevant to the use case of media monitoring we treat - as it requires much more fine-grained story clusters about particular events. To evaluate our approach, we adapted a dataset constructed for the different purpose of binary classification of joining cluster pairs. We processed it to become a collection of articles annotated with monolingual and crosslingual cluster labels. Statistics about this dataset are given in Table TABREF30 . As described further, we tune the hyper-parameter INLINEFORM0 on the development set. As for the hyper-parameters related to the timestamp features, we fixed INLINEFORM1 and tuned INLINEFORM2 on the development set, yielding INLINEFORM3 . To compute IDF scores (which are global numbers computed across a corpus), we used a different and much larger dataset that we collected from Deutsche Welle's news website (http://www.dw.com/). The dataset consists of 77,268, 118,045 and 134,243 documents for Spanish, English and German, respectively. The conclusions from our experiments are: (a) the weighting of the similarity metric features using SVM significantly outperforms unsupervised baselines such as CluStream (Table TABREF35 ); (b) the SVM approach significantly helps to learn when to create a new cluster, compared to simple grid search for the optimal INLINEFORM0 (Table TABREF39 ); (c) separating the feature space into one for monolingual clusters in the form of keywords and the other for crosslingual clusters based on crosslingual embeddings significantly helps performance. Monolingual Results In our first set of experiments, we report results on monolingual clustering for each language separately. Monolingual clustering of a stream of documents is an important problem that has been inspected by others, such as by ahmed2011unified and by aggarwal2006framework. We compare our results to our own implementation of the online micro-clustering routine presented by aggarwal2006framework, which shall be referred to as CluStream. We note that CluStream of aggarwal2006framework has been a widely used state-of-the-art system in media monitoring companies as well as academia, and serves as a strong baseline to this day. In our preliminary experiments, we also evaluated an online latent semantic analysis method, in which the centroids we keep for the function INLINEFORM0 (see § SECREF3 ) are the average of reduced dimensional vectors of the incoming documents as generated by an incremental singular value decomposition (SVD) of a document-term matrix that is updated after each incoming document. However, we discovered that online LSA performs significantly worse than representing the documents the way is described in § SECREF4 . Furthermore, it was also significantly slower than our algorithm due to the time it took to perform singular value decomposition. Table TABREF35 gives the final monolingual results on the three datasets. For English, we see that the significant improvement we get using our algorithm over the algorithm of aggarwal2006framework is due to an increased recall score. We also note that the trained models surpass the baseline for all languages, and that the timestamp feature (denoted by TS), while not required to beat the baseline, has a very relevant contribution in all cases. Although the results for both the baseline and our models seem to differ across languages, one can verify a consistent improvement from the latter to the former, suggesting that the score differences should be mostly tied to the different difficulty found across the datasets for each language. The presented scores show that our learning framework generalizes well to different languages and enables high quality clustering results. To investigate the impact of the timestamp features, we ran an additional experiment using only the same three timestamp features as used in the best model on the English dataset. This experiment yielded scores of INLINEFORM0 , INLINEFORM1 and INLINEFORM2 , which lead us to conclude that while these features are not competitive when used alone (hence temporal information by itself is not sufficient to predict the clusters), they contribute significantly to recall with the final feature ensemble. We note that as described in § SECREF3 , the optimization of the INLINEFORM0 parameter is part of the development process. The parameter INLINEFORM1 is a similarity threshold used to decide when an incoming document should merge to the best cluster or create a new one. We tune INLINEFORM2 on the development set for each language, and the sensitivity to it is demonstrated in Figure FIGREF36 (this process is further referred to as INLINEFORM3 ). Although applying grid-search on this parameter is the most immediate approach to this problem, we experimented with a different method which yielded superior results: as described further, we discuss how to do this process with an additional classifier (denoted SVM-merge), which captures more information about the incoming documents and the existing clusters. Additionally, we also experimented with computing the monolingual clusters with the same embeddings as used in the crosslingual clustering phase, which yielded poor results. In particular, this system achieved INLINEFORM0 score of INLINEFORM1 for English, which is below the bag-of-words baseline presented in Table TABREF35 . This result supports the approach we then followed of having two separate feature spaces for the monolingual and crosslingual clustering systems, where the monolingual space is discrete and the crosslingual space is based on embeddings. To investigate the importance of each feature, we now consider in Table TABREF37 the accuracy of the SVM ranker for English as described in § SECREF19 . We note that adding features increases the accuracy of the SVM ranker, especially the timestamp features. However, the timestamp feature actually interferes with our optimization of INLINEFORM0 to identify when new clusters are needed, although they improve the SVM reranking accuracy. We speculate this is true because high accuracy in the reranking problem does not necessarily help with identifying when new clusters need to be opened. To investigate this issue, we experimented with a different technique to learn when to create a new cluster. To this end, we trained another SVM classifier just to learn this decision, this time a binary classifier using LIBLINEAR BIBREF4 , by passing the max of the similarity of each feature between the incoming document and the current clustering pool as the input feature vector. This way, the classifier learns when the current clusters, as a whole, are of a different news story than the incoming document. As presented in Table TABREF39 , this method, which we refer to as SVM-merge, solved the issue of searching for the optimal INLINEFORM0 parameter for the SVM-rank model with timestamps, by greatly improving the F INLINEFORM1 score in respect to the original grid-search approach ( INLINEFORM2 ). Crosslingual Results As mentioned in § SECREF3 , crosslingual embeddings are used for crosslingual clustering. We experimented with the crosslingual embeddings of gardner2015translation and ammar2016massively. In our preliminary experiments we found that the former worked better for our use-case than the latter. We test two different scenarios for optimizing the similarity threshold INLINEFORM0 for the crosslingual case. Table TABREF41 shows the results for these experiments. First, we consider the simpler case of adjusting a global INLINEFORM1 parameter for the crosslingual distances, as also described for the monolingual case. As shown, this method works poorly, since the INLINEFORM2 grid-search could not find a reasonable INLINEFORM3 which worked well for every possible language pair. Subsequently, we also consider the case of using English as a pivot language (see § SECREF3 ), where distances for every other language are only compared to English, and crosslingual clustering decisions are made only based on this distance. This yielded our best crosslingual score of INLINEFORM0 , confirming that crosslingual similarity is of higher quality between each language and English, for the embeddings we used. This score represents only a small degradation in respect to the monolingual results, since clustering across different languages is a harder problem. Related Work Early research efforts, such as the TDT program BIBREF5 , have studied news clustering for some time. The problem of online monolingual clustering algorithms (for English) has also received a fair amount of attention in the literature. One of the earlier papers by aggarwal2006framework introduced a two-step clustering system with both offline and online components, where the online model is based on a streaming implementation of INLINEFORM0 -means and a bag-of-words document representation. Other authors have experimented with distributed representations, such as ahmed2011unified, who cluster news into storylines using Markov chain Monte Carlo methods, rehureklrec who used incremental Singular Value Decomposition (SVD) to find relevant topics from streaming data, and sato2017distributed who used the paragraph vector model BIBREF6 in an offline clustering setting. More recently, crosslingual linking of clusters has been discussed by rupnik2016news in the context of linking existing clusters from the Event Registry BIBREF7 in a batch fashion, and by steinberger2016mediagist who also present a batch clustering linking system. However, these are not “truly” online crosslingual clustering systems since they only decide on the linking of already-built monolingual clusters. In particular, rupnik2016news compute distances of document pairs across clusters using nearest neighbors, which might not scale well in an online setting. As detailed before, we adapted the cluster-linking dataset from rupnik2016news to evaluate our online crosslingual clustering approach. Preliminary work makes use of deep learning techniques BIBREF8 , BIBREF9 to cluster documents while learning their representations, but not in an online or multilingual fashion, and with a very small number of cluster labels (4, in the case of the text benchmark). In our work, we studied the problem of monolingual and crosslingual clustering, having experimented several directions and methods and the impact they have on the final clustering quality. We described the first system which aggregates news articles into fine-grained story clusters across different languages in a completely online and scalable fashion from a continuous stream. Conclusion We described a method for monolingual and crosslingual clustering of an incoming stream of documents. The method works by maintaining centroids for the monolingual and crosslingual clusters, where a monolingual cluster groups a set of documents and a crosslingual cluster groups a set of monolingual clusters. We presented an online crosslingual clustering method which auto-corrects past decisions in an efficient way. We showed that our method gives state-of-the-art results on a multilingual news article dataset for English, Spanish and German. Finally, we discussed how to leverage different SVM training procedures for ranking and classification to improve monolingual and crosslingual clustering decisions. Our system is integrated in a larger media monitoring project BIBREF10 , BIBREF11 and solving the use-cases of monitors and journalists, having been validated with qualitative user testing. Acknowledgments We would like to thank Esma Balkır, Nikos Papasarantopoulos, Afonso Mendes, Shashi Narayan and the anonymous reviewers for their feedback. This project was supported by the European H2020 project SUMMA, grant agreement 688139 (see http://www.summa-project.eu) and by a grant from Bloomberg.
What are the sources of the datasets?
2c78993524ca62bf1f525b60f2220a374d0e3535
What Ġare Ġthe Ġsources Ġof Ġthe Ġdatasets ?
rup nik 2016 news
<s> What Ġare Ġthe Ġsources Ġof Ġthe Ġdatasets ? </s> Introduction </s> Following Ġdeveloping Ġnews Ġstories Ġis Ġimperative Ġto Ġmaking Ġreal - time Ġdecisions Ġon Ġimportant Ġpolitical Ġand Ġpublic Ġsafety Ġmatters . ĠGiven Ġthe Ġabundance Ġof Ġmedia Ġproviders Ġand Ġlanguages , Ġthis Ġendeavor Ġis Ġan Ġextremely Ġdifficult Ġtask . ĠAs Ġsuch , Ġthere Ġis Ġa Ġstrong Ġdemand Ġfor Ġautomatic Ġclust ering Ġof Ġnews Ġstreams , Ġso Ġthat Ġthey Ġcan Ġbe Ġorganized Ġinto Ġstories Ġor Ġthemes Ġfor Ġfurther Ġprocessing . ĠPer forming Ġthis Ġtask Ġin Ġan Ġonline Ġand Ġefficient Ġmanner Ġis Ġa Ġchallenging Ġproblem , Ġnot Ġonly Ġfor Ġnews wire , Ġbut Ġalso Ġfor Ġscientific Ġarticles , Ġonline Ġreviews , Ġforum Ġposts , Ġblogs , Ġand Ġmicro blogs . </s> A Ġkey Ġchallenge Ġin Ġhandling Ġdocument Ġstreams Ġis Ġthat Ġthe Ġstory Ġclusters Ġmust Ġbe Ġgenerated Ġon Ġthe Ġfly Ġin Ġan Ġonline Ġfashion : Ġthis Ġrequires Ġhandling Ġdocuments Ġone - by - one Ġas Ġthey Ġappear Ġin Ġthe Ġdocument Ġstream . ĠIn Ġthis Ġpaper , Ġwe Ġprovide Ġa Ġtreatment Ġto Ġthe Ġproblem Ġof Ġonline Ġdocument Ġclust ering , Ġi . e . Ġthe Ġtask Ġof Ġclust ering Ġa Ġstream Ġof Ġdocuments Ġinto Ġthemes . ĠFor Ġexample , Ġfor Ġnews Ġarticles , Ġwe Ġwould Ġwant Ġto Ġcluster Ġthem Ġinto Ġrelated Ġnews Ġstories . </s> To Ġthis Ġend , Ġwe Ġintroduce Ġa Ġsystem Ġwhich Ġaggreg ates Ġnews Ġarticles Ġinto Ġfine - gr ained Ġstory Ġclusters Ġacross Ġdifferent Ġlanguages Ġin Ġa Ġcompletely Ġonline Ġand Ġscalable Ġfashion Ġfrom Ġa Ġcontinuous Ġstream . ĠOur Ġclust ering Ġapproach Ġis Ġpart Ġof Ġa Ġlarger Ġmedia Ġmonitoring Ġproject Ġto Ġsolve Ġthe Ġproblem Ġof Ġmonitoring Ġmassive Ġtext Ġand ĠTV / Radio Ġstreams Ġ( speech - to - text ). ĠIn Ġparticular , Ġmedia Ġmonitors Ġwrite Ġintelligence Ġreports Ġabout Ġthe Ġmost Ġrelevant Ġevents , Ġand Ġbeing Ġable Ġto Ġsearch , Ġvisualize Ġand Ġexplore Ġnews Ġclusters Ġassists Ġin Ġgathering Ġmore Ġinsight Ġabout Ġa Ġparticular Ġstory . ĠSince Ġrelevant Ġevents Ġmay Ġbe Ġspawned Ġfrom Ġany Ġpart Ġof Ġthe Ġworld Ġ( and Ġfrom Ġmany Ġmult ilingual Ġsources ), Ġit Ġbecomes Ġimperative Ġto Ġcluster Ġnews Ġacross Ġdifferent Ġlanguages . </s> In Ġterms Ġof Ġgran ularity , Ġthe Ġtype Ġof Ġstory Ġclusters Ġwe Ġare Ġinterested Ġin Ġare Ġthe Ġgroup Ġof Ġarticles Ġwhich , Ġfor Ġexample Ġ: Ġ( i ) ĠNarr ate Ġrecent Ġair - st rikes Ġin ĠEastern ĠGh out a Ġ( Syria ); Ġ( ii ) ĠDesc ribe Ġthe Ġrecent Ġlaunch Ġof ĠSpace ĠX 's ĠFalcon ĠHeavy Ġrocket . </s> Problem ĠForm ulation </s> We Ġfocus Ġon Ġclust ering Ġof Ġa Ġstream Ġof Ġdocuments , Ġwhere Ġthe Ġnumber Ġof Ġclusters Ġis Ġnot Ġfixed Ġand Ġlearned Ġautomatically . ĠWe Ġdenote Ġby ĠIN LINE FORM 0 Ġa Ġ( pot entially Ġinfinite ) Ġspace Ġof Ġmult ilingual Ġdocuments . ĠEach Ġdocument ĠIN LINE FORM 1 Ġis Ġassociated Ġwith Ġa Ġlanguage Ġin Ġwhich Ġit Ġis Ġwritten Ġthrough Ġa Ġfunction ĠIN LINE FORM 2 Ġwhere ĠIN LINE FORM 3 Ġis Ġa Ġset Ġof Ġlanguages . ĠFor Ġexample , ĠIN LINE FORM 4 Ġcould Ġreturn ĠEnglish , ĠSpanish Ġor ĠGerman . Ġ( In Ġthe Ġrest Ġof Ġthe Ġpaper , Ġfor Ġan Ġinteger ĠIN LINE FORM 5 Ġ, Ġwe Ġdenote Ġby ĠIN LINE FORM 6 Ġthe Ġset ĠIN LINE FORM 7 Ġ.) </s> We Ġare Ġinterested Ġin Ġassoci ating Ġeach Ġdocument Ġwith Ġa Ġmon oling ual Ġcluster Ġvia Ġthe Ġfunction ĠIN LINE FORM 0 Ġ, Ġwhich Ġreturns Ġthe Ġcluster Ġlabel Ġgiven Ġa Ġdocument . ĠThis Ġis Ġdone Ġindependently Ġfor Ġeach Ġlanguage , Ġsuch Ġthat Ġthe Ġspace Ġof Ġindices Ġwe Ġuse Ġfor Ġeach Ġlanguage Ġis Ġseparate . </s> Furthermore , Ġwe Ġinter lace Ġthe Ġproblem Ġof Ġmon oling ual Ġclust ering Ġwith Ġcross ling ual Ġclust ering . ĠThis Ġmeans Ġthat Ġas Ġpart Ġof Ġour Ġproblem Ġformulation Ġwe Ġare Ġalso Ġinterested Ġin Ġa Ġfunction ĠIN LINE FORM 0 Ġthat Ġassociates Ġeach Ġmon oling ual Ġcluster Ġwith Ġa Ġcross ling ual Ġcluster , Ġsuch Ġthat Ġeach Ġcross ling ual Ġcluster Ġonly Ġgroups Ġone Ġmon oling ual Ġcluster Ġper Ġdifferent Ġlanguage , Ġat Ġa Ġgiven Ġtime . ĠThe Ġcross ling ual Ġcluster Ġfor Ġa Ġdocument ĠIN LINE FORM 1 Ġis ĠIN LINE FORM 2 Ġ. ĠAs Ġsuch , Ġa Ġcross ling ual Ġcluster Ġgroups Ġtogether Ġmon oling ual Ġclusters , Ġat Ġmost Ġone Ġfor Ġeach Ġdifferent Ġlanguage . </s> Int uitive ly , Ġbuilding Ġboth Ġmon oling ual Ġand Ġcross ling ual Ġclusters Ġallows Ġthe Ġsystem Ġto Ġleverage Ġhigh - pre cision Ġmon oling ual Ġfeatures Ġ( e . g ., Ġwords , Ġnamed Ġentities ) Ġto Ġcluster Ġdocuments Ġof Ġthe Ġsame Ġlanguage , Ġwhile Ġsimpl ifying Ġthe Ġtask Ġof Ġcross ling ual Ġclust ering Ġto Ġthe Ġcomputation Ġof Ġsimilarity Ġscores Ġacross Ġmon oling ual Ġclusters Ġ- Ġwhich Ġis Ġa Ġsmaller Ġproblem Ġspace , Ġsince Ġthere Ġare Ġ( by Ġdefinition ) Ġless Ġclusters Ġthan Ġarticles . ĠWe Ġvalidate Ġthis Ġchoice Ġin ĠÂ§ ĠSEC REF 5 Ġ. </s> The ĠCl ust ering ĠAl gorithm </s> Each Ġdocument ĠIN LINE FORM 0 Ġis Ġrepresented Ġby Ġtwo Ġvectors Ġin ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ. ĠThe Ġfirst Ġvector Ġexists Ġin Ġa ĠâĢ ľ mon oling ual Ġspace âĢ Ŀ Ġ( of Ġdimension ality ĠIN LINE FORM 3 Ġ) Ġand Ġis Ġbased Ġon Ġa Ġbag - of - words Ġrepresentation Ġof Ġthe Ġdocument . ĠThe Ġsecond Ġvector Ġexists Ġin Ġa ĠâĢ ľ cross ling ual Ġspace âĢ Ŀ Ġ( of Ġdimension ality ĠIN LINE FORM 4 Ġ) Ġwhich Ġis Ġcommon Ġto Ġall Ġlanguages . ĠMore Ġdetails Ġabout Ġthese Ġrepresentations Ġare Ġdiscussed Ġin ĠÂ§ ĠSEC REF 4 Ġ. </s> Document ĠRepresent ation </s> In Ġthis Ġsection , Ġwe Ġgive Ġmore Ġdetails Ġabout Ġthe Ġway Ġwe Ġconstruct Ġthe Ġdocument Ġrepresentations Ġin Ġthe Ġmon oling ual Ġand Ġcross ling ual Ġspaces . ĠIn Ġparticular , Ġwe Ġintroduce Ġthe Ġdefinition Ġof Ġthe Ġsimilarity Ġfunctions ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġthat Ġwere Ġreferred Ġin ĠÂ§ ĠSEC REF 3 Ġ. </s> Similar ity ĠMet rics </s> Our Ġsimilarity Ġmetric Ġcomp utes Ġweighted Ġcos ine Ġsimilarity Ġon Ġthe Ġdifferent Ġsub ve ctors , Ġboth Ġin Ġthe Ġcase Ġof Ġmon oling ual Ġclust ering Ġand Ġcross ling ual Ġclust ering . ĠForm ally , Ġfor Ġthe Ġmon oling ual Ġcase , Ġthe Ġsimilarity Ġis Ġgiven Ġby Ġa Ġfunction Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> and Ġis Ġcomputed Ġon Ġthe ĠTF - ID F Ġsub ve ctors Ġwhere ĠIN LINE FORM 0 Ġis Ġthe Ġnumber Ġof Ġsub ve ctors Ġfor Ġthe Ġrelevant Ġdocument Ġrepresentation . ĠFor Ġthe Ġcross ling ual Ġcase , Ġwe Ġdiscuss Ġbelow Ġthe Ġfunction ĠIN LINE FORM 1 Ġ, Ġwhich Ġhas Ġa Ġsimilar Ġstructure . </s> Here , ĠIN LINE FORM 0 Ġis Ġthe ĠIN LINE FORM 1 Ġth Ġdocument Ġin Ġthe Ġstream Ġand ĠIN LINE FORM 2 Ġis Ġa Ġmon oling ual Ġcluster . ĠThe Ġfunction ĠIN LINE FORM 3 Ġreturns Ġthe Ġcos ine Ġsimilarity Ġbetween Ġthe Ġdocument Ġrepresentation Ġof Ġthe ĠIN LINE FORM 4 Ġth Ġdocument Ġand Ġthe Ġcent roid Ġfor Ġcluster ĠIN LINE FORM 5 Ġ. ĠThe Ġvector ĠIN LINE FORM 6 Ġdenotes Ġthe Ġweights Ġthrough Ġwhich Ġeach Ġof Ġthe Ġcos ine Ġsimilarity Ġvalues Ġfor Ġeach Ġsub ve ctors Ġare Ġweighted , Ġwhereas ĠIN LINE FORM 7 Ġdenotes Ġthe Ġweights Ġfor Ġthe Ġtimestamp Ġfeatures , Ġas Ġdetailed Ġfurther . ĠDetails Ġon Ġlearning Ġthe Ġweights ĠIN LINE FORM 8 Ġand ĠIN LINE FORM 9 Ġare Ġdiscussed Ġin ĠÂ§ ĠSEC REF 26 Ġ. </s> The Ġfunction ĠIN LINE FORM 0 Ġthat Ġmaps Ġa Ġpair Ġof Ġdocument Ġand Ġcluster Ġto ĠIN LINE FORM 1 Ġis Ġdefined Ġas Ġfollows . ĠLet ĠDIS PLAY FORM 0 </s> for Ġa Ġgiven ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġ. ĠFor Ġeach Ġdocument ĠIN LINE FORM 2 Ġand Ġcluster ĠIN LINE FORM 3 Ġ, Ġwe Ġgenerate Ġthe Ġfollowing Ġthree - dimensional Ġvector ĠIN LINE FORM 4 Ġ: </s> IN LINE FORM 0 Ġwhere ĠIN LINE FORM 1 Ġis Ġthe Ġtimestamp Ġfor Ġdocument ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġis Ġthe Ġtimestamp Ġfor Ġthe Ġnewest Ġdocument Ġin Ġcluster ĠIN LINE FORM 4 Ġ. </s> IN LINE FORM 0 Ġwhere ĠIN LINE FORM 1 Ġis Ġthe Ġaverage Ġtimestamp Ġfor Ġall Ġdocuments Ġin Ġcluster ĠIN LINE FORM 2 Ġ. </s> IN LINE FORM 0 Ġwhere ĠIN LINE FORM 1 Ġis Ġthe Ġtimestamp Ġfor Ġthe Ġoldest Ġdocument Ġin Ġcluster ĠIN LINE FORM 2 Ġ. </s> These Ġthree Ġtimestamp Ġfeatures Ġmodel Ġthe Ġtime Ġaspect Ġof Ġthe Ġonline Ġstream Ġof Ġnews Ġdata Ġand Ġhelp Ġdis amb ig uate Ġclust ering Ġdecisions , Ġsince Ġtime Ġis Ġa Ġvaluable Ġindicator Ġthat Ġa Ġnews Ġstory Ġhas Ġchanged , Ġeven Ġif Ġa Ġcluster Ġrepresentation Ġhas Ġa Ġreasonable Ġmatch Ġin Ġthe Ġtextual Ġfeatures Ġwith Ġthe Ġincoming Ġdocument . ĠThe Ġsame Ġway Ġa Ġnews Ġstory Ġbecomes Ġpopular Ġand Ġfades Ġover Ġtime ĠB IB REF 2 Ġ, Ġwe Ġmodel Ġthe Ġprobability Ġof Ġa Ġdocument Ġbelonging Ġto Ġa Ġcluster Ġ( in Ġterms Ġof Ġtimestamp Ġdifference ) Ġwith Ġa Ġprobability Ġdistribution . </s> For Ġthe Ġcase Ġof Ġcross ling ual Ġclust ering , Ġwe Ġintroduce ĠIN LINE FORM 0 Ġ, Ġwhich Ġhas Ġa Ġsimilar Ġdefinition Ġto ĠIN LINE FORM 1 Ġ, Ġonly Ġinstead Ġof Ġpassing Ġdocument / cl uster Ġsimilarity Ġfeature Ġvectors , Ġwe Ġpass Ġcluster / cl uster Ġsimilarities , Ġacross Ġall Ġlanguage Ġpairs . ĠFurthermore , Ġthe Ġfeatures Ġare Ġthe Ġcross ling ual Ġembed ding Ġvectors Ġof Ġthe Ġsections Ġtitle , Ġbody Ġand Ġboth Ġcombined Ġ( similar ly Ġto Ġthe Ġmon oling ual Ġcase ) Ġand Ġthe Ġtimestamp Ġfeatures . ĠFor Ġden oting Ġthe Ġcluster Ġtimestamp , Ġwe Ġuse Ġthe Ġaverage Ġtim est amps Ġof Ġall Ġarticles Ġin Ġit . </s> Learning Ġto ĠRank ĠCand idates </s> In ĠÂ§ ĠSEC REF 19 Ġwe Ġintroduced ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġas Ġthe Ġweight Ġvectors Ġfor Ġthe Ġseveral Ġdocument Ġrepresentation Ġfeatures . ĠWe Ġexperiment Ġwith Ġboth Ġsetting Ġthese Ġweights Ġto Ġjust Ġ1 Ġ( ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġ) Ġand Ġalso Ġlearning Ġthese Ġweights Ġusing Ġsupport Ġvector Ġmachines Ġ( S V Ms ). ĠTo Ġgenerate Ġthe ĠS VM Ġtraining Ġdata , Ġwe Ġsimulate Ġthe Ġexecution Ġof Ġthe Ġalgorithm Ġon Ġa Ġtraining Ġdata Ġpartition Ġ( which Ġwe Ġdo Ġnot Ġget Ġevaluated Ġon ) Ġand Ġin Ġwhich Ġthe Ġgold Ġstandard Ġlabels Ġare Ġgiven . ĠWe Ġrun Ġthe Ġalgorithm Ġusing Ġonly Ġthe Ġfirst Ġsub vector ĠIN LINE FORM 4 Ġ, Ġwhich Ġis Ġthe ĠTF - ID F Ġvector Ġwith Ġthe Ġwords Ġof Ġthe Ġdocument Ġin Ġthe Ġbody Ġand Ġtitle . ĠFor Ġeach Ġincoming Ġdocument , Ġwe Ġcreate Ġa Ġcollection Ġof Ġpositive Ġexamples , Ġfor Ġthe Ġdocument Ġand Ġthe Ġclusters Ġwhich Ġshare Ġat Ġleast Ġone Ġdocument Ġin Ġthe Ġgold Ġlabeling . ĠWe Ġthen Ġgenerate Ġ20 Ġnegative Ġexamples Ġfor Ġthe Ġdocument Ġfrom Ġthe Ġ20 Ġbest - match ing Ġclusters Ġwhich Ġare Ġnot Ġcorrect . ĠTo Ġfind Ġout Ġthe Ġbest - match ing Ġclusters , Ġwe Ġrank Ġthem Ġaccording Ġto Ġtheir Ġsimilarity Ġto Ġthe Ġinput Ġdocument Ġusing Ġonly Ġthe Ġfirst Ġsub vector ĠIN LINE FORM 5 Ġ. </s> Using Ġthis Ġscheme Ġwe Ġgenerate Ġa Ġcollection Ġof Ġranking Ġexamples Ġ( one Ġfor Ġeach Ġdocument Ġin Ġthe Ġdataset , Ġwith Ġthe Ġranking Ġof Ġthe Ġbest Ġcluster Ġmatches ), Ġwhich Ġare Ġthen Ġtrained Ġusing Ġthe ĠSV MR ank Ġalgorithm ĠB IB REF 3 Ġ. ĠWe Ġrun Ġ5 - fold Ġcross - valid ation Ġon Ġthis Ġdata Ġto Ġselect Ġthe Ġbest Ġmodel , Ġand Ġtrain Ġboth Ġa Ġseparate Ġmodel Ġfor Ġeach Ġlanguage Ġaccording Ġto ĠIN LINE FORM 0 Ġand Ġa Ġcross ling ual Ġmodel Ġaccording Ġto ĠIN LINE FORM 1 Ġ. </s> Exper iments </s> Our Ġsystem Ġwas Ġdesigned Ġto Ġcluster Ġdocuments Ġfrom Ġa Ġ( pot entially Ġinfinite ) Ġreal - word Ġdata Ġstream . ĠThe Ġdatasets Ġtypically Ġused Ġin Ġthe Ġliterature Ġ( TD T , ĠReuters ) Ġhave Ġa Ġsmall Ġnumber Ġof Ġclusters Ġ( ĠIN LINE FORM 0 Ġ20 ) Ġwith Ġcoarse Ġtopics Ġ( econom y , Ġsociety , Ġetc .), Ġand Ġtherefore Ġare Ġnot Ġrelevant Ġto Ġthe Ġuse Ġcase Ġof Ġmedia Ġmonitoring Ġwe Ġtreat Ġ- Ġas Ġit Ġrequires Ġmuch Ġmore Ġfine - gr ained Ġstory Ġclusters Ġabout Ġparticular Ġevents . ĠTo Ġevaluate Ġour Ġapproach , Ġwe Ġadapted Ġa Ġdataset Ġconstructed Ġfor Ġthe Ġdifferent Ġpurpose Ġof Ġbinary Ġclassification Ġof Ġjoining Ġcluster Ġpairs . ĠWe Ġprocessed Ġit Ġto Ġbecome Ġa Ġcollection Ġof Ġarticles Ġannot ated Ġwith Ġmon oling ual Ġand Ġcross ling ual Ġcluster Ġlabels . </s> Statistics Ġabout Ġthis Ġdataset Ġare Ġgiven Ġin ĠTable ĠT AB REF 30 Ġ. ĠAs Ġdescribed Ġfurther , Ġwe Ġtune Ġthe Ġhyper - param eter ĠIN LINE FORM 0 Ġon Ġthe Ġdevelopment Ġset . ĠAs Ġfor Ġthe Ġhyper - param eters Ġrelated Ġto Ġthe Ġtimestamp Ġfeatures , Ġwe Ġfixed ĠIN LINE FORM 1 Ġand Ġtuned ĠIN LINE FORM 2 Ġon Ġthe Ġdevelopment Ġset , Ġyielding ĠIN LINE FORM 3 Ġ. ĠTo Ġcompute ĠIDF Ġscores Ġ( which Ġare Ġglobal Ġnumbers Ġcomputed Ġacross Ġa Ġcorpus ), Ġwe Ġused Ġa Ġdifferent Ġand Ġmuch Ġlarger Ġdataset Ġthat Ġwe Ġcollected Ġfrom ĠDeutsche ĠW elle 's Ġnews Ġwebsite Ġ( http :// www . d w . com / ). ĠThe Ġdataset Ġconsists Ġof Ġ77 , 268 , Ġ118 , 045 Ġand Ġ134 , 243 Ġdocuments Ġfor ĠSpanish , ĠEnglish Ġand ĠGerman , Ġrespectively . </s> The Ġconclusions Ġfrom Ġour Ġexperiments Ġare : Ġ( a ) Ġthe Ġweight ing Ġof Ġthe Ġsimilarity Ġmetric Ġfeatures Ġusing ĠS VM Ġsignificantly Ġoutper forms Ġun super vised Ġbas elines Ġsuch Ġas ĠCl u Stream Ġ( Table ĠT AB REF 35 Ġ); Ġ( b ) Ġthe ĠS VM Ġapproach Ġsignificantly Ġhelps Ġto Ġlearn Ġwhen Ġto Ġcreate Ġa Ġnew Ġcluster , Ġcompared Ġto Ġsimple Ġgrid Ġsearch Ġfor Ġthe Ġoptimal ĠIN LINE FORM 0 Ġ( Table ĠT AB REF 39 Ġ); Ġ( c ) Ġseparating Ġthe Ġfeature Ġspace Ġinto Ġone Ġfor Ġmon oling ual Ġclusters Ġin Ġthe Ġform Ġof Ġkeywords Ġand Ġthe Ġother Ġfor Ġcross ling ual Ġclusters Ġbased Ġon Ġcross ling ual Ġembed d ings Ġsignificantly Ġhelps Ġperformance . </s> Mon oling ual ĠResults </s> In Ġour Ġfirst Ġset Ġof Ġexperiments , Ġwe Ġreport Ġresults Ġon Ġmon oling ual Ġclust ering Ġfor Ġeach Ġlanguage Ġseparately . ĠMon oling ual Ġclust ering Ġof Ġa Ġstream Ġof Ġdocuments Ġis Ġan Ġimportant Ġproblem Ġthat Ġhas Ġbeen Ġinspected Ġby Ġothers , Ġsuch Ġas Ġby Ġah med 2011 un ified Ġand Ġby Ġagg ar wal 2006 framework . ĠWe Ġcompare Ġour Ġresults Ġto Ġour Ġown Ġimplementation Ġof Ġthe Ġonline Ġmicro - cl ust ering Ġroutine Ġpresented Ġby Ġagg ar wal 2006 framework , Ġwhich Ġshall Ġbe Ġreferred Ġto Ġas ĠCl u Stream . ĠWe Ġnote Ġthat ĠCl u Stream Ġof Ġagg ar wal 2006 framework Ġhas Ġbeen Ġa Ġwidely Ġused Ġstate - of - the - art Ġsystem Ġin Ġmedia Ġmonitoring Ġcompanies Ġas Ġwell Ġas Ġacademia , Ġand Ġserves Ġas Ġa Ġstrong Ġbaseline Ġto Ġthis Ġday . </s> In Ġour Ġpreliminary Ġexperiments , Ġwe Ġalso Ġevaluated Ġan Ġonline Ġlatent Ġsemantic Ġanalysis Ġmethod , Ġin Ġwhich Ġthe Ġcent ro ids Ġwe Ġkeep Ġfor Ġthe Ġfunction ĠIN LINE FORM 0 Ġ( see ĠÂ§ ĠSEC REF 3 Ġ) Ġare Ġthe Ġaverage Ġof Ġreduced Ġdimensional Ġvectors Ġof Ġthe Ġincoming Ġdocuments Ġas Ġgenerated Ġby Ġan Ġincremental Ġsingular Ġvalue Ġdecom position Ġ( S VD ) Ġof Ġa Ġdocument - term Ġmatrix Ġthat Ġis Ġupdated Ġafter Ġeach Ġincoming Ġdocument . ĠHowever , Ġwe Ġdiscovered Ġthat Ġonline ĠL SA Ġperforms Ġsignificantly Ġworse Ġthan Ġrepresenting Ġthe Ġdocuments Ġthe Ġway Ġis Ġdescribed Ġin ĠÂ§ ĠSEC REF 4 Ġ. ĠFurthermore , Ġit Ġwas Ġalso Ġsignificantly Ġslower Ġthan Ġour Ġalgorithm Ġdue Ġto Ġthe Ġtime Ġit Ġtook Ġto Ġperform Ġsingular Ġvalue Ġdecom position . </s> Table ĠT AB REF 35 Ġgives Ġthe Ġfinal Ġmon oling ual Ġresults Ġon Ġthe Ġthree Ġdatasets . ĠFor ĠEnglish , Ġwe Ġsee Ġthat Ġthe Ġsignificant Ġimprovement Ġwe Ġget Ġusing Ġour Ġalgorithm Ġover Ġthe Ġalgorithm Ġof Ġagg ar wal 2006 framework Ġis Ġdue Ġto Ġan Ġincreased Ġrecall Ġscore . ĠWe Ġalso Ġnote Ġthat Ġthe Ġtrained Ġmodels Ġsurpass Ġthe Ġbaseline Ġfor Ġall Ġlanguages , Ġand Ġthat Ġthe Ġtimestamp Ġfeature Ġ( den oted Ġby ĠTS ), Ġwhile Ġnot Ġrequired Ġto Ġbeat Ġthe Ġbaseline , Ġhas Ġa Ġvery Ġrelevant Ġcontribution Ġin Ġall Ġcases . ĠAlthough Ġthe Ġresults Ġfor Ġboth Ġthe Ġbaseline Ġand Ġour Ġmodels Ġseem Ġto Ġdiffer Ġacross Ġlanguages , Ġone Ġcan Ġverify Ġa Ġconsistent Ġimprovement Ġfrom Ġthe Ġlatter Ġto Ġthe Ġformer , Ġsuggesting Ġthat Ġthe Ġscore Ġdifferences Ġshould Ġbe Ġmostly Ġtied Ġto Ġthe Ġdifferent Ġdifficulty Ġfound Ġacross Ġthe Ġdatasets Ġfor Ġeach Ġlanguage . ĠThe Ġpresented Ġscores Ġshow Ġthat Ġour Ġlearning Ġframework Ġgeneral izes Ġwell Ġto Ġdifferent Ġlanguages Ġand Ġenables Ġhigh Ġquality Ġclust ering Ġresults . </s> To Ġinvestigate Ġthe Ġimpact Ġof Ġthe Ġtimestamp Ġfeatures , Ġwe Ġran Ġan Ġadditional Ġexperiment Ġusing Ġonly Ġthe Ġsame Ġthree Ġtimestamp Ġfeatures Ġas Ġused Ġin Ġthe Ġbest Ġmodel Ġon Ġthe ĠEnglish Ġdataset . ĠThis Ġexperiment Ġyielded Ġscores Ġof ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ, Ġwhich Ġlead Ġus Ġto Ġconclude Ġthat Ġwhile Ġthese Ġfeatures Ġare Ġnot Ġcompetitive Ġwhen Ġused Ġalone Ġ( hen ce Ġtemporal Ġinformation Ġby Ġitself Ġis Ġnot Ġsufficient Ġto Ġpredict Ġthe Ġclusters ), Ġthey Ġcontribute Ġsignificantly Ġto Ġrecall Ġwith Ġthe Ġfinal Ġfeature Ġensemble . </s> We Ġnote Ġthat Ġas Ġdescribed Ġin ĠÂ§ ĠSEC REF 3 Ġ, Ġthe Ġoptimization Ġof Ġthe ĠIN LINE FORM 0 Ġparameter Ġis Ġpart Ġof Ġthe Ġdevelopment Ġprocess . ĠThe Ġparameter ĠIN LINE FORM 1 Ġis Ġa Ġsimilarity Ġthreshold Ġused Ġto Ġdecide Ġwhen Ġan Ġincoming Ġdocument Ġshould Ġmerge Ġto Ġthe Ġbest Ġcluster Ġor Ġcreate Ġa Ġnew Ġone . ĠWe Ġtune ĠIN LINE FORM 2 Ġon Ġthe Ġdevelopment Ġset Ġfor Ġeach Ġlanguage , Ġand Ġthe Ġsensitivity Ġto Ġit Ġis Ġdemonstrated Ġin ĠFigure ĠFIG REF 36 Ġ( this Ġprocess Ġis Ġfurther Ġreferred Ġto Ġas ĠIN LINE FORM 3 Ġ). ĠAlthough Ġapplying Ġgrid - search Ġon Ġthis Ġparameter Ġis Ġthe Ġmost Ġimmediate Ġapproach Ġto Ġthis Ġproblem , Ġwe Ġexperimented Ġwith Ġa Ġdifferent Ġmethod Ġwhich Ġyielded Ġsuperior Ġresults : Ġas Ġdescribed Ġfurther , Ġwe Ġdiscuss Ġhow Ġto Ġdo Ġthis Ġprocess Ġwith Ġan Ġadditional Ġclass ifier Ġ( den oted ĠS VM - mer ge ), Ġwhich Ġcaptures Ġmore Ġinformation Ġabout Ġthe Ġincoming Ġdocuments Ġand Ġthe Ġexisting Ġclusters . </s> Additionally , Ġwe Ġalso Ġexperimented Ġwith Ġcomputing Ġthe Ġmon oling ual Ġclusters Ġwith Ġthe Ġsame Ġembed d ings Ġas Ġused Ġin Ġthe Ġcross ling ual Ġclust ering Ġphase , Ġwhich Ġyielded Ġpoor Ġresults . ĠIn Ġparticular , Ġthis Ġsystem Ġachieved ĠIN LINE FORM 0 Ġscore Ġof ĠIN LINE FORM 1 Ġfor ĠEnglish , Ġwhich Ġis Ġbelow Ġthe Ġbag - of - words Ġbaseline Ġpresented Ġin ĠTable ĠT AB REF 35 Ġ. ĠThis Ġresult Ġsupports Ġthe Ġapproach Ġwe Ġthen Ġfollowed Ġof Ġhaving Ġtwo Ġseparate Ġfeature Ġspaces Ġfor Ġthe Ġmon oling ual Ġand Ġcross ling ual Ġclust ering Ġsystems , Ġwhere Ġthe Ġmon oling ual Ġspace Ġis Ġdiscrete Ġand Ġthe Ġcross ling ual Ġspace Ġis Ġbased Ġon Ġembed d ings . </s> To Ġinvestigate Ġthe Ġimportance Ġof Ġeach Ġfeature , Ġwe Ġnow Ġconsider Ġin ĠTable ĠT AB REF 37 Ġthe Ġaccuracy Ġof Ġthe ĠS VM Ġrank er Ġfor ĠEnglish Ġas Ġdescribed Ġin ĠÂ§ ĠSEC REF 19 Ġ. ĠWe Ġnote Ġthat Ġadding Ġfeatures Ġincreases Ġthe Ġaccuracy Ġof Ġthe ĠS VM Ġrank er , Ġespecially Ġthe Ġtimestamp Ġfeatures . ĠHowever , Ġthe Ġtimestamp Ġfeature Ġactually Ġinter fe res Ġwith Ġour Ġoptimization Ġof ĠIN LINE FORM 0 Ġto Ġidentify Ġwhen Ġnew Ġclusters Ġare Ġneeded , Ġalthough Ġthey Ġimprove Ġthe ĠS VM Ġre ranking Ġaccuracy . ĠWe Ġspeculate Ġthis Ġis Ġtrue Ġbecause Ġhigh Ġaccuracy Ġin Ġthe Ġre ranking Ġproblem Ġdoes Ġnot Ġnecessarily Ġhelp Ġwith Ġidentifying Ġwhen Ġnew Ġclusters Ġneed Ġto Ġbe Ġopened . </s> To Ġinvestigate Ġthis Ġissue , Ġwe Ġexperimented Ġwith Ġa Ġdifferent Ġtechnique Ġto Ġlearn Ġwhen Ġto Ġcreate Ġa Ġnew Ġcluster . ĠTo Ġthis Ġend , Ġwe Ġtrained Ġanother ĠS VM Ġclass ifier Ġjust Ġto Ġlearn Ġthis Ġdecision , Ġthis Ġtime Ġa Ġbinary Ġclass ifier Ġusing ĠLI BL INE AR ĠB IB REF 4 Ġ, Ġby Ġpassing Ġthe Ġmax Ġof Ġthe Ġsimilarity Ġof Ġeach Ġfeature Ġbetween Ġthe Ġincoming Ġdocument Ġand Ġthe Ġcurrent Ġclust ering Ġpool Ġas Ġthe Ġinput Ġfeature Ġvector . ĠThis Ġway , Ġthe Ġclass ifier Ġlearns Ġwhen Ġthe Ġcurrent Ġclusters , Ġas Ġa Ġwhole , Ġare Ġof Ġa Ġdifferent Ġnews Ġstory Ġthan Ġthe Ġincoming Ġdocument . ĠAs Ġpresented Ġin ĠTable ĠT AB REF 39 Ġ, Ġthis Ġmethod , Ġwhich Ġwe Ġrefer Ġto Ġas ĠS VM - mer ge , Ġsolved Ġthe Ġissue Ġof Ġsearching Ġfor Ġthe Ġoptimal ĠIN LINE FORM 0 Ġparameter Ġfor Ġthe ĠS VM - rank Ġmodel Ġwith Ġtim est amps , Ġby Ġgreatly Ġimproving Ġthe ĠF ĠIN LINE FORM 1 Ġscore Ġin Ġrespect Ġto Ġthe Ġoriginal Ġgrid - search Ġapproach Ġ( ĠIN LINE FORM 2 Ġ). </s> Cross ling ual ĠResults </s> As Ġmentioned Ġin ĠÂ§ ĠSEC REF 3 Ġ, Ġcross ling ual Ġembed d ings Ġare Ġused Ġfor Ġcross ling ual Ġclust ering . ĠWe Ġexperimented Ġwith Ġthe Ġcross ling ual Ġembed d ings Ġof Ġgard ner 2015 translation Ġand Ġam mar 2016 mass ively . ĠIn Ġour Ġpreliminary Ġexperiments Ġwe Ġfound Ġthat Ġthe Ġformer Ġworked Ġbetter Ġfor Ġour Ġuse - case Ġthan Ġthe Ġlatter . </s> We Ġtest Ġtwo Ġdifferent Ġscenarios Ġfor Ġoptimizing Ġthe Ġsimilarity Ġthreshold ĠIN LINE FORM 0 Ġfor Ġthe Ġcross ling ual Ġcase . ĠTable ĠT AB REF 41 Ġshows Ġthe Ġresults Ġfor Ġthese Ġexperiments . ĠFirst , Ġwe Ġconsider Ġthe Ġsimpler Ġcase Ġof Ġadjusting Ġa Ġglobal ĠIN LINE FORM 1 Ġparameter Ġfor Ġthe Ġcross ling ual Ġdistances , Ġas Ġalso Ġdescribed Ġfor Ġthe Ġmon oling ual Ġcase . ĠAs Ġshown , Ġthis Ġmethod Ġworks Ġpoorly , Ġsince Ġthe ĠIN LINE FORM 2 Ġgrid - search Ġcould Ġnot Ġfind Ġa Ġreasonable ĠIN LINE FORM 3 Ġwhich Ġworked Ġwell Ġfor Ġevery Ġpossible Ġlanguage Ġpair . </s> Sub sequently , Ġwe Ġalso Ġconsider Ġthe Ġcase Ġof Ġusing ĠEnglish Ġas Ġa Ġpivot Ġlanguage Ġ( see ĠÂ§ ĠSEC REF 3 Ġ), Ġwhere Ġdistances Ġfor Ġevery Ġother Ġlanguage Ġare Ġonly Ġcompared Ġto ĠEnglish , Ġand Ġcross ling ual Ġclust ering Ġdecisions Ġare Ġmade Ġonly Ġbased Ġon Ġthis Ġdistance . ĠThis Ġyielded Ġour Ġbest Ġcross ling ual Ġscore Ġof ĠIN LINE FORM 0 Ġ, Ġconfirming Ġthat Ġcross ling ual Ġsimilarity Ġis Ġof Ġhigher Ġquality Ġbetween Ġeach Ġlanguage Ġand ĠEnglish , Ġfor Ġthe Ġembed d ings Ġwe Ġused . ĠThis Ġscore Ġrepresents Ġonly Ġa Ġsmall Ġdegradation Ġin Ġrespect Ġto Ġthe Ġmon oling ual Ġresults , Ġsince Ġclust ering Ġacross Ġdifferent Ġlanguages Ġis Ġa Ġharder Ġproblem . </s> Related ĠWork </s> Early Ġresearch Ġefforts , Ġsuch Ġas Ġthe ĠTD T Ġprogram ĠB IB REF 5 Ġ, Ġhave Ġstudied Ġnews Ġclust ering Ġfor Ġsome Ġtime . ĠThe Ġproblem Ġof Ġonline Ġmon oling ual Ġclust ering Ġalgorithms Ġ( for ĠEnglish ) Ġhas Ġalso Ġreceived Ġa Ġfair Ġamount Ġof Ġattention Ġin Ġthe Ġliterature . ĠOne Ġof Ġthe Ġearlier Ġpapers Ġby Ġagg ar wal 2006 framework Ġintroduced Ġa Ġtwo - step Ġclust ering Ġsystem Ġwith Ġboth Ġoffline Ġand Ġonline Ġcomponents , Ġwhere Ġthe Ġonline Ġmodel Ġis Ġbased Ġon Ġa Ġstreaming Ġimplementation Ġof ĠIN LINE FORM 0 Ġ- me ans Ġand Ġa Ġbag - of - words Ġdocument Ġrepresentation . ĠOther Ġauthors Ġhave Ġexperimented Ġwith Ġdistributed Ġrepresentations , Ġsuch Ġas Ġah med 2011 un ified , Ġwho Ġcluster Ġnews Ġinto Ġstorylines Ġusing ĠMark ov Ġchain ĠMonte ĠCarlo Ġmethods , Ġre h ure kl rec Ġwho Ġused Ġincremental ĠSing ular ĠValue ĠDec om position Ġ( S VD ) Ġto Ġfind Ġrelevant Ġtopics Ġfrom Ġstreaming Ġdata , Ġand Ġsat o 2017 dist ributed Ġwho Ġused Ġthe Ġparagraph Ġvector Ġmodel ĠB IB REF 6 Ġin Ġan Ġoffline Ġclust ering Ġsetting . </s> More Ġrecently , Ġcross ling ual Ġlinking Ġof Ġclusters Ġhas Ġbeen Ġdiscussed Ġby Ġr up nik 2016 news Ġin Ġthe Ġcontext Ġof Ġlinking Ġexisting Ġclusters Ġfrom Ġthe ĠEvent ĠRegistry ĠB IB REF 7 Ġin Ġa Ġbatch Ġfashion , Ġand Ġby Ġste in berger 2016 medi ag ist Ġwho Ġalso Ġpresent Ġa Ġbatch Ġclust ering Ġlinking Ġsystem . ĠHowever , Ġthese Ġare Ġnot ĠâĢ ľ t ruly âĢ Ŀ Ġonline Ġcross ling ual Ġclust ering Ġsystems Ġsince Ġthey Ġonly Ġdecide Ġon Ġthe Ġlinking Ġof Ġalready - built Ġmon oling ual Ġclusters . ĠIn Ġparticular , Ġr up nik 2016 news Ġcompute Ġdistances Ġof Ġdocument Ġpairs Ġacross Ġclusters Ġusing Ġnearest Ġneighbors , Ġwhich Ġmight Ġnot Ġscale Ġwell Ġin Ġan Ġonline Ġsetting . ĠAs Ġdetailed Ġbefore , Ġwe Ġadapted Ġthe Ġcluster - l inking Ġdataset Ġfrom Ġr up nik 2016 news Ġto Ġevaluate Ġour Ġonline Ġcross ling ual Ġclust ering Ġapproach . ĠPrel iminary Ġwork Ġmakes Ġuse Ġof Ġdeep Ġlearning Ġtechniques ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġto Ġcluster Ġdocuments Ġwhile Ġlearning Ġtheir Ġrepresentations , Ġbut Ġnot Ġin Ġan Ġonline Ġor Ġmult ilingual Ġfashion , Ġand Ġwith Ġa Ġvery Ġsmall Ġnumber Ġof Ġcluster Ġlabels Ġ( 4 , Ġin Ġthe Ġcase Ġof Ġthe Ġtext Ġbenchmark ). </s> In Ġour Ġwork , Ġwe Ġstudied Ġthe Ġproblem Ġof Ġmon oling ual Ġand Ġcross ling ual Ġclust ering , Ġhaving Ġexperimented Ġseveral Ġdirections Ġand Ġmethods Ġand Ġthe Ġimpact Ġthey Ġhave Ġon Ġthe Ġfinal Ġclust ering Ġquality . ĠWe Ġdescribed Ġthe Ġfirst Ġsystem Ġwhich Ġaggreg ates Ġnews Ġarticles Ġinto Ġfine - gr ained Ġstory Ġclusters Ġacross Ġdifferent Ġlanguages Ġin Ġa Ġcompletely Ġonline Ġand Ġscalable Ġfashion Ġfrom Ġa Ġcontinuous Ġstream . </s> Conclusion </s> We Ġdescribed Ġa Ġmethod Ġfor Ġmon oling ual Ġand Ġcross ling ual Ġclust ering Ġof Ġan Ġincoming Ġstream Ġof Ġdocuments . ĠThe Ġmethod Ġworks Ġby Ġmaintaining Ġcent ro ids Ġfor Ġthe Ġmon oling ual Ġand Ġcross ling ual Ġclusters , Ġwhere Ġa Ġmon oling ual Ġcluster Ġgroups Ġa Ġset Ġof Ġdocuments Ġand Ġa Ġcross ling ual Ġcluster Ġgroups Ġa Ġset Ġof Ġmon oling ual Ġclusters . ĠWe Ġpresented Ġan Ġonline Ġcross ling ual Ġclust ering Ġmethod Ġwhich Ġauto - correct s Ġpast Ġdecisions Ġin Ġan Ġefficient Ġway . ĠWe Ġshowed Ġthat Ġour Ġmethod Ġgives Ġstate - of - the - art Ġresults Ġon Ġa Ġmult ilingual Ġnews Ġarticle Ġdataset Ġfor ĠEnglish , ĠSpanish Ġand ĠGerman . ĠFinally , Ġwe Ġdiscussed Ġhow Ġto Ġleverage Ġdifferent ĠS VM Ġtraining Ġprocedures Ġfor Ġranking Ġand Ġclassification Ġto Ġimprove Ġmon oling ual Ġand Ġcross ling ual Ġclust ering Ġdecisions . ĠOur Ġsystem Ġis Ġintegrated Ġin Ġa Ġlarger Ġmedia Ġmonitoring Ġproject ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġand Ġsolving Ġthe Ġuse - cases Ġof Ġmonitors Ġand Ġjournalists , Ġhaving Ġbeen Ġvalidated Ġwith Ġqualitative Ġuser Ġtesting . </s> Acknowled gments </s> We Ġwould Ġlike Ġto Ġthank ĠEs ma ĠBalk Ä± r , ĠNik os ĠPap as arant opoulos , ĠAf onso ĠMend es , ĠSh ashi ĠNar ayan Ġand Ġthe Ġanonymous Ġreviewers Ġfor Ġtheir Ġfeedback . ĠThis Ġproject Ġwas Ġsupported Ġby Ġthe ĠEuropean ĠH 2020 Ġproject ĠSUM MA , Ġgrant Ġagreement Ġ6 88 139 Ġ( see Ġhttp :// www . sum ma - project . eu ) Ġand Ġby Ġa Ġgrant Ġfrom ĠBloomberg .