By assigning a political bias label to each news article and training only on left-biased or right-biased outlets of both disinformation and mainstream domains
ABSTRACTIVE
we also account for political biases inherent to different news sources, referring to the procedure proposed in BIBREF2 to label different outlets. Overall we show that we are able to classify credible vs non-credible diffusion networks (and consequently news articles) with high accuracy (AUROC up to 94%), even when accounting for the political bias of sources (and training only on left-biased or right-biased articles). We observe that the layer of mentions alone conveys useful information for the classification, denoting a different usage of this functionality when sharing news belonging to the two news domains. We also show that most discriminative features, which are relative to the breadth and depth of largest cascades in different layers, are the same across the two countries.
EXTRACTIVE
As it is reported that conservatives and liberals exhibit different behaviors on online social platforms BIBREF19BIBREF20BIBREF21, we further assigned a political bias label to different US outlets (and therefore news articles) following the procedure described in BIBREF2. In order to assess the robustness of our method, we performed classification experiments by training only on left-biased (or right-biased) outlets of both disinformation and mainstream domains and testing on the entire set of sources, as well as excluding particular sources that outweigh the others in terms of samples to avoid over-fitting.
We perform classification experiments with an off-the-shelf Logistic Regression model on two different datasets of mainstream and disinformation news shared on Twitter respectively in the United States and in Italy during 2019. In the former case we also account for political biases inherent to different news sources, referring to the procedure proposed in BIBREF2 to label different outlets. Overall we show that we are able to classify credible vs non-credible diffusion networks (and consequently news articles) with high accuracy (AUROC up to 94%), even when accounting for the political bias of sources (and training only on left-biased or right-biased articles). We observe that the layer of mentions alone conveys useful information for the classification, denoting a different usage of this functionality when sharing news belonging to the two news domains. We also show that most discriminative features, which are relative to the breadth and depth of largest cascades in different layers, are the same across the two countries.
2002.12612
Introduction Ġand Ġrelated Ġwork </s> In Ġrecent Ġyears Ġthere Ġhas Ġbeen Ġincreasing Ġinterest Ġon Ġthe Ġissue Ġof Ġdisinformation Ġspreading Ġon Ġonline Ġsocial Ġmedia . ĠGlobal Ġconcern Ġover Ġfalse Ġ( or Ġ" fake ") Ġnews Ġas Ġa Ġthreat Ġto Ġmodern Ġdemocracies Ġhas Ġbeen Ġfrequently Ġraised âĢĵ ever Ġsince Ġ2016 ĠUS ĠPresidential Ġelections âĢĵ in Ġcorrespondence Ġof Ġevents Ġof Ġpolitical Ġrelevance , Ġwhere Ġthe Ġproliferation Ġof Ġmanipulated Ġand Ġlow - c red ibility Ġcontent Ġattempts Ġto Ġdrive Ġand Ġinfluence Ġpeople Ġopinions ĠB IB REF 0 BI B REF 1 BI B REF 2 BI B REF 3 . </s> Researchers Ġhave Ġhighlighted Ġseveral Ġdrivers Ġfor Ġthe Ġdiffusion Ġof Ġsuch Ġmalicious Ġphenomenon , Ġwhich Ġinclude Ġhuman Ġfactors Ġ( conf irmation Ġbias ĠB IB REF 4 , Ġnaive Ġrealism ĠB IB REF 5 ), Ġalgorith mic Ġbiases Ġ( filter Ġbubble Ġeffect ĠB IB REF 0 ), Ġthe Ġpresence Ġof Ġdeceptive Ġagents Ġon Ġsocial Ġplatforms Ġ( bots Ġand Ġtrolls ĠB IB REF 6 ) Ġand , Ġlast ly , Ġthe Ġformation Ġof Ġecho Ġchambers ĠB IB REF 7 Ġwhere Ġpeople Ġpolar ize Ġtheir Ġopinions Ġas Ġthey Ġare Ġinsulated Ġfrom Ġcontrary Ġperspectives . </s> The Ġproblem Ġof Ġautomatically Ġdetecting Ġonline Ġdisinformation Ġnews Ġhas Ġbeen Ġtypically Ġformulated Ġas Ġa Ġbinary Ġclassification Ġtask Ġ( i . e . Ġcredible Ġvs Ġnon - c redible Ġarticles ), Ġand Ġtackled Ġwith Ġa Ġvariety Ġof Ġdifferent Ġtechniques , Ġbased Ġon Ġtraditional Ġmachine Ġlearning Ġand / or Ġdeep Ġlearning , Ġwhich Ġmainly Ġdiffer Ġin Ġthe Ġdataset Ġand Ġthe Ġfeatures Ġthey Ġemploy Ġto Ġperform Ġthe Ġclassification . ĠWe Ġmay Ġdistinguish Ġthree Ġapproaches : Ġthose Ġbuilt Ġon Ġcontent - based Ġfeatures , Ġthose Ġbased Ġon Ġfeatures Ġextracted Ġfrom Ġthe Ġsocial Ġcontext , Ġand Ġthose Ġwhich Ġcombine Ġboth Ġaspects . ĠA Ġfew Ġmain Ġchallenges Ġhinder Ġthe Ġtask , Ġnamely Ġthe Ġimpossibility Ġto Ġmanually Ġverify Ġall Ġnews Ġitems , Ġthe Ġlack Ġof Ġgold - standard Ġdatasets Ġand Ġthe Ġadvers arial Ġsetting Ġin Ġwhich Ġmalicious Ġcontent Ġis Ġcreated ĠB IB REF 3 BI B REF 6 . </s> In Ġthis Ġwork Ġwe Ġfollow Ġthe Ġdirection Ġpointed Ġout Ġin Ġa Ġfew Ġrecent Ġcontributions Ġon Ġthe Ġdiffusion Ġof Ġdisinformation Ġcompared Ġto Ġtraditional Ġand Ġobjective Ġinformation . ĠThese Ġhave Ġshown Ġthat Ġfalse Ġnews Ġspread Ġfaster Ġand Ġdeeper Ġthan Ġtrue Ġnews ĠB IB REF 8 , Ġand Ġthat Ġsocial Ġbots Ġand Ġecho Ġchambers Ġplay Ġan Ġimportant Ġrole Ġin Ġthe Ġdiffusion Ġof Ġmalicious Ġcontent ĠB IB REF 6 , ĠB IB REF 7 . ĠTherefore Ġwe Ġfocus Ġon Ġthe Ġanalysis Ġof Ġspreading Ġpatterns Ġwhich Ġnaturally Ġarise Ġon Ġsocial Ġplatforms Ġas Ġa Ġconsequence Ġof Ġmultiple Ġinteractions Ġbetween Ġusers , Ġdue Ġto Ġthe Ġincreasing Ġtrend Ġin Ġonline Ġsharing Ġof Ġnews ĠB IB REF 0 . </s> A Ġdeep Ġlearning Ġframework Ġfor Ġdetection Ġof Ġfake Ġnews Ġcasc ades Ġis Ġprovided Ġin ĠB IB REF 9 , Ġwhere Ġthe Ġauthors Ġrefer Ġto ĠB IB REF 8 Ġin Ġorder Ġto Ġcollect ĠTwitter Ġcasc ades Ġpertaining Ġto Ġverified Ġfalse Ġand Ġtrue Ġrumors . ĠThey Ġemploy Ġgeometric Ġdeep Ġlearning , Ġa Ġnovel Ġparadigm Ġfor Ġgraph - based Ġstructures , Ġto Ġclassify Ġcasc ades Ġbased Ġon Ġfour Ġcategories Ġof Ġfeatures , Ġsuch Ġas Ġuser Ġprofile , Ġuser Ġactivity , Ġnetwork Ġand Ġspreading , Ġand Ġcontent . ĠThey Ġalso Ġobserve Ġthat Ġa Ġfew Ġhours Ġof Ġpropagation Ġare Ġsufficient Ġto Ġdistinguish Ġfalse Ġnews Ġfrom Ġtrue Ġnews Ġwith Ġhigh Ġaccuracy . ĠDiff usion Ġcasc ades Ġon ĠWe ibo Ġand ĠTwitter Ġare Ġanalyzed Ġin ĠB IB REF 10 , Ġwhere Ġauthors Ġfocus Ġon Ġhighlighting Ġdifferent Ġtop ological Ġproperties , Ġsuch Ġas Ġthe Ġnumber Ġof Ġhops Ġfrom Ġthe Ġsource Ġor Ġthe Ġheterogeneity Ġof Ġthe Ġnetwork , Ġto Ġshow Ġthat Ġfake Ġnews Ġshape Ġdiffusion Ġnetworks Ġwhich Ġare Ġhighly Ġdifferent Ġfrom Ġcredible Ġnews , Ġeven Ġat Ġearly Ġstages Ġof Ġpropagation . </s> In Ġthis Ġwork , Ġwe Ġconsider Ġthe Ġresults Ġof ĠB IB REF 11 Ġas Ġour Ġbaseline . ĠThe Ġauthors Ġuse Ġoff - the - she lf Ġmachine Ġlearning Ġclass ifiers Ġto Ġaccurately Ġclassify Ġnews Ġarticles Ġleveraging ĠTwitter Ġdiffusion Ġnetworks . ĠTo Ġthis Ġaim , Ġthey Ġconsider Ġa Ġset Ġof Ġbasic Ġfeatures Ġwhich Ġcan Ġbe Ġqual itatively Ġinterpreted Ġw . r . t Ġto Ġthe Ġsocial Ġbehavior Ġof Ġusers Ġsharing Ġcredible Ġvs Ġnon - c redible Ġinformation . ĠTheir Ġmethodology Ġis Ġoverall Ġin Ġaccordance Ġwith ĠB IB REF 12 , Ġwhere Ġauthors Ġsuccessfully Ġdetect ĠTwitter Ġast rot ur fing Ġcontent , Ġi . e . Ġpolitical Ġcampaigns Ġdisguised Ġas Ġspontaneous Ġgrassroots , Ġwith Ġa Ġmachine Ġlearning Ġframework Ġbased Ġon Ġnetwork Ġfeatures . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġclassification Ġframework Ġbased Ġon Ġa Ġmulti - layer Ġformulation Ġof ĠTwitter Ġdiffusion Ġnetworks . ĠFor Ġeach Ġarticle Ġwe Ġdis ent angle Ġdifferent Ġsocial Ġinteractions Ġon ĠTwitter , Ġnamely Ġtweets , Ġret we ets , Ġmentions , Ġreplies Ġand Ġquotes , Ġto Ġaccordingly Ġbuild Ġa Ġdiffusion Ġnetwork Ġcomposed Ġof Ġmultiple Ġlayers Ġ( on Ġfor Ġeach Ġtype Ġof Ġinteraction ), Ġand Ġwe Ġcompute Ġstructural Ġfeatures Ġseparately Ġfor Ġeach Ġlayer . ĠWe Ġpick Ġa Ġset Ġof Ġglobal Ġnetwork Ġproperties Ġfrom Ġthe Ġnetwork Ġscience Ġtool box Ġwhich Ġcan Ġbe Ġqual itatively Ġexplained Ġin Ġterms Ġof Ġsocial Ġdimensions Ġand Ġallow Ġus Ġto Ġencode Ġdifferent Ġnetworks Ġwith Ġa Ġtuple Ġof Ġfeatures . ĠThese Ġinclude Ġtraditional Ġindicators , Ġe . g . Ġnetwork Ġdensity , Ġnumber Ġof Ġstrong / weak Ġconnected Ġcomponents Ġand Ġdiameter , Ġand Ġmore Ġelaborated Ġones Ġsuch Ġas Ġmain ĠK - core Ġnumber ĠB IB REF 13 Ġand Ġstructural Ġvir ality ĠB IB REF 14 . ĠOur Ġmain Ġresearch Ġquestion Ġis Ġwhether Ġthe Ġuse Ġof Ġa Ġmulti - layer , Ġdis ent angled Ġnetwork Ġyields Ġa Ġsignificant Ġadvance Ġin Ġterms Ġof Ġclassification Ġaccuracy Ġover Ġa Ġconventional Ġsingle - layer Ġdiffusion Ġnetwork . ĠAdditionally , Ġwe Ġare Ġinterested Ġin Ġunderstanding Ġwhich Ġof Ġthe Ġabove Ġfeatures , Ġand Ġin Ġwhich Ġlayer , Ġare Ġmost Ġeffective Ġin Ġthe Ġclassification Ġtask . </s> We Ġperform Ġclassification Ġexperiments Ġwith Ġan Ġoff - the - she lf ĠLog istic ĠReg ression Ġmodel Ġon Ġtwo Ġdifferent Ġdatasets Ġof Ġmainstream Ġand Ġdisinformation Ġnews Ġshared Ġon ĠTwitter Ġrespectively Ġin Ġthe ĠUnited ĠStates Ġand Ġin ĠItaly Ġduring Ġ2019 . ĠIn Ġthe Ġformer Ġcase Ġwe Ġalso Ġaccount Ġfor Ġpolitical Ġbiases Ġinherent Ġto Ġdifferent Ġnews Ġsources , Ġreferring Ġto Ġthe Ġprocedure Ġproposed Ġin ĠB IB REF 2 Ġto Ġlabel Ġdifferent Ġoutlets . ĠOverall Ġwe Ġshow Ġthat Ġwe Ġare Ġable Ġto Ġclassify Ġcredible Ġvs Ġnon - c redible Ġdiffusion Ġnetworks Ġ( and Ġconsequently Ġnews Ġarticles ) Ġwith Ġhigh Ġaccuracy Ġ( A UR OC Ġup Ġto Ġ94 %), Ġeven Ġwhen Ġaccounting Ġfor Ġthe Ġpolitical Ġbias Ġof Ġsources Ġ( and Ġtraining Ġonly Ġon Ġleft - biased Ġor Ġright - biased Ġarticles ). ĠWe Ġobserve Ġthat Ġthe Ġlayer Ġof Ġmentions Ġalone Ġconve ys Ġuseful Ġinformation Ġfor Ġthe Ġclassification , Ġden oting Ġa Ġdifferent Ġusage Ġof Ġthis Ġfunctionality Ġwhen Ġsharing Ġnews Ġbelonging Ġto Ġthe Ġtwo Ġnews Ġdomains . ĠWe Ġalso Ġshow Ġthat Ġmost Ġdiscrim in ative Ġfeatures , Ġwhich Ġare Ġrelative Ġto Ġthe Ġbreadth Ġand Ġdepth Ġof Ġlargest Ġcasc ades Ġin Ġdifferent Ġlayers , Ġare Ġthe Ġsame Ġacross Ġthe Ġtwo Ġcountries . </s> The Ġoutline Ġof Ġthis Ġpaper Ġis Ġthe Ġfollowing : Ġwe Ġfirst Ġformulate Ġthe Ġproblem Ġand Ġdescribe Ġdata Ġcollection , Ġnetwork Ġrepresentation Ġand Ġstructural Ġproperties Ġemployed Ġfor Ġthe Ġclassification ; Ġthen Ġwe Ġprovide Ġexperimental Ġresults âĢĵ class ification Ġperformances , Ġlayer Ġand Ġfeature Ġimportance Ġanalyses Ġand Ġa Ġtemporal Ġclassification Ġevaluation âĢĵ and Ġfinally Ġwe Ġdraw Ġconclusions Ġand Ġfuture Ġdirections . </s> Method ology Ġ: :: ĠDis information Ġand Ġmainstream Ġnews </s> In Ġthis Ġwork Ġwe Ġformulate Ġour Ġclassification Ġproblem Ġas Ġfollows : Ġgiven Ġtwo Ġclasses Ġof Ġnews Ġarticles , Ġrespectively Ġ$ D $ Ġ( dis information ) Ġand Ġ$ M $ Ġ( main stream ), Ġa Ġset Ġof Ġnews Ġarticles Ġ$ A _ i $ Ġand Ġassociated Ġclass Ġlabels Ġ$ C _ i Ġ\ in Ġ\ l brace ĠD , M \ r brace Ġ$ , Ġand Ġa Ġset Ġof Ġtweets Ġ$\ Pi Ġ_ i = \ l brace ĠT _ i ^ 1 , ĠT _ i ^ 2 , Ġ... \ r brace Ġ$ Ġeach Ġof Ġwhich Ġcontains Ġan ĠUniform ĠResource ĠLoc ator Ġ( URL ) Ġpointing Ġexplicitly Ġto Ġarticle Ġ$ A _ i $, Ġpredict Ġthe Ġclass Ġ$ C _ i $ Ġof Ġeach Ġarticle Ġ$ A _ i $. ĠThere Ġis Ġhuge Ġdebate Ġand Ġcontroversy Ġon Ġa Ġproper Ġtax onomy Ġof Ġmalicious Ġand Ġdeceptive Ġinformation ĠB IB REF 1 BI B REF 2 BI B REF 15 BI B REF 16 BI B REF 17 BI B REF 3 BI B REF 11 . ĠIn Ġthis Ġwork Ġwe Ġprefer Ġthe Ġterm Ġdisinformation Ġto Ġthe Ġmore Ġspecific Ġfake Ġnews Ġto Ġrefer Ġto Ġa Ġvariety Ġof Ġmisleading Ġand Ġharmful Ġinformation . ĠTherefore , Ġwe Ġfollow Ġa Ġsource - based Ġapproach , Ġa Ġconsolidated Ġstrategy Ġalso Ġadopted Ġby ĠB IB REF 6 BI B REF 16 BI B REF 2 BI B REF 1 , Ġin Ġorder Ġto Ġobtain Ġrelevant Ġdata Ġfor Ġour Ġanalysis . ĠWe Ġcollected : </s> Dis information Ġarticles , Ġpublished Ġby Ġwebsites Ġwhich Ġare Ġwell - known Ġfor Ġproducing Ġlow - c red ibility Ġcontent , Ġfalse Ġand Ġmisleading Ġnews Ġreports Ġas Ġwell Ġas Ġextreme Ġpropaganda Ġand Ġhoax es Ġand Ġflagged Ġas Ġsuch Ġby Ġreputable Ġjournalists Ġand Ġfact - check ers ; </s> Main stream Ġnews , Ġreferring Ġto Ġtraditional Ġnews Ġoutlets Ġwhich Ġdeliver Ġfactual Ġand Ġcredible Ġinformation . </s> We Ġbelieve Ġthat Ġthis Ġis Ġcurrently Ġthe Ġmost Ġreliable Ġclassification Ġapproach , Ġbut Ġit Ġentails Ġobvious Ġlimitations , Ġas Ġdisinformation Ġoutlets Ġmay Ġalso Ġpublish Ġtrue Ġstories Ġand Ġlikewise Ġmisinformation Ġis Ġsometimes Ġreported Ġon Ġmainstream Ġmedia . ĠAlso , Ġgiven Ġthe Ġchoice Ġof Ġnews Ġsources , Ġwe Ġcannot Ġtest Ġwhether Ġour Ġmethodology Ġis Ġable Ġto Ġclassify Ġdisinformation Ġvs Ġfactual Ġbut Ġnot Ġmainstream Ġnews Ġwhich Ġare Ġpublished Ġon Ġniche , Ġnon - dis information Ġoutlets . </s> Method ology Ġ: :: ĠUS Ġdataset </s> We Ġcollected Ġtweets Ġassociated Ġto Ġa Ġdozen ĠUS Ġmainstream Ġnews Ġwebsites , Ġi . e . Ġmost Ġtrusted Ġsources Ġdescribed Ġin ĠB IB REF 18 , Ġwith Ġthe ĠStreaming ĠAPI , Ġand Ġwe Ġreferred Ġto ĠHo axy ĠAPI ĠB IB REF 16 Ġfor Ġwhat Ġconcerns Ġtweets Ġcontaining Ġlinks Ġto Ġ100 + ĠUS Ġdisinformation Ġoutlets . ĠWe Ġfiltered Ġout Ġarticles Ġassociated Ġto Ġless Ġthan Ġ50 Ġtweets . ĠThe Ġresulting Ġdataset Ġcontains Ġoverall Ġ$\ sim Ġ$ 1 . 7 Ġmillion Ġtweets Ġfor Ġmainstream Ġnews , Ġcollected Ġin Ġa Ġperiod Ġof Ġthree Ġweeks Ġ( February Ġ25 th , Ġ2019 - March Ġ18 th , Ġ2019 ), Ġwhich Ġare Ġassociated Ġto Ġ6 , 978 Ġnews Ġarticles , Ġand Ġ$\ sim Ġ$ 1 . 6 Ġmillion Ġtweets Ġfor Ġdisinformation , Ġcollected Ġin Ġa Ġperiod Ġof Ġthree Ġmonths Ġ( January Ġ1 st , Ġ2019 - March Ġ18 th , Ġ2019 ) Ġfor Ġsake Ġof Ġbalance Ġof Ġthe Ġtwo Ġclasses , Ġwhich Ġhold Ġ5 , 775 Ġdistinct Ġarticles . ĠDiff usion Ġcens oring Ġeffects ĠB IB REF 14 Ġwere Ġcorrectly Ġtaken Ġinto Ġaccount Ġin Ġboth Ġcollection Ġprocedures . ĠWe Ġprovide Ġin ĠFigure ĠFIG REF 4 Ġthe Ġdistribution Ġof Ġarticles Ġby Ġsource Ġand Ġpolitical Ġbias Ġfor Ġboth Ġnews Ġdomains . </s> As Ġit Ġis Ġreported Ġthat Ġconservatives Ġand Ġliberals Ġexhibit Ġdifferent Ġbehaviors Ġon Ġonline Ġsocial Ġplatforms ĠB IB REF 19 BI B REF 20 BI B REF 21 , Ġwe Ġfurther Ġassigned Ġa Ġpolitical Ġbias Ġlabel Ġto Ġdifferent ĠUS Ġoutlets Ġ( and Ġtherefore Ġnews Ġarticles ) Ġfollowing Ġthe Ġprocedure Ġdescribed Ġin ĠB IB REF 2 . ĠIn Ġorder Ġto Ġassess Ġthe Ġrobust ness Ġof Ġour Ġmethod , Ġwe Ġperformed Ġclassification Ġexperiments Ġby Ġtraining Ġonly Ġon Ġleft - biased Ġ( or Ġright - biased ) Ġoutlets Ġof Ġboth Ġdisinformation Ġand Ġmainstream Ġdomains Ġand Ġtesting Ġon Ġthe Ġentire Ġset Ġof Ġsources , Ġas Ġwell Ġas Ġexcluding Ġparticular Ġsources Ġthat Ġoutweigh Ġthe Ġothers Ġin Ġterms Ġof Ġsamples Ġto Ġavoid Ġover - fitting . </s> Method ology Ġ: :: ĠItalian Ġdataset </s> For Ġwhat Ġconcerns Ġthe ĠItalian Ġscenario Ġwe Ġfirst Ġcollected Ġtweets Ġwith Ġthe ĠStreaming ĠAPI Ġin Ġa Ġ3 - week Ġperiod Ġ( April Ġ19 th , Ġ2019 - May Ġ5 th , Ġ2019 ), Ġfiltering Ġthose Ġcontaining ĠURLs Ġpointing Ġto ĠItalian Ġofficial Ġnewspapers Ġwebsites Ġas Ġdescribed Ġin ĠB IB REF 22 ; Ġthese Ġcorrespond Ġto Ġthe Ġlist Ġprovided Ġby Ġthe Ġassociation Ġfor Ġthe Ġverification Ġof Ġnewspaper Ġcirculation Ġin ĠItaly Ġ( Acc ert ament i ĠDiff usion e ĠSt ampa ). ĠWe Ġinstead Ġreferred Ġto Ġthe Ġdataset Ġprovided Ġby ĠB IB REF 23 Ġto Ġobtain Ġa Ġset Ġof Ġtweets , Ġcollected Ġcontinuously Ġsince ĠJanuary Ġ2019 Ġusing Ġthe Ġsame ĠTwitter Ġendpoint , Ġwhich Ġcontain ĠURLs Ġto Ġ60 + ĠItalian Ġdisinformation Ġwebsites . ĠIn Ġorder Ġto Ġget Ġbalanced Ġclasses Ġ( April Ġ5 th , Ġ2019 - May Ġ5 th , Ġ2019 ), Ġwe Ġretained Ġdata Ġcollected Ġin Ġa Ġlonger Ġperiod Ġw . r . t Ġto Ġmainstream Ġnews . ĠIn Ġboth Ġcases Ġwe Ġfiltered Ġout Ġarticles Ġwith Ġless Ġthan Ġ50 Ġtweets ; Ġoverall Ġthis Ġdataset Ġcontains Ġ$\ sim Ġ$ 160 k Ġmainstream Ġtweets , Ġcorresponding Ġto Ġ227 Ġnews Ġarticles , Ġand Ġ$\ sim Ġ$ 100 k Ġdisinformation Ġtweets , Ġcorresponding Ġto Ġ237 Ġnews Ġarticles . ĠWe Ġprovide Ġin ĠFigure ĠFIG REF 5 Ġthe Ġdistribution Ġof Ġarticles Ġaccording Ġto Ġdistinct Ġsources Ġfor Ġboth Ġnews Ġdomains . ĠAs Ġin Ġthe ĠUS Ġdataset , Ġwe Ġtook Ġinto Ġaccount Ġcens oring Ġeffects ĠB IB REF 14 Ġby Ġexcluding Ġtweets Ġpublished Ġbefore Ġ( left - cens oring ) Ġor Ġafter Ġtwo Ġweeks Ġ( right - cens oring ) Ġfrom Ġthe Ġbeginning Ġof Ġthe Ġcollection Ġprocess . </s> The Ġdifferent Ġvolumes Ġof Ġnews Ġshared Ġon ĠTwitter Ġin Ġthe Ġtwo Ġcountries Ġare Ġdue Ġboth Ġto Ġthe Ġdifferent Ġpopulation Ġsize Ġof ĠUS Ġand ĠItaly Ġ( 320 Ġvs Ġ60 Ġmillions ) Ġbut Ġalso Ġto Ġthe Ġdifferent Ġusage Ġof ĠTwitter Ġplatform Ġ( and Ġsocial Ġmedia Ġin Ġgeneral ) Ġfor Ġnews Ġconsumption ĠB IB REF 24 . ĠBoth Ġdatasets Ġanalyzed Ġin Ġthis Ġwork Ġare Ġavailable Ġfrom Ġthe Ġauthors Ġon Ġrequest . </s> A Ġcrucial Ġaspect Ġin Ġour Ġapproach Ġis Ġthe Ġcapability Ġto Ġfully Ġcapturing Ġsharing Ġcasc ades Ġon ĠTwitter Ġassociated Ġto Ġnews Ġarticles . ĠIt Ġhas Ġbeen Ġreported ĠB IB REF 25 Ġthat Ġthe ĠTwitter Ġstreaming Ġendpoint Ġfilters Ġout Ġtweets Ġmatching Ġa Ġgiven Ġquery Ġif Ġthey Ġexceed Ġ1 % Ġof Ġthe Ġglobal Ġdaily Ġvolume Ġof Ġshared Ġtweets , Ġwhich Ġnowadays Ġis Ġapproximately Ġ$ 5 \ cd ot Ġ10 ^ 8 $ ; Ġhowever , Ġas Ġwe Ġalways Ġcollected Ġless Ġthan Ġ$ 10 ^ 6 $ Ġtweets Ġper Ġday , Ġwe Ġdid Ġnot Ġincur Ġin Ġthis Ġissue Ġand Ġwe Ġthus Ġgathered Ġ100 % Ġof Ġtweets Ġmatching Ġour Ġquery . </s> Method ology Ġ: :: ĠBuilding Ġdiffusion Ġnetworks </s> We Ġbuilt ĠTwitter Ġdiffusion Ġnetworks Ġfollowing Ġan Ġapproach Ġwidely Ġadopted Ġin Ġthe Ġliterature ĠB IB REF 6 BI B REF 17 BI B REF 2 . ĠWe Ġremark Ġthat Ġthere Ġis Ġan Ġunavoidable Ġlimitation Ġin ĠTwitter ĠStreaming ĠAPI , Ġwhich Ġdoes Ġnot Ġallow Ġto Ġretrieve Ġtrue Ġre - t weet ing Ġcasc ades Ġbecause Ġre - t we ets Ġalways Ġpoint Ġto Ġthe Ġoriginal Ġsource Ġand Ġnot Ġto Ġintermediate Ġre - t weet ing Ġusers ĠB IB REF 8 BI B REF 14 ; Ġthus Ġwe Ġadopt Ġthe Ġonly Ġviable Ġapproach Ġbased Ġon ĠTwitter 's Ġpublic Ġavailability Ġof Ġdata . ĠBesides , Ġby Ġdis ent angling Ġdifferent Ġinteractions Ġwith Ġmultiple Ġlayers Ġwe Ġpotentially Ġreduce Ġthe Ġimpact Ġof Ġthis Ġlimitation Ġon Ġthe Ġglobal Ġnetwork Ġproperties Ġcompared Ġto Ġthe Ġsingle - layer Ġapproach Ġused Ġin Ġour Ġbaseline . </s> Using Ġthe Ġnotation Ġdescribed Ġin ĠB IB REF 26 . Ġwe Ġemploy Ġa Ġmulti - layer Ġrepresentation Ġfor ĠTwitter Ġdiffusion Ġnetworks . ĠSoci ologists Ġhave Ġindeed Ġrecognized Ġdecades Ġago Ġthat Ġit Ġis Ġcrucial Ġto Ġstudy Ġsocial Ġsystems Ġby Ġconstructing Ġmultiple Ġsocial Ġnetworks Ġwhere Ġdifferent Ġtypes Ġof Ġties Ġamong Ġsame Ġindividuals Ġare Ġused ĠB IB REF 27 . ĠTherefore , Ġfor Ġeach Ġnews Ġarticle Ġwe Ġbuilt Ġa Ġmulti - layer Ġdiffusion Ġnetwork Ġcomposed Ġof Ġfour Ġdifferent Ġlayers , Ġone Ġfor Ġeach Ġtype Ġof Ġsocial Ġinteraction Ġon ĠTwitter Ġplatform , Ġnamely Ġretweet Ġ( RT ), Ġreply Ġ( R ), Ġquote Ġ( Q ) Ġand Ġmention Ġ( M ), Ġas Ġshown Ġin ĠFigure ĠFIG REF 11 . ĠThese Ġnetworks Ġare Ġnot Ġnecessarily Ġnode - aligned , Ġi . e . Ġusers Ġmight Ġbe Ġmissing Ġin Ġsome Ġlayers . ĠWe Ġdo Ġnot Ġinsert Ġ" d ummy " Ġnodes Ġto Ġrepresent Ġall Ġusers Ġas Ġit Ġwould Ġhave Ġsevere Ġimpact Ġon Ġthe Ġglobal Ġnetwork Ġproperties Ġ( e . g . Ġnumber Ġof Ġweak ly Ġconnected Ġcomponents ). ĠAlternatively Ġone Ġmay Ġlook Ġat Ġeach Ġmulti - layer Ġdiffusion Ġnetwork Ġas Ġan Ġensemble Ġof Ġindividual Ġgraphs ĠB IB REF 26 ; Ġsince Ġglobal Ġnetwork Ġproperties Ġare Ġcomputed Ġseparately Ġfor Ġeach Ġlayer , Ġthey Ġare Ġnot Ġaffected Ġby Ġthe Ġpresence Ġof Ġany Ġinter - layer Ġedges . </s> In Ġour Ġmulti - layer Ġrepresentation , Ġeach Ġlayer Ġis Ġa Ġdirected Ġgraph Ġwhere Ġwe Ġadd Ġedges Ġand Ġnodes Ġfor Ġeach Ġtweet Ġof Ġthe Ġlayer Ġtype , Ġe . g . Ġfor Ġthe ĠRT Ġlayer : Ġwhenever Ġuser Ġ$ a $ Ġret we ets Ġaccount Ġ$ b $ Ġwe Ġfirst Ġadd Ġnodes Ġ$ a $ Ġand Ġ$ b $ Ġif Ġnot Ġalready Ġpresent Ġin Ġthe ĠRT Ġlayer , Ġthen Ġwe Ġbuild Ġan Ġedge Ġthat Ġgoes Ġfrom Ġ$ b $ Ġto Ġ$ a $ Ġif Ġit Ġdoes Ġnot Ġexists Ġor Ġwe Ġincrement Ġthe Ġweight Ġby Ġ1 . ĠSimilarly Ġfor Ġthe Ġother Ġlayers : Ġfor Ġthe ĠR Ġlayer Ġedges Ġgo Ġfrom Ġuser Ġ$ a $ Ġ( who Ġreplies ) Ġto Ġuser Ġ$ b $, Ġfor Ġthe ĠQ Ġlayer Ġedges Ġgo Ġfrom Ġuser Ġ$ b $ Ġ( who Ġis Ġquoted Ġby ) Ġto Ġuser Ġ$ a $ Ġand Ġfor Ġthe ĠM Ġlayer Ġedges Ġgo Ġfrom Ġuser Ġ$ a $ Ġ( who Ġmentions ) Ġto Ġuser Ġ$ b $. </s> Note Ġthat , Ġby Ġconstruction , Ġour Ġlayers Ġdo Ġnot Ġinclude Ġisolated Ġnodes ; Ġthey Ġcorrespond Ġto Ġ" pure Ġtweets ", Ġi . e . Ġtweets Ġwhich Ġhave Ġnot Ġoriginated Ġany Ġinteractions Ġwith Ġother Ġusers . ĠHowever , Ġthey Ġare Ġpresent Ġin Ġour Ġdataset , Ġand Ġtheir Ġnumber Ġis Ġexploited Ġfor Ġclassification , Ġas Ġdescribed Ġbelow . </s> Method ology Ġ: :: ĠGlobal Ġnetwork Ġproperties </s> We Ġused Ġa Ġset Ġof Ġglobal Ġnetwork Ġindicators Ġwhich Ġallow Ġus Ġto Ġencode Ġeach Ġnetwork Ġlayer Ġby Ġa Ġtuple Ġof Ġfeatures . ĠThen Ġwe Ġsimply Ġconc aten ated Ġtu ples Ġas Ġto Ġrepresent Ġeach Ġmulti - layer Ġnetwork Ġwith Ġa Ġsingle Ġfeature Ġvector . ĠWe Ġused Ġthe Ġfollowing Ġglobal Ġnetwork Ġproperties : </s> Number Ġof ĠStrong ly ĠConnect ed ĠComponents Ġ( S CC ): Ġa ĠStrong ly ĠConnect ed ĠComponent Ġof Ġa Ġdirected Ġgraph Ġis Ġa Ġmaximal Ġ( sub ) graph Ġwhere Ġfor Ġeach Ġpair Ġof Ġvert ices Ġ$ u , v $ Ġthere Ġis Ġa Ġpath Ġin Ġeach Ġdirection Ġ($ u \ right arrow Ġv $, Ġ$ v \ right arrow Ġu $ ). </s> Size Ġof Ġthe ĠL arg est ĠStrong ly ĠConnect ed ĠComponent Ġ( LS CC ): Ġthe Ġnumber Ġof Ġnodes Ġin Ġthe Ġlargest Ġstrongly Ġconnected Ġcomponent Ġof Ġa Ġgiven Ġgraph . </s> Number Ġof ĠWeak ly ĠConnect ed ĠComponents Ġ( W CC ): Ġa ĠWeak ly ĠConnect ed ĠComponent Ġof Ġa Ġdirected Ġgraph Ġis Ġa Ġmaximal Ġ( sub ) graph Ġwhere Ġfor Ġeach Ġpair Ġof Ġvert ices Ġ$( u , Ġv ) $ Ġthere Ġis Ġa Ġpath Ġ$ u Ġ\ left right arrow Ġv $ Ġignoring Ġedge Ġdirections . </s> Size Ġof Ġthe ĠL arg est ĠWeak ly ĠConnect ed ĠComponent Ġ( L W CC ): Ġthe Ġnumber Ġof Ġnodes Ġin Ġthe Ġlargest Ġweak ly Ġconnected Ġcomponent Ġof Ġa Ġgiven Ġgraph . </s> D iameter Ġof Ġthe ĠL arg est ĠWeak ly ĠConnect ed ĠComponent Ġ( DW CC ): Ġthe Ġlargest Ġdistance Ġ( length Ġof Ġthe Ġshortest Ġpath ) Ġbetween Ġtwo Ġnodes Ġin Ġthe Ġ( und irect ed Ġversion Ġof ) Ġlargest Ġweak ly Ġconnected Ġcomponent Ġof Ġa Ġgraph . </s> Average ĠCl ust ering ĠCo efficient Ġ( CC ): Ġthe Ġaverage Ġof Ġthe Ġlocal Ġclust ering Ġcoefficients Ġof Ġall Ġnodes Ġin Ġa Ġgraph ; Ġthe Ġlocal Ġclust ering Ġcoefficient Ġof Ġa Ġnode Ġquant ifies Ġhow Ġclose Ġits Ġneighbours Ġare Ġto Ġbeing Ġa Ġcomplete Ġgraph Ġ( or Ġa Ġcl ique ). ĠIt Ġis Ġcomputed Ġaccording Ġto ĠB IB REF 28 . </s> Main ĠK - core ĠNumber Ġ( KC ): Ġa ĠK - core ĠB IB REF 13 Ġof Ġa Ġgraph Ġis Ġa Ġmaximal Ġsub - graph Ġthat Ġcontains Ġnodes Ġof Ġinternal Ġdegree Ġ$ k $ Ġor Ġmore ; Ġthe Ġmain ĠK - core Ġnumber Ġis Ġthe Ġhighest Ġvalue Ġof Ġ$ k $ Ġ( in Ġdirected Ġgraphs Ġthe Ġtotal Ġdegree Ġis Ġconsidered ). </s> D ensity Ġ( d ): Ġthe Ġdensity Ġfor Ġdirected Ġgraphs Ġis Ġ$ d = \ frac { | E | }{ | V || V - 1 | } $, Ġwhere Ġ$ | E | $ Ġis Ġthe Ġnumber Ġof Ġedges Ġand Ġ$ | N | $ Ġis Ġthe Ġnumber Ġof Ġvert ices Ġin Ġthe Ġgraph ; Ġthe Ġdensity Ġequals Ġ0 Ġfor Ġa Ġgraph Ġwithout Ġedges Ġand Ġ1 Ġfor Ġa Ġcomplete Ġgraph . </s> Struct ural Ġvir ality Ġof Ġthe Ġlargest Ġweak ly Ġconnected Ġcomponent Ġ( S V ): Ġthis Ġmeasure Ġis Ġdefined Ġin ĠB IB REF 14 Ġas Ġthe Ġaverage Ġdistance Ġbetween Ġall Ġpairs Ġof Ġnodes Ġin Ġa Ġcascade Ġtree Ġor , Ġequival ently , Ġas Ġthe Ġaverage Ġdepth Ġof Ġnodes , Ġaveraged Ġover Ġall Ġnodes Ġin Ġturn Ġacting Ġas Ġa Ġroot ; Ġfor Ġ$ | V | Ġ> Ġ1 $ Ġvert ices , Ġ$ S V = \ frac { 1 }{ | V || V - 1 | }\ sum Ġ_ i \ sum Ġ_ j Ġd _{ ij } $ Ġwhere Ġ$ d _{ ij } $ Ġdenotes Ġthe Ġlength Ġof Ġthe Ġshortest Ġpath Ġbetween Ġnodes Ġ$ i $ Ġand Ġ$ j $. ĠThis Ġis Ġequivalent Ġto Ġcompute Ġthe ĠWi ener 's Ġindex ĠB IB REF 29 Ġof Ġthe Ġgraph Ġand Ġmultiply Ġit Ġby Ġa Ġfactor Ġ$\ frac { 1 }{ | V || V - 1 | } $. ĠIn Ġour Ġcase Ġwe Ġcomputed Ġit Ġfor Ġthe Ġund irect ed Ġequivalent Ġgraph Ġof Ġthe Ġlargest Ġweak ly Ġconnected Ġcomponent , Ġsetting Ġit Ġto Ġ0 Ġwhenever Ġ$ V = 1 $. </s> We Ġused Ġnetwork x ĠPython Ġpackage ĠB IB REF 30 Ġto Ġcompute Ġall Ġfeatures . ĠWhenever Ġa Ġlayer Ġis Ġempty . Ġwe Ġsimply Ġset Ġto Ġ0 Ġall Ġits Ġfeatures . ĠIn Ġaddition Ġto Ġcomputing Ġthe Ġabove Ġnine Ġfeatures Ġfor Ġeach Ġlayer , Ġwe Ġadded Ġtwo Ġindicators Ġfor Ġencoding Ġinformation Ġabout Ġpure Ġtweets , Ġnamely Ġthe Ġnumber ĠT Ġof Ġpure Ġtweets Ġ( containing ĠURLs Ġto Ġa Ġgiven Ġnews Ġarticle ) Ġand Ġthe Ġnumber ĠU Ġof Ġunique Ġusers Ġauthor ing Ġthose Ġtweets . ĠTherefore , Ġa Ġsingle Ġdiffusion Ġnetwork Ġis Ġrepresented Ġby Ġa Ġvector Ġwith Ġ$ 9 \ cd ot Ġ4 + 2 = 38 $ Ġentries . </s> Method ology Ġ: :: ĠInterpret ation Ġof Ġnetwork Ġfeatures Ġand Ġlayers </s> A fore mentioned Ġnetwork Ġproperties Ġcan Ġbe Ġqual itatively Ġexplained Ġin Ġterms Ġof Ġsocial Ġfootprints Ġas Ġfollows : ĠS CC Ġcorrelates Ġwith Ġthe Ġsize Ġof Ġthe Ġdiffusion Ġnetwork , Ġas Ġthe Ġpropagation Ġof Ġnews Ġoccurs Ġin Ġa Ġbroadcast Ġmanner Ġmost Ġof Ġthe Ġtime , Ġi . e . Ġre - t we ets Ġdominate Ġon Ġother Ġinteractions , Ġwhile ĠLS CC Ġallows Ġto Ġdistinguish Ġcases Ġwhere Ġsuch Ġmono - direction ality Ġis Ġsomehow Ġbroken . ĠW CC Ġequals Ġ( approximately ) Ġthe Ġnumber Ġof Ġdistinct Ġdiffusion Ġcasc ades Ġpertaining Ġto Ġeach Ġnews Ġarticle , Ġwith Ġexceptions Ġcorresponding Ġto Ġthose Ġcases Ġwhere Ġsome Ġcasc ades Ġmerge Ġtogether Ġvia ĠTwitter Ġinteractions Ġsuch Ġas Ġmentions , Ġquotes Ġand Ġreplies , Ġand Ġaccordingly ĠLW CC Ġand ĠDW CC Ġequals Ġthe Ġsize Ġand Ġthe Ġdepth Ġof Ġthe Ġlargest Ġcascade . ĠCC Ġcorresponds Ġto Ġthe Ġlevel Ġof Ġconnected ness Ġof Ġneighboring Ġusers Ġin Ġa Ġgiven Ġdiffusion Ġnetwork Ġwhereas ĠKC Ġidentifies Ġthe Ġset Ġof Ġmost Ġinfluential Ġusers Ġin Ġa Ġnetwork Ġand Ġdescribes Ġthe Ġefficiency Ġof Ġinformation Ġspreading ĠB IB REF 17 . ĠFinally , Ġd Ġdescribes Ġthe Ġproportions Ġof Ġpotential Ġconnections Ġbetween Ġusers Ġwhich Ġare Ġactually Ġactivated Ġand ĠSV Ġindicates Ġwhether Ġa Ġnews Ġitem Ġhas Ġgained Ġpopularity Ġwith Ġa Ġsingle Ġand Ġlarge Ġbroadcast Ġor Ġin Ġa Ġmore Ġviral Ġfashion Ġthrough Ġmultiple Ġgenerations . </s> For Ġwhat Ġconcerns Ġdifferent ĠTwitter Ġactions , Ġusers Ġprimarily Ġinteract Ġwith Ġeach Ġother Ġusing Ġret we ets Ġand Ġmentions ĠB IB REF 20 . </s> The Ġformer Ġare Ġthe Ġmain Ġengagement Ġactivity Ġand Ġact Ġas Ġa Ġform Ġof Ġendorsement , Ġallowing Ġusers Ġto Ġreb road cast Ġcontent Ġgenerated Ġby Ġother Ġusers ĠB IB REF 31 . ĠBesides , Ġwhen Ġnode ĠB Ġret we ets Ġnode ĠA Ġwe Ġhave Ġan Ġimplicit Ġconfirmation Ġthat Ġinformation Ġfrom ĠA Ġappeared Ġin ĠB 's ĠTwitter Ġfeed ĠB IB REF 12 . ĠQu otes Ġare Ġsimply Ġa Ġspecial Ġcase Ġof Ġret we ets Ġwith Ġcomments . </s> M ent ions Ġusually Ġinclude Ġpersonal Ġconversations Ġas Ġthey Ġallow Ġsomeone Ġto Ġaddress Ġa Ġspecific Ġuser Ġor Ġto Ġrefer Ġto Ġan Ġindividual Ġin Ġthe Ġthird Ġperson ; Ġin Ġthe Ġfirst Ġcase Ġthey Ġare Ġlocated Ġat Ġthe Ġbeginning Ġof Ġa Ġtweet Ġand Ġthey Ġare Ġknown Ġas Ġreplies , Ġotherwise Ġthey Ġare Ġput Ġin Ġthe Ġbody Ġof Ġa Ġtweet ĠB IB REF 20 . ĠThe Ġnetwork Ġof Ġmentions Ġis Ġusually Ġseen Ġas Ġa Ġstronger Ġversion Ġof Ġinteractions Ġbetween ĠTwitter Ġusers , Ġcompared Ġto Ġthe Ġtraditional Ġgraph Ġof Ġfollower / follow ing Ġrelationships ĠB IB REF 32 . </s> Exper iments Ġ: :: ĠSetup </s> We Ġperformed Ġclassification Ġexperiments Ġusing Ġa Ġbasic Ġoff - the - she lf Ġclass ifier , Ġnamely ĠLog istic ĠReg ression Ġ( LR ) Ġwith ĠL 2 Ġpenalty ; Ġthis Ġalso Ġallows Ġus Ġto Ġcompare Ġresults Ġwith Ġour Ġbaseline . ĠWe Ġapplied Ġa Ġstandard ization Ġof Ġthe Ġfeatures Ġand Ġwe Ġused Ġthe Ġdefault Ġconfiguration Ġfor Ġparameters Ġas Ġdescribed Ġin Ġsc ik it - learn Ġpackage ĠB IB REF 33 . ĠWe Ġalso Ġtested Ġother Ġclass ifiers Ġ( such Ġas ĠK - Ne arest ĠNeigh bors , ĠSupport ĠVector ĠMachines Ġand ĠRandom ĠForest ) Ġbut Ġwe Ġomit Ġresults Ġas Ġthey Ġgive Ġcomparable Ġperformances . ĠWe Ġremark Ġthat Ġour Ġgoal Ġis Ġto Ġshow Ġthat Ġa Ġvery Ġsimple Ġmachine Ġlearning Ġframework , Ġwith Ġno Ġparameter Ġtuning Ġand Ġoptimization , Ġallows Ġfor Ġaccurate Ġresults Ġwith Ġour Ġnetwork - based Ġapproach . </s> We Ġused Ġthe Ġfollowing Ġevaluation Ġmetrics Ġto Ġassess Ġthe Ġperformances Ġof Ġdifferent Ġclass ifiers Ġ( TP = true Ġpositives , ĠFP = false Ġpositives , ĠFN = false Ġnegatives ): </s> Pre cision Ġ= Ġ$\ frac { TP }{ TP + FP } $, Ġthe Ġability Ġof Ġa Ġclass ifier Ġnot Ġto Ġlabel Ġas Ġpositive Ġa Ġnegative Ġsample . </s> Rec all Ġ= Ġ$\ frac { TP }{ TP + FN } $, Ġthe Ġability Ġof Ġa Ġclass ifier Ġto Ġretrieve Ġall Ġpositive Ġsamples . </s> F 1 - score Ġ= Ġ$ 2 Ġ\ frac {\ mb ox { Pre cision } Ġ\ cd ot Ġ\ mb ox { Rec all }} {\ mb ox { Pre cision } Ġ+ Ġ\ mb ox { Rec all }} $, Ġthe Ġharmonic Ġaverage Ġof ĠPrecision Ġand ĠRecall . </s> Area ĠUnder Ġthe ĠReceiver ĠOperating ĠCharacter istic Ġcurve Ġ( A UR OC ); Ġthe ĠReceiver ĠOperating ĠCharacter istic Ġ( R OC ) Ġcurve ĠB IB REF 34 , Ġwhich Ġplots Ġthe ĠTP Ġrate Ġversus Ġthe ĠFP Ġrate , Ġshows Ġthe Ġability Ġof Ġa Ġclass ifier Ġto Ġdiscriminate Ġpositive Ġsamples Ġfrom Ġnegative Ġones Ġas Ġits Ġthreshold Ġis Ġvaried ; Ġthe ĠA UR OC Ġvalue Ġis Ġin Ġthe Ġrange Ġ$ [ 0 , Ġ1 ] $, Ġwith Ġthe Ġrandom Ġbaseline Ġclass ifier Ġholding ĠA UR OC $ = 0 . 5 $ Ġand Ġthe Ġideal Ġperfect Ġclass ifier ĠA UR OC $ = 1 $ ; Ġthus Ġlarger ĠA UR OC Ġvalues Ġ( and Ġst eeper ĠR OC s ) Ġcorrespond Ġto Ġbetter Ġclass ifiers . </s> In Ġparticular Ġwe Ġcomputed Ġso - called Ġmacro Ġaverage âĢĵ simple Ġun weight ed Ġmean âĢĵ of Ġthese Ġmetrics Ġevaluated Ġconsidering Ġboth Ġlabels Ġ( dis information Ġand Ġmainstream ). ĠWe Ġemployed Ġstrat ified Ġshuffle Ġsplit Ġcross Ġvalidation Ġ( with Ġ10 Ġfolds ) Ġto Ġevaluate Ġperformances . </s> Finally , Ġwe Ġpartition ed Ġnetworks Ġaccording Ġto Ġthe Ġtotal Ġnumber Ġof Ġunique Ġusers Ġinvolved Ġin Ġthe Ġsharing , Ġi . e . Ġthe Ġnumber Ġof Ġnodes Ġin Ġthe Ġaggreg ated Ġnetwork Ġrepresented Ġwith Ġa Ġsingle - layer Ġrepresentation Ġconsidering Ġtogether Ġall Ġlayers Ġand Ġalso Ġpure Ġtweets . ĠA Ġbreakdown Ġof Ġboth Ġdatasets Ġaccording Ġto Ġsize Ġclass Ġ( and Ġpolitical Ġbiases Ġfor Ġthe ĠUS Ġscenario ) Ġis Ġprovided Ġin ĠTable Ġ1 Ġand ĠTable Ġ2 . </s> Exper iments Ġ: :: ĠClassification Ġperformances </s> In ĠTable Ġ3 Ġwe Ġfirst Ġprovide Ġclassification Ġperformances Ġon Ġthe ĠUS Ġdataset Ġfor Ġthe ĠLR Ġclass ifier Ġevaluated Ġon Ġthe Ġsize Ġclass Ġdescribed Ġin ĠTable Ġ1 . ĠWe Ġcan Ġobserve Ġthat Ġin Ġall Ġinstances Ġour Ġmethodology Ġperforms Ġbetter Ġthan Ġa Ġrandom Ġclass ifier Ġ( 50 % ĠA UR OC ), Ġwith ĠA UR OC Ġvalues Ġabove Ġ85 % Ġin Ġall Ġcases . </s> For Ġwhat Ġconcerns Ġpolitical Ġbiases , Ġas Ġthe Ġclasses Ġof Ġmainstream Ġand Ġdisinformation Ġnetworks Ġare Ġnot Ġbalanced Ġ( e . g ., Ġ1 , 292 Ġmainstream Ġand Ġ4 , 149 Ġdisinformation Ġnetworks Ġwith Ġright Ġbias ) Ġwe Ġemploy Ġa ĠBalanced ĠRandom ĠForest Ġwith Ġdefault Ġparameters Ġ( as Ġprovided Ġin Ġim ble arn ĠPython Ġpackage ĠB IB REF 35 ). ĠIn Ġorder Ġto Ġtest Ġthe Ġrobust ness Ġof Ġour Ġmethodology , Ġwe Ġtrained Ġonly Ġon Ġleft - biased Ġnetworks Ġor Ġright - biased Ġnetworks Ġand Ġtested Ġon Ġthe Ġentire Ġset Ġof Ġsources Ġ( relative Ġto Ġthe ĠUS Ġdataset ); Ġwe Ġprovide Ġa Ġcomparison Ġof ĠA UR OC Ġvalues Ġfor Ġboth Ġbiases Ġin ĠFigure Ġ4 . ĠWe Ġcan Ġnotice Ġthat Ġour Ġmulti - layer Ġapproach Ġstill Ġentails Ġsignificant Ġresults , Ġthus Ġshowing Ġthat Ġit Ġcan Ġaccurately Ġdistinguish Ġmainstream Ġnews Ġfrom Ġdisinformation Ġregardless Ġof Ġthe Ġpolitical Ġbias . ĠWe Ġfurther Ġcorrobor ated Ġthis Ġresult Ġwith Ġadditional Ġclassification Ġexperiments , Ġthat Ġshow Ġsimilar Ġperformances , Ġin Ġwhich Ġwe Ġexcluded Ġfrom Ġthe Ġtraining / test Ġset Ġtwo Ġspecific Ġsources Ġ( one Ġat Ġa Ġtime Ġand Ġboth Ġat Ġthe Ġsame Ġtime ) Ġthat Ġoutweigh Ġthe Ġothers Ġin Ġterms Ġof Ġdata Ġsamples âĢĵ respect ively Ġ" bre itbart . com " Ġfor Ġright - biased Ġsources Ġand Ġ" polit icus usa . com " Ġfor Ġleft - biased Ġones . </s> We Ġperformed Ġclassification Ġexperiments Ġon Ġthe ĠItalian Ġdataset Ġusing Ġthe ĠLR Ġclass ifier Ġand Ġdifferent Ġsize Ġclasses Ġ( we Ġexcluded Ġ$ [ 1000 , Ġ+ \ in fty Ġ) $ Ġwhich Ġis Ġempty ); Ġwe Ġshow Ġresults Ġfor Ġdifferent Ġevaluation Ġmetrics Ġin ĠTable Ġ3 . ĠWe Ġcan Ġsee Ġthat Ġdespite Ġthe Ġlimited Ġnumber Ġof Ġsamples Ġ( one Ġorder Ġof Ġmagnitude Ġsmaller Ġthan Ġthe ĠUS Ġdataset ) Ġthe Ġperformances Ġare Ġoverall Ġin Ġaccordance Ġwith Ġthe ĠUS Ġscenario . ĠAs Ġshown Ġin ĠTable Ġ4 , Ġwe Ġobtain Ġresults Ġwhich Ġare Ġmuch Ġbetter Ġthan Ġour Ġbaseline Ġin Ġall Ġsize Ġclasses Ġ( see ĠTable Ġ4 ): </s> In Ġthe ĠUS Ġdataset Ġour Ġmulti - layer Ġmethodology Ġperforms Ġmuch Ġbetter Ġin Ġall Ġsize Ġclasses Ġexcept Ġfor Ġlarge Ġnetworks Ġ($ [ 1000 , Ġ+ \ in fty Ġ) $ Ġsize Ġclass ), Ġreaching Ġup Ġto Ġ13 % Ġimprovement Ġon Ġsmaller Ġnetworks Ġ($ [ 0 , Ġ100 ) $ Ġsize Ġclass ); </s> In Ġthe ĠIT Ġdataset Ġour Ġmulti - layer Ġmethodology Ġoutper forms Ġthe Ġbaseline Ġin Ġall Ġsize Ġclasses , Ġwith Ġthe Ġmaximum Ġperformance Ġgain Ġ( 20 %) Ġon Ġmedium Ġnetworks Ġ($ [ 100 , Ġ1000 ) $ Ġsize Ġclass ); Ġthe Ġbaseline Ġgenerally Ġreaches Ġbad Ġperformances Ġcompared Ġto Ġthe ĠUS Ġscenario . </s> Overall , Ġour Ġperformances Ġare Ġcomparable Ġwith Ġthose Ġachieved Ġby Ġtwo Ġstate - of - the - art Ġdeep Ġlearning Ġmodels Ġfor Ġ" fake Ġnews " Ġdetection ĠB IB REF 9 BI B REF 36 . </s> Exper iments Ġ: :: ĠLayer Ġimportance Ġanalysis </s> In Ġorder Ġto Ġunderstand Ġthe Ġimpact Ġof Ġeach Ġlayer Ġon Ġthe Ġperformances Ġof Ġclass ifiers , Ġwe Ġperformed Ġadditional Ġexperiments Ġconsidering Ġseparately Ġeach Ġlayer Ġ( we Ġignored ĠT Ġand ĠU Ġfeatures Ġrelative Ġto Ġpure Ġtweets ). ĠIn ĠTable Ġ5 Ġwe Ġshow Ġmetrics Ġfor Ġeach Ġlayer Ġand Ġall Ġsize Ġclasses , Ġcomputed Ġwith Ġa Ġ10 - fold Ġstrat ified Ġshuffle Ġsplit Ġcross Ġvalidation , Ġevaluated Ġon Ġthe ĠUS Ġdataset ; Ġin ĠFigure Ġ5 Ġwe Ġshow ĠA UR OC Ġvalues Ġfor Ġeach Ġlayer Ġcompared Ġwith Ġthe Ġgeneral Ġmulti - layer Ġapproach . ĠWe Ġcan Ġnotice Ġthat Ġboth ĠQ Ġand ĠM Ġlayers Ġalone Ġcapture Ġadequately Ġthe Ġdiscrepancies Ġof Ġthe Ġtwo Ġdistinct Ġnews Ġdomains Ġin Ġthe ĠUnited ĠStates Ġas Ġthey Ġobtain Ġgood Ġresults Ġwith ĠA UR OC Ġvalues Ġin Ġthe Ġrange Ġ75 %- 86 %; Ġthese Ġare Ġcomparable Ġwith Ġthose Ġof Ġthe Ġmulti - layer Ġapproach Ġwhich , Ġnevertheless , Ġoutper forms Ġthem Ġacross Ġall Ġsize Ġclasses . </s> We Ġobtained Ġsimilar Ġperformances Ġfor Ġthe ĠItalian Ġdataset , Ġas Ġthe ĠM Ġlayer Ġob tains Ġcomparable Ġperformances Ġw . r . t Ġmulti - layer Ġapproach Ġwith ĠA UR OC Ġvalues Ġin Ġthe Ġrange Ġ72 %- 82 %. ĠWe Ġdo Ġnot Ġshow Ġthese Ġresults Ġfor Ġsake Ġof Ġconc is eness . </s> Exper iments Ġ: :: ĠFeature Ġimportance Ġanalysis </s> We Ġfurther Ġinvestigated Ġthe Ġimportance Ġof Ġeach Ġfeature Ġby Ġperforming Ġa Ġ$\ chi Ġ^ 2 $ Ġtest , Ġwith Ġ10 - fold Ġstrat ified Ġshuffle Ġsplit Ġcross Ġvalidation , Ġconsidering Ġthe Ġentire Ġrange Ġof Ġnetwork Ġsizes Ġ$ [ 0 , Ġ+ \ in fty Ġ) $. ĠWe Ġshow Ġthe ĠTop - 5 Ġmost Ġdiscrim in ative Ġfeatures Ġfor Ġeach Ġcountry Ġin ĠTable Ġ6 . </s> We Ġcan Ġnotice Ġthe Ġexact Ġsame Ġset Ġof Ġfeatures Ġ( with Ġdifferent Ġrelative Ġorder ings Ġin Ġthe ĠTop - 3 ) Ġin Ġboth Ġcountries ; Ġthese Ġcorrespond Ġto Ġtwo Ġglobal Ġnetwork Ġproper tie âĢĵ L W CC , Ġwhich Ġindicates Ġthe Ġsize Ġof Ġthe Ġlargest Ġcascade Ġin Ġthe Ġlayer , Ġand ĠS CC , Ġwhich Ġcorrelates Ġwith Ġthe Ġsize Ġof Ġthe Ġnetwork âĢĵ associated Ġto Ġthe Ġsame Ġset Ġof Ġlayers Ġ( Quotes , ĠRet we ets Ġand ĠMent ions ). </s> We Ġfurther Ġperformed Ġa Ġ$\ chi Ġ^ 2 $ Ġtest Ġto Ġhighlight Ġthe Ġmost Ġdiscrim in ative Ġfeatures Ġin Ġthe ĠM Ġlayer Ġof Ġboth Ġcountries , Ġwhich Ġperformed Ġequally Ġwell Ġin Ġthe Ġclassification Ġtask Ġas Ġpreviously Ġhighlighted ; Ġalso Ġin Ġthis Ġcase Ġwe Ġfocused Ġon Ġthe Ġentire Ġrange Ġof Ġnetwork Ġsizes Ġ$ [ 0 , Ġ+ \ in fty Ġ) $. ĠInterestingly , Ġwe Ġdiscovered Ġexactly Ġthe Ġsame Ġset Ġof ĠTop - 3 Ġfeatures Ġin Ġboth Ġcountries , Ġnamely ĠLW CC , ĠS CC Ġand ĠDW CC Ġ( which Ġindicates Ġthe Ġdepth Ġof Ġthe Ġlargest Ġcascade Ġin Ġthe Ġlayer ). </s> An Ġinspection Ġof Ġthe Ġdistributions Ġof Ġall Ġaforementioned Ġfeatures Ġrevealed Ġthat Ġdisinformation Ġnews Ġexhibit Ġon Ġaverage Ġlarger Ġvalues Ġthan Ġmainstream Ġnews . </s> We Ġcan Ġqual itatively Ġsum Ġup Ġthese Ġresults Ġas Ġfollows : </s> Sh aring Ġpatterns Ġin Ġthe Ġtwo Ġnews Ġdomains Ġexhibit Ġdiscrepancies Ġwhich Ġmight Ġbe Ġcountry - independent Ġand Ġdue Ġto Ġthe Ġcontent Ġthat Ġis Ġbeing Ġshared . </s> Inter actions Ġin Ġdisinformation Ġsharing Ġcasc ades Ġtends Ġto Ġbe Ġbroader Ġand Ġdeeper Ġthan Ġin Ġmainstream Ġnews , Ġas Ġwidely Ġreported Ġin Ġthe Ġliterature ĠB IB REF 8 BI B REF 2 BI B REF 7 . </s> Users Ġlikely Ġmake Ġa Ġdifferent Ġusage Ġof Ġmentions Ġwhen Ġsharing Ġnews Ġbelonging Ġto Ġthe Ġtwo Ġdomains , Ġconsequently Ġshaping Ġdifferent Ġsharing Ġpatterns . </s> Exper iments Ġ: :: ĠTem poral Ġanalysis </s> Similar Ġto ĠB IB REF 9 , Ġwe Ġcarried Ġout Ġadditional Ġexperiments Ġto Ġanswer Ġthe Ġfollowing Ġquestion : Ġhow Ġlong Ġdo Ġwe Ġneed Ġto Ġobserve Ġa Ġnews Ġspreading Ġon ĠTwitter Ġin Ġorder Ġto Ġaccurately Ġclassify Ġit Ġas Ġdisinformation Ġor Ġmainstream ? </s> With Ġthis Ġgoal , Ġwe Ġbuilt Ġseveral Ġversions Ġof Ġour Ġoriginal Ġdataset Ġof Ġmulti - layer Ġnetworks Ġby Ġconsidering Ġin Ġturn Ġthe Ġfollowing Ġlif etimes : Ġ1 Ġhour , Ġ6 Ġhours , Ġ12 Ġhours , Ġ1 Ġday , Ġ2 Ġdays , Ġ3 Ġdays Ġand Ġ7 Ġdays ; Ġfor Ġeach Ġcase , Ġwe Ġcomputed Ġthe Ġglobal Ġnetwork Ġproperties Ġof Ġthe Ġcorresponding Ġnetwork Ġand Ġevaluated Ġthe ĠLR Ġclass ifier Ġwith Ġ10 - fold Ġcross Ġvalidation , Ġseparately Ġfor Ġeach Ġlifetime Ġ( and Ġconsidering Ġalways Ġthe Ġentire Ġset Ġof Ġnetworks ). ĠWe Ġshow Ġcorresponding ĠA UR OC Ġvalues Ġfor Ġboth ĠUS Ġand ĠIT Ġdatasets Ġin ĠFigure Ġ6 . </s> We Ġcan Ġsee Ġthat Ġin Ġboth Ġcountries Ġnews Ġdiffusion Ġnetworks Ġcan Ġbe Ġaccurately Ġclassified Ġafter Ġjust Ġa Ġfew Ġhours Ġof Ġspreading , Ġwith ĠA UR OC Ġvalues Ġwhich Ġare Ġlarger Ġthan Ġ80 % Ġafter Ġonly Ġ6 Ġhours Ġof Ġdiffusion . ĠThese Ġresults Ġare Ġvery Ġpromising Ġand Ġsuggest Ġthat Ġarticles Ġpertaining Ġto Ġthe Ġtwo Ġnews Ġdomains Ġexhibit Ġdiscrepancies Ġin Ġtheir Ġsharing Ġpatterns Ġthat Ġcan Ġbe Ġtimely Ġexploited Ġin Ġorder Ġto Ġrapidly Ġdetect Ġmisleading Ġitems Ġfrom Ġfactual Ġinformation . </s> Con clusions </s> In Ġthis Ġwork Ġwe Ġtackled Ġthe Ġproblem Ġof Ġthe Ġautomatic Ġclassification Ġof Ġnews Ġarticles Ġin Ġtwo Ġdomains , Ġnamely Ġmainstream Ġand Ġdisinformation Ġnews , Ġwith Ġa Ġlanguage - independent Ġapproach Ġwhich Ġis Ġbased Ġsolely Ġon Ġthe Ġdiffusion Ġof Ġnews Ġitems Ġon ĠTwitter Ġsocial Ġplatform . ĠWe Ġdis ent angled Ġdifferent Ġtypes Ġof Ġinteractions Ġon ĠTwitter Ġto Ġaccordingly Ġbuild Ġa Ġmulti - layer Ġrepresentation Ġof Ġnews Ġdiffusion Ġnetworks , Ġand Ġwe Ġcomputed Ġa Ġset Ġof Ġglobal Ġnetwork Ġproperties âĢĵ separ ately Ġfor Ġeach Ġlayer âĢĵ in Ġorder Ġto Ġencode Ġeach Ġnetwork Ġwith Ġa Ġtuple Ġof Ġfeatures . ĠOur Ġgoal Ġwas Ġto Ġinvestigate Ġwhether Ġa Ġmulti - layer Ġrepresentation Ġperforms Ġbetter Ġthan Ġone Ġlayer ĠB IB REF 11 , Ġand Ġto Ġunderstand Ġwhich Ġof Ġthe Ġfeatures , Ġobserved Ġat Ġgiven Ġlayers , Ġare Ġmost Ġeffective Ġin Ġthe Ġclassification Ġtask . </s> Exper iments Ġwith Ġan Ġoff - the - she lf Ġclass ifier Ġsuch Ġas ĠLog istic ĠReg ression Ġon Ġdatasets Ġpertaining Ġto Ġtwo Ġdifferent Ġmedia Ġlandscapes Ġ( US Ġand ĠItaly ) Ġyield Ġvery Ġaccurate Ġclassification Ġresults Ġ( A UR OC Ġup Ġto Ġ94 %), Ġeven Ġwhen Ġaccounting Ġfor Ġthe Ġdifferent Ġpolitical Ġbias Ġof Ġnews Ġsources , Ġwhich Ġare Ġfar Ġbetter Ġthan Ġour Ġbaseline ĠB IB REF 11 Ġwith Ġimprovements Ġup Ġto Ġ20 %. ĠClassification Ġperformances Ġusing Ġsingle Ġlayers Ġshow Ġthat Ġthe Ġlayer Ġof Ġmentions Ġalone Ġentails Ġbetter Ġperformance Ġw . r . t Ġother Ġlayers Ġin Ġboth Ġcountries . </s> We Ġalso Ġhighlighted Ġthe Ġmost Ġdiscrim in ative Ġfeatures Ġacross Ġdifferent Ġlayers Ġin Ġboth Ġcountries ; Ġthe Ġresults Ġsuggest Ġthat Ġdifferences Ġbetween Ġthe Ġtwo Ġnews Ġdomains Ġmight Ġbe Ġcountry - independent Ġbut Ġrather Ġdue Ġonly Ġto Ġthe Ġtyp ology Ġof Ġcontent Ġshared , Ġand Ġthat Ġdisinformation Ġnews Ġshape Ġbroader Ġand Ġdeeper Ġcasc ades . </s> Additional Ġexperiments Ġinvolving Ġthe Ġtemporal Ġevolution Ġof ĠTwitter Ġdiffusion Ġnetworks Ġshow Ġthat Ġour Ġmethodology Ġcan Ġaccurate Ġclassify Ġmainstream Ġand Ġdisinformation Ġnews Ġafter Ġa Ġfew Ġhours Ġof Ġpropagation Ġon Ġthe Ġplatform . </s> Overall , Ġour Ġresults Ġprove Ġthat Ġthe Ġtop ological Ġfeatures Ġof Ġmulti - layer Ġdiffusion Ġnetworks Ġmight Ġbe Ġeffectively Ġexploited Ġto Ġdetect Ġonline Ġdisinformation . ĠWe Ġdo Ġnot Ġdeny Ġthe Ġpresence Ġof Ġdeceptive Ġefforts Ġto Ġorche strate Ġthe Ġregular Ġspread Ġof Ġinformation Ġon Ġsocial Ġmedia Ġvia Ġcontent Ġamplification Ġand Ġmanipulation ĠB IB REF 37 BI B REF 38 . ĠOn Ġthe Ġcontrary , Ġwe Ġpost ulate Ġthat Ġsuch Ġhidden Ġforces Ġmight Ġplay Ġto Ġaccent uate Ġthe Ġdiscrepancies Ġbetween Ġthe Ġdiffusion Ġpatterns Ġof Ġdisinformation Ġand Ġmainstream Ġnews Ġ( and Ġthus Ġto Ġmake Ġour Ġmethodology Ġeffective ). </s> In Ġthe Ġfuture Ġwe Ġaim Ġto Ġfurther Ġinvestigate Ġthree Ġdirections : Ġ( 1 ) Ġemploy Ġtemporal Ġnetworks Ġto Ġrepresent Ġnews Ġdiffusion Ġand Ġapply Ġclassification Ġtechniques Ġthat Ġtake Ġinto Ġaccount Ġthe Ġsequential Ġaspect Ġof Ġdata Ġ( e . g . Ġrecurrent Ġneural Ġnetworks ); Ġ( 2 ) Ġcarry Ġout Ġan Ġextensive Ġcomparison Ġof Ġthe Ġdiffusion Ġof Ġdisinformation Ġand Ġmainstream Ġnews Ġacross Ġcountries Ġto Ġinvestigate Ġdeeper Ġthe Ġpresence Ġof Ġdifferences Ġand Ġsimilarities Ġin Ġsharing Ġpatterns ; Ġ( 3 ) Ġleverage Ġour Ġnetwork - based Ġfeatures Ġin Ġaddition Ġto Ġstate - of - the - art Ġtext - based Ġapproaches Ġfor Ġ" fake Ġnews " Ġdet e Ġc tion Ġin Ġorder Ġto Ġdeliver Ġa Ġreal - world Ġsystem Ġto Ġdetect Ġmisleading Ġand Ġharmful Ġinformation Ġspreading Ġon Ġsocial Ġmedia .
Introduction and related work In recent years there has been increasing interest on the issue of disinformation spreading on online social media. Global concern over false (or "fake") news as a threat to modern democracies has been frequently raised–ever since 2016 US Presidential elections–in correspondence of events of political relevance, where the proliferation of manipulated and low-credibility content attempts to drive and influence people opinions BIBREF0BIBREF1BIBREF2BIBREF3. Researchers have highlighted several drivers for the diffusion of such malicious phenomenon, which include human factors (confirmation bias BIBREF4, naive realism BIBREF5), algorithmic biases (filter bubble effect BIBREF0), the presence of deceptive agents on social platforms (bots and trolls BIBREF6) and, lastly, the formation of echo chambers BIBREF7 where people polarize their opinions as they are insulated from contrary perspectives. The problem of automatically detecting online disinformation news has been typically formulated as a binary classification task (i.e. credible vs non-credible articles), and tackled with a variety of different techniques, based on traditional machine learning and/or deep learning, which mainly differ in the dataset and the features they employ to perform the classification. We may distinguish three approaches: those built on content-based features, those based on features extracted from the social context, and those which combine both aspects. A few main challenges hinder the task, namely the impossibility to manually verify all news items, the lack of gold-standard datasets and the adversarial setting in which malicious content is created BIBREF3BIBREF6. In this work we follow the direction pointed out in a few recent contributions on the diffusion of disinformation compared to traditional and objective information. These have shown that false news spread faster and deeper than true news BIBREF8, and that social bots and echo chambers play an important role in the diffusion of malicious content BIBREF6, BIBREF7. Therefore we focus on the analysis of spreading patterns which naturally arise on social platforms as a consequence of multiple interactions between users, due to the increasing trend in online sharing of news BIBREF0. A deep learning framework for detection of fake news cascades is provided in BIBREF9, where the authors refer to BIBREF8 in order to collect Twitter cascades pertaining to verified false and true rumors. They employ geometric deep learning, a novel paradigm for graph-based structures, to classify cascades based on four categories of features, such as user profile, user activity, network and spreading, and content. They also observe that a few hours of propagation are sufficient to distinguish false news from true news with high accuracy. Diffusion cascades on Weibo and Twitter are analyzed in BIBREF10, where authors focus on highlighting different topological properties, such as the number of hops from the source or the heterogeneity of the network, to show that fake news shape diffusion networks which are highly different from credible news, even at early stages of propagation. In this work, we consider the results of BIBREF11 as our baseline. The authors use off-the-shelf machine learning classifiers to accurately classify news articles leveraging Twitter diffusion networks. To this aim, they consider a set of basic features which can be qualitatively interpreted w.r.t to the social behavior of users sharing credible vs non-credible information. Their methodology is overall in accordance with BIBREF12, where authors successfully detect Twitter astroturfing content, i.e. political campaigns disguised as spontaneous grassroots, with a machine learning framework based on network features. In this paper, we propose a classification framework based on a multi-layer formulation of Twitter diffusion networks. For each article we disentangle different social interactions on Twitter, namely tweets, retweets, mentions, replies and quotes, to accordingly build a diffusion network composed of multiple layers (on for each type of interaction), and we compute structural features separately for each layer. We pick a set of global network properties from the network science toolbox which can be qualitatively explained in terms of social dimensions and allow us to encode different networks with a tuple of features. These include traditional indicators, e.g. network density, number of strong/weak connected components and diameter, and more elaborated ones such as main K-core number BIBREF13 and structural virality BIBREF14. Our main research question is whether the use of a multi-layer, disentangled network yields a significant advance in terms of classification accuracy over a conventional single-layer diffusion network. Additionally, we are interested in understanding which of the above features, and in which layer, are most effective in the classification task. We perform classification experiments with an off-the-shelf Logistic Regression model on two different datasets of mainstream and disinformation news shared on Twitter respectively in the United States and in Italy during 2019. In the former case we also account for political biases inherent to different news sources, referring to the procedure proposed in BIBREF2 to label different outlets. Overall we show that we are able to classify credible vs non-credible diffusion networks (and consequently news articles) with high accuracy (AUROC up to 94%), even when accounting for the political bias of sources (and training only on left-biased or right-biased articles). We observe that the layer of mentions alone conveys useful information for the classification, denoting a different usage of this functionality when sharing news belonging to the two news domains. We also show that most discriminative features, which are relative to the breadth and depth of largest cascades in different layers, are the same across the two countries. The outline of this paper is the following: we first formulate the problem and describe data collection, network representation and structural properties employed for the classification; then we provide experimental results–classification performances, layer and feature importance analyses and a temporal classification evaluation–and finally we draw conclusions and future directions. Methodology ::: Disinformation and mainstream news In this work we formulate our classification problem as follows: given two classes of news articles, respectively $D$ (disinformation) and $M$ (mainstream), a set of news articles $A_i$ and associated class labels $C_i \in \lbrace D,M\rbrace $, and a set of tweets $\Pi _i=\lbrace T_i^1, T_i^2, ...\rbrace $ each of which contains an Uniform Resource Locator (URL) pointing explicitly to article $A_i$, predict the class $C_i$ of each article $A_i$. There is huge debate and controversy on a proper taxonomy of malicious and deceptive information BIBREF1BIBREF2BIBREF15BIBREF16BIBREF17BIBREF3BIBREF11. In this work we prefer the term disinformation to the more specific fake news to refer to a variety of misleading and harmful information. Therefore, we follow a source-based approach, a consolidated strategy also adopted by BIBREF6BIBREF16BIBREF2BIBREF1, in order to obtain relevant data for our analysis. We collected: Disinformation articles, published by websites which are well-known for producing low-credibility content, false and misleading news reports as well as extreme propaganda and hoaxes and flagged as such by reputable journalists and fact-checkers; Mainstream news, referring to traditional news outlets which deliver factual and credible information. We believe that this is currently the most reliable classification approach, but it entails obvious limitations, as disinformation outlets may also publish true stories and likewise misinformation is sometimes reported on mainstream media. Also, given the choice of news sources, we cannot test whether our methodology is able to classify disinformation vs factual but not mainstream news which are published on niche, non-disinformation outlets. Methodology ::: US dataset We collected tweets associated to a dozen US mainstream news websites, i.e. most trusted sources described in BIBREF18, with the Streaming API, and we referred to Hoaxy API BIBREF16 for what concerns tweets containing links to 100+ US disinformation outlets. We filtered out articles associated to less than 50 tweets. The resulting dataset contains overall $\sim $1.7 million tweets for mainstream news, collected in a period of three weeks (February 25th, 2019-March 18th, 2019), which are associated to 6,978 news articles, and $\sim $1.6 million tweets for disinformation, collected in a period of three months (January 1st, 2019-March 18th, 2019) for sake of balance of the two classes, which hold 5,775 distinct articles. Diffusion censoring effects BIBREF14 were correctly taken into account in both collection procedures. We provide in Figure FIGREF4 the distribution of articles by source and political bias for both news domains. As it is reported that conservatives and liberals exhibit different behaviors on online social platforms BIBREF19BIBREF20BIBREF21, we further assigned a political bias label to different US outlets (and therefore news articles) following the procedure described in BIBREF2. In order to assess the robustness of our method, we performed classification experiments by training only on left-biased (or right-biased) outlets of both disinformation and mainstream domains and testing on the entire set of sources, as well as excluding particular sources that outweigh the others in terms of samples to avoid over-fitting. Methodology ::: Italian dataset For what concerns the Italian scenario we first collected tweets with the Streaming API in a 3-week period (April 19th, 2019-May 5th, 2019), filtering those containing URLs pointing to Italian official newspapers websites as described in BIBREF22; these correspond to the list provided by the association for the verification of newspaper circulation in Italy (Accertamenti Diffusione Stampa). We instead referred to the dataset provided by BIBREF23 to obtain a set of tweets, collected continuously since January 2019 using the same Twitter endpoint, which contain URLs to 60+ Italian disinformation websites. In order to get balanced classes (April 5th, 2019-May 5th, 2019), we retained data collected in a longer period w.r.t to mainstream news. In both cases we filtered out articles with less than 50 tweets; overall this dataset contains $\sim $160k mainstream tweets, corresponding to 227 news articles, and $\sim $100k disinformation tweets, corresponding to 237 news articles. We provide in Figure FIGREF5 the distribution of articles according to distinct sources for both news domains. As in the US dataset, we took into account censoring effects BIBREF14 by excluding tweets published before (left-censoring) or after two weeks (right-censoring) from the beginning of the collection process. The different volumes of news shared on Twitter in the two countries are due both to the different population size of US and Italy (320 vs 60 millions) but also to the different usage of Twitter platform (and social media in general) for news consumption BIBREF24. Both datasets analyzed in this work are available from the authors on request. A crucial aspect in our approach is the capability to fully capturing sharing cascades on Twitter associated to news articles. It has been reported BIBREF25 that the Twitter streaming endpoint filters out tweets matching a given query if they exceed 1% of the global daily volume of shared tweets, which nowadays is approximately $5\cdot 10^8$; however, as we always collected less than $10^6$ tweets per day, we did not incur in this issue and we thus gathered 100% of tweets matching our query. Methodology ::: Building diffusion networks We built Twitter diffusion networks following an approach widely adopted in the literature BIBREF6BIBREF17BIBREF2. We remark that there is an unavoidable limitation in Twitter Streaming API, which does not allow to retrieve true re-tweeting cascades because re-tweets always point to the original source and not to intermediate re-tweeting users BIBREF8BIBREF14; thus we adopt the only viable approach based on Twitter's public availability of data. Besides, by disentangling different interactions with multiple layers we potentially reduce the impact of this limitation on the global network properties compared to the single-layer approach used in our baseline. Using the notation described in BIBREF26. we employ a multi-layer representation for Twitter diffusion networks. Sociologists have indeed recognized decades ago that it is crucial to study social systems by constructing multiple social networks where different types of ties among same individuals are used BIBREF27. Therefore, for each news article we built a multi-layer diffusion network composed of four different layers, one for each type of social interaction on Twitter platform, namely retweet (RT), reply (R), quote (Q) and mention (M), as shown in Figure FIGREF11. These networks are not necessarily node-aligned, i.e. users might be missing in some layers. We do not insert "dummy" nodes to represent all users as it would have severe impact on the global network properties (e.g. number of weakly connected components). Alternatively one may look at each multi-layer diffusion network as an ensemble of individual graphs BIBREF26; since global network properties are computed separately for each layer, they are not affected by the presence of any inter-layer edges. In our multi-layer representation, each layer is a directed graph where we add edges and nodes for each tweet of the layer type, e.g. for the RT layer: whenever user $a$ retweets account $b$ we first add nodes $a$ and $b$ if not already present in the RT layer, then we build an edge that goes from $b$ to $a$ if it does not exists or we increment the weight by 1. Similarly for the other layers: for the R layer edges go from user $a$ (who replies) to user $b$, for the Q layer edges go from user $b$ (who is quoted by) to user $a$ and for the M layer edges go from user $a$ (who mentions) to user $b$. Note that, by construction, our layers do not include isolated nodes; they correspond to "pure tweets", i.e. tweets which have not originated any interactions with other users. However, they are present in our dataset, and their number is exploited for classification, as described below. Methodology ::: Global network properties We used a set of global network indicators which allow us to encode each network layer by a tuple of features. Then we simply concatenated tuples as to represent each multi-layer network with a single feature vector. We used the following global network properties: Number of Strongly Connected Components (SCC): a Strongly Connected Component of a directed graph is a maximal (sub)graph where for each pair of vertices $u,v$ there is a path in each direction ($u\rightarrow v$, $v\rightarrow u$). Size of the Largest Strongly Connected Component (LSCC): the number of nodes in the largest strongly connected component of a given graph. Number of Weakly Connected Components (WCC): a Weakly Connected Component of a directed graph is a maximal (sub)graph where for each pair of vertices $(u, v)$ there is a path $u \leftrightarrow v$ ignoring edge directions. Size of the Largest Weakly Connected Component (LWCC): the number of nodes in the largest weakly connected component of a given graph. Diameter of the Largest Weakly Connected Component (DWCC): the largest distance (length of the shortest path) between two nodes in the (undirected version of) largest weakly connected component of a graph. Average Clustering Coefficient (CC): the average of the local clustering coefficients of all nodes in a graph; the local clustering coefficient of a node quantifies how close its neighbours are to being a complete graph (or a clique). It is computed according to BIBREF28. Main K-core Number (KC): a K-core BIBREF13 of a graph is a maximal sub-graph that contains nodes of internal degree $k$ or more; the main K-core number is the highest value of $k$ (in directed graphs the total degree is considered). Density (d): the density for directed graphs is $d=\frac{|E|}{|V||V-1|}$, where $|E|$ is the number of edges and $|N|$ is the number of vertices in the graph; the density equals 0 for a graph without edges and 1 for a complete graph. Structural virality of the largest weakly connected component (SV): this measure is defined in BIBREF14 as the average distance between all pairs of nodes in a cascade tree or, equivalently, as the average depth of nodes, averaged over all nodes in turn acting as a root; for $|V| > 1$ vertices, $SV=\frac{1}{|V||V-1|}\sum _i\sum _j d_{ij}$ where $d_{ij}$ denotes the length of the shortest path between nodes $i$ and $j$. This is equivalent to compute the Wiener's index BIBREF29 of the graph and multiply it by a factor $\frac{1}{|V||V-1|}$. In our case we computed it for the undirected equivalent graph of the largest weakly connected component, setting it to 0 whenever $V=1$. We used networkx Python package BIBREF30 to compute all features. Whenever a layer is empty. we simply set to 0 all its features. In addition to computing the above nine features for each layer, we added two indicators for encoding information about pure tweets, namely the number T of pure tweets (containing URLs to a given news article) and the number U of unique users authoring those tweets. Therefore, a single diffusion network is represented by a vector with $9\cdot 4+2=38$ entries. Methodology ::: Interpretation of network features and layers Aforementioned network properties can be qualitatively explained in terms of social footprints as follows: SCC correlates with the size of the diffusion network, as the propagation of news occurs in a broadcast manner most of the time, i.e. re-tweets dominate on other interactions, while LSCC allows to distinguish cases where such mono-directionality is somehow broken. WCC equals (approximately) the number of distinct diffusion cascades pertaining to each news article, with exceptions corresponding to those cases where some cascades merge together via Twitter interactions such as mentions, quotes and replies, and accordingly LWCC and DWCC equals the size and the depth of the largest cascade. CC corresponds to the level of connectedness of neighboring users in a given diffusion network whereas KC identifies the set of most influential users in a network and describes the efficiency of information spreading BIBREF17. Finally, d describes the proportions of potential connections between users which are actually activated and SV indicates whether a news item has gained popularity with a single and large broadcast or in a more viral fashion through multiple generations. For what concerns different Twitter actions, users primarily interact with each other using retweets and mentions BIBREF20. The former are the main engagement activity and act as a form of endorsement, allowing users to rebroadcast content generated by other users BIBREF31. Besides, when node B retweets node A we have an implicit confirmation that information from A appeared in B's Twitter feed BIBREF12. Quotes are simply a special case of retweets with comments. Mentions usually include personal conversations as they allow someone to address a specific user or to refer to an individual in the third person; in the first case they are located at the beginning of a tweet and they are known as replies, otherwise they are put in the body of a tweet BIBREF20. The network of mentions is usually seen as a stronger version of interactions between Twitter users, compared to the traditional graph of follower/following relationships BIBREF32. Experiments ::: Setup We performed classification experiments using a basic off-the-shelf classifier, namely Logistic Regression (LR) with L2 penalty; this also allows us to compare results with our baseline. We applied a standardization of the features and we used the default configuration for parameters as described in scikit-learn package BIBREF33. We also tested other classifiers (such as K-Nearest Neighbors, Support Vector Machines and Random Forest) but we omit results as they give comparable performances. We remark that our goal is to show that a very simple machine learning framework, with no parameter tuning and optimization, allows for accurate results with our network-based approach. We used the following evaluation metrics to assess the performances of different classifiers (TP=true positives, FP=false positives, FN=false negatives): Precision = $\frac{TP}{TP+FP}$, the ability of a classifier not to label as positive a negative sample. Recall = $\frac{TP}{TP+FN}$, the ability of a classifier to retrieve all positive samples. F1-score = $2 \frac{\mbox{Precision} \cdot \mbox{Recall}}{\mbox{Precision} + \mbox{Recall}}$, the harmonic average of Precision and Recall. Area Under the Receiver Operating Characteristic curve (AUROC); the Receiver Operating Characteristic (ROC) curve BIBREF34, which plots the TP rate versus the FP rate, shows the ability of a classifier to discriminate positive samples from negative ones as its threshold is varied; the AUROC value is in the range $[0, 1]$, with the random baseline classifier holding AUROC$=0.5$ and the ideal perfect classifier AUROC$=1$; thus larger AUROC values (and steeper ROCs) correspond to better classifiers. In particular we computed so-called macro average–simple unweighted mean–of these metrics evaluated considering both labels (disinformation and mainstream). We employed stratified shuffle split cross validation (with 10 folds) to evaluate performances. Finally, we partitioned networks according to the total number of unique users involved in the sharing, i.e. the number of nodes in the aggregated network represented with a single-layer representation considering together all layers and also pure tweets. A breakdown of both datasets according to size class (and political biases for the US scenario) is provided in Table 1 and Table 2. Experiments ::: Classification performances In Table 3 we first provide classification performances on the US dataset for the LR classifier evaluated on the size class described in Table 1. We can observe that in all instances our methodology performs better than a random classifier (50% AUROC), with AUROC values above 85% in all cases. For what concerns political biases, as the classes of mainstream and disinformation networks are not balanced (e.g., 1,292 mainstream and 4,149 disinformation networks with right bias) we employ a Balanced Random Forest with default parameters (as provided in imblearn Python package BIBREF35). In order to test the robustness of our methodology, we trained only on left-biased networks or right-biased networks and tested on the entire set of sources (relative to the US dataset); we provide a comparison of AUROC values for both biases in Figure 4. We can notice that our multi-layer approach still entails significant results, thus showing that it can accurately distinguish mainstream news from disinformation regardless of the political bias. We further corroborated this result with additional classification experiments, that show similar performances, in which we excluded from the training/test set two specific sources (one at a time and both at the same time) that outweigh the others in terms of data samples–respectively "breitbart.com" for right-biased sources and "politicususa.com" for left-biased ones. We performed classification experiments on the Italian dataset using the LR classifier and different size classes (we excluded $[1000, +\infty )$ which is empty); we show results for different evaluation metrics in Table 3. We can see that despite the limited number of samples (one order of magnitude smaller than the US dataset) the performances are overall in accordance with the US scenario. As shown in Table 4, we obtain results which are much better than our baseline in all size classes (see Table 4): In the US dataset our multi-layer methodology performs much better in all size classes except for large networks ($[1000, +\infty )$ size class), reaching up to 13% improvement on smaller networks ($[0, 100)$ size class); In the IT dataset our multi-layer methodology outperforms the baseline in all size classes, with the maximum performance gain (20%) on medium networks ($[100, 1000)$ size class); the baseline generally reaches bad performances compared to the US scenario. Overall, our performances are comparable with those achieved by two state-of-the-art deep learning models for "fake news" detection BIBREF9BIBREF36. Experiments ::: Layer importance analysis In order to understand the impact of each layer on the performances of classifiers, we performed additional experiments considering separately each layer (we ignored T and U features relative to pure tweets). In Table 5 we show metrics for each layer and all size classes, computed with a 10-fold stratified shuffle split cross validation, evaluated on the US dataset; in Figure 5 we show AUROC values for each layer compared with the general multi-layer approach. We can notice that both Q and M layers alone capture adequately the discrepancies of the two distinct news domains in the United States as they obtain good results with AUROC values in the range 75%-86%; these are comparable with those of the multi-layer approach which, nevertheless, outperforms them across all size classes. We obtained similar performances for the Italian dataset, as the M layer obtains comparable performances w.r.t multi-layer approach with AUROC values in the range 72%-82%. We do not show these results for sake of conciseness. Experiments ::: Feature importance analysis We further investigated the importance of each feature by performing a $\chi ^2$ test, with 10-fold stratified shuffle split cross validation, considering the entire range of network sizes $[0, +\infty )$. We show the Top-5 most discriminative features for each country in Table 6. We can notice the exact same set of features (with different relative orderings in the Top-3) in both countries; these correspond to two global network propertie–LWCC, which indicates the size of the largest cascade in the layer, and SCC, which correlates with the size of the network–associated to the same set of layers (Quotes, Retweets and Mentions). We further performed a $\chi ^2$ test to highlight the most discriminative features in the M layer of both countries, which performed equally well in the classification task as previously highlighted; also in this case we focused on the entire range of network sizes $[0, +\infty )$. Interestingly, we discovered exactly the same set of Top-3 features in both countries, namely LWCC, SCC and DWCC (which indicates the depth of the largest cascade in the layer). An inspection of the distributions of all aforementioned features revealed that disinformation news exhibit on average larger values than mainstream news. We can qualitatively sum up these results as follows: Sharing patterns in the two news domains exhibit discrepancies which might be country-independent and due to the content that is being shared. Interactions in disinformation sharing cascades tends to be broader and deeper than in mainstream news, as widely reported in the literature BIBREF8BIBREF2BIBREF7. Users likely make a different usage of mentions when sharing news belonging to the two domains, consequently shaping different sharing patterns. Experiments ::: Temporal analysis Similar to BIBREF9, we carried out additional experiments to answer the following question: how long do we need to observe a news spreading on Twitter in order to accurately classify it as disinformation or mainstream? With this goal, we built several versions of our original dataset of multi-layer networks by considering in turn the following lifetimes: 1 hour, 6 hours, 12 hours, 1 day, 2 days, 3 days and 7 days; for each case, we computed the global network properties of the corresponding network and evaluated the LR classifier with 10-fold cross validation, separately for each lifetime (and considering always the entire set of networks). We show corresponding AUROC values for both US and IT datasets in Figure 6. We can see that in both countries news diffusion networks can be accurately classified after just a few hours of spreading, with AUROC values which are larger than 80% after only 6 hours of diffusion. These results are very promising and suggest that articles pertaining to the two news domains exhibit discrepancies in their sharing patterns that can be timely exploited in order to rapidly detect misleading items from factual information. Conclusions In this work we tackled the problem of the automatic classification of news articles in two domains, namely mainstream and disinformation news, with a language-independent approach which is based solely on the diffusion of news items on Twitter social platform. We disentangled different types of interactions on Twitter to accordingly build a multi-layer representation of news diffusion networks, and we computed a set of global network properties–separately for each layer–in order to encode each network with a tuple of features. Our goal was to investigate whether a multi-layer representation performs better than one layer BIBREF11, and to understand which of the features, observed at given layers, are most effective in the classification task. Experiments with an off-the-shelf classifier such as Logistic Regression on datasets pertaining to two different media landscapes (US and Italy) yield very accurate classification results (AUROC up to 94%), even when accounting for the different political bias of news sources, which are far better than our baseline BIBREF11 with improvements up to 20%. Classification performances using single layers show that the layer of mentions alone entails better performance w.r.t other layers in both countries. We also highlighted the most discriminative features across different layers in both countries; the results suggest that differences between the two news domains might be country-independent but rather due only to the typology of content shared, and that disinformation news shape broader and deeper cascades. Additional experiments involving the temporal evolution of Twitter diffusion networks show that our methodology can accurate classify mainstream and disinformation news after a few hours of propagation on the platform. Overall, our results prove that the topological features of multi-layer diffusion networks might be effectively exploited to detect online disinformation. We do not deny the presence of deceptive efforts to orchestrate the regular spread of information on social media via content amplification and manipulation BIBREF37BIBREF38. On the contrary, we postulate that such hidden forces might play to accentuate the discrepancies between the diffusion patterns of disinformation and mainstream news (and thus to make our methodology effective). In the future we aim to further investigate three directions: (1) employ temporal networks to represent news diffusion and apply classification techniques that take into account the sequential aspect of data (e.g. recurrent neural networks); (2) carry out an extensive comparison of the diffusion of disinformation and mainstream news across countries to investigate deeper the presence of differences and similarities in sharing patterns; (3) leverage our network-based features in addition to state-of-the-art text-based approaches for "fake news" dete ction in order to deliver a real-world system to detect misleading and harmful information spreading on social media.
How is the political bias of different sources included in the model?
dc2a2c177cd5df6da5d03e6e74262bf424850ec9
How Ġis Ġthe Ġpolitical Ġbias Ġof Ġdifferent Ġsources Ġincluded Ġin Ġthe Ġmodel ?
By Ġassigning Ġa Ġpolitical Ġbias Ġlabel Ġto Ġeach Ġnews Ġarticle Ġand Ġtraining Ġonly Ġon Ġleft - biased Ġor Ġright - biased Ġoutlets Ġof Ġboth Ġdisinformation Ġand Ġmainstream Ġdomains
<s> How Ġis Ġthe Ġpolitical Ġbias Ġof Ġdifferent Ġsources Ġincluded Ġin Ġthe Ġmodel ? </s> Introduction Ġand Ġrelated Ġwork </s> In Ġrecent Ġyears Ġthere Ġhas Ġbeen Ġincreasing Ġinterest Ġon Ġthe Ġissue Ġof Ġdisinformation Ġspreading Ġon Ġonline Ġsocial Ġmedia . ĠGlobal Ġconcern Ġover Ġfalse Ġ( or Ġ" fake ") Ġnews Ġas Ġa Ġthreat Ġto Ġmodern Ġdemocracies Ġhas Ġbeen Ġfrequently Ġraised âĢĵ ever Ġsince Ġ2016 ĠUS ĠPresidential Ġelections âĢĵ in Ġcorrespondence Ġof Ġevents Ġof Ġpolitical Ġrelevance , Ġwhere Ġthe Ġproliferation Ġof Ġmanipulated Ġand Ġlow - c red ibility Ġcontent Ġattempts Ġto Ġdrive Ġand Ġinfluence Ġpeople Ġopinions ĠB IB REF 0 BI B REF 1 BI B REF 2 BI B REF 3 . </s> Researchers Ġhave Ġhighlighted Ġseveral Ġdrivers Ġfor Ġthe Ġdiffusion Ġof Ġsuch Ġmalicious Ġphenomenon , Ġwhich Ġinclude Ġhuman Ġfactors Ġ( conf irmation Ġbias ĠB IB REF 4 , Ġnaive Ġrealism ĠB IB REF 5 ), Ġalgorith mic Ġbiases Ġ( filter Ġbubble Ġeffect ĠB IB REF 0 ), Ġthe Ġpresence Ġof Ġdeceptive Ġagents Ġon Ġsocial Ġplatforms Ġ( bots Ġand Ġtrolls ĠB IB REF 6 ) Ġand , Ġlast ly , Ġthe Ġformation Ġof Ġecho Ġchambers ĠB IB REF 7 Ġwhere Ġpeople Ġpolar ize Ġtheir Ġopinions Ġas Ġthey Ġare Ġinsulated Ġfrom Ġcontrary Ġperspectives . </s> The Ġproblem Ġof Ġautomatically Ġdetecting Ġonline Ġdisinformation Ġnews Ġhas Ġbeen Ġtypically Ġformulated Ġas Ġa Ġbinary Ġclassification Ġtask Ġ( i . e . Ġcredible Ġvs Ġnon - c redible Ġarticles ), Ġand Ġtackled Ġwith Ġa Ġvariety Ġof Ġdifferent Ġtechniques , Ġbased Ġon Ġtraditional Ġmachine Ġlearning Ġand / or Ġdeep Ġlearning , Ġwhich Ġmainly Ġdiffer Ġin Ġthe Ġdataset Ġand Ġthe Ġfeatures Ġthey Ġemploy Ġto Ġperform Ġthe Ġclassification . ĠWe Ġmay Ġdistinguish Ġthree Ġapproaches : Ġthose Ġbuilt Ġon Ġcontent - based Ġfeatures , Ġthose Ġbased Ġon Ġfeatures Ġextracted Ġfrom Ġthe Ġsocial Ġcontext , Ġand Ġthose Ġwhich Ġcombine Ġboth Ġaspects . ĠA Ġfew Ġmain Ġchallenges Ġhinder Ġthe Ġtask , Ġnamely Ġthe Ġimpossibility Ġto Ġmanually Ġverify Ġall Ġnews Ġitems , Ġthe Ġlack Ġof Ġgold - standard Ġdatasets Ġand Ġthe Ġadvers arial Ġsetting Ġin Ġwhich Ġmalicious Ġcontent Ġis Ġcreated ĠB IB REF 3 BI B REF 6 . </s> In Ġthis Ġwork Ġwe Ġfollow Ġthe Ġdirection Ġpointed Ġout Ġin Ġa Ġfew Ġrecent Ġcontributions Ġon Ġthe Ġdiffusion Ġof Ġdisinformation Ġcompared Ġto Ġtraditional Ġand Ġobjective Ġinformation . ĠThese Ġhave Ġshown Ġthat Ġfalse Ġnews Ġspread Ġfaster Ġand Ġdeeper Ġthan Ġtrue Ġnews ĠB IB REF 8 , Ġand Ġthat Ġsocial Ġbots Ġand Ġecho Ġchambers Ġplay Ġan Ġimportant Ġrole Ġin Ġthe Ġdiffusion Ġof Ġmalicious Ġcontent ĠB IB REF 6 , ĠB IB REF 7 . ĠTherefore Ġwe Ġfocus Ġon Ġthe Ġanalysis Ġof Ġspreading Ġpatterns Ġwhich Ġnaturally Ġarise Ġon Ġsocial Ġplatforms Ġas Ġa Ġconsequence Ġof Ġmultiple Ġinteractions Ġbetween Ġusers , Ġdue Ġto Ġthe Ġincreasing Ġtrend Ġin Ġonline Ġsharing Ġof Ġnews ĠB IB REF 0 . </s> A Ġdeep Ġlearning Ġframework Ġfor Ġdetection Ġof Ġfake Ġnews Ġcasc ades Ġis Ġprovided Ġin ĠB IB REF 9 , Ġwhere Ġthe Ġauthors Ġrefer Ġto ĠB IB REF 8 Ġin Ġorder Ġto Ġcollect ĠTwitter Ġcasc ades Ġpertaining Ġto Ġverified Ġfalse Ġand Ġtrue Ġrumors . ĠThey Ġemploy Ġgeometric Ġdeep Ġlearning , Ġa Ġnovel Ġparadigm Ġfor Ġgraph - based Ġstructures , Ġto Ġclassify Ġcasc ades Ġbased Ġon Ġfour Ġcategories Ġof Ġfeatures , Ġsuch Ġas Ġuser Ġprofile , Ġuser Ġactivity , Ġnetwork Ġand Ġspreading , Ġand Ġcontent . ĠThey Ġalso Ġobserve Ġthat Ġa Ġfew Ġhours Ġof Ġpropagation Ġare Ġsufficient Ġto Ġdistinguish Ġfalse Ġnews Ġfrom Ġtrue Ġnews Ġwith Ġhigh Ġaccuracy . ĠDiff usion Ġcasc ades Ġon ĠWe ibo Ġand ĠTwitter Ġare Ġanalyzed Ġin ĠB IB REF 10 , Ġwhere Ġauthors Ġfocus Ġon Ġhighlighting Ġdifferent Ġtop ological Ġproperties , Ġsuch Ġas Ġthe Ġnumber Ġof Ġhops Ġfrom Ġthe Ġsource Ġor Ġthe Ġheterogeneity Ġof Ġthe Ġnetwork , Ġto Ġshow Ġthat Ġfake Ġnews Ġshape Ġdiffusion Ġnetworks Ġwhich Ġare Ġhighly Ġdifferent Ġfrom Ġcredible Ġnews , Ġeven Ġat Ġearly Ġstages Ġof Ġpropagation . </s> In Ġthis Ġwork , Ġwe Ġconsider Ġthe Ġresults Ġof ĠB IB REF 11 Ġas Ġour Ġbaseline . ĠThe Ġauthors Ġuse Ġoff - the - she lf Ġmachine Ġlearning Ġclass ifiers Ġto Ġaccurately Ġclassify Ġnews Ġarticles Ġleveraging ĠTwitter Ġdiffusion Ġnetworks . ĠTo Ġthis Ġaim , Ġthey Ġconsider Ġa Ġset Ġof Ġbasic Ġfeatures Ġwhich Ġcan Ġbe Ġqual itatively Ġinterpreted Ġw . r . t Ġto Ġthe Ġsocial Ġbehavior Ġof Ġusers Ġsharing Ġcredible Ġvs Ġnon - c redible Ġinformation . ĠTheir Ġmethodology Ġis Ġoverall Ġin Ġaccordance Ġwith ĠB IB REF 12 , Ġwhere Ġauthors Ġsuccessfully Ġdetect ĠTwitter Ġast rot ur fing Ġcontent , Ġi . e . Ġpolitical Ġcampaigns Ġdisguised Ġas Ġspontaneous Ġgrassroots , Ġwith Ġa Ġmachine Ġlearning Ġframework Ġbased Ġon Ġnetwork Ġfeatures . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġclassification Ġframework Ġbased Ġon Ġa Ġmulti - layer Ġformulation Ġof ĠTwitter Ġdiffusion Ġnetworks . ĠFor Ġeach Ġarticle Ġwe Ġdis ent angle Ġdifferent Ġsocial Ġinteractions Ġon ĠTwitter , Ġnamely Ġtweets , Ġret we ets , Ġmentions , Ġreplies Ġand Ġquotes , Ġto Ġaccordingly Ġbuild Ġa Ġdiffusion Ġnetwork Ġcomposed Ġof Ġmultiple Ġlayers Ġ( on Ġfor Ġeach Ġtype Ġof Ġinteraction ), Ġand Ġwe Ġcompute Ġstructural Ġfeatures Ġseparately Ġfor Ġeach Ġlayer . ĠWe Ġpick Ġa Ġset Ġof Ġglobal Ġnetwork Ġproperties Ġfrom Ġthe Ġnetwork Ġscience Ġtool box Ġwhich Ġcan Ġbe Ġqual itatively Ġexplained Ġin Ġterms Ġof Ġsocial Ġdimensions Ġand Ġallow Ġus Ġto Ġencode Ġdifferent Ġnetworks Ġwith Ġa Ġtuple Ġof Ġfeatures . ĠThese Ġinclude Ġtraditional Ġindicators , Ġe . g . Ġnetwork Ġdensity , Ġnumber Ġof Ġstrong / weak Ġconnected Ġcomponents Ġand Ġdiameter , Ġand Ġmore Ġelaborated Ġones Ġsuch Ġas Ġmain ĠK - core Ġnumber ĠB IB REF 13 Ġand Ġstructural Ġvir ality ĠB IB REF 14 . ĠOur Ġmain Ġresearch Ġquestion Ġis Ġwhether Ġthe Ġuse Ġof Ġa Ġmulti - layer , Ġdis ent angled Ġnetwork Ġyields Ġa Ġsignificant Ġadvance Ġin Ġterms Ġof Ġclassification Ġaccuracy Ġover Ġa Ġconventional Ġsingle - layer Ġdiffusion Ġnetwork . ĠAdditionally , Ġwe Ġare Ġinterested Ġin Ġunderstanding Ġwhich Ġof Ġthe Ġabove Ġfeatures , Ġand Ġin Ġwhich Ġlayer , Ġare Ġmost Ġeffective Ġin Ġthe Ġclassification Ġtask . </s> We Ġperform Ġclassification Ġexperiments Ġwith Ġan Ġoff - the - she lf ĠLog istic ĠReg ression Ġmodel Ġon Ġtwo Ġdifferent Ġdatasets Ġof Ġmainstream Ġand Ġdisinformation Ġnews Ġshared Ġon ĠTwitter Ġrespectively Ġin Ġthe ĠUnited ĠStates Ġand Ġin ĠItaly Ġduring Ġ2019 . ĠIn Ġthe Ġformer Ġcase Ġwe Ġalso Ġaccount Ġfor Ġpolitical Ġbiases Ġinherent Ġto Ġdifferent Ġnews Ġsources , Ġreferring Ġto Ġthe Ġprocedure Ġproposed Ġin ĠB IB REF 2 Ġto Ġlabel Ġdifferent Ġoutlets . ĠOverall Ġwe Ġshow Ġthat Ġwe Ġare Ġable Ġto Ġclassify Ġcredible Ġvs Ġnon - c redible Ġdiffusion Ġnetworks Ġ( and Ġconsequently Ġnews Ġarticles ) Ġwith Ġhigh Ġaccuracy Ġ( A UR OC Ġup Ġto Ġ94 %), Ġeven Ġwhen Ġaccounting Ġfor Ġthe Ġpolitical Ġbias Ġof Ġsources Ġ( and Ġtraining Ġonly Ġon Ġleft - biased Ġor Ġright - biased Ġarticles ). ĠWe Ġobserve Ġthat Ġthe Ġlayer Ġof Ġmentions Ġalone Ġconve ys Ġuseful Ġinformation Ġfor Ġthe Ġclassification , Ġden oting Ġa Ġdifferent Ġusage Ġof Ġthis Ġfunctionality Ġwhen Ġsharing Ġnews Ġbelonging Ġto Ġthe Ġtwo Ġnews Ġdomains . ĠWe Ġalso Ġshow Ġthat Ġmost Ġdiscrim in ative Ġfeatures , Ġwhich Ġare Ġrelative Ġto Ġthe Ġbreadth Ġand Ġdepth Ġof Ġlargest Ġcasc ades Ġin Ġdifferent Ġlayers , Ġare Ġthe Ġsame Ġacross Ġthe Ġtwo Ġcountries . </s> The Ġoutline Ġof Ġthis Ġpaper Ġis Ġthe Ġfollowing : Ġwe Ġfirst Ġformulate Ġthe Ġproblem Ġand Ġdescribe Ġdata Ġcollection , Ġnetwork Ġrepresentation Ġand Ġstructural Ġproperties Ġemployed Ġfor Ġthe Ġclassification ; Ġthen Ġwe Ġprovide Ġexperimental Ġresults âĢĵ class ification Ġperformances , Ġlayer Ġand Ġfeature Ġimportance Ġanalyses Ġand Ġa Ġtemporal Ġclassification Ġevaluation âĢĵ and Ġfinally Ġwe Ġdraw Ġconclusions Ġand Ġfuture Ġdirections . </s> Method ology Ġ: :: ĠDis information Ġand Ġmainstream Ġnews </s> In Ġthis Ġwork Ġwe Ġformulate Ġour Ġclassification Ġproblem Ġas Ġfollows : Ġgiven Ġtwo Ġclasses Ġof Ġnews Ġarticles , Ġrespectively Ġ$ D $ Ġ( dis information ) Ġand Ġ$ M $ Ġ( main stream ), Ġa Ġset Ġof Ġnews Ġarticles Ġ$ A _ i $ Ġand Ġassociated Ġclass Ġlabels Ġ$ C _ i Ġ\ in Ġ\ l brace ĠD , M \ r brace Ġ$ , Ġand Ġa Ġset Ġof Ġtweets Ġ$\ Pi Ġ_ i = \ l brace ĠT _ i ^ 1 , ĠT _ i ^ 2 , Ġ... \ r brace Ġ$ Ġeach Ġof Ġwhich Ġcontains Ġan ĠUniform ĠResource ĠLoc ator Ġ( URL ) Ġpointing Ġexplicitly Ġto Ġarticle Ġ$ A _ i $, Ġpredict Ġthe Ġclass Ġ$ C _ i $ Ġof Ġeach Ġarticle Ġ$ A _ i $. ĠThere Ġis Ġhuge Ġdebate Ġand Ġcontroversy Ġon Ġa Ġproper Ġtax onomy Ġof Ġmalicious Ġand Ġdeceptive Ġinformation ĠB IB REF 1 BI B REF 2 BI B REF 15 BI B REF 16 BI B REF 17 BI B REF 3 BI B REF 11 . ĠIn Ġthis Ġwork Ġwe Ġprefer Ġthe Ġterm Ġdisinformation Ġto Ġthe Ġmore Ġspecific Ġfake Ġnews Ġto Ġrefer Ġto Ġa Ġvariety Ġof Ġmisleading Ġand Ġharmful Ġinformation . ĠTherefore , Ġwe Ġfollow Ġa Ġsource - based Ġapproach , Ġa Ġconsolidated Ġstrategy Ġalso Ġadopted Ġby ĠB IB REF 6 BI B REF 16 BI B REF 2 BI B REF 1 , Ġin Ġorder Ġto Ġobtain Ġrelevant Ġdata Ġfor Ġour Ġanalysis . ĠWe Ġcollected : </s> Dis information Ġarticles , Ġpublished Ġby Ġwebsites Ġwhich Ġare Ġwell - known Ġfor Ġproducing Ġlow - c red ibility Ġcontent , Ġfalse Ġand Ġmisleading Ġnews Ġreports Ġas Ġwell Ġas Ġextreme Ġpropaganda Ġand Ġhoax es Ġand Ġflagged Ġas Ġsuch Ġby Ġreputable Ġjournalists Ġand Ġfact - check ers ; </s> Main stream Ġnews , Ġreferring Ġto Ġtraditional Ġnews Ġoutlets Ġwhich Ġdeliver Ġfactual Ġand Ġcredible Ġinformation . </s> We Ġbelieve Ġthat Ġthis Ġis Ġcurrently Ġthe Ġmost Ġreliable Ġclassification Ġapproach , Ġbut Ġit Ġentails Ġobvious Ġlimitations , Ġas Ġdisinformation Ġoutlets Ġmay Ġalso Ġpublish Ġtrue Ġstories Ġand Ġlikewise Ġmisinformation Ġis Ġsometimes Ġreported Ġon Ġmainstream Ġmedia . ĠAlso , Ġgiven Ġthe Ġchoice Ġof Ġnews Ġsources , Ġwe Ġcannot Ġtest Ġwhether Ġour Ġmethodology Ġis Ġable Ġto Ġclassify Ġdisinformation Ġvs Ġfactual Ġbut Ġnot Ġmainstream Ġnews Ġwhich Ġare Ġpublished Ġon Ġniche , Ġnon - dis information Ġoutlets . </s> Method ology Ġ: :: ĠUS Ġdataset </s> We Ġcollected Ġtweets Ġassociated Ġto Ġa Ġdozen ĠUS Ġmainstream Ġnews Ġwebsites , Ġi . e . Ġmost Ġtrusted Ġsources Ġdescribed Ġin ĠB IB REF 18 , Ġwith Ġthe ĠStreaming ĠAPI , Ġand Ġwe Ġreferred Ġto ĠHo axy ĠAPI ĠB IB REF 16 Ġfor Ġwhat Ġconcerns Ġtweets Ġcontaining Ġlinks Ġto Ġ100 + ĠUS Ġdisinformation Ġoutlets . ĠWe Ġfiltered Ġout Ġarticles Ġassociated Ġto Ġless Ġthan Ġ50 Ġtweets . ĠThe Ġresulting Ġdataset Ġcontains Ġoverall Ġ$\ sim Ġ$ 1 . 7 Ġmillion Ġtweets Ġfor Ġmainstream Ġnews , Ġcollected Ġin Ġa Ġperiod Ġof Ġthree Ġweeks Ġ( February Ġ25 th , Ġ2019 - March Ġ18 th , Ġ2019 ), Ġwhich Ġare Ġassociated Ġto Ġ6 , 978 Ġnews Ġarticles , Ġand Ġ$\ sim Ġ$ 1 . 6 Ġmillion Ġtweets Ġfor Ġdisinformation , Ġcollected Ġin Ġa Ġperiod Ġof Ġthree Ġmonths Ġ( January Ġ1 st , Ġ2019 - March Ġ18 th , Ġ2019 ) Ġfor Ġsake Ġof Ġbalance Ġof Ġthe Ġtwo Ġclasses , Ġwhich Ġhold Ġ5 , 775 Ġdistinct Ġarticles . ĠDiff usion Ġcens oring Ġeffects ĠB IB REF 14 Ġwere Ġcorrectly Ġtaken Ġinto Ġaccount Ġin Ġboth Ġcollection Ġprocedures . ĠWe Ġprovide Ġin ĠFigure ĠFIG REF 4 Ġthe Ġdistribution Ġof Ġarticles Ġby Ġsource Ġand Ġpolitical Ġbias Ġfor Ġboth Ġnews Ġdomains . </s> As Ġit Ġis Ġreported Ġthat Ġconservatives Ġand Ġliberals Ġexhibit Ġdifferent Ġbehaviors Ġon Ġonline Ġsocial Ġplatforms ĠB IB REF 19 BI B REF 20 BI B REF 21 , Ġwe Ġfurther Ġassigned Ġa Ġpolitical Ġbias Ġlabel Ġto Ġdifferent ĠUS Ġoutlets Ġ( and Ġtherefore Ġnews Ġarticles ) Ġfollowing Ġthe Ġprocedure Ġdescribed Ġin ĠB IB REF 2 . ĠIn Ġorder Ġto Ġassess Ġthe Ġrobust ness Ġof Ġour Ġmethod , Ġwe Ġperformed Ġclassification Ġexperiments Ġby Ġtraining Ġonly Ġon Ġleft - biased Ġ( or Ġright - biased ) Ġoutlets Ġof Ġboth Ġdisinformation Ġand Ġmainstream Ġdomains Ġand Ġtesting Ġon Ġthe Ġentire Ġset Ġof Ġsources , Ġas Ġwell Ġas Ġexcluding Ġparticular Ġsources Ġthat Ġoutweigh Ġthe Ġothers Ġin Ġterms Ġof Ġsamples Ġto Ġavoid Ġover - fitting . </s> Method ology Ġ: :: ĠItalian Ġdataset </s> For Ġwhat Ġconcerns Ġthe ĠItalian Ġscenario Ġwe Ġfirst Ġcollected Ġtweets Ġwith Ġthe ĠStreaming ĠAPI Ġin Ġa Ġ3 - week Ġperiod Ġ( April Ġ19 th , Ġ2019 - May Ġ5 th , Ġ2019 ), Ġfiltering Ġthose Ġcontaining ĠURLs Ġpointing Ġto ĠItalian Ġofficial Ġnewspapers Ġwebsites Ġas Ġdescribed Ġin ĠB IB REF 22 ; Ġthese Ġcorrespond Ġto Ġthe Ġlist Ġprovided Ġby Ġthe Ġassociation Ġfor Ġthe Ġverification Ġof Ġnewspaper Ġcirculation Ġin ĠItaly Ġ( Acc ert ament i ĠDiff usion e ĠSt ampa ). ĠWe Ġinstead Ġreferred Ġto Ġthe Ġdataset Ġprovided Ġby ĠB IB REF 23 Ġto Ġobtain Ġa Ġset Ġof Ġtweets , Ġcollected Ġcontinuously Ġsince ĠJanuary Ġ2019 Ġusing Ġthe Ġsame ĠTwitter Ġendpoint , Ġwhich Ġcontain ĠURLs Ġto Ġ60 + ĠItalian Ġdisinformation Ġwebsites . ĠIn Ġorder Ġto Ġget Ġbalanced Ġclasses Ġ( April Ġ5 th , Ġ2019 - May Ġ5 th , Ġ2019 ), Ġwe Ġretained Ġdata Ġcollected Ġin Ġa Ġlonger Ġperiod Ġw . r . t Ġto Ġmainstream Ġnews . ĠIn Ġboth Ġcases Ġwe Ġfiltered Ġout Ġarticles Ġwith Ġless Ġthan Ġ50 Ġtweets ; Ġoverall Ġthis Ġdataset Ġcontains Ġ$\ sim Ġ$ 160 k Ġmainstream Ġtweets , Ġcorresponding Ġto Ġ227 Ġnews Ġarticles , Ġand Ġ$\ sim Ġ$ 100 k Ġdisinformation Ġtweets , Ġcorresponding Ġto Ġ237 Ġnews Ġarticles . ĠWe Ġprovide Ġin ĠFigure ĠFIG REF 5 Ġthe Ġdistribution Ġof Ġarticles Ġaccording Ġto Ġdistinct Ġsources Ġfor Ġboth Ġnews Ġdomains . ĠAs Ġin Ġthe ĠUS Ġdataset , Ġwe Ġtook Ġinto Ġaccount Ġcens oring Ġeffects ĠB IB REF 14 Ġby Ġexcluding Ġtweets Ġpublished Ġbefore Ġ( left - cens oring ) Ġor Ġafter Ġtwo Ġweeks Ġ( right - cens oring ) Ġfrom Ġthe Ġbeginning Ġof Ġthe Ġcollection Ġprocess . </s> The Ġdifferent Ġvolumes Ġof Ġnews Ġshared Ġon ĠTwitter Ġin Ġthe Ġtwo Ġcountries Ġare Ġdue Ġboth Ġto Ġthe Ġdifferent Ġpopulation Ġsize Ġof ĠUS Ġand ĠItaly Ġ( 320 Ġvs Ġ60 Ġmillions ) Ġbut Ġalso Ġto Ġthe Ġdifferent Ġusage Ġof ĠTwitter Ġplatform Ġ( and Ġsocial Ġmedia Ġin Ġgeneral ) Ġfor Ġnews Ġconsumption ĠB IB REF 24 . ĠBoth Ġdatasets Ġanalyzed Ġin Ġthis Ġwork Ġare Ġavailable Ġfrom Ġthe Ġauthors Ġon Ġrequest . </s> A Ġcrucial Ġaspect Ġin Ġour Ġapproach Ġis Ġthe Ġcapability Ġto Ġfully Ġcapturing Ġsharing Ġcasc ades Ġon ĠTwitter Ġassociated Ġto Ġnews Ġarticles . ĠIt Ġhas Ġbeen Ġreported ĠB IB REF 25 Ġthat Ġthe ĠTwitter Ġstreaming Ġendpoint Ġfilters Ġout Ġtweets Ġmatching Ġa Ġgiven Ġquery Ġif Ġthey Ġexceed Ġ1 % Ġof Ġthe Ġglobal Ġdaily Ġvolume Ġof Ġshared Ġtweets , Ġwhich Ġnowadays Ġis Ġapproximately Ġ$ 5 \ cd ot Ġ10 ^ 8 $ ; Ġhowever , Ġas Ġwe Ġalways Ġcollected Ġless Ġthan Ġ$ 10 ^ 6 $ Ġtweets Ġper Ġday , Ġwe Ġdid Ġnot Ġincur Ġin Ġthis Ġissue Ġand Ġwe Ġthus Ġgathered Ġ100 % Ġof Ġtweets Ġmatching Ġour Ġquery . </s> Method ology Ġ: :: ĠBuilding Ġdiffusion Ġnetworks </s> We Ġbuilt ĠTwitter Ġdiffusion Ġnetworks Ġfollowing Ġan Ġapproach Ġwidely Ġadopted Ġin Ġthe Ġliterature ĠB IB REF 6 BI B REF 17 BI B REF 2 . ĠWe Ġremark Ġthat Ġthere Ġis Ġan Ġunavoidable Ġlimitation Ġin ĠTwitter ĠStreaming ĠAPI , Ġwhich Ġdoes Ġnot Ġallow Ġto Ġretrieve Ġtrue Ġre - t weet ing Ġcasc ades Ġbecause Ġre - t we ets Ġalways Ġpoint Ġto Ġthe Ġoriginal Ġsource Ġand Ġnot Ġto Ġintermediate Ġre - t weet ing Ġusers ĠB IB REF 8 BI B REF 14 ; Ġthus Ġwe Ġadopt Ġthe Ġonly Ġviable Ġapproach Ġbased Ġon ĠTwitter 's Ġpublic Ġavailability Ġof Ġdata . ĠBesides , Ġby Ġdis ent angling Ġdifferent Ġinteractions Ġwith Ġmultiple Ġlayers Ġwe Ġpotentially Ġreduce Ġthe Ġimpact Ġof Ġthis Ġlimitation Ġon Ġthe Ġglobal Ġnetwork Ġproperties Ġcompared Ġto Ġthe Ġsingle - layer Ġapproach Ġused Ġin Ġour Ġbaseline . </s> Using Ġthe Ġnotation Ġdescribed Ġin ĠB IB REF 26 . Ġwe Ġemploy Ġa Ġmulti - layer Ġrepresentation Ġfor ĠTwitter Ġdiffusion Ġnetworks . ĠSoci ologists Ġhave Ġindeed Ġrecognized Ġdecades Ġago Ġthat Ġit Ġis Ġcrucial Ġto Ġstudy Ġsocial Ġsystems Ġby Ġconstructing Ġmultiple Ġsocial Ġnetworks Ġwhere Ġdifferent Ġtypes Ġof Ġties Ġamong Ġsame Ġindividuals Ġare Ġused ĠB IB REF 27 . ĠTherefore , Ġfor Ġeach Ġnews Ġarticle Ġwe Ġbuilt Ġa Ġmulti - layer Ġdiffusion Ġnetwork Ġcomposed Ġof Ġfour Ġdifferent Ġlayers , Ġone Ġfor Ġeach Ġtype Ġof Ġsocial Ġinteraction Ġon ĠTwitter Ġplatform , Ġnamely Ġretweet Ġ( RT ), Ġreply Ġ( R ), Ġquote Ġ( Q ) Ġand Ġmention Ġ( M ), Ġas Ġshown Ġin ĠFigure ĠFIG REF 11 . ĠThese Ġnetworks Ġare Ġnot Ġnecessarily Ġnode - aligned , Ġi . e . Ġusers Ġmight Ġbe Ġmissing Ġin Ġsome Ġlayers . ĠWe Ġdo Ġnot Ġinsert Ġ" d ummy " Ġnodes Ġto Ġrepresent Ġall Ġusers Ġas Ġit Ġwould Ġhave Ġsevere Ġimpact Ġon Ġthe Ġglobal Ġnetwork Ġproperties Ġ( e . g . Ġnumber Ġof Ġweak ly Ġconnected Ġcomponents ). ĠAlternatively Ġone Ġmay Ġlook Ġat Ġeach Ġmulti - layer Ġdiffusion Ġnetwork Ġas Ġan Ġensemble Ġof Ġindividual Ġgraphs ĠB IB REF 26 ; Ġsince Ġglobal Ġnetwork Ġproperties Ġare Ġcomputed Ġseparately Ġfor Ġeach Ġlayer , Ġthey Ġare Ġnot Ġaffected Ġby Ġthe Ġpresence Ġof Ġany Ġinter - layer Ġedges . </s> In Ġour Ġmulti - layer Ġrepresentation , Ġeach Ġlayer Ġis Ġa Ġdirected Ġgraph Ġwhere Ġwe Ġadd Ġedges Ġand Ġnodes Ġfor Ġeach Ġtweet Ġof Ġthe Ġlayer Ġtype , Ġe . g . Ġfor Ġthe ĠRT Ġlayer : Ġwhenever Ġuser Ġ$ a $ Ġret we ets Ġaccount Ġ$ b $ Ġwe Ġfirst Ġadd Ġnodes Ġ$ a $ Ġand Ġ$ b $ Ġif Ġnot Ġalready Ġpresent Ġin Ġthe ĠRT Ġlayer , Ġthen Ġwe Ġbuild Ġan Ġedge Ġthat Ġgoes Ġfrom Ġ$ b $ Ġto Ġ$ a $ Ġif Ġit Ġdoes Ġnot Ġexists Ġor Ġwe Ġincrement Ġthe Ġweight Ġby Ġ1 . ĠSimilarly Ġfor Ġthe Ġother Ġlayers : Ġfor Ġthe ĠR Ġlayer Ġedges Ġgo Ġfrom Ġuser Ġ$ a $ Ġ( who Ġreplies ) Ġto Ġuser Ġ$ b $, Ġfor Ġthe ĠQ Ġlayer Ġedges Ġgo Ġfrom Ġuser Ġ$ b $ Ġ( who Ġis Ġquoted Ġby ) Ġto Ġuser Ġ$ a $ Ġand Ġfor Ġthe ĠM Ġlayer Ġedges Ġgo Ġfrom Ġuser Ġ$ a $ Ġ( who Ġmentions ) Ġto Ġuser Ġ$ b $. </s> Note Ġthat , Ġby Ġconstruction , Ġour Ġlayers Ġdo Ġnot Ġinclude Ġisolated Ġnodes ; Ġthey Ġcorrespond Ġto Ġ" pure Ġtweets ", Ġi . e . Ġtweets Ġwhich Ġhave Ġnot Ġoriginated Ġany Ġinteractions Ġwith Ġother Ġusers . ĠHowever , Ġthey Ġare Ġpresent Ġin Ġour Ġdataset , Ġand Ġtheir Ġnumber Ġis Ġexploited Ġfor Ġclassification , Ġas Ġdescribed Ġbelow . </s> Method ology Ġ: :: ĠGlobal Ġnetwork Ġproperties </s> We Ġused Ġa Ġset Ġof Ġglobal Ġnetwork Ġindicators Ġwhich Ġallow Ġus Ġto Ġencode Ġeach Ġnetwork Ġlayer Ġby Ġa Ġtuple Ġof Ġfeatures . ĠThen Ġwe Ġsimply Ġconc aten ated Ġtu ples Ġas Ġto Ġrepresent Ġeach Ġmulti - layer Ġnetwork Ġwith Ġa Ġsingle Ġfeature Ġvector . ĠWe Ġused Ġthe Ġfollowing Ġglobal Ġnetwork Ġproperties : </s> Number Ġof ĠStrong ly ĠConnect ed ĠComponents Ġ( S CC ): Ġa ĠStrong ly ĠConnect ed ĠComponent Ġof Ġa Ġdirected Ġgraph Ġis Ġa Ġmaximal Ġ( sub ) graph Ġwhere Ġfor Ġeach Ġpair Ġof Ġvert ices Ġ$ u , v $ Ġthere Ġis Ġa Ġpath Ġin Ġeach Ġdirection Ġ($ u \ right arrow Ġv $, Ġ$ v \ right arrow Ġu $ ). </s> Size Ġof Ġthe ĠL arg est ĠStrong ly ĠConnect ed ĠComponent Ġ( LS CC ): Ġthe Ġnumber Ġof Ġnodes Ġin Ġthe Ġlargest Ġstrongly Ġconnected Ġcomponent Ġof Ġa Ġgiven Ġgraph . </s> Number Ġof ĠWeak ly ĠConnect ed ĠComponents Ġ( W CC ): Ġa ĠWeak ly ĠConnect ed ĠComponent Ġof Ġa Ġdirected Ġgraph Ġis Ġa Ġmaximal Ġ( sub ) graph Ġwhere Ġfor Ġeach Ġpair Ġof Ġvert ices Ġ$( u , Ġv ) $ Ġthere Ġis Ġa Ġpath Ġ$ u Ġ\ left right arrow Ġv $ Ġignoring Ġedge Ġdirections . </s> Size Ġof Ġthe ĠL arg est ĠWeak ly ĠConnect ed ĠComponent Ġ( L W CC ): Ġthe Ġnumber Ġof Ġnodes Ġin Ġthe Ġlargest Ġweak ly Ġconnected Ġcomponent Ġof Ġa Ġgiven Ġgraph . </s> D iameter Ġof Ġthe ĠL arg est ĠWeak ly ĠConnect ed ĠComponent Ġ( DW CC ): Ġthe Ġlargest Ġdistance Ġ( length Ġof Ġthe Ġshortest Ġpath ) Ġbetween Ġtwo Ġnodes Ġin Ġthe Ġ( und irect ed Ġversion Ġof ) Ġlargest Ġweak ly Ġconnected Ġcomponent Ġof Ġa Ġgraph . </s> Average ĠCl ust ering ĠCo efficient Ġ( CC ): Ġthe Ġaverage Ġof Ġthe Ġlocal Ġclust ering Ġcoefficients Ġof Ġall Ġnodes Ġin Ġa Ġgraph ; Ġthe Ġlocal Ġclust ering Ġcoefficient Ġof Ġa Ġnode Ġquant ifies Ġhow Ġclose Ġits Ġneighbours Ġare Ġto Ġbeing Ġa Ġcomplete Ġgraph Ġ( or Ġa Ġcl ique ). ĠIt Ġis Ġcomputed Ġaccording Ġto ĠB IB REF 28 . </s> Main ĠK - core ĠNumber Ġ( KC ): Ġa ĠK - core ĠB IB REF 13 Ġof Ġa Ġgraph Ġis Ġa Ġmaximal Ġsub - graph Ġthat Ġcontains Ġnodes Ġof Ġinternal Ġdegree Ġ$ k $ Ġor Ġmore ; Ġthe Ġmain ĠK - core Ġnumber Ġis Ġthe Ġhighest Ġvalue Ġof Ġ$ k $ Ġ( in Ġdirected Ġgraphs Ġthe Ġtotal Ġdegree Ġis Ġconsidered ). </s> D ensity Ġ( d ): Ġthe Ġdensity Ġfor Ġdirected Ġgraphs Ġis Ġ$ d = \ frac { | E | }{ | V || V - 1 | } $, Ġwhere Ġ$ | E | $ Ġis Ġthe Ġnumber Ġof Ġedges Ġand Ġ$ | N | $ Ġis Ġthe Ġnumber Ġof Ġvert ices Ġin Ġthe Ġgraph ; Ġthe Ġdensity Ġequals Ġ0 Ġfor Ġa Ġgraph Ġwithout Ġedges Ġand Ġ1 Ġfor Ġa Ġcomplete Ġgraph . </s> Struct ural Ġvir ality Ġof Ġthe Ġlargest Ġweak ly Ġconnected Ġcomponent Ġ( S V ): Ġthis Ġmeasure Ġis Ġdefined Ġin ĠB IB REF 14 Ġas Ġthe Ġaverage Ġdistance Ġbetween Ġall Ġpairs Ġof Ġnodes Ġin Ġa Ġcascade Ġtree Ġor , Ġequival ently , Ġas Ġthe Ġaverage Ġdepth Ġof Ġnodes , Ġaveraged Ġover Ġall Ġnodes Ġin Ġturn Ġacting Ġas Ġa Ġroot ; Ġfor Ġ$ | V | Ġ> Ġ1 $ Ġvert ices , Ġ$ S V = \ frac { 1 }{ | V || V - 1 | }\ sum Ġ_ i \ sum Ġ_ j Ġd _{ ij } $ Ġwhere Ġ$ d _{ ij } $ Ġdenotes Ġthe Ġlength Ġof Ġthe Ġshortest Ġpath Ġbetween Ġnodes Ġ$ i $ Ġand Ġ$ j $. ĠThis Ġis Ġequivalent Ġto Ġcompute Ġthe ĠWi ener 's Ġindex ĠB IB REF 29 Ġof Ġthe Ġgraph Ġand Ġmultiply Ġit Ġby Ġa Ġfactor Ġ$\ frac { 1 }{ | V || V - 1 | } $. ĠIn Ġour Ġcase Ġwe Ġcomputed Ġit Ġfor Ġthe Ġund irect ed Ġequivalent Ġgraph Ġof Ġthe Ġlargest Ġweak ly Ġconnected Ġcomponent , Ġsetting Ġit Ġto Ġ0 Ġwhenever Ġ$ V = 1 $. </s> We Ġused Ġnetwork x ĠPython Ġpackage ĠB IB REF 30 Ġto Ġcompute Ġall Ġfeatures . ĠWhenever Ġa Ġlayer Ġis Ġempty . Ġwe Ġsimply Ġset Ġto Ġ0 Ġall Ġits Ġfeatures . ĠIn Ġaddition Ġto Ġcomputing Ġthe Ġabove Ġnine Ġfeatures Ġfor Ġeach Ġlayer , Ġwe Ġadded Ġtwo Ġindicators Ġfor Ġencoding Ġinformation Ġabout Ġpure Ġtweets , Ġnamely Ġthe Ġnumber ĠT Ġof Ġpure Ġtweets Ġ( containing ĠURLs Ġto Ġa Ġgiven Ġnews Ġarticle ) Ġand Ġthe Ġnumber ĠU Ġof Ġunique Ġusers Ġauthor ing Ġthose Ġtweets . ĠTherefore , Ġa Ġsingle Ġdiffusion Ġnetwork Ġis Ġrepresented Ġby Ġa Ġvector Ġwith Ġ$ 9 \ cd ot Ġ4 + 2 = 38 $ Ġentries . </s> Method ology Ġ: :: ĠInterpret ation Ġof Ġnetwork Ġfeatures Ġand Ġlayers </s> A fore mentioned Ġnetwork Ġproperties Ġcan Ġbe Ġqual itatively Ġexplained Ġin Ġterms Ġof Ġsocial Ġfootprints Ġas Ġfollows : ĠS CC Ġcorrelates Ġwith Ġthe Ġsize Ġof Ġthe Ġdiffusion Ġnetwork , Ġas Ġthe Ġpropagation Ġof Ġnews Ġoccurs Ġin Ġa Ġbroadcast Ġmanner Ġmost Ġof Ġthe Ġtime , Ġi . e . Ġre - t we ets Ġdominate Ġon Ġother Ġinteractions , Ġwhile ĠLS CC Ġallows Ġto Ġdistinguish Ġcases Ġwhere Ġsuch Ġmono - direction ality Ġis Ġsomehow Ġbroken . ĠW CC Ġequals Ġ( approximately ) Ġthe Ġnumber Ġof Ġdistinct Ġdiffusion Ġcasc ades Ġpertaining Ġto Ġeach Ġnews Ġarticle , Ġwith Ġexceptions Ġcorresponding Ġto Ġthose Ġcases Ġwhere Ġsome Ġcasc ades Ġmerge Ġtogether Ġvia ĠTwitter Ġinteractions Ġsuch Ġas Ġmentions , Ġquotes Ġand Ġreplies , Ġand Ġaccordingly ĠLW CC Ġand ĠDW CC Ġequals Ġthe Ġsize Ġand Ġthe Ġdepth Ġof Ġthe Ġlargest Ġcascade . ĠCC Ġcorresponds Ġto Ġthe Ġlevel Ġof Ġconnected ness Ġof Ġneighboring Ġusers Ġin Ġa Ġgiven Ġdiffusion Ġnetwork Ġwhereas ĠKC Ġidentifies Ġthe Ġset Ġof Ġmost Ġinfluential Ġusers Ġin Ġa Ġnetwork Ġand Ġdescribes Ġthe Ġefficiency Ġof Ġinformation Ġspreading ĠB IB REF 17 . ĠFinally , Ġd Ġdescribes Ġthe Ġproportions Ġof Ġpotential Ġconnections Ġbetween Ġusers Ġwhich Ġare Ġactually Ġactivated Ġand ĠSV Ġindicates Ġwhether Ġa Ġnews Ġitem Ġhas Ġgained Ġpopularity Ġwith Ġa Ġsingle Ġand Ġlarge Ġbroadcast Ġor Ġin Ġa Ġmore Ġviral Ġfashion Ġthrough Ġmultiple Ġgenerations . </s> For Ġwhat Ġconcerns Ġdifferent ĠTwitter Ġactions , Ġusers Ġprimarily Ġinteract Ġwith Ġeach Ġother Ġusing Ġret we ets Ġand Ġmentions ĠB IB REF 20 . </s> The Ġformer Ġare Ġthe Ġmain Ġengagement Ġactivity Ġand Ġact Ġas Ġa Ġform Ġof Ġendorsement , Ġallowing Ġusers Ġto Ġreb road cast Ġcontent Ġgenerated Ġby Ġother Ġusers ĠB IB REF 31 . ĠBesides , Ġwhen Ġnode ĠB Ġret we ets Ġnode ĠA Ġwe Ġhave Ġan Ġimplicit Ġconfirmation Ġthat Ġinformation Ġfrom ĠA Ġappeared Ġin ĠB 's ĠTwitter Ġfeed ĠB IB REF 12 . ĠQu otes Ġare Ġsimply Ġa Ġspecial Ġcase Ġof Ġret we ets Ġwith Ġcomments . </s> M ent ions Ġusually Ġinclude Ġpersonal Ġconversations Ġas Ġthey Ġallow Ġsomeone Ġto Ġaddress Ġa Ġspecific Ġuser Ġor Ġto Ġrefer Ġto Ġan Ġindividual Ġin Ġthe Ġthird Ġperson ; Ġin Ġthe Ġfirst Ġcase Ġthey Ġare Ġlocated Ġat Ġthe Ġbeginning Ġof Ġa Ġtweet Ġand Ġthey Ġare Ġknown Ġas Ġreplies , Ġotherwise Ġthey Ġare Ġput Ġin Ġthe Ġbody Ġof Ġa Ġtweet ĠB IB REF 20 . ĠThe Ġnetwork Ġof Ġmentions Ġis Ġusually Ġseen Ġas Ġa Ġstronger Ġversion Ġof Ġinteractions Ġbetween ĠTwitter Ġusers , Ġcompared Ġto Ġthe Ġtraditional Ġgraph Ġof Ġfollower / follow ing Ġrelationships ĠB IB REF 32 . </s> Exper iments Ġ: :: ĠSetup </s> We Ġperformed Ġclassification Ġexperiments Ġusing Ġa Ġbasic Ġoff - the - she lf Ġclass ifier , Ġnamely ĠLog istic ĠReg ression Ġ( LR ) Ġwith ĠL 2 Ġpenalty ; Ġthis Ġalso Ġallows Ġus Ġto Ġcompare Ġresults Ġwith Ġour Ġbaseline . ĠWe Ġapplied Ġa Ġstandard ization Ġof Ġthe Ġfeatures Ġand Ġwe Ġused Ġthe Ġdefault Ġconfiguration Ġfor Ġparameters Ġas Ġdescribed Ġin Ġsc ik it - learn Ġpackage ĠB IB REF 33 . ĠWe Ġalso Ġtested Ġother Ġclass ifiers Ġ( such Ġas ĠK - Ne arest ĠNeigh bors , ĠSupport ĠVector ĠMachines Ġand ĠRandom ĠForest ) Ġbut Ġwe Ġomit Ġresults Ġas Ġthey Ġgive Ġcomparable Ġperformances . ĠWe Ġremark Ġthat Ġour Ġgoal Ġis Ġto Ġshow Ġthat Ġa Ġvery Ġsimple Ġmachine Ġlearning Ġframework , Ġwith Ġno Ġparameter Ġtuning Ġand Ġoptimization , Ġallows Ġfor Ġaccurate Ġresults Ġwith Ġour Ġnetwork - based Ġapproach . </s> We Ġused Ġthe Ġfollowing Ġevaluation Ġmetrics Ġto Ġassess Ġthe Ġperformances Ġof Ġdifferent Ġclass ifiers Ġ( TP = true Ġpositives , ĠFP = false Ġpositives , ĠFN = false Ġnegatives ): </s> Pre cision Ġ= Ġ$\ frac { TP }{ TP + FP } $, Ġthe Ġability Ġof Ġa Ġclass ifier Ġnot Ġto Ġlabel Ġas Ġpositive Ġa Ġnegative Ġsample . </s> Rec all Ġ= Ġ$\ frac { TP }{ TP + FN } $, Ġthe Ġability Ġof Ġa Ġclass ifier Ġto Ġretrieve Ġall Ġpositive Ġsamples . </s> F 1 - score Ġ= Ġ$ 2 Ġ\ frac {\ mb ox { Pre cision } Ġ\ cd ot Ġ\ mb ox { Rec all }} {\ mb ox { Pre cision } Ġ+ Ġ\ mb ox { Rec all }} $, Ġthe Ġharmonic Ġaverage Ġof ĠPrecision Ġand ĠRecall . </s> Area ĠUnder Ġthe ĠReceiver ĠOperating ĠCharacter istic Ġcurve Ġ( A UR OC ); Ġthe ĠReceiver ĠOperating ĠCharacter istic Ġ( R OC ) Ġcurve ĠB IB REF 34 , Ġwhich Ġplots Ġthe ĠTP Ġrate Ġversus Ġthe ĠFP Ġrate , Ġshows Ġthe Ġability Ġof Ġa Ġclass ifier Ġto Ġdiscriminate Ġpositive Ġsamples Ġfrom Ġnegative Ġones Ġas Ġits Ġthreshold Ġis Ġvaried ; Ġthe ĠA UR OC Ġvalue Ġis Ġin Ġthe Ġrange Ġ$ [ 0 , Ġ1 ] $, Ġwith Ġthe Ġrandom Ġbaseline Ġclass ifier Ġholding ĠA UR OC $ = 0 . 5 $ Ġand Ġthe Ġideal Ġperfect Ġclass ifier ĠA UR OC $ = 1 $ ; Ġthus Ġlarger ĠA UR OC Ġvalues Ġ( and Ġst eeper ĠR OC s ) Ġcorrespond Ġto Ġbetter Ġclass ifiers . </s> In Ġparticular Ġwe Ġcomputed Ġso - called Ġmacro Ġaverage âĢĵ simple Ġun weight ed Ġmean âĢĵ of Ġthese Ġmetrics Ġevaluated Ġconsidering Ġboth Ġlabels Ġ( dis information Ġand Ġmainstream ). ĠWe Ġemployed Ġstrat ified Ġshuffle Ġsplit Ġcross Ġvalidation Ġ( with Ġ10 Ġfolds ) Ġto Ġevaluate Ġperformances . </s> Finally , Ġwe Ġpartition ed Ġnetworks Ġaccording Ġto Ġthe Ġtotal Ġnumber Ġof Ġunique Ġusers Ġinvolved Ġin Ġthe Ġsharing , Ġi . e . Ġthe Ġnumber Ġof Ġnodes Ġin Ġthe Ġaggreg ated Ġnetwork Ġrepresented Ġwith Ġa Ġsingle - layer Ġrepresentation Ġconsidering Ġtogether Ġall Ġlayers Ġand Ġalso Ġpure Ġtweets . ĠA Ġbreakdown Ġof Ġboth Ġdatasets Ġaccording Ġto Ġsize Ġclass Ġ( and Ġpolitical Ġbiases Ġfor Ġthe ĠUS Ġscenario ) Ġis Ġprovided Ġin ĠTable Ġ1 Ġand ĠTable Ġ2 . </s> Exper iments Ġ: :: ĠClassification Ġperformances </s> In ĠTable Ġ3 Ġwe Ġfirst Ġprovide Ġclassification Ġperformances Ġon Ġthe ĠUS Ġdataset Ġfor Ġthe ĠLR Ġclass ifier Ġevaluated Ġon Ġthe Ġsize Ġclass Ġdescribed Ġin ĠTable Ġ1 . ĠWe Ġcan Ġobserve Ġthat Ġin Ġall Ġinstances Ġour Ġmethodology Ġperforms Ġbetter Ġthan Ġa Ġrandom Ġclass ifier Ġ( 50 % ĠA UR OC ), Ġwith ĠA UR OC Ġvalues Ġabove Ġ85 % Ġin Ġall Ġcases . </s> For Ġwhat Ġconcerns Ġpolitical Ġbiases , Ġas Ġthe Ġclasses Ġof Ġmainstream Ġand Ġdisinformation Ġnetworks Ġare Ġnot Ġbalanced Ġ( e . g ., Ġ1 , 292 Ġmainstream Ġand Ġ4 , 149 Ġdisinformation Ġnetworks Ġwith Ġright Ġbias ) Ġwe Ġemploy Ġa ĠBalanced ĠRandom ĠForest Ġwith Ġdefault Ġparameters Ġ( as Ġprovided Ġin Ġim ble arn ĠPython Ġpackage ĠB IB REF 35 ). ĠIn Ġorder Ġto Ġtest Ġthe Ġrobust ness Ġof Ġour Ġmethodology , Ġwe Ġtrained Ġonly Ġon Ġleft - biased Ġnetworks Ġor Ġright - biased Ġnetworks Ġand Ġtested Ġon Ġthe Ġentire Ġset Ġof Ġsources Ġ( relative Ġto Ġthe ĠUS Ġdataset ); Ġwe Ġprovide Ġa Ġcomparison Ġof ĠA UR OC Ġvalues Ġfor Ġboth Ġbiases Ġin ĠFigure Ġ4 . ĠWe Ġcan Ġnotice Ġthat Ġour Ġmulti - layer Ġapproach Ġstill Ġentails Ġsignificant Ġresults , Ġthus Ġshowing Ġthat Ġit Ġcan Ġaccurately Ġdistinguish Ġmainstream Ġnews Ġfrom Ġdisinformation Ġregardless Ġof Ġthe Ġpolitical Ġbias . ĠWe Ġfurther Ġcorrobor ated Ġthis Ġresult Ġwith Ġadditional Ġclassification Ġexperiments , Ġthat Ġshow Ġsimilar Ġperformances , Ġin Ġwhich Ġwe Ġexcluded Ġfrom Ġthe Ġtraining / test Ġset Ġtwo Ġspecific Ġsources Ġ( one Ġat Ġa Ġtime Ġand Ġboth Ġat Ġthe Ġsame Ġtime ) Ġthat Ġoutweigh Ġthe Ġothers Ġin Ġterms Ġof Ġdata Ġsamples âĢĵ respect ively Ġ" bre itbart . com " Ġfor Ġright - biased Ġsources Ġand Ġ" polit icus usa . com " Ġfor Ġleft - biased Ġones . </s> We Ġperformed Ġclassification Ġexperiments Ġon Ġthe ĠItalian Ġdataset Ġusing Ġthe ĠLR Ġclass ifier Ġand Ġdifferent Ġsize Ġclasses Ġ( we Ġexcluded Ġ$ [ 1000 , Ġ+ \ in fty Ġ) $ Ġwhich Ġis Ġempty ); Ġwe Ġshow Ġresults Ġfor Ġdifferent Ġevaluation Ġmetrics Ġin ĠTable Ġ3 . ĠWe Ġcan Ġsee Ġthat Ġdespite Ġthe Ġlimited Ġnumber Ġof Ġsamples Ġ( one Ġorder Ġof Ġmagnitude Ġsmaller Ġthan Ġthe ĠUS Ġdataset ) Ġthe Ġperformances Ġare Ġoverall Ġin Ġaccordance Ġwith Ġthe ĠUS Ġscenario . ĠAs Ġshown Ġin ĠTable Ġ4 , Ġwe Ġobtain Ġresults Ġwhich Ġare Ġmuch Ġbetter Ġthan Ġour Ġbaseline Ġin Ġall Ġsize Ġclasses Ġ( see ĠTable Ġ4 ): </s> In Ġthe ĠUS Ġdataset Ġour Ġmulti - layer Ġmethodology Ġperforms Ġmuch Ġbetter Ġin Ġall Ġsize Ġclasses Ġexcept Ġfor Ġlarge Ġnetworks Ġ($ [ 1000 , Ġ+ \ in fty Ġ) $ Ġsize Ġclass ), Ġreaching Ġup Ġto Ġ13 % Ġimprovement Ġon Ġsmaller Ġnetworks Ġ($ [ 0 , Ġ100 ) $ Ġsize Ġclass ); </s> In Ġthe ĠIT Ġdataset Ġour Ġmulti - layer Ġmethodology Ġoutper forms Ġthe Ġbaseline Ġin Ġall Ġsize Ġclasses , Ġwith Ġthe Ġmaximum Ġperformance Ġgain Ġ( 20 %) Ġon Ġmedium Ġnetworks Ġ($ [ 100 , Ġ1000 ) $ Ġsize Ġclass ); Ġthe Ġbaseline Ġgenerally Ġreaches Ġbad Ġperformances Ġcompared Ġto Ġthe ĠUS Ġscenario . </s> Overall , Ġour Ġperformances Ġare Ġcomparable Ġwith Ġthose Ġachieved Ġby Ġtwo Ġstate - of - the - art Ġdeep Ġlearning Ġmodels Ġfor Ġ" fake Ġnews " Ġdetection ĠB IB REF 9 BI B REF 36 . </s> Exper iments Ġ: :: ĠLayer Ġimportance Ġanalysis </s> In Ġorder Ġto Ġunderstand Ġthe Ġimpact Ġof Ġeach Ġlayer Ġon Ġthe Ġperformances Ġof Ġclass ifiers , Ġwe Ġperformed Ġadditional Ġexperiments Ġconsidering Ġseparately Ġeach Ġlayer Ġ( we Ġignored ĠT Ġand ĠU Ġfeatures Ġrelative Ġto Ġpure Ġtweets ). ĠIn ĠTable Ġ5 Ġwe Ġshow Ġmetrics Ġfor Ġeach Ġlayer Ġand Ġall Ġsize Ġclasses , Ġcomputed Ġwith Ġa Ġ10 - fold Ġstrat ified Ġshuffle Ġsplit Ġcross Ġvalidation , Ġevaluated Ġon Ġthe ĠUS Ġdataset ; Ġin ĠFigure Ġ5 Ġwe Ġshow ĠA UR OC Ġvalues Ġfor Ġeach Ġlayer Ġcompared Ġwith Ġthe Ġgeneral Ġmulti - layer Ġapproach . ĠWe Ġcan Ġnotice Ġthat Ġboth ĠQ Ġand ĠM Ġlayers Ġalone Ġcapture Ġadequately Ġthe Ġdiscrepancies Ġof Ġthe Ġtwo Ġdistinct Ġnews Ġdomains Ġin Ġthe ĠUnited ĠStates Ġas Ġthey Ġobtain Ġgood Ġresults Ġwith ĠA UR OC Ġvalues Ġin Ġthe Ġrange Ġ75 %- 86 %; Ġthese Ġare Ġcomparable Ġwith Ġthose Ġof Ġthe Ġmulti - layer Ġapproach Ġwhich , Ġnevertheless , Ġoutper forms Ġthem Ġacross Ġall Ġsize Ġclasses . </s> We Ġobtained Ġsimilar Ġperformances Ġfor Ġthe ĠItalian Ġdataset , Ġas Ġthe ĠM Ġlayer Ġob tains Ġcomparable Ġperformances Ġw . r . t Ġmulti - layer Ġapproach Ġwith ĠA UR OC Ġvalues Ġin Ġthe Ġrange Ġ72 %- 82 %. ĠWe Ġdo Ġnot Ġshow Ġthese Ġresults Ġfor Ġsake Ġof Ġconc is eness . </s> Exper iments Ġ: :: ĠFeature Ġimportance Ġanalysis </s> We Ġfurther Ġinvestigated Ġthe Ġimportance Ġof Ġeach Ġfeature Ġby Ġperforming Ġa Ġ$\ chi Ġ^ 2 $ Ġtest , Ġwith Ġ10 - fold Ġstrat ified Ġshuffle Ġsplit Ġcross Ġvalidation , Ġconsidering Ġthe Ġentire Ġrange Ġof Ġnetwork Ġsizes Ġ$ [ 0 , Ġ+ \ in fty Ġ) $. ĠWe Ġshow Ġthe ĠTop - 5 Ġmost Ġdiscrim in ative Ġfeatures Ġfor Ġeach Ġcountry Ġin ĠTable Ġ6 . </s> We Ġcan Ġnotice Ġthe Ġexact Ġsame Ġset Ġof Ġfeatures Ġ( with Ġdifferent Ġrelative Ġorder ings Ġin Ġthe ĠTop - 3 ) Ġin Ġboth Ġcountries ; Ġthese Ġcorrespond Ġto Ġtwo Ġglobal Ġnetwork Ġproper tie âĢĵ L W CC , Ġwhich Ġindicates Ġthe Ġsize Ġof Ġthe Ġlargest Ġcascade Ġin Ġthe Ġlayer , Ġand ĠS CC , Ġwhich Ġcorrelates Ġwith Ġthe Ġsize Ġof Ġthe Ġnetwork âĢĵ associated Ġto Ġthe Ġsame Ġset Ġof Ġlayers Ġ( Quotes , ĠRet we ets Ġand ĠMent ions ). </s> We Ġfurther Ġperformed Ġa Ġ$\ chi Ġ^ 2 $ Ġtest Ġto Ġhighlight Ġthe Ġmost Ġdiscrim in ative Ġfeatures Ġin Ġthe ĠM Ġlayer Ġof Ġboth Ġcountries , Ġwhich Ġperformed Ġequally Ġwell Ġin Ġthe Ġclassification Ġtask Ġas Ġpreviously Ġhighlighted ; Ġalso Ġin Ġthis Ġcase Ġwe Ġfocused Ġon Ġthe Ġentire Ġrange Ġof Ġnetwork Ġsizes Ġ$ [ 0 , Ġ+ \ in fty Ġ) $. ĠInterestingly , Ġwe Ġdiscovered Ġexactly Ġthe Ġsame Ġset Ġof ĠTop - 3 Ġfeatures Ġin Ġboth Ġcountries , Ġnamely ĠLW CC , ĠS CC Ġand ĠDW CC Ġ( which Ġindicates Ġthe Ġdepth Ġof Ġthe Ġlargest Ġcascade Ġin Ġthe Ġlayer ). </s> An Ġinspection Ġof Ġthe Ġdistributions Ġof Ġall Ġaforementioned Ġfeatures Ġrevealed Ġthat Ġdisinformation Ġnews Ġexhibit Ġon Ġaverage Ġlarger Ġvalues Ġthan Ġmainstream Ġnews . </s> We Ġcan Ġqual itatively Ġsum Ġup Ġthese Ġresults Ġas Ġfollows : </s> Sh aring Ġpatterns Ġin Ġthe Ġtwo Ġnews Ġdomains Ġexhibit Ġdiscrepancies Ġwhich Ġmight Ġbe Ġcountry - independent Ġand Ġdue Ġto Ġthe Ġcontent Ġthat Ġis Ġbeing Ġshared . </s> Inter actions Ġin Ġdisinformation Ġsharing Ġcasc ades Ġtends Ġto Ġbe Ġbroader Ġand Ġdeeper Ġthan Ġin Ġmainstream Ġnews , Ġas Ġwidely Ġreported Ġin Ġthe Ġliterature ĠB IB REF 8 BI B REF 2 BI B REF 7 . </s> Users Ġlikely Ġmake Ġa Ġdifferent Ġusage Ġof Ġmentions Ġwhen Ġsharing Ġnews Ġbelonging Ġto Ġthe Ġtwo Ġdomains , Ġconsequently Ġshaping Ġdifferent Ġsharing Ġpatterns . </s> Exper iments Ġ: :: ĠTem poral Ġanalysis </s> Similar Ġto ĠB IB REF 9 , Ġwe Ġcarried Ġout Ġadditional Ġexperiments Ġto Ġanswer Ġthe Ġfollowing Ġquestion : Ġhow Ġlong Ġdo Ġwe Ġneed Ġto Ġobserve Ġa Ġnews Ġspreading Ġon ĠTwitter Ġin Ġorder Ġto Ġaccurately Ġclassify Ġit Ġas Ġdisinformation Ġor Ġmainstream ? </s> With Ġthis Ġgoal , Ġwe Ġbuilt Ġseveral Ġversions Ġof Ġour Ġoriginal Ġdataset Ġof Ġmulti - layer Ġnetworks Ġby Ġconsidering Ġin Ġturn Ġthe Ġfollowing Ġlif etimes : Ġ1 Ġhour , Ġ6 Ġhours , Ġ12 Ġhours , Ġ1 Ġday , Ġ2 Ġdays , Ġ3 Ġdays Ġand Ġ7 Ġdays ; Ġfor Ġeach Ġcase , Ġwe Ġcomputed Ġthe Ġglobal Ġnetwork Ġproperties Ġof Ġthe Ġcorresponding Ġnetwork Ġand Ġevaluated Ġthe ĠLR Ġclass ifier Ġwith Ġ10 - fold Ġcross Ġvalidation , Ġseparately Ġfor Ġeach Ġlifetime Ġ( and Ġconsidering Ġalways Ġthe Ġentire Ġset Ġof Ġnetworks ). ĠWe Ġshow Ġcorresponding ĠA UR OC Ġvalues Ġfor Ġboth ĠUS Ġand ĠIT Ġdatasets Ġin ĠFigure Ġ6 . </s> We Ġcan Ġsee Ġthat Ġin Ġboth Ġcountries Ġnews Ġdiffusion Ġnetworks Ġcan Ġbe Ġaccurately Ġclassified Ġafter Ġjust Ġa Ġfew Ġhours Ġof Ġspreading , Ġwith ĠA UR OC Ġvalues Ġwhich Ġare Ġlarger Ġthan Ġ80 % Ġafter Ġonly Ġ6 Ġhours Ġof Ġdiffusion . ĠThese Ġresults Ġare Ġvery Ġpromising Ġand Ġsuggest Ġthat Ġarticles Ġpertaining Ġto Ġthe Ġtwo Ġnews Ġdomains Ġexhibit Ġdiscrepancies Ġin Ġtheir Ġsharing Ġpatterns Ġthat Ġcan Ġbe Ġtimely Ġexploited Ġin Ġorder Ġto Ġrapidly Ġdetect Ġmisleading Ġitems Ġfrom Ġfactual Ġinformation . </s> Con clusions </s> In Ġthis Ġwork Ġwe Ġtackled Ġthe Ġproblem Ġof Ġthe Ġautomatic Ġclassification Ġof Ġnews Ġarticles Ġin Ġtwo Ġdomains , Ġnamely Ġmainstream Ġand Ġdisinformation Ġnews , Ġwith Ġa Ġlanguage - independent Ġapproach Ġwhich Ġis Ġbased Ġsolely Ġon Ġthe Ġdiffusion Ġof Ġnews Ġitems Ġon ĠTwitter Ġsocial Ġplatform . ĠWe Ġdis ent angled Ġdifferent Ġtypes Ġof Ġinteractions Ġon ĠTwitter Ġto Ġaccordingly Ġbuild Ġa Ġmulti - layer Ġrepresentation Ġof Ġnews Ġdiffusion Ġnetworks , Ġand Ġwe Ġcomputed Ġa Ġset Ġof Ġglobal Ġnetwork Ġproperties âĢĵ separ ately Ġfor Ġeach Ġlayer âĢĵ in Ġorder Ġto Ġencode Ġeach Ġnetwork Ġwith Ġa Ġtuple Ġof Ġfeatures . ĠOur Ġgoal Ġwas Ġto Ġinvestigate Ġwhether Ġa Ġmulti - layer Ġrepresentation Ġperforms Ġbetter Ġthan Ġone Ġlayer ĠB IB REF 11 , Ġand Ġto Ġunderstand Ġwhich Ġof Ġthe Ġfeatures , Ġobserved Ġat Ġgiven Ġlayers , Ġare Ġmost Ġeffective Ġin Ġthe Ġclassification Ġtask . </s> Exper iments Ġwith Ġan Ġoff - the - she lf Ġclass ifier Ġsuch Ġas ĠLog istic ĠReg ression Ġon Ġdatasets Ġpertaining Ġto Ġtwo Ġdifferent Ġmedia Ġlandscapes Ġ( US Ġand ĠItaly ) Ġyield Ġvery Ġaccurate Ġclassification Ġresults Ġ( A UR OC Ġup Ġto Ġ94 %), Ġeven Ġwhen Ġaccounting Ġfor Ġthe Ġdifferent Ġpolitical Ġbias Ġof Ġnews Ġsources , Ġwhich Ġare Ġfar Ġbetter Ġthan Ġour Ġbaseline ĠB IB REF 11 Ġwith Ġimprovements Ġup Ġto Ġ20 %. ĠClassification Ġperformances Ġusing Ġsingle Ġlayers Ġshow Ġthat Ġthe Ġlayer Ġof Ġmentions Ġalone Ġentails Ġbetter Ġperformance Ġw . r . t Ġother Ġlayers Ġin Ġboth Ġcountries . </s> We Ġalso Ġhighlighted Ġthe Ġmost Ġdiscrim in ative Ġfeatures Ġacross Ġdifferent Ġlayers Ġin Ġboth Ġcountries ; Ġthe Ġresults Ġsuggest Ġthat Ġdifferences Ġbetween Ġthe Ġtwo Ġnews Ġdomains Ġmight Ġbe Ġcountry - independent Ġbut Ġrather Ġdue Ġonly Ġto Ġthe Ġtyp ology Ġof Ġcontent Ġshared , Ġand Ġthat Ġdisinformation Ġnews Ġshape Ġbroader Ġand Ġdeeper Ġcasc ades . </s> Additional Ġexperiments Ġinvolving Ġthe Ġtemporal Ġevolution Ġof ĠTwitter Ġdiffusion Ġnetworks Ġshow Ġthat Ġour Ġmethodology Ġcan Ġaccurate Ġclassify Ġmainstream Ġand Ġdisinformation Ġnews Ġafter Ġa Ġfew Ġhours Ġof Ġpropagation Ġon Ġthe Ġplatform . </s> Overall , Ġour Ġresults Ġprove Ġthat Ġthe Ġtop ological Ġfeatures Ġof Ġmulti - layer Ġdiffusion Ġnetworks Ġmight Ġbe Ġeffectively Ġexploited Ġto Ġdetect Ġonline Ġdisinformation . ĠWe Ġdo Ġnot Ġdeny Ġthe Ġpresence Ġof Ġdeceptive Ġefforts Ġto Ġorche strate Ġthe Ġregular Ġspread Ġof Ġinformation Ġon Ġsocial Ġmedia Ġvia Ġcontent Ġamplification Ġand Ġmanipulation ĠB IB REF 37 BI B REF 38 . ĠOn Ġthe Ġcontrary , Ġwe Ġpost ulate Ġthat Ġsuch Ġhidden Ġforces Ġmight Ġplay Ġto Ġaccent uate Ġthe Ġdiscrepancies Ġbetween Ġthe Ġdiffusion Ġpatterns Ġof Ġdisinformation Ġand Ġmainstream Ġnews Ġ( and Ġthus Ġto Ġmake Ġour Ġmethodology Ġeffective ). </s> In Ġthe Ġfuture Ġwe Ġaim Ġto Ġfurther Ġinvestigate Ġthree Ġdirections : Ġ( 1 ) Ġemploy Ġtemporal Ġnetworks Ġto Ġrepresent Ġnews Ġdiffusion Ġand Ġapply Ġclassification Ġtechniques Ġthat Ġtake Ġinto Ġaccount Ġthe Ġsequential Ġaspect Ġof Ġdata Ġ( e . g . Ġrecurrent Ġneural Ġnetworks ); Ġ( 2 ) Ġcarry Ġout Ġan Ġextensive Ġcomparison Ġof Ġthe Ġdiffusion Ġof Ġdisinformation Ġand Ġmainstream Ġnews Ġacross Ġcountries Ġto Ġinvestigate Ġdeeper Ġthe Ġpresence Ġof Ġdifferences Ġand Ġsimilarities Ġin Ġsharing Ġpatterns ; Ġ( 3 ) Ġleverage Ġour Ġnetwork - based Ġfeatures Ġin Ġaddition Ġto Ġstate - of - the - art Ġtext - based Ġapproaches Ġfor Ġ" fake Ġnews " Ġdet e Ġc tion Ġin Ġorder Ġto Ġdeliver Ġa Ġreal - world Ġsystem Ġto Ġdetect Ġmisleading Ġand Ġharmful Ġinformation Ġspreading Ġon Ġsocial Ġmedia .