Social Honeypot dataset (public) and Weibo dataset (self-collected); yes
ABSTRACTIVE
Social Honeypot, which is not of high quality
ABSTRACTIVE
We use one public dataset Social Honeypot dataset and one self-collected dataset Weibo dataset to validate the effectiveness of our proposed features. Preprocessing: Before directly performing the experiments on the employed datasets, we first delete some accounts with few posts in the two employed since the number of tweets is highly indicative of spammers. For the English Honeypot dataset, we remove stopwords, punctuations, non-ASCII words and apply stemming. For the Chinese Weibo dataset, we perform segmentation with "Jieba", a Chinese text segmentation tool. After preprocessing steps, the Weibo dataset contains 2197 legitimate users and 802 spammers, and the honeypot dataset contains 2218 legitimate users and 2947 spammers. It is worth mentioning that the Honeypot dataset has been slashed because most of the Twitter accounts only have limited number of posts, which are not enough to show their interest inclination.
We use one public dataset Social Honeypot dataset and one self-collected dataset Weibo dataset to validate the effectiveness of our proposed features. Preprocessing: Before directly performing the experiments on the employed datasets, we first delete some accounts with few posts in the two employed since the number of tweets is highly indicative of spammers. For the English Honeypot dataset, we remove stopwords, punctuations, non-ASCII words and apply stemming. For the Chinese Weibo dataset, we perform segmentation with "Jieba", a Chinese text segmentation tool. After preprocessing steps, the Weibo dataset contains 2197 legitimate users and 802 spammers, and the honeypot dataset contains 2218 legitimate users and 2947 spammers. It is worth mentioning that the Honeypot dataset has been slashed because most of the Twitter accounts only have limited number of posts, which are not enough to show their interest inclination. To compare our extracted features with previously used features for spammer detection, we use three most discriminative feature sets according to Lee et al. lee2011seven( 4 ). Two classifiers (Adaboost and SVM) are selected to conduct feature performance comparisons. Using Adaboost, our LOSS+GOSS features outperform all other features except for UFN which is 2% higher than ours with regard to precision on the Honeypot dataset. It is caused by the incorrectly classified spammers who are mostly news source after our manual check. They keep posting all kinds of news pieces covering diverse topics, which is similar to the behavior of fake accounts. However, UFN based on friendship networks is more useful for public accounts who possess large number of followers. The best recall value of our LOSS+GOSS features using SVM is up to 6% higher than the results by other feature groups. Regarding F1-score, our features outperform all other features. To further show the advantages of our proposed features, we compare our combined LOSS+GOSS with the combination of all the features from Lee et al. lee2011seven (UFN+UC+UH). It's obvious that LOSS+GOSS have a great advantage over UFN+UC+UH in terms of recall and F1-score. Moreover, by combining our LOSS+GOSS features and UFN+UC+UH features together, we obtained another 7.1% and 2.3% performance gain with regard to precision and F1-score by Adaboost. Though there is a slight decline in terms of recall. By SVM, we get comparative results on recall and F1-score but about 3.5% improvement on precision.
1604.08504
Introduction </s> Micro blog ging Ġsuch Ġas ĠTwitter Ġand ĠWe ibo Ġis Ġa Ġpopular Ġsocial Ġnetworking Ġservice , Ġwhich Ġallows Ġusers Ġto Ġpost Ġmessages Ġup Ġto Ġ140 Ġcharacters . ĠThere Ġare Ġmillions Ġof Ġactive Ġusers Ġon Ġthe Ġplatform Ġwho Ġstay Ġconnected Ġwith Ġfriends . ĠUnfortunately , Ġsp ammers Ġalso Ġuse Ġit Ġas Ġa Ġtool Ġto Ġpost Ġmalicious Ġlinks , Ġsend Ġuns olicited Ġmessages Ġto Ġlegitimate Ġusers , Ġetc . ĠA Ġcertain Ġamount Ġof Ġsp ammers Ġcould Ġsway Ġthe Ġpublic Ġopinion Ġand Ġcause Ġdistrust Ġof Ġthe Ġsocial Ġplatform . ĠDespite Ġthe Ġuse Ġof Ġrigid Ġanti - sp am Ġrules , Ġhuman - like Ġsp ammers Ġwhose Ġhome pages Ġhaving Ġphotos , Ġdetailed Ġprofiles Ġetc . Ġhave Ġemerged . ĠUnlike Ġprevious Ġ" simple " Ġsp ammers , Ġwhose Ġtweets Ġcontain Ġonly Ġmalicious Ġlinks , Ġthose Ġ" smart " Ġsp ammers Ġare Ġmore Ġdifficult Ġto Ġdistinguish Ġfrom Ġlegitimate Ġusers Ġvia Ġcontent - based Ġfeatures Ġalone ĠB IB REF 0 Ġ. </s> There Ġis Ġa Ġconsiderable Ġamount Ġof Ġprevious Ġwork Ġon Ġspam mer Ġdetection Ġon Ġsocial Ġplatforms . ĠRes earcher Ġfrom ĠTwitter ĠInc . ĠB IB REF 1 Ġcollect Ġbot Ġaccounts Ġand Ġperform Ġanalysis Ġon Ġthe Ġuser Ġbehavior Ġand Ġuser Ġprofile Ġfeatures . ĠLee Ġet Ġal . Ġle e 2011 seven Ġuse Ġthe Ġso - called Ġsocial Ġhoney pot Ġby Ġall uring Ġsocial Ġsp ammers ' Ġretweet Ġto Ġbuild Ġa Ġbenchmark Ġdataset , Ġwhich Ġhas Ġbeen Ġextensively Ġexplored Ġin Ġour Ġpaper . ĠSome Ġresearchers Ġfocus Ġon Ġthe Ġclust ering Ġof Ġur ls Ġin Ġtweets Ġand Ġnetwork Ġgraph Ġof Ġsocial Ġsp ammers ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, Ġshowing Ġthe Ġpower Ġof Ġsocial Ġrelationship Ġfeatures . As Ġfor Ġcontent Ġinformation Ġmodeling , ĠB IB REF 6 Ġapply Ġimproved Ġsparse Ġlearning Ġmethods . ĠHowever , Ġfew Ġstudies Ġhave Ġadopted Ġtopic - based Ġfeatures . ĠSome Ġresearchers ĠB IB REF 7 Ġdiscuss Ġtopic Ġcharacteristics Ġof Ġspam ming Ġposts , Ġindicating Ġthat Ġsp ammers Ġare Ġhighly Ġlikely Ġto Ġdwell Ġon Ġsome Ġcertain Ġtopics Ġsuch Ġas Ġpromotion . ĠBut Ġthis Ġmay Ġnot Ġbe Ġapplicable Ġto Ġthe Ġcurrent Ġscenario Ġof Ġsmart Ġsp ammers . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġan Ġefficient Ġfeature Ġextraction Ġmethod . ĠIn Ġthis Ġmethod , Ġtwo Ġnew Ġtopic - based Ġfeatures Ġare Ġextracted Ġand Ġused Ġto Ġdiscriminate Ġhuman - like Ġsp ammers Ġfrom Ġlegitimate Ġusers . ĠWe Ġconsider Ġthe Ġhistorical Ġtweets Ġof Ġeach Ġuser Ġas Ġa Ġdocument Ġand Ġuse Ġthe ĠLat ent ĠDir ich let ĠAll ocation Ġ( L DA ) Ġmodel Ġto Ġcompute Ġthe Ġtopic Ġdistribution Ġfor Ġeach Ġuser . ĠBased Ġon Ġthe Ġcalculated Ġtopic Ġprobability , Ġtwo Ġtopic - based Ġfeatures , Ġthe ĠLocal ĠOut lier ĠStandard ĠScore Ġ( L OSS ) Ġwhich Ġcaptures Ġthe Ġuser 's Ġinterests Ġon Ġdifferent Ġtopics Ġand Ġthe ĠGlobal ĠOut lier ĠStandard ĠScore Ġ( G OSS ) Ġwhich Ġreveals Ġthe Ġuser 's Ġinterests Ġon Ġspecific Ġtopic Ġin Ġcomparison Ġwith Ġother Ġusers ', Ġare Ġextracted . ĠThe Ġtwo Ġfeatures Ġcontain Ġboth Ġlocal Ġand Ġglobal Ġinformation , Ġand Ġthe Ġcombination Ġof Ġthem Ġcan Ġdistinguish Ġhuman - like Ġsp ammers Ġeffectively . </s> To Ġthe Ġbest Ġof Ġour Ġknowledge , Ġit Ġis Ġthe Ġfirst Ġtime Ġthat Ġfeatures Ġbased Ġon Ġtopic Ġdistributions Ġare Ġused Ġin Ġspam mer Ġclassification . ĠExperimental Ġresults Ġon Ġone Ġpublic Ġdataset Ġand Ġone Ġself - col lected Ġdataset Ġfurther Ġvalidate Ġthat Ġthe Ġtwo Ġsets Ġof Ġextracted Ġtopic - based Ġfeatures Ġget Ġexcellent Ġperformance Ġon Ġhuman - like Ġspam mer Ġclassification Ġproblem Ġcompared Ġwith Ġother Ġstate - of - the - art Ġmethods . ĠIn Ġaddition , Ġwe Ġbuild Ġa ĠWe ibo Ġdataset , Ġwhich Ġcontains Ġboth Ġlegitimate Ġusers Ġand Ġsp ammers . </s> To Ġsummarize , Ġour Ġmajor Ġcontributions Ġare Ġtwo - fold : </s> In Ġthe Ġfollowing Ġsections , Ġwe Ġfirst Ġpropose Ġthe Ġtopic - based Ġfeatures Ġextraction Ġmethod Ġin ĠSection Ġ2 , Ġand Ġthen Ġintroduce Ġthe Ġtwo Ġdatasets Ġin ĠSection Ġ3 . ĠExperimental Ġresults Ġare Ġdiscussed Ġin ĠSection Ġ4 , Ġand Ġwe Ġconclude Ġthe Ġpaper Ġin ĠSection Ġ5 . ĠFuture Ġwork Ġis Ġpresented Ġin ĠSection Ġ6 . </s> Method ology </s> In Ġthis Ġsection , Ġwe Ġfirst Ġprovide Ġsome Ġobservations Ġwe Ġobtained Ġafter Ġcarefully Ġexploring Ġthe Ġsocial Ġnetwork , Ġthen Ġthe ĠL DA Ġmodel Ġis Ġintroduced . ĠBased Ġon Ġthe ĠL DA Ġmodel , Ġthe Ġways Ġto Ġobtain Ġthe Ġtopic Ġprobability Ġvector Ġfor Ġeach Ġuser Ġand Ġthe Ġtwo Ġtopic - based Ġfeatures Ġare Ġprovided . </s> Obs ervation </s> After Ġexploring Ġthe Ġhome pages Ġof Ġa Ġsubstantial Ġnumber Ġof Ġsp ammers , Ġwe Ġhave Ġtwo Ġobservations . Ġ1 ) Ġsocial Ġsp ammers Ġcan Ġbe Ġdivided Ġinto Ġtwo Ġcategories . ĠOne Ġis Ġcontent Ġpoll uters , Ġand Ġtheir Ġtweets Ġare Ġall Ġabout Ġcertain Ġkinds Ġof Ġadvertisement Ġand Ġcampaign . ĠThe Ġother Ġis Ġfake Ġaccounts , Ġand Ġtheir Ġtweets Ġresemble Ġlegitimate Ġusers ' Ġbut Ġit Ġseems Ġthey Ġare Ġsimply Ġrandom Ġcopies Ġof Ġothers Ġto Ġavoid Ġbeing Ġdetected Ġby Ġanti - sp am Ġrules . Ġ2 ) ĠFor Ġlegitimate Ġusers , Ġcontent Ġpoll uters Ġand Ġfake Ġaccounts , Ġthey Ġshow Ġdifferent Ġpatterns Ġon Ġtopics Ġwhich Ġinterest Ġthem . </s> Leg itimate Ġusers Ġmainly Ġfocus Ġon Ġlimited Ġtopics Ġwhich Ġinterest Ġhim . ĠThey Ġseldom Ġpost Ġcontents Ġunrelated Ġto Ġtheir Ġconcern . </s> Content Ġpoll uters Ġconcentrate Ġon Ġcertain Ġtopics . </s> Fake Ġaccounts Ġfocus Ġon Ġa Ġwide Ġrange Ġof Ġtopics Ġdue Ġto Ġrandom Ġcopying Ġand Ġretweet ing Ġof Ġother Ġusers ' Ġtweets . </s> Sp ammers Ġand Ġlegitimate Ġusers Ġshow Ġdifferent Ġinterests Ġon Ġsome Ġtopics Ġe . g . Ġcommercial , Ġweather , Ġetc . </s> To Ġbetter Ġillustrate Ġour Ġobservation , ĠFigure . Ġ1 Ġshows Ġthe Ġtopic Ġdistribution Ġof Ġsp ammers Ġand Ġlegitimate Ġusers Ġin Ġtwo Ġemployed Ġdatasets ( the ĠHoney pot Ġdataset Ġand ĠWe ibo Ġdataset ). ĠWe Ġcan Ġsee Ġthat Ġon Ġboth Ġtopics Ġ( topic - 3 Ġand Ġtopic - 11 ) Ġthere Ġexists Ġobvious Ġdifference Ġbetween Ġthe Ġred Ġbars Ġand Ġgreen Ġbars , Ġrepresenting Ġsp ammers Ġand Ġlegitimate Ġusers . ĠOn Ġthe ĠHoney pot Ġdataset , Ġsp ammers Ġhave Ġa Ġnarrower Ġshape Ġof Ġdistribution Ġ( the Ġoutl iers Ġon Ġthe Ġred Ġbar Ġtail Ġare Ġnot Ġcounted ) Ġthan Ġthat Ġof Ġlegitimate Ġusers . ĠThis Ġis Ġbecause Ġthere Ġare Ġmore Ġcontent Ġpoll uters Ġthan Ġfake Ġaccounts . ĠIn Ġother Ġword , Ġsp ammers Ġin Ġthis Ġdataset Ġtend Ġto Ġconcentrate Ġon Ġlimited Ġtopics . ĠWhile Ġon Ġthe ĠWe ibo Ġdataset , Ġfake Ġaccounts Ġwho Ġare Ġinterested Ġin Ġdifferent Ġtopics Ġtake Ġlarge Ġproportion Ġof Ġsp ammers . ĠTheir Ġdistribution Ġis Ġmore Ġflat Ġ( i . e . Ġred Ġbars ) Ġthan Ġthat Ġof Ġthe Ġlegitimate Ġusers . ĠTherefore Ġwe Ġcan Ġdetect Ġsp ammers Ġby Ġmeans Ġof Ġthe Ġdifference Ġof Ġtheir Ġtopic Ġdistribution Ġpatterns . </s> L DA Ġmodel </s> Ble i Ġet Ġal . ble i 2003 lat ent Ġfirst Ġpresented ĠLat ent ĠDir ich let ĠAll ocation ( L DA ) Ġas Ġan Ġexample Ġof Ġtopic Ġmodel . </s> Each Ġdocument Ġ$ i $ Ġis Ġdeemed Ġas Ġa Ġbag Ġof Ġwords Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ Ġand Ġ$ M $ Ġis Ġthe Ġnumber Ġof Ġwords . ĠEach Ġword Ġis Ġattributable Ġto Ġone Ġof Ġthe Ġdocument 's Ġtopics Ġ$ Z = \ left \ l brace Ġ Ġz _{ i 1 }, z _{ i 2 }, ... , z _{ i K }\ right \ r brace Ġ$ Ġand Ġ$ K $ Ġis Ġthe Ġnumber Ġof Ġtopics . Ġ$\ ps i Ġ_ { k } $ Ġis Ġa Ġmult in omial Ġdistribution Ġover Ġwords Ġfor Ġtopic Ġ$ k $ Ġ. Ġ$\ the ta Ġ_ i $ Ġis Ġanother Ġmult in omial Ġdistribution Ġover Ġtopics Ġfor Ġdocument Ġ$ i $ Ġ. ĠThe Ġsmoot hed Ġgener ative Ġmodel Ġis Ġillustrated Ġin ĠFigure . Ġ2 Ġ. Ġ$\ alpha Ġ$ Ġand Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 0 Ġare Ġhyper Ġparameter Ġthat Ġaffect Ġscarcity Ġof Ġthe Ġdocument - topic Ġand Ġtopic - word Ġdistributions . ĠIn Ġthis Ġpaper , Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 1 Ġ, Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 2 Ġand Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 3 Ġare Ġempir ically Ġset Ġto Ġ0 . 3 , Ġ0 . 01 Ġand Ġ15 . ĠThe Ġentire Ġcontent Ġof Ġeach ĠTwitter Ġuser Ġis Ġregarded Ġas Ġone Ġdocument . ĠWe Ġadopt ĠGibbs ĠSam pling ĠB IB REF 8 Ġto Ġspeed Ġup Ġthe Ġinference Ġof ĠL DA . ĠBased Ġon ĠL DA , Ġwe Ġcan Ġget Ġthe Ġtopic Ġprobabilities Ġfor Ġall Ġusers Ġin Ġthe Ġemployed Ġdataset Ġas : Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 4 Ġ, Ġwhere Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 5 Ġis Ġthe Ġnumber Ġof Ġusers . ĠEach Ġelement Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 6 Ġis Ġa Ġtopic Ġprobability Ġvector Ġfor Ġthe Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 7 Ġdocument . Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 8 Ġis Ġthe Ġraw Ġtopic Ġprobability Ġvector Ġand Ġour Ġfeatures Ġare Ġdeveloped Ġon Ġtop Ġof Ġit . </s> Topic - based ĠFeatures </s> Using Ġthe ĠL DA Ġmodel , Ġeach Ġperson Ġin Ġthe Ġdataset Ġis Ġwith Ġa Ġtopic Ġprobability Ġvector Ġ$ X _ i $ Ġ. ĠAss ume Ġ$ x _{ ik }\ in ĠX _{ i } $ Ġdenotes Ġthe Ġlikelihood Ġthat Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġtweet Ġaccount Ġfavors Ġ$\ em ph Ġ{ k } ^{ th } $ Ġtopic Ġin Ġthe Ġdataset . ĠOur Ġtopic Ġbased Ġfeatures Ġcan Ġbe Ġcalculated Ġas Ġbelow . </s> Global ĠOut lier ĠStandard ĠScore Ġmeasures Ġthe Ġdegree Ġthat Ġa Ġuser 's Ġtweet Ġcontent Ġis Ġrelated Ġto Ġa Ġcertain Ġtopic Ġcompared Ġto Ġthe Ġother Ġusers . ĠSpecifically , Ġthe Ġ" G OSS " Ġscore Ġof Ġuser Ġ$ i $ Ġon Ġtopic Ġ$ k $ Ġcan Ġbe Ġcalculated Ġas ĠE q .( Ġ12 Ġ): </s> $$ \ cent ering Ġ\ begin { array }{ ll } Ġ\ mu Ġ\ left ( x _{ k }\ right )= \ frac {\ sum Ġ_ { i = 1 } ^{ n } Ġx _{ ik }} { n }, \\ ĠG OSS \ left ( x _{ ik }\ right )= \ frac { x _{ ik } - \ mu Ġ\ left ( x _ k \ right ) }{ \ sq rt {\ unders et { i }{ \ sum Ġ} \ left ( x _{ ik } - \ mu Ġ\ left ( x _{ k }\ right )\ right ) ^{ 2 }} }. Ġ\ end { array } $$ Ġ Ġ Ġ( E q . Ġ12 ) </s> The Ġvalue Ġof Ġ$ G OSS \ left ( x _{ ik }\ right ) $ Ġindicates Ġthe Ġinteresting Ġdegree Ġof Ġthis Ġperson Ġto Ġthe Ġ$\ em ph Ġ{ k } ^{ th } $ Ġtopic . ĠSpecifically , Ġif Ġ$ G OSS \ left ( x _{ ik }\ right ) $ Ġ> Ġ$ G OSS \ left ( x _{ j k }\ right ) $ Ġ, Ġit Ġmeans Ġthat Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson Ġhas Ġmore Ġinterest Ġin Ġtopic Ġ$ k $ Ġthan Ġthe Ġ$\ em ph Ġ{ j } ^{ th } $ Ġperson . ĠIf Ġthe Ġvalue Ġ$ G OSS \ left ( x _{ ik }\ right ) $ Ġis Ġextremely Ġhigh Ġor Ġlow , Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson Ġshowing Ġextreme Ġinterest Ġor Ġno Ġinterest Ġon Ġtopic Ġ$ k $ Ġwhich Ġwill Ġprobably Ġbe Ġa Ġdistinctive Ġpattern Ġin Ġthe Ġf ow ll owing Ġclass f ication . ĠTherefore , Ġthe Ġtopics Ġinterested Ġor Ġdisliked Ġby Ġthe Ġ$\ em ph Ġ{ k } ^{ th } $ 0 Ġperson Ġcan Ġbe Ġmanifested Ġby Ġ$\ em ph Ġ{ k } ^{ th } $ 1 Ġ, Ġfrom Ġwhich Ġthe Ġpattern Ġof Ġthe Ġinterested Ġtopics Ġwith Ġregarding Ġto Ġthis Ġperson Ġis Ġfound . ĠDen ote Ġ$\ em ph Ġ{ k } ^{ th } $ 2 Ġour Ġfirst Ġtopic - based Ġfeature , Ġand Ġit Ġhopefully Ġcan Ġget Ġgood Ġperformance Ġon Ġspam mer Ġdetection . </s> Local ĠOut lier ĠStandard ĠScore Ġmeasures Ġthe Ġdegree Ġof Ġinterest Ġsomeone Ġshows Ġto Ġa Ġcertain Ġtopic Ġby Ġconsidering Ġhis Ġown Ġhomepage Ġcontent Ġonly . ĠFor Ġinstance , Ġthe Ġ" L OSS " Ġscore Ġof Ġaccount Ġ$ i $ Ġon Ġtopic Ġ$ k $ Ġcan Ġbe Ġcalculated Ġas ĠE q .( Ġ13 Ġ): </s> $$ \ cent ering Ġ\ begin { array }{ ll } Ġ\ mu Ġ\ left ( x _{ i }\ right )= \ frac {\ sum Ġ_ { k = 1 } ^{ K } Ġx _{ ik }} { K }, \\ ĠL OSS \ left ( x _{ ik }\ right )= \ frac { x _{ ik } - \ mu Ġ\ left ( x _ i \ right ) }{ \ sq rt {\ unders et { k }{ \ sum Ġ} \ left ( x _{ ik } - \ mu Ġ\ left ( x _{ i }\ right )\ right ) ^{ 2 }} }. Ġ\ end { array } $$ Ġ Ġ Ġ( E q . Ġ13 ) </s> $ \ mu Ġ( x _ i ) $ Ġrepresents Ġthe Ġaveraged Ġinteresting Ġdegree Ġfor Ġall Ġtopics Ġwith Ġregarding Ġto Ġ$\ em ph Ġ{ i } ^{ th } $ Ġuser Ġand Ġhis Ġtweet Ġcontent . ĠSimilarly Ġto Ġ$ G OSS $ Ġ, Ġthe Ġtopics Ġinterested Ġor Ġdisliked Ġby Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson Ġvia Ġconsidering Ġhis Ġsingle Ġpost Ġinformation Ġcan Ġbe Ġmanifested Ġby Ġ$ f _{ L OSS } ^{ i } =[ L OSS ( x _{ i 1 } )\ cd ots ĠL OSS ( x _{ i K } )] $ Ġ, Ġand Ġ$ L OSS $ Ġbecomes Ġour Ġsecond Ġtopic - based Ġfeatures Ġfor Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson . </s> Dat as et </s> We Ġuse Ġone Ġpublic Ġdataset ĠSocial ĠHoney pot Ġdataset Ġand Ġone Ġself - col lected Ġdataset ĠWe ibo Ġdataset Ġto Ġvalidate Ġthe Ġeffectiveness Ġof Ġour Ġproposed Ġfeatures . </s> Social ĠHoney pot ĠDat as et : ĠLee Ġet Ġal . Ġle e 2010 dev ils Ġcreated Ġand Ġdeployed Ġ60 Ġseed Ġsocial Ġaccounts Ġon ĠTwitter Ġto Ġattract Ġsp ammers Ġby Ġreporting Ġback Ġwhat Ġaccounts Ġinteract Ġwith Ġthem . ĠThey Ġcollected Ġ19 , 276 Ġlegitimate Ġusers Ġand Ġ22 , 223 Ġsp ammers Ġin Ġtheir Ġdatasets Ġalong Ġwith Ġtheir Ġtweet Ġcontent Ġin Ġ7 Ġmonths . ĠThis Ġis Ġour Ġfirst Ġtest Ġdataset . </s> Our ĠWe ibo ĠDat as et : ĠSina ĠWe ibo Ġis Ġone Ġof Ġthe Ġmost Ġfamous Ġsocial Ġplatforms Ġin ĠChina . ĠIt Ġhas Ġimplemented Ġmany Ġfeatures Ġfrom ĠTwitter . ĠThe Ġ2 197 Ġlegitimate Ġuser Ġaccounts Ġin Ġthis Ġdataset Ġare Ġprovided Ġby Ġthe ĠTian chi ĠCompetition Ġheld Ġby ĠSina ĠWe ibo . ĠThe Ġsp ammers Ġare Ġall Ġpurchased Ġcommercially Ġfrom Ġmultiple Ġvendors Ġon Ġthe ĠInternet . ĠWe Ġchecked Ġthem Ġmanually Ġand Ġcollected Ġ802 Ġsuitable Ġ" smart " Ġsp ammers Ġaccounts . </s> Pre processing : ĠBefore Ġdirectly Ġperforming Ġthe Ġexperiments Ġon Ġthe Ġemployed Ġdatasets , Ġwe Ġfirst Ġdelete Ġsome Ġaccounts Ġwith Ġfew Ġposts Ġin Ġthe Ġtwo Ġemployed Ġsince Ġthe Ġnumber Ġof Ġtweets Ġis Ġhighly Ġindicative Ġof Ġsp ammers . ĠFor Ġthe ĠEnglish ĠHoney pot Ġdataset , Ġwe Ġremove Ġstop words , Ġpunct uations , Ġnon - ASC II Ġwords Ġand Ġapply Ġstemming . ĠFor Ġthe ĠChinese ĠWe ibo Ġdataset , Ġwe Ġperform Ġsegment ation Ġwith Ġ" J ie ba ", Ġa ĠChinese Ġtext Ġsegment ation Ġtool . ĠAfter Ġpre processing Ġsteps , Ġthe ĠWe ibo Ġdataset Ġcontains Ġ2 197 Ġlegitimate Ġusers Ġand Ġ802 Ġsp ammers , Ġand Ġthe Ġhoney pot Ġdataset Ġcontains Ġ22 18 Ġlegitimate Ġusers Ġand Ġ29 47 Ġsp ammers . ĠIt Ġis Ġworth Ġmentioning Ġthat Ġthe ĠHoney pot Ġdataset Ġhas Ġbeen Ġslashed Ġbecause Ġmost Ġof Ġthe ĠTwitter Ġaccounts Ġonly Ġhave Ġlimited Ġnumber Ġof Ġposts , Ġwhich Ġare Ġnot Ġenough Ġto Ġshow Ġtheir Ġinterest Ġinclination . </s> E val uation ĠMet rics </s> The Ġevaluating Ġindicators Ġin Ġour Ġmodel Ġare Ġshow Ġin Ġ2 Ġ. ĠWe Ġcalculate Ġprecision , Ġrecall Ġand ĠF 1 - score Ġ( i . e . ĠF 1 Ġscore ) Ġas Ġin ĠE q . Ġ( Ġ19 Ġ). ĠPrecision Ġis Ġthe Ġratio Ġof Ġselected Ġaccounts Ġthat Ġare Ġsp ammers . ĠRecall Ġis Ġthe Ġratio Ġof Ġsp ammers Ġthat Ġare Ġdetected Ġso . ĠF 1 - score Ġis Ġthe Ġharmonic Ġmean Ġof Ġprecision Ġand Ġrecall . </s> $$ pre cision Ġ= \ frac { TP }{ TP + FP }, Ġrecall Ġ= \ frac { TP }{ TP + FN }\ non umber Ġ\\ ĠF 1 - score Ġ= Ġ\ frac { 2 \ times Ġprecision Ġ\ times Ġrecall }{ pre cision Ġ+ Ġrecall } $$ Ġ Ġ Ġ( E q . Ġ19 ) </s> Performance ĠCompar isons Ġwith ĠBas eline </s> Three Ġbaseline Ġclassification Ġmethods : ĠSupport ĠVector ĠMachines Ġ( S VM ), ĠAd abo ost , Ġand ĠRandom ĠFore sts Ġare Ġadopted Ġto Ġevaluate Ġour Ġextracted Ġfeatures . ĠWe Ġtest Ġeach Ġclassification Ġalgorithm Ġwith Ġsc ik it - learn ĠB IB REF 9 Ġand Ġrun Ġa Ġ10 - fold Ġcross Ġvalidation . ĠOn Ġeach Ġdataset , Ġthe Ġemployed Ġclass ifiers Ġare Ġtrained Ġwith Ġindividual Ġfeature Ġfirst , Ġand Ġthen Ġwith Ġthe Ġcombination Ġof Ġthe Ġtwo Ġfeatures . ĠFrom Ġ1 Ġ, Ġwe Ġcan Ġsee Ġthat ĠG OSS + L OSS Ġachieves Ġthe Ġbest Ġperformance Ġon ĠF 1 - score Ġamong Ġall Ġothers . ĠBesides , Ġthe Ġclassification Ġby Ġcombination Ġof ĠL OSS Ġand ĠG OSS Ġcan Ġincrease Ġaccuracy Ġby Ġmore Ġthan Ġ3 % Ġcompared Ġwith Ġraw Ġtopic Ġdistribution Ġprobability . </s> Compar ison Ġwith ĠOther ĠFeatures </s> To Ġcompare Ġour Ġextracted Ġfeatures Ġwith Ġpreviously Ġused Ġfeatures Ġfor Ġspam mer Ġdetection , Ġwe Ġuse Ġthree Ġmost Ġdiscrim in ative Ġfeature Ġsets Ġaccording Ġto ĠLee Ġet Ġal . Ġle e 2011 seven ( Ġ4 Ġ). ĠTwo Ġclass ifiers Ġ( Ad abo ost Ġand ĠS VM ) Ġare Ġselected Ġto Ġconduct Ġfeature Ġperformance Ġcomparisons . ĠUsing ĠAd abo ost , Ġour ĠL OSS + G OSS Ġfeatures Ġoutper form Ġall Ġother Ġfeatures Ġexcept Ġfor ĠU FN Ġwhich Ġis Ġ2 % Ġhigher Ġthan Ġours Ġwith Ġregard Ġto Ġprecision Ġon Ġthe ĠHoney pot Ġdataset . ĠIt Ġis Ġcaused Ġby Ġthe Ġincorrectly Ġclassified Ġsp ammers Ġwho Ġare Ġmostly Ġnews Ġsource Ġafter Ġour Ġmanual Ġcheck . ĠThey Ġkeep Ġposting Ġall Ġkinds Ġof Ġnews Ġpieces Ġcovering Ġdiverse Ġtopics , Ġwhich Ġis Ġsimilar Ġto Ġthe Ġbehavior Ġof Ġfake Ġaccounts . ĠHowever , ĠU FN Ġbased Ġon Ġfriendship Ġnetworks Ġis Ġmore Ġuseful Ġfor Ġpublic Ġaccounts Ġwho Ġpossess Ġlarge Ġnumber Ġof Ġfollowers . ĠThe Ġbest Ġrecall Ġvalue Ġof Ġour ĠL OSS + G OSS Ġfeatures Ġusing ĠS VM Ġis Ġup Ġto Ġ6 % Ġhigher Ġthan Ġthe Ġresults Ġby Ġother Ġfeature Ġgroups . ĠRegarding ĠF 1 - score , Ġour Ġfeatures Ġoutper form Ġall Ġother Ġfeatures . ĠTo Ġfurther Ġshow Ġthe Ġadvantages Ġof Ġour Ġproposed Ġfeatures , Ġwe Ġcompare Ġour Ġcombined ĠL OSS + G OSS Ġwith Ġthe Ġcombination Ġof Ġall Ġthe Ġfeatures Ġfrom ĠLee Ġet Ġal . Ġle e 2011 seven Ġ( UF N + UC + U H ). ĠIt 's Ġobvious Ġthat ĠL OSS + G OSS Ġhave Ġa Ġgreat Ġadvantage Ġover ĠU FN + UC + U H Ġin Ġterms Ġof Ġrecall Ġand ĠF 1 - score . ĠMoreover , Ġby Ġcombining Ġour ĠL OSS + G OSS Ġfeatures Ġand ĠU FN + UC + U H Ġfeatures Ġtogether , Ġwe Ġobtained Ġanother Ġ7 . 1 % Ġand Ġ2 . 3 % Ġperformance Ġgain Ġwith Ġregard Ġto Ġprecision Ġand ĠF 1 - score Ġby ĠAd abo ost . ĠThough Ġthere Ġis Ġa Ġslight Ġdecline Ġin Ġterms Ġof Ġrecall . ĠBy ĠS VM , Ġwe Ġget Ġcomparative Ġresults Ġon Ġrecall Ġand ĠF 1 - score Ġbut Ġabout Ġ3 . 5 % Ġimprovement Ġon Ġprecision . </s> Conclusion </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġnovel Ġfeature Ġextraction Ġmethod Ġto Ġeffectively Ġdetect Ġ" smart " Ġsp ammers Ġwho Ġpost Ġseemingly Ġlegitimate Ġtweets Ġand Ġare Ġthus Ġdifficult Ġto Ġidentify Ġby Ġexisting Ġspam mer Ġclassification Ġmethods . ĠUsing Ġthe ĠL DA Ġmodel , Ġwe Ġobtain Ġthe Ġtopic Ġprobability Ġfor Ġeach ĠTwitter Ġuser . ĠBy Ġutilizing Ġthe Ġtopic Ġprobability Ġresult , Ġwe Ġextract Ġour Ġtwo Ġtopic - based Ġfeatures : ĠG OSS Ġand ĠL OSS Ġwhich Ġrepresent Ġthe Ġaccount Ġwith Ġglobal Ġand Ġlocal Ġinformation . ĠExperimental Ġresults Ġon Ġa Ġpublic Ġdataset Ġand Ġa Ġself - built ĠChinese Ġmicro blog Ġdataset Ġvalidate Ġthe Ġeffectiveness Ġof Ġthe Ġproposed Ġfeatures . </s> Future ĠWork </s> In Ġfuture Ġwork , Ġthe Ġcombination Ġmethod Ġof Ġlocal Ġand Ġglobal Ġinformation Ġcan Ġbe Ġfurther Ġimproved Ġto Ġmaximize Ġtheir Ġindividual Ġstrengths . ĠWe Ġwill Ġalso Ġapply Ġdecision Ġtheory Ġto Ġenhancing Ġthe Ġperformance Ġof Ġour Ġproposed Ġfeatures . ĠMoreover , Ġwe Ġare Ġalso Ġbuilding Ġlarger Ġdatasets Ġon Ġboth ĠTwitter Ġand ĠWe ibo Ġto Ġvalidate Ġour Ġmethod . ĠMoreover , Ġlarger Ġdatasets Ġon Ġboth ĠTwitter Ġand ĠWe ibo Ġwill Ġbe Ġbuilt Ġto Ġfurther Ġvalidate Ġour Ġmethod .
Introduction Microblogging such as Twitter and Weibo is a popular social networking service, which allows users to post messages up to 140 characters. There are millions of active users on the platform who stay connected with friends. Unfortunately, spammers also use it as a tool to post malicious links, send unsolicited messages to legitimate users, etc. A certain amount of spammers could sway the public opinion and cause distrust of the social platform. Despite the use of rigid anti-spam rules, human-like spammers whose homepages having photos, detailed profiles etc. have emerged. Unlike previous "simple" spammers, whose tweets contain only malicious links, those "smart" spammers are more difficult to distinguish from legitimate users via content-based features alone BIBREF0 . There is a considerable amount of previous work on spammer detection on social platforms. Researcher from Twitter Inc. BIBREF1 collect bot accounts and perform analysis on the user behavior and user profile features. Lee et al. lee2011seven use the so-called social honeypot by alluring social spammers' retweet to build a benchmark dataset, which has been extensively explored in our paper. Some researchers focus on the clustering of urls in tweets and network graph of social spammers BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , showing the power of social relationship features.As for content information modeling, BIBREF6 apply improved sparse learning methods. However, few studies have adopted topic-based features. Some researchers BIBREF7 discuss topic characteristics of spamming posts, indicating that spammers are highly likely to dwell on some certain topics such as promotion. But this may not be applicable to the current scenario of smart spammers. In this paper, we propose an efficient feature extraction method. In this method, two new topic-based features are extracted and used to discriminate human-like spammers from legitimate users. We consider the historical tweets of each user as a document and use the Latent Dirichlet Allocation (LDA) model to compute the topic distribution for each user. Based on the calculated topic probability, two topic-based features, the Local Outlier Standard Score (LOSS) which captures the user's interests on different topics and the Global Outlier Standard Score (GOSS) which reveals the user's interests on specific topic in comparison with other users', are extracted. The two features contain both local and global information, and the combination of them can distinguish human-like spammers effectively. To the best of our knowledge, it is the first time that features based on topic distributions are used in spammer classification. Experimental results on one public dataset and one self-collected dataset further validate that the two sets of extracted topic-based features get excellent performance on human-like spammer classification problem compared with other state-of-the-art methods. In addition, we build a Weibo dataset, which contains both legitimate users and spammers. To summarize, our major contributions are two-fold: In the following sections, we first propose the topic-based features extraction method in Section 2, and then introduce the two datasets in Section 3. Experimental results are discussed in Section 4, and we conclude the paper in Section 5. Future work is presented in Section 6. Methodology In this section, we first provide some observations we obtained after carefully exploring the social network, then the LDA model is introduced. Based on the LDA model, the ways to obtain the topic probability vector for each user and the two topic-based features are provided. Observation After exploring the homepages of a substantial number of spammers, we have two observations. 1) social spammers can be divided into two categories. One is content polluters, and their tweets are all about certain kinds of advertisement and campaign. The other is fake accounts, and their tweets resemble legitimate users' but it seems they are simply random copies of others to avoid being detected by anti-spam rules. 2) For legitimate users, content polluters and fake accounts, they show different patterns on topics which interest them. Legitimate users mainly focus on limited topics which interest him. They seldom post contents unrelated to their concern. Content polluters concentrate on certain topics. Fake accounts focus on a wide range of topics due to random copying and retweeting of other users' tweets. Spammers and legitimate users show different interests on some topics e.g. commercial, weather, etc. To better illustrate our observation, Figure. 1 shows the topic distribution of spammers and legitimate users in two employed datasets(the Honeypot dataset and Weibo dataset). We can see that on both topics (topic-3 and topic-11) there exists obvious difference between the red bars and green bars, representing spammers and legitimate users. On the Honeypot dataset, spammers have a narrower shape of distribution (the outliers on the red bar tail are not counted) than that of legitimate users. This is because there are more content polluters than fake accounts. In other word, spammers in this dataset tend to concentrate on limited topics. While on the Weibo dataset, fake accounts who are interested in different topics take large proportion of spammers. Their distribution is more flat (i.e. red bars) than that of the legitimate users. Therefore we can detect spammers by means of the difference of their topic distribution patterns. LDA model Blei et al.blei2003latent first presented Latent Dirichlet Allocation(LDA) as an example of topic model. Each document $i$ is deemed as a bag of words $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $ and $M$ is the number of words. Each word is attributable to one of the document's topics $Z=\left\lbrace  z_{i1},z_{i2},...,z_{iK}\right\rbrace $ and $K$ is the number of topics. $\psi _{k}$ is a multinomial distribution over words for topic $k$ . $\theta _i$ is another multinomial distribution over topics for document $i$ . The smoothed generative model is illustrated in Figure. 2 . $\alpha $ and $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $0 are hyper parameter that affect scarcity of the document-topic and topic-word distributions. In this paper, $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $1 , $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $2 and $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $3 are empirically set to 0.3, 0.01 and 15. The entire content of each Twitter user is regarded as one document. We adopt Gibbs Sampling BIBREF8 to speed up the inference of LDA. Based on LDA, we can get the topic probabilities for all users in the employed dataset as: $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $4 , where $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $5 is the number of users. Each element $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $6 is a topic probability vector for the $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $7 document. $W=\left\lbrace  w_{i1},w_{i2},...,w_{iM}\right\rbrace $8 is the raw topic probability vector and our features are developed on top of it. Topic-based Features Using the LDA model, each person in the dataset is with a topic probability vector $X_i$ . Assume $x_{ik}\in X_{i}$ denotes the likelihood that the $\emph {i}^{th}$ tweet account favors $\emph {k}^{th}$ topic in the dataset. Our topic based features can be calculated as below. Global Outlier Standard Score measures the degree that a user's tweet content is related to a certain topic compared to the other users. Specifically, the "GOSS" score of user $i$ on topic $k$ can be calculated as Eq.( 12 ): $$\centering \begin{array}{ll} \mu \left(x_{k}\right)=\frac{\sum _{i=1}^{n} x_{ik}}{n},\\ GOSS\left(x_{ik}\right)=\frac{x_{ik}-\mu \left(x_k\right)}{\sqrt{\underset{i}{\sum }\left(x_{ik}-\mu \left(x_{k}\right)\right)^{2}}}. \end{array}$$   (Eq. 12) The value of $GOSS\left(x_{ik}\right)$ indicates the interesting degree of this person to the $\emph {k}^{th}$ topic. Specifically, if $GOSS\left(x_{ik}\right)$ > $GOSS\left(x_{jk}\right)$ , it means that the $\emph {i}^{th}$ person has more interest in topic $k$ than the $\emph {j}^{th}$ person. If the value $GOSS\left(x_{ik}\right)$ is extremely high or low, the $\emph {i}^{th}$ person showing extreme interest or no interest on topic $k$ which will probably be a distinctive pattern in the fowllowing classfication. Therefore, the topics interested or disliked by the $\emph {k}^{th}$0 person can be manifested by $\emph {k}^{th}$1 , from which the pattern of the interested topics with regarding to this person is found. Denote $\emph {k}^{th}$2 our first topic-based feature, and it hopefully can get good performance on spammer detection. Local Outlier Standard Score measures the degree of interest someone shows to a certain topic by considering his own homepage content only. For instance, the "LOSS" score of account $i$ on topic $k$ can be calculated as Eq.( 13 ): $$\centering \begin{array}{ll} \mu \left(x_{i}\right)=\frac{\sum _{k=1}^{K} x_{ik}}{K},\\ LOSS\left(x_{ik}\right)=\frac{x_{ik}-\mu \left(x_i\right)}{\sqrt{\underset{k}{\sum }\left(x_{ik}-\mu \left(x_{i}\right)\right)^{2}}}. \end{array}$$   (Eq. 13) $\mu (x_i)$ represents the averaged interesting degree for all topics with regarding to $\emph {i}^{th}$ user and his tweet content. Similarly to $GOSS$ , the topics interested or disliked by the $\emph {i}^{th}$ person via considering his single post information can be manifested by $f_{LOSS}^{i}=[LOSS(x_{i1})\cdots LOSS(x_{iK})]$ , and $LOSS$ becomes our second topic-based features for the $\emph {i}^{th}$ person. Dataset We use one public dataset Social Honeypot dataset and one self-collected dataset Weibo dataset to validate the effectiveness of our proposed features. Social Honeypot Dataset: Lee et al. lee2010devils created and deployed 60 seed social accounts on Twitter to attract spammers by reporting back what accounts interact with them. They collected 19,276 legitimate users and 22,223 spammers in their datasets along with their tweet content in 7 months. This is our first test dataset. Our Weibo Dataset: Sina Weibo is one of the most famous social platforms in China. It has implemented many features from Twitter. The 2197 legitimate user accounts in this dataset are provided by the Tianchi Competition held by Sina Weibo. The spammers are all purchased commercially from multiple vendors on the Internet. We checked them manually and collected 802 suitable "smart" spammers accounts. Preprocessing: Before directly performing the experiments on the employed datasets, we first delete some accounts with few posts in the two employed since the number of tweets is highly indicative of spammers. For the English Honeypot dataset, we remove stopwords, punctuations, non-ASCII words and apply stemming. For the Chinese Weibo dataset, we perform segmentation with "Jieba", a Chinese text segmentation tool. After preprocessing steps, the Weibo dataset contains 2197 legitimate users and 802 spammers, and the honeypot dataset contains 2218 legitimate users and 2947 spammers. It is worth mentioning that the Honeypot dataset has been slashed because most of the Twitter accounts only have limited number of posts, which are not enough to show their interest inclination. Evaluation Metrics The evaluating indicators in our model are show in 2 . We calculate precision, recall and F1-score (i.e. F1 score) as in Eq. ( 19 ). Precision is the ratio of selected accounts that are spammers. Recall is the ratio of spammers that are detected so. F1-score is the harmonic mean of precision and recall. $$precision =\frac{TP}{TP+FP}, recall =\frac{TP}{TP+FN}\nonumber \\ F1-score = \frac{2\times precision \times recall}{precision + recall}$$   (Eq. 19) Performance Comparisons with Baseline Three baseline classification methods: Support Vector Machines (SVM), Adaboost, and Random Forests are adopted to evaluate our extracted features. We test each classification algorithm with scikit-learn BIBREF9 and run a 10-fold cross validation. On each dataset, the employed classifiers are trained with individual feature first, and then with the combination of the two features. From 1 , we can see that GOSS+LOSS achieves the best performance on F1-score among all others. Besides, the classification by combination of LOSS and GOSS can increase accuracy by more than 3% compared with raw topic distribution probability. Comparison with Other Features To compare our extracted features with previously used features for spammer detection, we use three most discriminative feature sets according to Lee et al. lee2011seven( 4 ). Two classifiers (Adaboost and SVM) are selected to conduct feature performance comparisons. Using Adaboost, our LOSS+GOSS features outperform all other features except for UFN which is 2% higher than ours with regard to precision on the Honeypot dataset. It is caused by the incorrectly classified spammers who are mostly news source after our manual check. They keep posting all kinds of news pieces covering diverse topics, which is similar to the behavior of fake accounts. However, UFN based on friendship networks is more useful for public accounts who possess large number of followers. The best recall value of our LOSS+GOSS features using SVM is up to 6% higher than the results by other feature groups. Regarding F1-score, our features outperform all other features. To further show the advantages of our proposed features, we compare our combined LOSS+GOSS with the combination of all the features from Lee et al. lee2011seven (UFN+UC+UH). It's obvious that LOSS+GOSS have a great advantage over UFN+UC+UH in terms of recall and F1-score. Moreover, by combining our LOSS+GOSS features and UFN+UC+UH features together, we obtained another 7.1% and 2.3% performance gain with regard to precision and F1-score by Adaboost. Though there is a slight decline in terms of recall. By SVM, we get comparative results on recall and F1-score but about 3.5% improvement on precision. Conclusion In this paper, we propose a novel feature extraction method to effectively detect "smart" spammers who post seemingly legitimate tweets and are thus difficult to identify by existing spammer classification methods. Using the LDA model, we obtain the topic probability for each Twitter user. By utilizing the topic probability result, we extract our two topic-based features: GOSS and LOSS which represent the account with global and local information. Experimental results on a public dataset and a self-built Chinese microblog dataset validate the effectiveness of the proposed features. Future Work In future work, the combination method of local and global information can be further improved to maximize their individual strengths. We will also apply decision theory to enhancing the performance of our proposed features. Moreover, we are also building larger datasets on both Twitter and Weibo to validate our method. Moreover, larger datasets on both Twitter and Weibo will be built to further validate our method.
What is the benchmark dataset and is its quality high?
a1645d0ba50e4c29f0feb806521093e7b1459081
What Ġis Ġthe Ġbenchmark Ġdataset Ġand Ġis Ġits Ġquality Ġhigh ?
Social ĠHoney pot Ġdataset Ġ( public ) Ġand ĠWe ibo Ġdataset Ġ( self - col lected ); Ġyes
<s> What Ġis Ġthe Ġbenchmark Ġdataset Ġand Ġis Ġits Ġquality Ġhigh ? </s> Introduction </s> Micro blog ging Ġsuch Ġas ĠTwitter Ġand ĠWe ibo Ġis Ġa Ġpopular Ġsocial Ġnetworking Ġservice , Ġwhich Ġallows Ġusers Ġto Ġpost Ġmessages Ġup Ġto Ġ140 Ġcharacters . ĠThere Ġare Ġmillions Ġof Ġactive Ġusers Ġon Ġthe Ġplatform Ġwho Ġstay Ġconnected Ġwith Ġfriends . ĠUnfortunately , Ġsp ammers Ġalso Ġuse Ġit Ġas Ġa Ġtool Ġto Ġpost Ġmalicious Ġlinks , Ġsend Ġuns olicited Ġmessages Ġto Ġlegitimate Ġusers , Ġetc . ĠA Ġcertain Ġamount Ġof Ġsp ammers Ġcould Ġsway Ġthe Ġpublic Ġopinion Ġand Ġcause Ġdistrust Ġof Ġthe Ġsocial Ġplatform . ĠDespite Ġthe Ġuse Ġof Ġrigid Ġanti - sp am Ġrules , Ġhuman - like Ġsp ammers Ġwhose Ġhome pages Ġhaving Ġphotos , Ġdetailed Ġprofiles Ġetc . Ġhave Ġemerged . ĠUnlike Ġprevious Ġ" simple " Ġsp ammers , Ġwhose Ġtweets Ġcontain Ġonly Ġmalicious Ġlinks , Ġthose Ġ" smart " Ġsp ammers Ġare Ġmore Ġdifficult Ġto Ġdistinguish Ġfrom Ġlegitimate Ġusers Ġvia Ġcontent - based Ġfeatures Ġalone ĠB IB REF 0 Ġ. </s> There Ġis Ġa Ġconsiderable Ġamount Ġof Ġprevious Ġwork Ġon Ġspam mer Ġdetection Ġon Ġsocial Ġplatforms . ĠRes earcher Ġfrom ĠTwitter ĠInc . ĠB IB REF 1 Ġcollect Ġbot Ġaccounts Ġand Ġperform Ġanalysis Ġon Ġthe Ġuser Ġbehavior Ġand Ġuser Ġprofile Ġfeatures . ĠLee Ġet Ġal . Ġle e 2011 seven Ġuse Ġthe Ġso - called Ġsocial Ġhoney pot Ġby Ġall uring Ġsocial Ġsp ammers ' Ġretweet Ġto Ġbuild Ġa Ġbenchmark Ġdataset , Ġwhich Ġhas Ġbeen Ġextensively Ġexplored Ġin Ġour Ġpaper . ĠSome Ġresearchers Ġfocus Ġon Ġthe Ġclust ering Ġof Ġur ls Ġin Ġtweets Ġand Ġnetwork Ġgraph Ġof Ġsocial Ġsp ammers ĠB IB REF 2 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, Ġshowing Ġthe Ġpower Ġof Ġsocial Ġrelationship Ġfeatures . As Ġfor Ġcontent Ġinformation Ġmodeling , ĠB IB REF 6 Ġapply Ġimproved Ġsparse Ġlearning Ġmethods . ĠHowever , Ġfew Ġstudies Ġhave Ġadopted Ġtopic - based Ġfeatures . ĠSome Ġresearchers ĠB IB REF 7 Ġdiscuss Ġtopic Ġcharacteristics Ġof Ġspam ming Ġposts , Ġindicating Ġthat Ġsp ammers Ġare Ġhighly Ġlikely Ġto Ġdwell Ġon Ġsome Ġcertain Ġtopics Ġsuch Ġas Ġpromotion . ĠBut Ġthis Ġmay Ġnot Ġbe Ġapplicable Ġto Ġthe Ġcurrent Ġscenario Ġof Ġsmart Ġsp ammers . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġan Ġefficient Ġfeature Ġextraction Ġmethod . ĠIn Ġthis Ġmethod , Ġtwo Ġnew Ġtopic - based Ġfeatures Ġare Ġextracted Ġand Ġused Ġto Ġdiscriminate Ġhuman - like Ġsp ammers Ġfrom Ġlegitimate Ġusers . ĠWe Ġconsider Ġthe Ġhistorical Ġtweets Ġof Ġeach Ġuser Ġas Ġa Ġdocument Ġand Ġuse Ġthe ĠLat ent ĠDir ich let ĠAll ocation Ġ( L DA ) Ġmodel Ġto Ġcompute Ġthe Ġtopic Ġdistribution Ġfor Ġeach Ġuser . ĠBased Ġon Ġthe Ġcalculated Ġtopic Ġprobability , Ġtwo Ġtopic - based Ġfeatures , Ġthe ĠLocal ĠOut lier ĠStandard ĠScore Ġ( L OSS ) Ġwhich Ġcaptures Ġthe Ġuser 's Ġinterests Ġon Ġdifferent Ġtopics Ġand Ġthe ĠGlobal ĠOut lier ĠStandard ĠScore Ġ( G OSS ) Ġwhich Ġreveals Ġthe Ġuser 's Ġinterests Ġon Ġspecific Ġtopic Ġin Ġcomparison Ġwith Ġother Ġusers ', Ġare Ġextracted . ĠThe Ġtwo Ġfeatures Ġcontain Ġboth Ġlocal Ġand Ġglobal Ġinformation , Ġand Ġthe Ġcombination Ġof Ġthem Ġcan Ġdistinguish Ġhuman - like Ġsp ammers Ġeffectively . </s> To Ġthe Ġbest Ġof Ġour Ġknowledge , Ġit Ġis Ġthe Ġfirst Ġtime Ġthat Ġfeatures Ġbased Ġon Ġtopic Ġdistributions Ġare Ġused Ġin Ġspam mer Ġclassification . ĠExperimental Ġresults Ġon Ġone Ġpublic Ġdataset Ġand Ġone Ġself - col lected Ġdataset Ġfurther Ġvalidate Ġthat Ġthe Ġtwo Ġsets Ġof Ġextracted Ġtopic - based Ġfeatures Ġget Ġexcellent Ġperformance Ġon Ġhuman - like Ġspam mer Ġclassification Ġproblem Ġcompared Ġwith Ġother Ġstate - of - the - art Ġmethods . ĠIn Ġaddition , Ġwe Ġbuild Ġa ĠWe ibo Ġdataset , Ġwhich Ġcontains Ġboth Ġlegitimate Ġusers Ġand Ġsp ammers . </s> To Ġsummarize , Ġour Ġmajor Ġcontributions Ġare Ġtwo - fold : </s> In Ġthe Ġfollowing Ġsections , Ġwe Ġfirst Ġpropose Ġthe Ġtopic - based Ġfeatures Ġextraction Ġmethod Ġin ĠSection Ġ2 , Ġand Ġthen Ġintroduce Ġthe Ġtwo Ġdatasets Ġin ĠSection Ġ3 . ĠExperimental Ġresults Ġare Ġdiscussed Ġin ĠSection Ġ4 , Ġand Ġwe Ġconclude Ġthe Ġpaper Ġin ĠSection Ġ5 . ĠFuture Ġwork Ġis Ġpresented Ġin ĠSection Ġ6 . </s> Method ology </s> In Ġthis Ġsection , Ġwe Ġfirst Ġprovide Ġsome Ġobservations Ġwe Ġobtained Ġafter Ġcarefully Ġexploring Ġthe Ġsocial Ġnetwork , Ġthen Ġthe ĠL DA Ġmodel Ġis Ġintroduced . ĠBased Ġon Ġthe ĠL DA Ġmodel , Ġthe Ġways Ġto Ġobtain Ġthe Ġtopic Ġprobability Ġvector Ġfor Ġeach Ġuser Ġand Ġthe Ġtwo Ġtopic - based Ġfeatures Ġare Ġprovided . </s> Obs ervation </s> After Ġexploring Ġthe Ġhome pages Ġof Ġa Ġsubstantial Ġnumber Ġof Ġsp ammers , Ġwe Ġhave Ġtwo Ġobservations . Ġ1 ) Ġsocial Ġsp ammers Ġcan Ġbe Ġdivided Ġinto Ġtwo Ġcategories . ĠOne Ġis Ġcontent Ġpoll uters , Ġand Ġtheir Ġtweets Ġare Ġall Ġabout Ġcertain Ġkinds Ġof Ġadvertisement Ġand Ġcampaign . ĠThe Ġother Ġis Ġfake Ġaccounts , Ġand Ġtheir Ġtweets Ġresemble Ġlegitimate Ġusers ' Ġbut Ġit Ġseems Ġthey Ġare Ġsimply Ġrandom Ġcopies Ġof Ġothers Ġto Ġavoid Ġbeing Ġdetected Ġby Ġanti - sp am Ġrules . Ġ2 ) ĠFor Ġlegitimate Ġusers , Ġcontent Ġpoll uters Ġand Ġfake Ġaccounts , Ġthey Ġshow Ġdifferent Ġpatterns Ġon Ġtopics Ġwhich Ġinterest Ġthem . </s> Leg itimate Ġusers Ġmainly Ġfocus Ġon Ġlimited Ġtopics Ġwhich Ġinterest Ġhim . ĠThey Ġseldom Ġpost Ġcontents Ġunrelated Ġto Ġtheir Ġconcern . </s> Content Ġpoll uters Ġconcentrate Ġon Ġcertain Ġtopics . </s> Fake Ġaccounts Ġfocus Ġon Ġa Ġwide Ġrange Ġof Ġtopics Ġdue Ġto Ġrandom Ġcopying Ġand Ġretweet ing Ġof Ġother Ġusers ' Ġtweets . </s> Sp ammers Ġand Ġlegitimate Ġusers Ġshow Ġdifferent Ġinterests Ġon Ġsome Ġtopics Ġe . g . Ġcommercial , Ġweather , Ġetc . </s> To Ġbetter Ġillustrate Ġour Ġobservation , ĠFigure . Ġ1 Ġshows Ġthe Ġtopic Ġdistribution Ġof Ġsp ammers Ġand Ġlegitimate Ġusers Ġin Ġtwo Ġemployed Ġdatasets ( the ĠHoney pot Ġdataset Ġand ĠWe ibo Ġdataset ). ĠWe Ġcan Ġsee Ġthat Ġon Ġboth Ġtopics Ġ( topic - 3 Ġand Ġtopic - 11 ) Ġthere Ġexists Ġobvious Ġdifference Ġbetween Ġthe Ġred Ġbars Ġand Ġgreen Ġbars , Ġrepresenting Ġsp ammers Ġand Ġlegitimate Ġusers . ĠOn Ġthe ĠHoney pot Ġdataset , Ġsp ammers Ġhave Ġa Ġnarrower Ġshape Ġof Ġdistribution Ġ( the Ġoutl iers Ġon Ġthe Ġred Ġbar Ġtail Ġare Ġnot Ġcounted ) Ġthan Ġthat Ġof Ġlegitimate Ġusers . ĠThis Ġis Ġbecause Ġthere Ġare Ġmore Ġcontent Ġpoll uters Ġthan Ġfake Ġaccounts . ĠIn Ġother Ġword , Ġsp ammers Ġin Ġthis Ġdataset Ġtend Ġto Ġconcentrate Ġon Ġlimited Ġtopics . ĠWhile Ġon Ġthe ĠWe ibo Ġdataset , Ġfake Ġaccounts Ġwho Ġare Ġinterested Ġin Ġdifferent Ġtopics Ġtake Ġlarge Ġproportion Ġof Ġsp ammers . ĠTheir Ġdistribution Ġis Ġmore Ġflat Ġ( i . e . Ġred Ġbars ) Ġthan Ġthat Ġof Ġthe Ġlegitimate Ġusers . ĠTherefore Ġwe Ġcan Ġdetect Ġsp ammers Ġby Ġmeans Ġof Ġthe Ġdifference Ġof Ġtheir Ġtopic Ġdistribution Ġpatterns . </s> L DA Ġmodel </s> Ble i Ġet Ġal . ble i 2003 lat ent Ġfirst Ġpresented ĠLat ent ĠDir ich let ĠAll ocation ( L DA ) Ġas Ġan Ġexample Ġof Ġtopic Ġmodel . </s> Each Ġdocument Ġ$ i $ Ġis Ġdeemed Ġas Ġa Ġbag Ġof Ġwords Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ Ġand Ġ$ M $ Ġis Ġthe Ġnumber Ġof Ġwords . ĠEach Ġword Ġis Ġattributable Ġto Ġone Ġof Ġthe Ġdocument 's Ġtopics Ġ$ Z = \ left \ l brace Ġ Ġz _{ i 1 }, z _{ i 2 }, ... , z _{ i K }\ right \ r brace Ġ$ Ġand Ġ$ K $ Ġis Ġthe Ġnumber Ġof Ġtopics . Ġ$\ ps i Ġ_ { k } $ Ġis Ġa Ġmult in omial Ġdistribution Ġover Ġwords Ġfor Ġtopic Ġ$ k $ Ġ. Ġ$\ the ta Ġ_ i $ Ġis Ġanother Ġmult in omial Ġdistribution Ġover Ġtopics Ġfor Ġdocument Ġ$ i $ Ġ. ĠThe Ġsmoot hed Ġgener ative Ġmodel Ġis Ġillustrated Ġin ĠFigure . Ġ2 Ġ. Ġ$\ alpha Ġ$ Ġand Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 0 Ġare Ġhyper Ġparameter Ġthat Ġaffect Ġscarcity Ġof Ġthe Ġdocument - topic Ġand Ġtopic - word Ġdistributions . ĠIn Ġthis Ġpaper , Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 1 Ġ, Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 2 Ġand Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 3 Ġare Ġempir ically Ġset Ġto Ġ0 . 3 , Ġ0 . 01 Ġand Ġ15 . ĠThe Ġentire Ġcontent Ġof Ġeach ĠTwitter Ġuser Ġis Ġregarded Ġas Ġone Ġdocument . ĠWe Ġadopt ĠGibbs ĠSam pling ĠB IB REF 8 Ġto Ġspeed Ġup Ġthe Ġinference Ġof ĠL DA . ĠBased Ġon ĠL DA , Ġwe Ġcan Ġget Ġthe Ġtopic Ġprobabilities Ġfor Ġall Ġusers Ġin Ġthe Ġemployed Ġdataset Ġas : Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 4 Ġ, Ġwhere Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 5 Ġis Ġthe Ġnumber Ġof Ġusers . ĠEach Ġelement Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 6 Ġis Ġa Ġtopic Ġprobability Ġvector Ġfor Ġthe Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 7 Ġdocument . Ġ$ W = \ left \ l brace Ġ Ġw _{ i 1 }, w _{ i 2 }, ... , w _{ i M }\ right \ r brace Ġ$ 8 Ġis Ġthe Ġraw Ġtopic Ġprobability Ġvector Ġand Ġour Ġfeatures Ġare Ġdeveloped Ġon Ġtop Ġof Ġit . </s> Topic - based ĠFeatures </s> Using Ġthe ĠL DA Ġmodel , Ġeach Ġperson Ġin Ġthe Ġdataset Ġis Ġwith Ġa Ġtopic Ġprobability Ġvector Ġ$ X _ i $ Ġ. ĠAss ume Ġ$ x _{ ik }\ in ĠX _{ i } $ Ġdenotes Ġthe Ġlikelihood Ġthat Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġtweet Ġaccount Ġfavors Ġ$\ em ph Ġ{ k } ^{ th } $ Ġtopic Ġin Ġthe Ġdataset . ĠOur Ġtopic Ġbased Ġfeatures Ġcan Ġbe Ġcalculated Ġas Ġbelow . </s> Global ĠOut lier ĠStandard ĠScore Ġmeasures Ġthe Ġdegree Ġthat Ġa Ġuser 's Ġtweet Ġcontent Ġis Ġrelated Ġto Ġa Ġcertain Ġtopic Ġcompared Ġto Ġthe Ġother Ġusers . ĠSpecifically , Ġthe Ġ" G OSS " Ġscore Ġof Ġuser Ġ$ i $ Ġon Ġtopic Ġ$ k $ Ġcan Ġbe Ġcalculated Ġas ĠE q .( Ġ12 Ġ): </s> $$ \ cent ering Ġ\ begin { array }{ ll } Ġ\ mu Ġ\ left ( x _{ k }\ right )= \ frac {\ sum Ġ_ { i = 1 } ^{ n } Ġx _{ ik }} { n }, \\ ĠG OSS \ left ( x _{ ik }\ right )= \ frac { x _{ ik } - \ mu Ġ\ left ( x _ k \ right ) }{ \ sq rt {\ unders et { i }{ \ sum Ġ} \ left ( x _{ ik } - \ mu Ġ\ left ( x _{ k }\ right )\ right ) ^{ 2 }} }. Ġ\ end { array } $$ Ġ Ġ Ġ( E q . Ġ12 ) </s> The Ġvalue Ġof Ġ$ G OSS \ left ( x _{ ik }\ right ) $ Ġindicates Ġthe Ġinteresting Ġdegree Ġof Ġthis Ġperson Ġto Ġthe Ġ$\ em ph Ġ{ k } ^{ th } $ Ġtopic . ĠSpecifically , Ġif Ġ$ G OSS \ left ( x _{ ik }\ right ) $ Ġ> Ġ$ G OSS \ left ( x _{ j k }\ right ) $ Ġ, Ġit Ġmeans Ġthat Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson Ġhas Ġmore Ġinterest Ġin Ġtopic Ġ$ k $ Ġthan Ġthe Ġ$\ em ph Ġ{ j } ^{ th } $ Ġperson . ĠIf Ġthe Ġvalue Ġ$ G OSS \ left ( x _{ ik }\ right ) $ Ġis Ġextremely Ġhigh Ġor Ġlow , Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson Ġshowing Ġextreme Ġinterest Ġor Ġno Ġinterest Ġon Ġtopic Ġ$ k $ Ġwhich Ġwill Ġprobably Ġbe Ġa Ġdistinctive Ġpattern Ġin Ġthe Ġf ow ll owing Ġclass f ication . ĠTherefore , Ġthe Ġtopics Ġinterested Ġor Ġdisliked Ġby Ġthe Ġ$\ em ph Ġ{ k } ^{ th } $ 0 Ġperson Ġcan Ġbe Ġmanifested Ġby Ġ$\ em ph Ġ{ k } ^{ th } $ 1 Ġ, Ġfrom Ġwhich Ġthe Ġpattern Ġof Ġthe Ġinterested Ġtopics Ġwith Ġregarding Ġto Ġthis Ġperson Ġis Ġfound . ĠDen ote Ġ$\ em ph Ġ{ k } ^{ th } $ 2 Ġour Ġfirst Ġtopic - based Ġfeature , Ġand Ġit Ġhopefully Ġcan Ġget Ġgood Ġperformance Ġon Ġspam mer Ġdetection . </s> Local ĠOut lier ĠStandard ĠScore Ġmeasures Ġthe Ġdegree Ġof Ġinterest Ġsomeone Ġshows Ġto Ġa Ġcertain Ġtopic Ġby Ġconsidering Ġhis Ġown Ġhomepage Ġcontent Ġonly . ĠFor Ġinstance , Ġthe Ġ" L OSS " Ġscore Ġof Ġaccount Ġ$ i $ Ġon Ġtopic Ġ$ k $ Ġcan Ġbe Ġcalculated Ġas ĠE q .( Ġ13 Ġ): </s> $$ \ cent ering Ġ\ begin { array }{ ll } Ġ\ mu Ġ\ left ( x _{ i }\ right )= \ frac {\ sum Ġ_ { k = 1 } ^{ K } Ġx _{ ik }} { K }, \\ ĠL OSS \ left ( x _{ ik }\ right )= \ frac { x _{ ik } - \ mu Ġ\ left ( x _ i \ right ) }{ \ sq rt {\ unders et { k }{ \ sum Ġ} \ left ( x _{ ik } - \ mu Ġ\ left ( x _{ i }\ right )\ right ) ^{ 2 }} }. Ġ\ end { array } $$ Ġ Ġ Ġ( E q . Ġ13 ) </s> $ \ mu Ġ( x _ i ) $ Ġrepresents Ġthe Ġaveraged Ġinteresting Ġdegree Ġfor Ġall Ġtopics Ġwith Ġregarding Ġto Ġ$\ em ph Ġ{ i } ^{ th } $ Ġuser Ġand Ġhis Ġtweet Ġcontent . ĠSimilarly Ġto Ġ$ G OSS $ Ġ, Ġthe Ġtopics Ġinterested Ġor Ġdisliked Ġby Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson Ġvia Ġconsidering Ġhis Ġsingle Ġpost Ġinformation Ġcan Ġbe Ġmanifested Ġby Ġ$ f _{ L OSS } ^{ i } =[ L OSS ( x _{ i 1 } )\ cd ots ĠL OSS ( x _{ i K } )] $ Ġ, Ġand Ġ$ L OSS $ Ġbecomes Ġour Ġsecond Ġtopic - based Ġfeatures Ġfor Ġthe Ġ$\ em ph Ġ{ i } ^{ th } $ Ġperson . </s> Dat as et </s> We Ġuse Ġone Ġpublic Ġdataset ĠSocial ĠHoney pot Ġdataset Ġand Ġone Ġself - col lected Ġdataset ĠWe ibo Ġdataset Ġto Ġvalidate Ġthe Ġeffectiveness Ġof Ġour Ġproposed Ġfeatures . </s> Social ĠHoney pot ĠDat as et : ĠLee Ġet Ġal . Ġle e 2010 dev ils Ġcreated Ġand Ġdeployed Ġ60 Ġseed Ġsocial Ġaccounts Ġon ĠTwitter Ġto Ġattract Ġsp ammers Ġby Ġreporting Ġback Ġwhat Ġaccounts Ġinteract Ġwith Ġthem . ĠThey Ġcollected Ġ19 , 276 Ġlegitimate Ġusers Ġand Ġ22 , 223 Ġsp ammers Ġin Ġtheir Ġdatasets Ġalong Ġwith Ġtheir Ġtweet Ġcontent Ġin Ġ7 Ġmonths . ĠThis Ġis Ġour Ġfirst Ġtest Ġdataset . </s> Our ĠWe ibo ĠDat as et : ĠSina ĠWe ibo Ġis Ġone Ġof Ġthe Ġmost Ġfamous Ġsocial Ġplatforms Ġin ĠChina . ĠIt Ġhas Ġimplemented Ġmany Ġfeatures Ġfrom ĠTwitter . ĠThe Ġ2 197 Ġlegitimate Ġuser Ġaccounts Ġin Ġthis Ġdataset Ġare Ġprovided Ġby Ġthe ĠTian chi ĠCompetition Ġheld Ġby ĠSina ĠWe ibo . ĠThe Ġsp ammers Ġare Ġall Ġpurchased Ġcommercially Ġfrom Ġmultiple Ġvendors Ġon Ġthe ĠInternet . ĠWe Ġchecked Ġthem Ġmanually Ġand Ġcollected Ġ802 Ġsuitable Ġ" smart " Ġsp ammers Ġaccounts . </s> Pre processing : ĠBefore Ġdirectly Ġperforming Ġthe Ġexperiments Ġon Ġthe Ġemployed Ġdatasets , Ġwe Ġfirst Ġdelete Ġsome Ġaccounts Ġwith Ġfew Ġposts Ġin Ġthe Ġtwo Ġemployed Ġsince Ġthe Ġnumber Ġof Ġtweets Ġis Ġhighly Ġindicative Ġof Ġsp ammers . ĠFor Ġthe ĠEnglish ĠHoney pot Ġdataset , Ġwe Ġremove Ġstop words , Ġpunct uations , Ġnon - ASC II Ġwords Ġand Ġapply Ġstemming . ĠFor Ġthe ĠChinese ĠWe ibo Ġdataset , Ġwe Ġperform Ġsegment ation Ġwith Ġ" J ie ba ", Ġa ĠChinese Ġtext Ġsegment ation Ġtool . ĠAfter Ġpre processing Ġsteps , Ġthe ĠWe ibo Ġdataset Ġcontains Ġ2 197 Ġlegitimate Ġusers Ġand Ġ802 Ġsp ammers , Ġand Ġthe Ġhoney pot Ġdataset Ġcontains Ġ22 18 Ġlegitimate Ġusers Ġand Ġ29 47 Ġsp ammers . ĠIt Ġis Ġworth Ġmentioning Ġthat Ġthe ĠHoney pot Ġdataset Ġhas Ġbeen Ġslashed Ġbecause Ġmost Ġof Ġthe ĠTwitter Ġaccounts Ġonly Ġhave Ġlimited Ġnumber Ġof Ġposts , Ġwhich Ġare Ġnot Ġenough Ġto Ġshow Ġtheir Ġinterest Ġinclination . </s> E val uation ĠMet rics </s> The Ġevaluating Ġindicators Ġin Ġour Ġmodel Ġare Ġshow Ġin Ġ2 Ġ. ĠWe Ġcalculate Ġprecision , Ġrecall Ġand ĠF 1 - score Ġ( i . e . ĠF 1 Ġscore ) Ġas Ġin ĠE q . Ġ( Ġ19 Ġ). ĠPrecision Ġis Ġthe Ġratio Ġof Ġselected Ġaccounts Ġthat Ġare Ġsp ammers . ĠRecall Ġis Ġthe Ġratio Ġof Ġsp ammers Ġthat Ġare Ġdetected Ġso . ĠF 1 - score Ġis Ġthe Ġharmonic Ġmean Ġof Ġprecision Ġand Ġrecall . </s> $$ pre cision Ġ= \ frac { TP }{ TP + FP }, Ġrecall Ġ= \ frac { TP }{ TP + FN }\ non umber Ġ\\ ĠF 1 - score Ġ= Ġ\ frac { 2 \ times Ġprecision Ġ\ times Ġrecall }{ pre cision Ġ+ Ġrecall } $$ Ġ Ġ Ġ( E q . Ġ19 ) </s> Performance ĠCompar isons Ġwith ĠBas eline </s> Three Ġbaseline Ġclassification Ġmethods : ĠSupport ĠVector ĠMachines Ġ( S VM ), ĠAd abo ost , Ġand ĠRandom ĠFore sts Ġare Ġadopted Ġto Ġevaluate Ġour Ġextracted Ġfeatures . ĠWe Ġtest Ġeach Ġclassification Ġalgorithm Ġwith Ġsc ik it - learn ĠB IB REF 9 Ġand Ġrun Ġa Ġ10 - fold Ġcross Ġvalidation . ĠOn Ġeach Ġdataset , Ġthe Ġemployed Ġclass ifiers Ġare Ġtrained Ġwith Ġindividual Ġfeature Ġfirst , Ġand Ġthen Ġwith Ġthe Ġcombination Ġof Ġthe Ġtwo Ġfeatures . ĠFrom Ġ1 Ġ, Ġwe Ġcan Ġsee Ġthat ĠG OSS + L OSS Ġachieves Ġthe Ġbest Ġperformance Ġon ĠF 1 - score Ġamong Ġall Ġothers . ĠBesides , Ġthe Ġclassification Ġby Ġcombination Ġof ĠL OSS Ġand ĠG OSS Ġcan Ġincrease Ġaccuracy Ġby Ġmore Ġthan Ġ3 % Ġcompared Ġwith Ġraw Ġtopic Ġdistribution Ġprobability . </s> Compar ison Ġwith ĠOther ĠFeatures </s> To Ġcompare Ġour Ġextracted Ġfeatures Ġwith Ġpreviously Ġused Ġfeatures Ġfor Ġspam mer Ġdetection , Ġwe Ġuse Ġthree Ġmost Ġdiscrim in ative Ġfeature Ġsets Ġaccording Ġto ĠLee Ġet Ġal . Ġle e 2011 seven ( Ġ4 Ġ). ĠTwo Ġclass ifiers Ġ( Ad abo ost Ġand ĠS VM ) Ġare Ġselected Ġto Ġconduct Ġfeature Ġperformance Ġcomparisons . ĠUsing ĠAd abo ost , Ġour ĠL OSS + G OSS Ġfeatures Ġoutper form Ġall Ġother Ġfeatures Ġexcept Ġfor ĠU FN Ġwhich Ġis Ġ2 % Ġhigher Ġthan Ġours Ġwith Ġregard Ġto Ġprecision Ġon Ġthe ĠHoney pot Ġdataset . ĠIt Ġis Ġcaused Ġby Ġthe Ġincorrectly Ġclassified Ġsp ammers Ġwho Ġare Ġmostly Ġnews Ġsource Ġafter Ġour Ġmanual Ġcheck . ĠThey Ġkeep Ġposting Ġall Ġkinds Ġof Ġnews Ġpieces Ġcovering Ġdiverse Ġtopics , Ġwhich Ġis Ġsimilar Ġto Ġthe Ġbehavior Ġof Ġfake Ġaccounts . ĠHowever , ĠU FN Ġbased Ġon Ġfriendship Ġnetworks Ġis Ġmore Ġuseful Ġfor Ġpublic Ġaccounts Ġwho Ġpossess Ġlarge Ġnumber Ġof Ġfollowers . ĠThe Ġbest Ġrecall Ġvalue Ġof Ġour ĠL OSS + G OSS Ġfeatures Ġusing ĠS VM Ġis Ġup Ġto Ġ6 % Ġhigher Ġthan Ġthe Ġresults Ġby Ġother Ġfeature Ġgroups . ĠRegarding ĠF 1 - score , Ġour Ġfeatures Ġoutper form Ġall Ġother Ġfeatures . ĠTo Ġfurther Ġshow Ġthe Ġadvantages Ġof Ġour Ġproposed Ġfeatures , Ġwe Ġcompare Ġour Ġcombined ĠL OSS + G OSS Ġwith Ġthe Ġcombination Ġof Ġall Ġthe Ġfeatures Ġfrom ĠLee Ġet Ġal . Ġle e 2011 seven Ġ( UF N + UC + U H ). ĠIt 's Ġobvious Ġthat ĠL OSS + G OSS Ġhave Ġa Ġgreat Ġadvantage Ġover ĠU FN + UC + U H Ġin Ġterms Ġof Ġrecall Ġand ĠF 1 - score . ĠMoreover , Ġby Ġcombining Ġour ĠL OSS + G OSS Ġfeatures Ġand ĠU FN + UC + U H Ġfeatures Ġtogether , Ġwe Ġobtained Ġanother Ġ7 . 1 % Ġand Ġ2 . 3 % Ġperformance Ġgain Ġwith Ġregard Ġto Ġprecision Ġand ĠF 1 - score Ġby ĠAd abo ost . ĠThough Ġthere Ġis Ġa Ġslight Ġdecline Ġin Ġterms Ġof Ġrecall . ĠBy ĠS VM , Ġwe Ġget Ġcomparative Ġresults Ġon Ġrecall Ġand ĠF 1 - score Ġbut Ġabout Ġ3 . 5 % Ġimprovement Ġon Ġprecision . </s> Conclusion </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġa Ġnovel Ġfeature Ġextraction Ġmethod Ġto Ġeffectively Ġdetect Ġ" smart " Ġsp ammers Ġwho Ġpost Ġseemingly Ġlegitimate Ġtweets Ġand Ġare Ġthus Ġdifficult Ġto Ġidentify Ġby Ġexisting Ġspam mer Ġclassification Ġmethods . ĠUsing Ġthe ĠL DA Ġmodel , Ġwe Ġobtain Ġthe Ġtopic Ġprobability Ġfor Ġeach ĠTwitter Ġuser . ĠBy Ġutilizing Ġthe Ġtopic Ġprobability Ġresult , Ġwe Ġextract Ġour Ġtwo Ġtopic - based Ġfeatures : ĠG OSS Ġand ĠL OSS Ġwhich Ġrepresent Ġthe Ġaccount Ġwith Ġglobal Ġand Ġlocal Ġinformation . ĠExperimental Ġresults Ġon Ġa Ġpublic Ġdataset Ġand Ġa Ġself - built ĠChinese Ġmicro blog Ġdataset Ġvalidate Ġthe Ġeffectiveness Ġof Ġthe Ġproposed Ġfeatures . </s> Future ĠWork </s> In Ġfuture Ġwork , Ġthe Ġcombination Ġmethod Ġof Ġlocal Ġand Ġglobal Ġinformation Ġcan Ġbe Ġfurther Ġimproved Ġto Ġmaximize Ġtheir Ġindividual Ġstrengths . ĠWe Ġwill Ġalso Ġapply Ġdecision Ġtheory Ġto Ġenhancing Ġthe Ġperformance Ġof Ġour Ġproposed Ġfeatures . ĠMoreover , Ġwe Ġare Ġalso Ġbuilding Ġlarger Ġdatasets Ġon Ġboth ĠTwitter Ġand ĠWe ibo Ġto Ġvalidate Ġour Ġmethod . ĠMoreover , Ġlarger Ġdatasets Ġon Ġboth ĠTwitter Ġand ĠWe ibo Ġwill Ġbe Ġbuilt Ġto Ġfurther Ġvalidate Ġour Ġmethod .