personal attack, racism, and sexism
EXTRACTIVE
racism, sexism, personal attack, not specifically about any single topic
EXTRACTIVE
Past works on cyberbullying detection have at least one of the following three bottlenecks. First (Bottleneck B1), they target only one particular social media platform. How these methods perform across other SMPs is unknown. Second (Bottleneck B2), they address only one topic of cyberbullying such as racism, and sexism. Depending on the topic, vocabulary and nature of cyberbullying changes. These models are not flexible in accommodating changes in the definition of cyberbullying. Third (Bottleneck B3), they rely on carefully handcrafted features such as swear word list and POS tagging. However, these handcrafted features are not robust against variations in writing style. In contrast to existing bottlenecks, this work targets three different types of social networks (Formspring: a Q&A forum, Twitter: microblogging, and Wikipedia: collaborative knowledge repository) for three topics of cyberbullying (personal attack, racism, and sexism) without doing any explicit feature engineering by developing deep learning based models along with transfer learning.
Please refer to Table TABREF7 for summary of datasets used. We performed experiments using large, diverse, manually annotated, and publicly available datasets for cyberbullying detection in social media. We cover three different types of social networks: teen oriented Q&A forum (Formspring), large microblogging platform (Twitter), and collaborative knowledge repository (Wikipedia talk pages). Each dataset addresses a different topic of cyberbullying. Twitter dataset contains examples of racism and sexism. Wikipedia dataset contains examples of personal attack. However, Formspring dataset is not specifically about any single topic. All three datasets have the problem of class imbalance where posts labeled as cyberbullying are in the minority as compared to neutral posts. Variation in the number of posts across datasets also affects vocabulary size that represents the number of distinct words encountered in the dataset. We measure the size of a post in terms of the number of words in the post. For each dataset, there are only a few posts with large size. We truncate such large posts to the size of post ranked at 95 percentile in that dataset. For example, in Wikipedia dataset, the largest post has 2846 words. However, size of post ranked at 95 percentile in that dataset is only 231. Any post larger than size 231 in Wikipedia dataset will be truncated by considering only first 231 words. This truncation affects only a small minority of posts in each dataset. However, it is required for efficiently training various models in our experiments. Details of each dataset are as follows.
1801.06482
Introduction </s> Cy ber bull ying Ġhas Ġbeen Ġdefined Ġby Ġthe ĠNational ĠCrime ĠPrevention ĠCouncil Ġas Ġthe Ġuse Ġof Ġthe ĠInternet , Ġcell Ġphones Ġor Ġother Ġdevices Ġto Ġsend Ġor Ġpost Ġtext Ġor Ġimages Ġintended Ġto Ġhurt Ġor Ġembarrass Ġanother Ġperson . ĠVarious Ġstudies Ġhave Ġestimated Ġthat Ġbetween Ġto Ġ10 % Ġto Ġ40 % Ġof Ġinternet Ġusers Ġare Ġvictims Ġof Ġcyber bull ying ĠB IB REF 0 Ġ. ĠEffects Ġof Ġcyber bull ying Ġcan Ġrange Ġfrom Ġtemporary Ġanxiety Ġto Ġsuicide ĠB IB REF 1 Ġ. ĠMany Ġhigh Ġprofile Ġincidents Ġhave Ġemphasized Ġthe Ġprevalence Ġof Ġcyber bull ying Ġon Ġsocial Ġmedia . ĠMost Ġrecently Ġin ĠOctober Ġ2017 , Ġa ĠSwedish Ġmodel ĠAr v ida ĠBy str Ã¶ m Ġwas Ġcyber bull ied Ġto Ġthe Ġextent Ġof Ġreceiving Ġrape Ġthreats Ġafter Ġshe Ġappeared Ġin Ġan Ġadvertisement Ġwith Ġhairy Ġlegs . </s> Det ection Ġof Ġcyber bull ying Ġin Ġsocial Ġmedia Ġis Ġa Ġchallenging Ġtask . ĠDefinition Ġof Ġwhat Ġconstitutes Ġcyber bull ying Ġis Ġquite Ġsubjective . ĠFor Ġexample , Ġfrequent Ġuse Ġof Ġswear Ġwords Ġmight Ġbe Ġconsidered Ġas Ġbullying Ġby Ġthe Ġgeneral Ġpopulation . ĠHowever , Ġfor Ġteen Ġoriented Ġsocial Ġmedia Ġplatforms Ġsuch Ġas ĠForm spring , Ġthis Ġdoes Ġnot Ġnecessarily Ġmean Ġbullying Ġ( Table ĠT AB REF 9 Ġ). ĠAcross Ġmultiple ĠS MP s , Ġcyber bull ies Ġattack Ġvictims Ġon Ġdifferent Ġtopics Ġsuch Ġas Ġrace , Ġreligion , Ġand Ġgender . ĠDepending Ġon Ġthe Ġtopic Ġof Ġcyber bull ying , Ġvocabulary Ġand Ġperceived Ġmeaning Ġof Ġwords Ġvary Ġsignificantly Ġacross ĠS MP s . ĠFor Ġexample , Ġin Ġour Ġexperiments Ġwe Ġfound Ġthat Ġfor Ġword Ġ` fat ', Ġthe Ġmost Ġsimilar Ġwords Ġas Ġper ĠTwitter Ġdataset Ġare Ġ` female ' Ġand Ġ` woman ' Ġ( Table ĠT AB REF 23 Ġ). ĠHowever , Ġother Ġtwo Ġdatasets Ġdo Ġnot Ġshow Ġsuch Ġparticular Ġbias Ġagainst Ġwomen . ĠThis Ġplatform Ġspecific Ġsemantic Ġsimilarity Ġbetween Ġwords Ġis Ġa Ġkey Ġaspect Ġof Ġcyber bull ying Ġdetection Ġacross ĠS MP s . ĠStyle Ġof Ġcommunication Ġvaries Ġsignificantly Ġacross ĠS MP s . ĠFor Ġexample , ĠTwitter Ġposts Ġare Ġshort Ġand Ġlack Ġanonymity . ĠWhereas Ġposts Ġon ĠQ & A Ġoriented ĠS MP s Ġare Ġlong Ġand Ġhave Ġoption Ġof Ġanonymity Ġ( Table ĠT AB REF 7 Ġ). ĠFast Ġevolving Ġwords Ġand Ġhasht ags Ġin Ġsocial Ġmedia Ġmake Ġit Ġdifficult Ġto Ġdetect Ġcyber bull ying Ġusing Ġswear Ġword Ġlist Ġbased Ġsimple Ġfiltering Ġapproaches . ĠThe Ġoption Ġof Ġanonymity Ġin Ġcertain Ġsocial Ġnetworks Ġalso Ġmakes Ġit Ġharder Ġto Ġidentify Ġcyber bull ying Ġas Ġprofile Ġand Ġhistory Ġof Ġthe Ġbully Ġmight Ġnot Ġbe Ġavailable . </s> Past Ġworks Ġon Ġcyber bull ying Ġdetection Ġhave Ġat Ġleast Ġone Ġof Ġthe Ġfollowing Ġthree Ġbott len ec ks . ĠFirst Ġ( Bott leneck ĠB 1 ), Ġthey Ġtarget Ġonly Ġone Ġparticular Ġsocial Ġmedia Ġplatform . ĠHow Ġthese Ġmethods Ġperform Ġacross Ġother ĠS MP s Ġis Ġunknown . ĠSecond Ġ( Bott leneck ĠB 2 ), Ġthey Ġaddress Ġonly Ġone Ġtopic Ġof Ġcyber bull ying Ġsuch Ġas Ġracism , Ġand Ġsexism . ĠDepending Ġon Ġthe Ġtopic , Ġvocabulary Ġand Ġnature Ġof Ġcyber bull ying Ġchanges . ĠThese Ġmodels Ġare Ġnot Ġflexible Ġin Ġaccommodating Ġchanges Ġin Ġthe Ġdefinition Ġof Ġcyber bull ying . ĠThird Ġ( Bott leneck ĠB 3 ), Ġthey Ġrely Ġon Ġcarefully Ġhand crafted Ġfeatures Ġsuch Ġas Ġswear Ġword Ġlist Ġand ĠPOS Ġtagging . ĠHowever , Ġthese Ġhand crafted Ġfeatures Ġare Ġnot Ġrobust Ġagainst Ġvariations Ġin Ġwriting Ġstyle . ĠIn Ġcontrast Ġto Ġexisting Ġbott len ec ks , Ġthis Ġwork Ġtargets Ġthree Ġdifferent Ġtypes Ġof Ġsocial Ġnetworks Ġ( Form spring : Ġa ĠQ & A Ġforum , ĠTwitter : Ġmicro blog ging , Ġand ĠWikipedia : Ġcollaborative Ġknowledge Ġrepository ) Ġfor Ġthree Ġtopics Ġof Ġcyber bull ying Ġ( personal Ġattack , Ġracism , Ġand Ġsexism ) Ġwithout Ġdoing Ġany Ġexplicit Ġfeature Ġengineering Ġby Ġdeveloping Ġdeep Ġlearning Ġbased Ġmodels Ġalong Ġwith Ġtransfer Ġlearning . </s> We Ġexperimented Ġwith Ġdiverse Ġtraditional Ġmachine Ġlearning Ġmodels Ġ( log istic Ġregression , Ġsupport Ġvector Ġmachine , Ġrandom Ġforest , Ġnaive ĠBay es ) Ġand Ġdeep Ġneural Ġnetwork Ġmodels Ġ( CNN , ĠL ST M , ĠBL ST M , ĠBL ST M Ġwith ĠAttention ) Ġusing Ġvariety Ġof Ġrepresentation Ġmethods Ġfor Ġwords Ġ( bag Ġof Ġcharacter Ġn - gram , Ġbag Ġof Ġword Ġun ig ram , ĠGl o Ve Ġembed d ings , ĠSS WE Ġembed d ings ). ĠSummary Ġof Ġour Ġfindings Ġand Ġresearch Ġcontributions Ġis Ġas Ġfollows . </s> Dat as ets </s> Please Ġrefer Ġto ĠTable ĠT AB REF 7 Ġfor Ġsummary Ġof Ġdatasets Ġused . ĠWe Ġperformed Ġexperiments Ġusing Ġlarge , Ġdiverse , Ġmanually Ġannot ated , Ġand Ġpublicly Ġavailable Ġdatasets Ġfor Ġcyber bull ying Ġdetection Ġin Ġsocial Ġmedia . ĠWe Ġcover Ġthree Ġdifferent Ġtypes Ġof Ġsocial Ġnetworks : Ġteen Ġoriented ĠQ & A Ġforum Ġ( Form spring ), Ġlarge Ġmicro blog ging Ġplatform Ġ( Twitter ), Ġand Ġcollaborative Ġknowledge Ġrepository Ġ( Wikipedia Ġtalk Ġpages ). ĠEach Ġdataset Ġaddresses Ġa Ġdifferent Ġtopic Ġof Ġcyber bull ying . ĠTwitter Ġdataset Ġcontains Ġexamples Ġof Ġracism Ġand Ġsexism . ĠWikipedia Ġdataset Ġcontains Ġexamples Ġof Ġpersonal Ġattack . ĠHowever , ĠForm spring Ġdataset Ġis Ġnot Ġspecifically Ġabout Ġany Ġsingle Ġtopic . ĠAll Ġthree Ġdatasets Ġhave Ġthe Ġproblem Ġof Ġclass Ġimbalance Ġwhere Ġposts Ġlabeled Ġas Ġcyber bull ying Ġare Ġin Ġthe Ġminority Ġas Ġcompared Ġto Ġneutral Ġposts . ĠVari ation Ġin Ġthe Ġnumber Ġof Ġposts Ġacross Ġdatasets Ġalso Ġaffects Ġvocabulary Ġsize Ġthat Ġrepresents Ġthe Ġnumber Ġof Ġdistinct Ġwords Ġencountered Ġin Ġthe Ġdataset . ĠWe Ġmeasure Ġthe Ġsize Ġof Ġa Ġpost Ġin Ġterms Ġof Ġthe Ġnumber Ġof Ġwords Ġin Ġthe Ġpost . ĠFor Ġeach Ġdataset , Ġthere Ġare Ġonly Ġa Ġfew Ġposts Ġwith Ġlarge Ġsize . ĠWe Ġtrunc ate Ġsuch Ġlarge Ġposts Ġto Ġthe Ġsize Ġof Ġpost Ġranked Ġat Ġ95 Ġpercentile Ġin Ġthat Ġdataset . ĠFor Ġexample , Ġin ĠWikipedia Ġdataset , Ġthe Ġlargest Ġpost Ġhas Ġ28 46 Ġwords . ĠHowever , Ġsize Ġof Ġpost Ġranked Ġat Ġ95 Ġpercentile Ġin Ġthat Ġdataset Ġis Ġonly Ġ231 . ĠAny Ġpost Ġlarger Ġthan Ġsize Ġ231 Ġin ĠWikipedia Ġdataset Ġwill Ġbe Ġtrunc ated Ġby Ġconsidering Ġonly Ġfirst Ġ231 Ġwords . ĠThis Ġtrunc ation Ġaffects Ġonly Ġa Ġsmall Ġminority Ġof Ġposts Ġin Ġeach Ġdataset . ĠHowever , Ġit Ġis Ġrequired Ġfor Ġefficiently Ġtraining Ġvarious Ġmodels Ġin Ġour Ġexperiments . ĠDetails Ġof Ġeach Ġdataset Ġare Ġas Ġfollows . </s> Form spring ĠB IB REF 2 Ġ: ĠIt Ġwas Ġa Ġquestion Ġand Ġanswer Ġbased Ġwebsite Ġwhere Ġusers Ġcould Ġopenly Ġinvite Ġothers Ġto Ġask Ġand Ġanswer Ġquestions . ĠThe Ġdataset Ġincludes Ġ12 K Ġannot ated Ġquestion Ġand Ġanswer Ġpairs . ĠEach Ġpost Ġis Ġmanually Ġlabeled Ġby Ġthree Ġworkers . ĠAmong Ġthese Ġpairs , Ġ8 25 Ġwere Ġlabeled Ġas Ġcontaining Ġcyber bull ying Ġcontent Ġby Ġat Ġleast Ġtwo ĠAmazon ĠMechanical Ġtur k Ġworkers . </s> Twitter ĠB IB REF 3 Ġ: ĠThis Ġdataset Ġincludes Ġ16 K Ġannot ated Ġtweets . ĠThe Ġauthors Ġboot stra pped Ġthe Ġcorpus Ġcollection , Ġby Ġperforming Ġan Ġinitial Ġmanual Ġsearch Ġof Ġcommon Ġslurs Ġand Ġterms Ġused Ġpertaining Ġto Ġreligious , Ġsexual , Ġgender , Ġand Ġethnic Ġminorities . ĠOf Ġthe Ġ16 K Ġtweets , Ġ3 117 Ġare Ġlabeled Ġas Ġsexist , Ġ1937 Ġas Ġracist , Ġand Ġthe Ġremaining Ġare Ġmarked Ġas Ġneither Ġsexist Ġnor Ġracist . </s> Wikipedia ĠB IB REF 4 Ġ: ĠFor Ġeach Ġpage Ġin ĠWikipedia , Ġa Ġcorresponding Ġtalk Ġpage Ġmaintains Ġthe Ġhistory Ġof Ġdiscussion Ġamong Ġusers Ġwho Ġparticipated Ġin Ġits Ġediting . ĠThis Ġdata Ġset Ġincludes Ġover Ġ100 k Ġlabeled Ġdiscussion Ġcomments Ġfrom ĠEnglish ĠWikipedia 's Ġtalk Ġpages . ĠEach Ġcomment Ġwas Ġlabeled Ġby Ġ10 Ġannot ators Ġvia ĠCrowd flower Ġon Ġwhether Ġit Ġcontains Ġa Ġpersonal Ġattack . ĠThere Ġare Ġtotal Ġ13 590 Ġcomments Ġlabeled Ġas Ġpersonal Ġattack . </s> Use Ġof ĠSw ear ĠWords Ġand ĠAn onym ity </s> Please Ġrefer Ġto ĠTable ĠT AB REF 9 Ġ. ĠWe Ġuse Ġthe Ġfollowing Ġshort Ġforms Ġin Ġthis Ġsection : ĠB = Bull ying , ĠS = Sw earing , ĠA = Anonymous . ĠSome Ġof Ġthe Ġvalues Ġfor ĠTwitter Ġdataset Ġare Ġundefined Ġas ĠTwitter Ġdoes Ġnot Ġallow Ġanonymous Ġpostings . ĠUse Ġof Ġswear Ġwords Ġhas Ġbeen Ġrepeatedly Ġlinked Ġto Ġcyber bull ying . ĠHowever , Ġpreliminary Ġanalysis Ġof Ġdatasets Ġreveals Ġthat Ġdepending Ġon Ġswear Ġword Ġusage Ġcan Ġneither Ġlead Ġto Ġhigh Ġprecision Ġnor Ġhigh Ġrecall Ġfor Ġcyber bull ying Ġdetection . ĠSw ear Ġword Ġlist Ġbased Ġmethods Ġwill Ġhave Ġlow Ġprecision Ġas ĠP ( B ĠIN LINE FORM 0 ĠS ) Ġis Ġnot Ġclose Ġto Ġ1 . ĠIn Ġfact , Ġfor Ġteen Ġoriented Ġsocial Ġnetwork ĠForm spring , Ġ78 % Ġof Ġthe Ġswearing Ġposts Ġare Ġnon - bull ying . ĠSw ear Ġwords Ġbased Ġfiltering Ġwill Ġbe Ġirritating Ġto Ġthe Ġusers Ġin Ġsuch ĠS MP s Ġwhere Ġswear Ġwords Ġare Ġused Ġcasually . ĠSw ear Ġword Ġlist Ġbased Ġmethods Ġwill Ġalso Ġhave Ġa Ġlow Ġrecall Ġas ĠP ( S ĠIN LINE FORM 1 ĠB ) Ġis Ġnot Ġclose Ġto Ġ1 . ĠFor ĠTwitter Ġdataset , Ġ82 % Ġof Ġbullying Ġposts Ġdo Ġnot Ġuse Ġany Ġswear Ġwords . ĠSuch Ġpassive - aggressive Ġcyber bull ying Ġwill Ġgo Ġundet ected Ġwith Ġswear Ġword Ġlist Ġbased Ġmethods . ĠAn onym ity Ġis Ġanother Ġclue Ġthat Ġis Ġused Ġfor Ġdetecting Ġcyber bull ying Ġas Ġbully Ġmight Ġprefer Ġto Ġhide Ġits Ġidentity . ĠAn onym ity Ġdefinitely Ġleads Ġto Ġincreased Ġuse Ġof Ġswear Ġwords Ġ( P ( S ĠIN LINE FORM 2 ĠA ) ĠIN LINE FORM 3 ĠP ( S )) Ġand Ġcyber bull ying Ġ( P ( B ĠIN LINE FORM 4 ĠA ) ĠIN LINE FORM 5 ĠP ( B ), Ġand ĠP ( B ĠIN LINE FORM 6 ĠA & S )) ĠIN LINE FORM 7 ĠP ( B )). ĠHowever , Ġsignificant Ġfraction Ġof Ġanonymous Ġposts Ġare Ġnon - bull ying Ġ( P ( B ĠIN LINE FORM 8 ĠA ) Ġnot Ġclose Ġto Ġ1 ) Ġand Ġmany Ġof Ġbullying Ġposts Ġare Ġnot Ġanonymous Ġ( P ( A ĠIN LINE FORM 9 ĠB ) Ġnot Ġclose Ġto Ġ1 ). ĠFurther , Ġanonymity Ġmight Ġnot Ġbe Ġallowed Ġby Ġmany ĠS MP s Ġsuch Ġas ĠTwitter . </s> Related ĠWork </s> Cy ber bull ying Ġis Ġrecognized Ġas Ġa Ġphenomenon Ġat Ġleast Ġsince Ġ2003 ĠB IB REF 5 Ġ. ĠUse Ġof Ġsocial Ġmedia Ġexploded Ġwith Ġlaunching Ġof Ġmultiple Ġplatforms Ġsuch Ġas ĠWikipedia Ġ( 2001 ), ĠMy Space Ġ( 2003 ), ĠOr k ut Ġ( 2004 ), ĠFacebook Ġ( 2004 ), Ġand ĠTwitter Ġ( 2005 ). ĠBy Ġ2006 , Ġresearchers Ġhad Ġpointed Ġthat Ġcyber bull ying Ġwas Ġas Ġserious Ġphenomenon Ġas Ġoffline Ġbullying ĠB IB REF 6 Ġ. ĠHowever , Ġautomatic Ġdetection Ġof Ġcyber bull ying Ġwas Ġaddressed Ġonly Ġsince Ġ2009 ĠB IB REF 7 Ġ. ĠAs Ġa Ġresearch Ġtopic , Ġcyber bull ying Ġdetection Ġis Ġa Ġtext Ġclassification Ġproblem . ĠMost Ġof Ġthe Ġexisting Ġworks Ġfit Ġin Ġthe Ġfollowing Ġtemplate : Ġget Ġtraining Ġdataset Ġfrom Ġsingle ĠS MP , Ġengineer Ġvariety Ġof Ġfeatures Ġwith Ġcertain Ġstyle Ġof Ġcyber bull ying Ġas Ġthe Ġtarget , Ġapply Ġa Ġfew Ġtraditional Ġmachine Ġlearning Ġmethods , Ġand Ġevaluate Ġsuccess Ġin Ġterms Ġof Ġmeasures Ġsuch Ġas ĠF 1 Ġscore Ġand Ġaccuracy . ĠThese Ġworks Ġheavily Ġrely Ġon Ġhand crafted Ġfeatures Ġsuch Ġas Ġuse Ġof Ġswear Ġwords . ĠThese Ġmethods Ġtend Ġto Ġhave Ġlow Ġprecision Ġfor Ġcyber bull ying Ġdetection Ġas Ġhand crafted Ġfeatures Ġare Ġnot Ġrobust Ġagainst Ġvariations Ġin Ġbullying Ġstyle Ġacross ĠS MP s Ġand Ġbullying Ġtopics . ĠOnly Ġrecently , Ġdeep Ġlearning Ġhas Ġbeen Ġapplied Ġfor Ġcyber bull ying Ġdetection ĠB IB REF 8 Ġ. ĠTable ĠT AB REF 27 Ġsummarizes Ġimportant Ġrelated Ġwork . </s> Deep ĠNeural ĠNetwork Ġ( D NN ) ĠBased ĠModels </s> We Ġexperimented Ġwith Ġfour ĠD NN Ġbased Ġmodels Ġfor Ġcyber bull ying Ġdetection : ĠCNN , ĠL ST M , ĠBL ST M , Ġand ĠBL ST M Ġwith Ġattention . ĠThese Ġmodels Ġare Ġlisted Ġin Ġthe Ġincreasing Ġcomplexity Ġof Ġtheir Ġneural Ġarchitecture Ġand Ġamount Ġof Ġinformation Ġused Ġby Ġthese Ġmodels . ĠPlease Ġrefer Ġto ĠFigure Ġ1 Ġfor Ġgeneral Ġarchitecture Ġthat Ġwe Ġhave Ġused Ġacross Ġfour Ġmodels . ĠVarious Ġmodels Ġdiffer Ġonly Ġin Ġthe ĠNeural ĠArchitecture Ġlayer Ġwhile Ġhaving Ġidentical Ġrest Ġof Ġthe Ġlayers . ĠCNN s Ġare Ġproviding Ġstate - of - the - results Ġon Ġextracting Ġcontextual Ġfeature Ġfor Ġclassification Ġtasks Ġin Ġimages , Ġvideos , Ġaud ios , Ġand Ġtext . ĠRecently , ĠCNN s Ġwere Ġused Ġfor Ġsentiment Ġclassification ĠB IB REF 9 Ġ. ĠLong ĠShort ĠTerm ĠMemory Ġnetworks Ġare Ġa Ġspecial Ġkind Ġof ĠR NN , Ġcapable Ġof Ġlearning Ġlong - term Ġdependencies . ĠTheir Ġability Ġto Ġuse Ġtheir Ġinternal Ġmemory Ġto Ġprocess Ġarbitrary Ġsequences Ġof Ġinputs Ġhas Ġbeen Ġfound Ġto Ġbe Ġeffective Ġfor Ġtext Ġclassification ĠB IB REF 10 Ġ. ĠBid irection al ĠL ST Ms ĠB IB REF 11 Ġfurther Ġincrease Ġthe Ġamount Ġof Ġinput Ġinformation Ġavailable Ġto Ġthe Ġnetwork Ġby Ġencoding Ġinformation Ġin Ġboth Ġforward Ġand Ġbackward Ġdirection . ĠBy Ġusing Ġtwo Ġdirections , Ġinput Ġinformation Ġfrom Ġboth Ġthe Ġpast Ġand Ġfuture Ġof Ġthe Ġcurrent Ġtime Ġframe Ġcan Ġbe Ġused . ĠAttention Ġmechanisms Ġallow Ġfor Ġa Ġmore Ġdirect Ġdependence Ġbetween Ġthe Ġstate Ġof Ġthe Ġmodel Ġat Ġdifferent Ġpoints Ġin Ġtime . ĠImport antly , Ġattention Ġmechanism Ġlets Ġthe Ġmodel Ġlearn Ġwhat Ġto Ġattend Ġto Ġbased Ġon Ġthe Ġinput Ġsentence Ġand Ġwhat Ġit Ġhas Ġproduced Ġso Ġfar . </s> The Ġembed ding Ġlayer Ġprocesses Ġa Ġfixed Ġsize Ġsequence Ġof Ġwords . ĠEach Ġword Ġis Ġrepresented Ġas Ġa Ġreal - valued Ġvector , Ġalso Ġknown Ġas Ġword Ġembed d ings . ĠWe Ġhave Ġexperimented Ġwith Ġthree Ġmethods Ġfor Ġinitial izing Ġword Ġembed d ings : Ġrandom , ĠGl o Ve ĠB IB REF 12 Ġ, Ġand ĠSS WE ĠB IB REF 13 Ġ. ĠDuring Ġthe Ġtraining , Ġmodel Ġimproves Ġupon Ġthe Ġinitial Ġword Ġembed d ings Ġto Ġlearn Ġtask Ġspecific Ġword Ġembed d ings . ĠWe Ġhave Ġobserved Ġthat Ġthese Ġtask Ġspecific Ġword Ġembed d ings Ġcapture Ġthe ĠS MP Ġspecific Ġand Ġtopic Ġspecific Ġstyle Ġof Ġcyber bull ying . ĠUsing ĠGl o Ve Ġvectors Ġover Ġrandom Ġvector Ġinitialization Ġhas Ġbeen Ġreported Ġto Ġimprove Ġperformance Ġfor Ġsome ĠN LP Ġtasks . ĠMost Ġof Ġthe Ġword Ġembed ding Ġmethods Ġsuch Ġas ĠGl o Ve , Ġconsider Ġonly Ġsynt actic Ġcontext Ġof Ġthe Ġword Ġwhile Ġignoring Ġthe Ġsentiment Ġconveyed Ġby Ġthe Ġtext . ĠSS WE Ġmethod Ġover comes Ġthis Ġproblem Ġby Ġincorporating Ġthe Ġtext Ġsentiment Ġas Ġone Ġof Ġthe Ġparameters Ġfor Ġword Ġembed ding Ġgeneration . ĠWe Ġexperimented Ġwith Ġvarious Ġdimension Ġsize Ġfor Ġword Ġembed d ings . ĠExperimental Ġresults Ġreported Ġhere Ġare Ġwith Ġdimension Ġsize Ġas Ġ50 . ĠThere Ġwas Ġno Ġsignificant Ġvariation Ġin Ġresults Ġwith Ġdimension Ġsize Ġranging Ġfrom Ġ30 Ġto Ġ200 . </s> To Ġavoid Ġover fitting , Ġwe Ġused Ġtwo Ġdrop out Ġlayers , Ġone Ġbefore Ġthe Ġneural Ġarchitecture Ġlayer Ġand Ġone Ġafter , Ġwith Ġdrop out Ġrates Ġof Ġ0 . 25 Ġand Ġ0 . 5 Ġrespectively . ĠFully Ġconnected Ġlayer Ġis Ġa Ġdense Ġoutput Ġlayer Ġwith Ġthe Ġnumber Ġof Ġneurons Ġequal Ġto Ġthe Ġnumber Ġof Ġclasses , Ġfollowed Ġby Ġsoft max Ġlayer Ġthat Ġprovides Ġsoft max Ġactivation . ĠAll Ġour Ġmodels Ġare Ġtrained Ġusing Ġback prop ag ation . ĠThe Ġoptim izer Ġused Ġfor Ġtraining Ġis ĠAdam Ġand Ġthe Ġloss Ġfunction Ġis Ġcateg orical Ġcross - ent ropy . ĠBesides Ġlearning Ġthe Ġnetwork Ġweights , Ġthese Ġmethods Ġalso Ġlearn Ġtask - specific Ġword Ġembed d ings Ġtuned Ġtowards Ġthe Ġbullying Ġlabels Ġ( See ĠSection ĠSEC REF 21 Ġ). ĠOur Ġcode Ġis Ġavailable Ġat : Ġhttps :// github . com / sw eta 20 / Detect ing - Cy ber bull ying - Across - S MP s . </s> Exper iments </s> Ex isting Ġworks Ġhave Ġheavily Ġrelied Ġon Ġtraditional Ġmachine Ġlearning Ġmodels Ġfor Ġcyber bull ying Ġdetection . ĠHowever , Ġthey Ġdo Ġnot Ġstudy Ġthe Ġperformance Ġof Ġthese Ġmodels Ġacross Ġmultiple ĠS MP s . ĠWe Ġexperimented Ġwith Ġfour Ġmodels : Ġlog istic Ġregression Ġ( LR ), Ġsupport Ġvector Ġmachine Ġ( S VM ), Ġrandom Ġforest Ġ( RF ), Ġand Ġnaive ĠBay es Ġ( NB ), Ġas Ġthese Ġare Ġused Ġin Ġprevious Ġworks Ġ( Table ĠT AB REF 27 Ġ). ĠWe Ġused Ġtwo Ġdata Ġrepresentation Ġmethods : Ġcharacter Ġn - gram Ġand Ġword Ġun ig ram . ĠPast Ġwork Ġin Ġthe Ġdomain Ġof Ġdetecting Ġabusive Ġlanguage Ġhave Ġshowed Ġthat Ġsimple Ġn - gram Ġfeatures Ġare Ġmore Ġpowerful Ġthan Ġlinguistic Ġand Ġsynt actic Ġfeatures , Ġhand - engine ered Ġlex icons , Ġand Ġword Ġand Ġparagraph Ġembed d ings ĠB IB REF 14 Ġ. ĠAs Ġcompared Ġto ĠD NN Ġmodels , Ġperformance Ġof Ġall Ġfour Ġtraditional Ġmachine Ġlearning Ġmodels Ġwas Ġsignificantly Ġlower . ĠPlease Ġrefer Ġto ĠTable ĠT AB REF 11 Ġ. </s> All ĠD NN Ġmodels Ġreported Ġhere Ġwere Ġimplemented Ġusing ĠKer as . ĠWe Ġpre - process Ġthe Ġdata , Ġsubject ing Ġit Ġto Ġstandard Ġoperations Ġof Ġremoval Ġof Ġstop Ġwords , Ġpunct uation Ġmarks Ġand Ġlower c asing , Ġbefore Ġannot ating Ġit Ġto Ġassigning Ġrespective Ġlabels Ġto Ġeach Ġcomment . ĠFor Ġeach Ġtrained Ġmodel , Ġwe Ġreport Ġits Ġperformance Ġafter Ġdoing Ġfive - fold Ġcross - valid ation . ĠWe Ġuse Ġfollowing Ġshort Ġforms . </s> Effect Ġof ĠOvers am pling ĠBull ying ĠInst ances </s> The Ġtraining Ġdatasets Ġhad Ġa Ġmajor Ġproblem Ġof Ġclass Ġimbalance Ġwith Ġposts Ġmarked Ġas Ġbullying Ġin Ġthe Ġminority . ĠAs Ġa Ġresult , Ġall Ġmodels Ġwere Ġbiased Ġtowards Ġlabeling Ġthe Ġposts Ġas Ġnon - bull ying . ĠTo Ġremove Ġthis Ġbias , Ġwe Ġovers am pled Ġthe Ġdata Ġfrom Ġbullying Ġclass Ġthr ice . ĠThat Ġis , Ġwe Ġreplicated Ġbullying Ġposts Ġthr ice Ġin Ġthe Ġtraining Ġdata . ĠThis Ġsignificantly Ġimproved Ġthe Ġperformance Ġof Ġall ĠD NN Ġmodels Ġwith Ġmajor Ġleap Ġin Ġall Ġthree Ġevaluation Ġmeasures . ĠTable ĠT AB REF 17 Ġshows Ġthe Ġeffect Ġof Ġovers am pling Ġfor Ġa Ġvariety Ġof Ġword Ġembed ding Ġmethods Ġwith ĠBL ST M ĠAttention Ġas Ġthe Ġdetection Ġmodel . ĠResults Ġfor Ġother Ġmodels Ġare Ġsimilar ĠB IB REF 15 Ġ. ĠWe Ġcan Ġnotice Ġthat Ġovers am pled Ġdatasets Ġ( F +, ĠT +, ĠW +) Ġhave Ġfar Ġbetter Ġperformance Ġthan Ġtheir Ġcounterparts Ġ( F , ĠT , ĠW Ġrespectively ). ĠOvers am pling Ġparticularly Ġhelps Ġthe Ġsmallest Ġdataset ĠForm spring Ġwhere Ġnumber Ġof Ġtraining Ġinstances Ġfor Ġbullying Ġclass Ġis Ġquite Ġsmall Ġ( 825 ) Ġas Ġcompared Ġto Ġother Ġtwo Ġdatasets Ġ( about Ġ5 K Ġand Ġ13 K ). ĠWe Ġalso Ġexperimented Ġwith Ġvarying Ġthe Ġreplication Ġrate Ġfor Ġbullying Ġposts ĠB IB REF 15 Ġ. ĠHowever , Ġwe Ġobserved Ġthat Ġfor Ġbullying Ġposts , Ġreplication Ġrate Ġof Ġthree Ġis Ġgood Ġenough . </s> Choice Ġof ĠInitial ĠWord ĠEmb edd ings Ġand ĠModel </s> Initial Ġword Ġembed d ings Ġdecide Ġdata Ġrepresentation Ġfor ĠD NN Ġmodels . ĠHowever Ġduring Ġthe Ġtraining , ĠD NN Ġmodels Ġmodify Ġthese Ġinitial Ġword Ġembed d ings Ġto Ġlearn Ġtask Ġspecific Ġword Ġembed d ings . ĠWe Ġhave Ġexperimented Ġwith Ġthree Ġmethods Ġto Ġinitialize Ġword Ġembed d ings . ĠPlease Ġrefer Ġto ĠTable ĠT AB REF 19 Ġ. ĠThis Ġtable Ġshows Ġthe Ġeffect Ġof Ġvarying Ġinitial Ġword Ġembed d ings Ġfor Ġmultiple ĠD NN Ġmodels Ġacross Ġdatasets . ĠWe Ġcan Ġnotice Ġthat Ġinitial Ġword Ġembed d ings Ġdo Ġnot Ġhave Ġa Ġsignificant Ġeffect Ġon Ġcyber bull ying Ġdetection Ġwhen Ġovers am pling Ġof Ġbullying Ġposts Ġis Ġdone Ġ( rows Ġcorresponding Ġto ĠF +, ĠT +, ĠW + ). ĠIn Ġthe Ġabsence Ġof Ġovers am pling Ġ( rows Ġcorresponding Ġto ĠF , ĠT ĠW ), Ġthere Ġis Ġa Ġgap Ġin Ġperformance Ġof Ġsimplest Ġ( CNN ) Ġand Ġmost Ġcomplex Ġ( BL ST M Ġwith Ġattention ) Ġmodels . ĠHowever , Ġthis Ġgap Ġgoes Ġon Ġreducing Ġwith Ġthe Ġincrease Ġin Ġthe Ġsize Ġof Ġdatasets . </s> Table ĠT AB REF 20 Ġcompares Ġthe Ġperformance Ġof Ġfour ĠD NN Ġmodels Ġfor Ġthree Ġevaluation Ġmeasures Ġwhile Ġusing ĠSS WE Ġas Ġthe Ġinitial Ġword Ġembed d ings . ĠWe Ġhave Ġnoticed Ġthat Ġmost Ġof Ġthe Ġtime ĠL ST M Ġperforms Ġweaker Ġthan Ġother Ġthree Ġmodels . ĠHowever , Ġperformance Ġgap Ġin Ġthe Ġother Ġthree Ġmodels Ġis Ġnot Ġsignificant . </s> Task ĠSpecific ĠWord ĠEmb edd ings </s> D NN Ġmodels Ġlearn Ġword Ġembed d ings Ġover Ġthe Ġtraining Ġdata . ĠThese Ġlearned Ġembed d ings Ġacross Ġmultiple Ġdatasets Ġshow Ġthe Ġdifference Ġin Ġnature Ġand Ġstyle Ġof Ġbullying Ġacross Ġcyber bull ying Ġtopics Ġand ĠS MP s . ĠHere Ġwe Ġreport Ġresults Ġfor ĠBL ST M Ġwith Ġattention Ġmodel . ĠResults Ġfor Ġother Ġmodels Ġare Ġsimilar . ĠWe Ġfirst Ġverify Ġthat Ġimportant Ġwords Ġfor Ġeach Ġtopic Ġof Ġcyber bull ying Ġform Ġclusters Ġin Ġthe Ġlearned Ġembed d ings . ĠTo Ġenable Ġthe Ġvisualization Ġof Ġgrouping , Ġwe Ġreduced Ġdimension ality Ġwith Ġt - S NE ĠB IB REF 16 Ġ, Ġa Ġwell - known Ġtechnique Ġfor Ġdimension ality Ġreduction Ġparticularly Ġwell Ġsuited Ġfor Ġvisualization Ġof Ġhigh Ġdimensional Ġdatasets . ĠPlease Ġrefer Ġto ĠTable ĠT AB REF 22 Ġ. ĠThis Ġtable Ġshows Ġimportant Ġclusters Ġobserved Ġin Ġt - S NE Ġprojection Ġof Ġlearned Ġword Ġembed d ings . ĠEach Ġcluster Ġshows Ġthat Ġwords Ġmost Ġrelevant Ġto Ġa Ġparticular Ġtopic Ġof Ġbullying Ġform Ġcluster . </s> We Ġalso Ġobserved Ġchanges Ġin Ġthe Ġmeanings Ġof Ġthe Ġwords Ġacross Ġtopics Ġof Ġcyber bull ying . ĠTable ĠT AB REF 23 Ġshows Ġmost Ġsimilar Ġwords Ġfor Ġa Ġgiven Ġquery Ġword Ġfor Ġtwo Ġdatasets . ĠTwitter Ġdataset Ġwhich Ġis Ġheavy Ġon Ġsexism Ġand Ġracism , Ġconsiders Ġword Ġslave Ġas Ġsimilar Ġto Ġtargets Ġof Ġracism Ġand Ġsexism . ĠHowever , ĠWikipedia Ġdataset Ġthat Ġis Ġabout Ġpersonal Ġattacks Ġdoes Ġnot Ġshow Ġsuch Ġbias . </s> Transfer ĠLearning </s> We Ġused Ġtransfer Ġlearning Ġto Ġcheck Ġif Ġthe Ġknowledge Ġgained Ġby ĠD NN Ġmodels Ġon Ġone Ġdataset Ġcan Ġbe Ġused Ġto Ġimprove Ġcyber bull ying Ġdetection Ġperformance Ġon Ġother Ġdatasets . ĠWe Ġreport Ġresults Ġwhere ĠBL ST M Ġwith Ġattention Ġis Ġused Ġas Ġthe ĠD NN Ġmodel . ĠResults Ġfor Ġother Ġmodels Ġare Ġsimilar ĠB IB REF 15 Ġ. ĠWe Ġexperimented Ġwith Ġfollowing Ġthree Ġflavors Ġof Ġtransfer Ġlearning . </s> Complete ĠTransfer ĠLearning Ġ( TL 1 ): ĠIn Ġthis Ġflavor , Ġa Ġmodel Ġtrained Ġon Ġone Ġdataset Ġwas Ġdirectly Ġused Ġto Ġdetect Ġcyber bull ying Ġin Ġother Ġdatasets Ġwithout Ġany Ġextra Ġtraining . ĠTL 1 Ġresulted Ġin Ġsignificantly Ġlow Ġrecall Ġindicating Ġthat Ġthree Ġdatasets Ġhave Ġdifferent Ġnature Ġof Ġcyber bull ying Ġwith Ġlow Ġoverlap Ġ( Table ĠT AB REF 25 Ġ). ĠHowever Ġprecision Ġwas Ġrelatively Ġhigher Ġfor ĠTL 1 , Ġindicating Ġthat ĠD NN Ġmodels Ġare Ġcautious Ġin Ġlabeling Ġa Ġpost Ġas Ġbully Ġ( Table ĠT AB REF 25 Ġ). ĠTL 1 Ġalso Ġhelps Ġto Ġmeasure Ġsimilarity Ġin Ġnature Ġof Ġcyber bull ying Ġacross Ġthree Ġdatasets . ĠWe Ġcan Ġobserve Ġthat Ġbullying Ġnature Ġin ĠForm spring Ġand ĠWikipedia Ġdatasets Ġis Ġmore Ġsimilar Ġto Ġeach Ġother Ġthan Ġthe ĠTwitter Ġdataset . ĠThis Ġcan Ġbe Ġinferred Ġfrom Ġthe Ġfact Ġthat Ġwith ĠTL 1 , Ġcyber bull ying Ġdetection Ġperformance Ġfor ĠForm spring Ġdataset Ġis Ġhigher Ġwhen Ġbase Ġmodel Ġis ĠWikipedia Ġ( pre cision Ġ= 0 . 51 Ġand Ġrecall = 0 . 66 ) as Ġcompared Ġto ĠTwitter Ġas Ġthe Ġbase Ġmodel Ġ( pre cision = 0 . 38 Ġand Ġrecall = 0 . 04 ). ĠSimilarly , Ġfor ĠWikipedia Ġdataset , ĠForm spring Ġacts Ġas Ġa Ġbetter Ġbase Ġmodel Ġthan ĠTwitter Ġwhile Ġusing ĠTL 1 Ġflavor Ġof Ġtransfer Ġlearning . ĠNature Ġof ĠS MP Ġmight Ġbe Ġa Ġfactor Ġbehind Ġthis Ġsimilarity Ġin Ġnature Ġof Ġcyber bull ying . ĠBoth ĠForm spring Ġand ĠWikipedia Ġare Ġtask Ġoriented Ġsocial Ġnetworks Ġ( Q & A Ġand Ġcollaborative Ġknowledge Ġrepository Ġrespectively ) Ġthat Ġallow Ġanonymity Ġand Ġlarger Ġposts . ĠWhereas Ġcommunication Ġon ĠTwitter Ġis Ġshort , Ġfree Ġof Ġanonymity Ġand Ġnot Ġoriented Ġtowards Ġa Ġparticular Ġtask . </s> Feature ĠLevel ĠTransfer ĠLearning Ġ( TL 2 ): ĠIn Ġthis Ġflavor , Ġa Ġmodel Ġwas Ġtrained Ġon Ġone Ġdataset Ġand Ġonly Ġlearned Ġword Ġembed d ings Ġwere Ġtransferred Ġto Ġanother Ġdataset Ġfor Ġtraining Ġa Ġnew Ġmodel . ĠAs Ġcompared Ġto ĠTL 1 , Ġrecall Ġscore Ġimproved Ġdramatically Ġwith ĠTL 2 Ġ( Table ĠT AB REF 25 Ġ). ĠImprovement Ġin Ġprecision Ġwas Ġalso Ġsignificant Ġ( Table ĠT AB REF 25 Ġ). ĠThese Ġimprovements Ġindicate Ġthat Ġlearned Ġword Ġembed d ings Ġare Ġan Ġessential Ġpart Ġof Ġknowledge Ġtransfer Ġacross Ġdatasets Ġfor Ġcyber bull ying Ġdetection . </s> Model ĠLevel ĠTransfer ĠLearning Ġ( TL 3 ): ĠIn Ġthis Ġflavor , Ġa Ġmodel Ġwas Ġtrained Ġon Ġone Ġdataset Ġand Ġlearned Ġword Ġembed d ings , Ġas Ġwell Ġas Ġnetwork Ġweights , Ġwere Ġtransferred Ġto Ġanother Ġdataset Ġfor Ġtraining Ġa Ġnew Ġmodel . ĠTL 3 Ġdoes Ġnot Ġresult Ġin Ġany Ġsignificant Ġimprovement Ġover ĠTL 2 . ĠThis Ġlack Ġof Ġimprovement Ġindicates Ġthat Ġtransfer Ġof Ġnetwork Ġweights Ġis Ġnot Ġessential Ġfor Ġcyber bull ying Ġdetection Ġand Ġlearned Ġword Ġembed d ings Ġis Ġthe Ġkey Ġknowledge Ġgained Ġby Ġthe ĠD NN Ġmodels . </s> D NN Ġbased Ġmodels Ġcoupled Ġwith Ġtransfer Ġlearning Ġbeat Ġthe Ġbest - known Ġresults Ġfor Ġall Ġthree Ġdatasets . ĠPrevious Ġbest ĠF 1 Ġscores Ġfor ĠWikipedia ĠB IB REF 4 Ġand ĠTwitter ĠB IB REF 8 Ġdatasets Ġwere Ġ0 . 68 Ġand Ġ0 . 93 Ġrespectively . ĠWe Ġachieve ĠF 1 Ġscores Ġof Ġ0 . 94 Ġfor Ġboth Ġthese Ġdatasets Ġusing ĠBL ST M Ġwith Ġattention Ġand Ġfeature Ġlevel Ġtransfer Ġlearning Ġ( Table ĠT AB REF 25 Ġ). ĠFor ĠForm spring Ġdataset , Ġauthors Ġhave Ġnot Ġreported ĠF 1 Ġscore . ĠTheir Ġmethod Ġhas Ġaccuracy Ġscore Ġof Ġ78 . 5 % ĠB IB REF 2 Ġ. ĠWe Ġachieve ĠF 1 Ġscore Ġof Ġ0 . 95 Ġwith Ġaccuracy Ġscore Ġof Ġ98 % Ġfor Ġthe Ġsame Ġdataset . </s> Conclusion Ġand ĠFuture ĠWork </s> We Ġhave Ġshown Ġthat ĠD NN Ġmodels Ġcan Ġbe Ġused Ġfor Ġcyber bull ying Ġdetection Ġon Ġvarious Ġtopics Ġacross Ġmultiple ĠS MP s Ġusing Ġthree Ġdatasets Ġand Ġfour ĠD NN Ġmodels . ĠThese Ġmodels Ġcoupled Ġwith Ġtransfer Ġlearning Ġbeat Ġstate Ġof Ġthe Ġart Ġresults Ġfor Ġall Ġthree Ġdatasets . ĠThese Ġmodels Ġcan Ġbe Ġfurther Ġimproved Ġwith Ġextra Ġdata Ġsuch Ġas Ġinformation Ġabout Ġthe Ġprofile Ġand Ġsocial Ġgraph Ġof Ġusers . ĠMost Ġof Ġthe Ġcurrent Ġdatasets Ġdo Ġnot Ġprovide Ġany Ġinformation Ġabout Ġthe Ġseverity Ġof Ġbullying . ĠIf Ġsuch Ġfine - gr ained Ġinformation Ġis Ġmade Ġavailable , Ġthen Ġcyber bull ying Ġdetection Ġmodels Ġcan Ġbe Ġfurther Ġimproved Ġto Ġtake Ġa Ġvariety Ġof Ġactions Ġdepending Ġon Ġthe Ġperceived Ġseriousness Ġof Ġthe Ġposts .
Introduction Cyberbullying has been defined by the National Crime Prevention Council as the use of the Internet, cell phones or other devices to send or post text or images intended to hurt or embarrass another person. Various studies have estimated that between to 10% to 40% of internet users are victims of cyberbullying BIBREF0 . Effects of cyberbullying can range from temporary anxiety to suicide BIBREF1 . Many high profile incidents have emphasized the prevalence of cyberbullying on social media. Most recently in October 2017, a Swedish model Arvida Byström was cyberbullied to the extent of receiving rape threats after she appeared in an advertisement with hairy legs. Detection of cyberbullying in social media is a challenging task. Definition of what constitutes cyberbullying is quite subjective. For example, frequent use of swear words might be considered as bullying by the general population. However, for teen oriented social media platforms such as Formspring, this does not necessarily mean bullying (Table TABREF9 ). Across multiple SMPs, cyberbullies attack victims on different topics such as race, religion, and gender. Depending on the topic of cyberbullying, vocabulary and perceived meaning of words vary significantly across SMPs. For example, in our experiments we found that for word `fat', the most similar words as per Twitter dataset are `female' and `woman' (Table TABREF23 ). However, other two datasets do not show such particular bias against women. This platform specific semantic similarity between words is a key aspect of cyberbullying detection across SMPs. Style of communication varies significantly across SMPs. For example, Twitter posts are short and lack anonymity. Whereas posts on Q&A oriented SMPs are long and have option of anonymity (Table TABREF7 ). Fast evolving words and hashtags in social media make it difficult to detect cyberbullying using swear word list based simple filtering approaches. The option of anonymity in certain social networks also makes it harder to identify cyberbullying as profile and history of the bully might not be available. Past works on cyberbullying detection have at least one of the following three bottlenecks. First (Bottleneck B1), they target only one particular social media platform. How these methods perform across other SMPs is unknown. Second (Bottleneck B2), they address only one topic of cyberbullying such as racism, and sexism. Depending on the topic, vocabulary and nature of cyberbullying changes. These models are not flexible in accommodating changes in the definition of cyberbullying. Third (Bottleneck B3), they rely on carefully handcrafted features such as swear word list and POS tagging. However, these handcrafted features are not robust against variations in writing style. In contrast to existing bottlenecks, this work targets three different types of social networks (Formspring: a Q&A forum, Twitter: microblogging, and Wikipedia: collaborative knowledge repository) for three topics of cyberbullying (personal attack, racism, and sexism) without doing any explicit feature engineering by developing deep learning based models along with transfer learning. We experimented with diverse traditional machine learning models (logistic regression, support vector machine, random forest, naive Bayes) and deep neural network models (CNN, LSTM, BLSTM, BLSTM with Attention) using variety of representation methods for words (bag of character n-gram, bag of word unigram, GloVe embeddings, SSWE embeddings). Summary of our findings and research contributions is as follows. Datasets Please refer to Table TABREF7 for summary of datasets used. We performed experiments using large, diverse, manually annotated, and publicly available datasets for cyberbullying detection in social media. We cover three different types of social networks: teen oriented Q&A forum (Formspring), large microblogging platform (Twitter), and collaborative knowledge repository (Wikipedia talk pages). Each dataset addresses a different topic of cyberbullying. Twitter dataset contains examples of racism and sexism. Wikipedia dataset contains examples of personal attack. However, Formspring dataset is not specifically about any single topic. All three datasets have the problem of class imbalance where posts labeled as cyberbullying are in the minority as compared to neutral posts. Variation in the number of posts across datasets also affects vocabulary size that represents the number of distinct words encountered in the dataset. We measure the size of a post in terms of the number of words in the post. For each dataset, there are only a few posts with large size. We truncate such large posts to the size of post ranked at 95 percentile in that dataset. For example, in Wikipedia dataset, the largest post has 2846 words. However, size of post ranked at 95 percentile in that dataset is only 231. Any post larger than size 231 in Wikipedia dataset will be truncated by considering only first 231 words. This truncation affects only a small minority of posts in each dataset. However, it is required for efficiently training various models in our experiments. Details of each dataset are as follows. Formspring BIBREF2 : It was a question and answer based website where users could openly invite others to ask and answer questions. The dataset includes 12K annotated question and answer pairs. Each post is manually labeled by three workers. Among these pairs, 825 were labeled as containing cyberbullying content by at least two Amazon Mechanical turk workers. Twitter BIBREF3 : This dataset includes 16K annotated tweets. The authors bootstrapped the corpus collection, by performing an initial manual search of common slurs and terms used pertaining to religious, sexual, gender, and ethnic minorities. Of the 16K tweets, 3117 are labeled as sexist, 1937 as racist, and the remaining are marked as neither sexist nor racist. Wikipedia BIBREF4 : For each page in Wikipedia, a corresponding talk page maintains the history of discussion among users who participated in its editing. This data set includes over 100k labeled discussion comments from English Wikipedia's talk pages. Each comment was labeled by 10 annotators via Crowdflower on whether it contains a personal attack. There are total 13590 comments labeled as personal attack. Use of Swear Words and Anonymity Please refer to Table TABREF9 . We use the following short forms in this section: B=Bullying, S=Swearing, A=Anonymous. Some of the values for Twitter dataset are undefined as Twitter does not allow anonymous postings. Use of swear words has been repeatedly linked to cyberbullying. However, preliminary analysis of datasets reveals that depending on swear word usage can neither lead to high precision nor high recall for cyberbullying detection. Swear word list based methods will have low precision as P(B INLINEFORM0 S) is not close to 1. In fact, for teen oriented social network Formspring, 78% of the swearing posts are non-bullying. Swear words based filtering will be irritating to the users in such SMPs where swear words are used casually. Swear word list based methods will also have a low recall as P(S INLINEFORM1 B) is not close to 1. For Twitter dataset, 82% of bullying posts do not use any swear words. Such passive-aggressive cyberbullying will go undetected with swear word list based methods. Anonymity is another clue that is used for detecting cyberbullying as bully might prefer to hide its identity. Anonymity definitely leads to increased use of swear words (P(S INLINEFORM2 A) INLINEFORM3 P(S)) and cyberbullying (P(B INLINEFORM4 A) INLINEFORM5 P(B), and P(B INLINEFORM6 A&S)) INLINEFORM7 P(B)). However, significant fraction of anonymous posts are non-bullying (P(B INLINEFORM8 A) not close to 1) and many of bullying posts are not anonymous (P(A INLINEFORM9 B) not close to 1). Further, anonymity might not be allowed by many SMPs such as Twitter. Related Work Cyberbullying is recognized as a phenomenon at least since 2003 BIBREF5 . Use of social media exploded with launching of multiple platforms such as Wikipedia (2001), MySpace (2003), Orkut (2004), Facebook (2004), and Twitter (2005). By 2006, researchers had pointed that cyberbullying was as serious phenomenon as offline bullying BIBREF6 . However, automatic detection of cyberbullying was addressed only since 2009 BIBREF7 . As a research topic, cyberbullying detection is a text classification problem. Most of the existing works fit in the following template: get training dataset from single SMP, engineer variety of features with certain style of cyberbullying as the target, apply a few traditional machine learning methods, and evaluate success in terms of measures such as F1 score and accuracy. These works heavily rely on handcrafted features such as use of swear words. These methods tend to have low precision for cyberbullying detection as handcrafted features are not robust against variations in bullying style across SMPs and bullying topics. Only recently, deep learning has been applied for cyberbullying detection BIBREF8 . Table TABREF27 summarizes important related work. Deep Neural Network (DNN) Based Models We experimented with four DNN based models for cyberbullying detection: CNN, LSTM, BLSTM, and BLSTM with attention. These models are listed in the increasing complexity of their neural architecture and amount of information used by these models. Please refer to Figure 1 for general architecture that we have used across four models. Various models differ only in the Neural Architecture layer while having identical rest of the layers. CNNs are providing state-of-the-results on extracting contextual feature for classification tasks in images, videos, audios, and text. Recently, CNNs were used for sentiment classification BIBREF9 . Long Short Term Memory networks are a special kind of RNN, capable of learning long-term dependencies. Their ability to use their internal memory to process arbitrary sequences of inputs has been found to be effective for text classification BIBREF10 . Bidirectional LSTMs BIBREF11 further increase the amount of input information available to the network by encoding information in both forward and backward direction. By using two directions, input information from both the past and future of the current time frame can be used. Attention mechanisms allow for a more direct dependence between the state of the model at different points in time. Importantly, attention mechanism lets the model learn what to attend to based on the input sentence and what it has produced so far. The embedding layer processes a fixed size sequence of words. Each word is represented as a real-valued vector, also known as word embeddings. We have experimented with three methods for initializing word embeddings: random, GloVe BIBREF12 , and SSWE BIBREF13 . During the training, model improves upon the initial word embeddings to learn task specific word embeddings. We have observed that these task specific word embeddings capture the SMP specific and topic specific style of cyberbullying. Using GloVe vectors over random vector initialization has been reported to improve performance for some NLP tasks. Most of the word embedding methods such as GloVe, consider only syntactic context of the word while ignoring the sentiment conveyed by the text. SSWE method overcomes this problem by incorporating the text sentiment as one of the parameters for word embedding generation. We experimented with various dimension size for word embeddings. Experimental results reported here are with dimension size as 50. There was no significant variation in results with dimension size ranging from 30 to 200. To avoid overfitting, we used two dropout layers, one before the neural architecture layer and one after, with dropout rates of 0.25 and 0.5 respectively. Fully connected layer is a dense output layer with the number of neurons equal to the number of classes, followed by softmax layer that provides softmax activation. All our models are trained using backpropagation. The optimizer used for training is Adam and the loss function is categorical cross-entropy. Besides learning the network weights, these methods also learn task-specific word embeddings tuned towards the bullying labels (See Section SECREF21 ). Our code is available at: https://github.com/sweta20/Detecting-Cyberbullying-Across-SMPs. Experiments Existing works have heavily relied on traditional machine learning models for cyberbullying detection. However, they do not study the performance of these models across multiple SMPs. We experimented with four models: logistic regression (LR), support vector machine (SVM), random forest (RF), and naive Bayes (NB), as these are used in previous works (Table TABREF27 ). We used two data representation methods: character n-gram and word unigram. Past work in the domain of detecting abusive language have showed that simple n-gram features are more powerful than linguistic and syntactic features, hand-engineered lexicons, and word and paragraph embeddings BIBREF14 . As compared to DNN models, performance of all four traditional machine learning models was significantly lower. Please refer to Table TABREF11 . All DNN models reported here were implemented using Keras. We pre-process the data, subjecting it to standard operations of removal of stop words, punctuation marks and lowercasing, before annotating it to assigning respective labels to each comment. For each trained model, we report its performance after doing five-fold cross-validation. We use following short forms. Effect of Oversampling Bullying Instances The training datasets had a major problem of class imbalance with posts marked as bullying in the minority. As a result, all models were biased towards labeling the posts as non-bullying. To remove this bias, we oversampled the data from bullying class thrice. That is, we replicated bullying posts thrice in the training data. This significantly improved the performance of all DNN models with major leap in all three evaluation measures. Table TABREF17 shows the effect of oversampling for a variety of word embedding methods with BLSTM Attention as the detection model. Results for other models are similar BIBREF15 . We can notice that oversampled datasets (F+, T+, W+) have far better performance than their counterparts (F, T, W respectively). Oversampling particularly helps the smallest dataset Formspring where number of training instances for bullying class is quite small (825) as compared to other two datasets (about 5K and 13K). We also experimented with varying the replication rate for bullying posts BIBREF15 . However, we observed that for bullying posts, replication rate of three is good enough. Choice of Initial Word Embeddings and Model Initial word embeddings decide data representation for DNN models. However during the training, DNN models modify these initial word embeddings to learn task specific word embeddings. We have experimented with three methods to initialize word embeddings. Please refer to Table TABREF19 . This table shows the effect of varying initial word embeddings for multiple DNN models across datasets. We can notice that initial word embeddings do not have a significant effect on cyberbullying detection when oversampling of bullying posts is done (rows corresponding to F+, T+, W+). In the absence of oversampling (rows corresponding to F, T W), there is a gap in performance of simplest (CNN) and most complex (BLSTM with attention) models. However, this gap goes on reducing with the increase in the size of datasets. Table TABREF20 compares the performance of four DNN models for three evaluation measures while using SSWE as the initial word embeddings. We have noticed that most of the time LSTM performs weaker than other three models. However, performance gap in the other three models is not significant. Task Specific Word Embeddings DNN models learn word embeddings over the training data. These learned embeddings across multiple datasets show the difference in nature and style of bullying across cyberbullying topics and SMPs. Here we report results for BLSTM with attention model. Results for other models are similar. We first verify that important words for each topic of cyberbullying form clusters in the learned embeddings. To enable the visualization of grouping, we reduced dimensionality with t-SNE BIBREF16 , a well-known technique for dimensionality reduction particularly well suited for visualization of high dimensional datasets. Please refer to Table TABREF22 . This table shows important clusters observed in t-SNE projection of learned word embeddings. Each cluster shows that words most relevant to a particular topic of bullying form cluster. We also observed changes in the meanings of the words across topics of cyberbullying. Table TABREF23 shows most similar words for a given query word for two datasets. Twitter dataset which is heavy on sexism and racism, considers word slave as similar to targets of racism and sexism. However, Wikipedia dataset that is about personal attacks does not show such bias. Transfer Learning We used transfer learning to check if the knowledge gained by DNN models on one dataset can be used to improve cyberbullying detection performance on other datasets. We report results where BLSTM with attention is used as the DNN model. Results for other models are similar BIBREF15 . We experimented with following three flavors of transfer learning. Complete Transfer Learning (TL1): In this flavor, a model trained on one dataset was directly used to detect cyberbullying in other datasets without any extra training. TL1 resulted in significantly low recall indicating that three datasets have different nature of cyberbullying with low overlap (Table TABREF25 ). However precision was relatively higher for TL1, indicating that DNN models are cautious in labeling a post as bully (Table TABREF25 ). TL1 also helps to measure similarity in nature of cyberbullying across three datasets. We can observe that bullying nature in Formspring and Wikipedia datasets is more similar to each other than the Twitter dataset. This can be inferred from the fact that with TL1, cyberbullying detection performance for Formspring dataset is higher when base model is Wikipedia (precision =0.51 and recall=0.66)as compared to Twitter as the base model (precision=0.38 and recall=0.04). Similarly, for Wikipedia dataset, Formspring acts as a better base model than Twitter while using TL1 flavor of transfer learning. Nature of SMP might be a factor behind this similarity in nature of cyberbullying. Both Formspring and Wikipedia are task oriented social networks (Q&A and collaborative knowledge repository respectively) that allow anonymity and larger posts. Whereas communication on Twitter is short, free of anonymity and not oriented towards a particular task. Feature Level Transfer Learning (TL2): In this flavor, a model was trained on one dataset and only learned word embeddings were transferred to another dataset for training a new model. As compared to TL1, recall score improved dramatically with TL2 (Table TABREF25 ). Improvement in precision was also significant (Table TABREF25 ). These improvements indicate that learned word embeddings are an essential part of knowledge transfer across datasets for cyberbullying detection. Model Level Transfer Learning (TL3): In this flavor, a model was trained on one dataset and learned word embeddings, as well as network weights, were transferred to another dataset for training a new model. TL3 does not result in any significant improvement over TL2. This lack of improvement indicates that transfer of network weights is not essential for cyberbullying detection and learned word embeddings is the key knowledge gained by the DNN models. DNN based models coupled with transfer learning beat the best-known results for all three datasets. Previous best F1 scores for Wikipedia BIBREF4 and Twitter BIBREF8 datasets were 0.68 and 0.93 respectively. We achieve F1 scores of 0.94 for both these datasets using BLSTM with attention and feature level transfer learning (Table TABREF25 ). For Formspring dataset, authors have not reported F1 score. Their method has accuracy score of 78.5% BIBREF2 . We achieve F1 score of 0.95 with accuracy score of 98% for the same dataset. Conclusion and Future Work We have shown that DNN models can be used for cyberbullying detection on various topics across multiple SMPs using three datasets and four DNN models. These models coupled with transfer learning beat state of the art results for all three datasets. These models can be further improved with extra data such as information about the profile and social graph of users. Most of the current datasets do not provide any information about the severity of bullying. If such fine-grained information is made available, then cyberbullying detection models can be further improved to take a variety of actions depending on the perceived seriousness of the posts.
What cyberbulling topics did they address?
7e38e0279a620d3df05ab9b5e2795044f18d4471
What Ġcyber bull ing Ġtopics Ġdid Ġthey Ġaddress ?
personal Ġattack , Ġracism , Ġand Ġsexism
<s> What Ġcyber bull ing Ġtopics Ġdid Ġthey Ġaddress ? </s> Introduction </s> Cy ber bull ying Ġhas Ġbeen Ġdefined Ġby Ġthe ĠNational ĠCrime ĠPrevention ĠCouncil Ġas Ġthe Ġuse Ġof Ġthe ĠInternet , Ġcell Ġphones Ġor Ġother Ġdevices Ġto Ġsend Ġor Ġpost Ġtext Ġor Ġimages Ġintended Ġto Ġhurt Ġor Ġembarrass Ġanother Ġperson . ĠVarious Ġstudies Ġhave Ġestimated Ġthat Ġbetween Ġto Ġ10 % Ġto Ġ40 % Ġof Ġinternet Ġusers Ġare Ġvictims Ġof Ġcyber bull ying ĠB IB REF 0 Ġ. ĠEffects Ġof Ġcyber bull ying Ġcan Ġrange Ġfrom Ġtemporary Ġanxiety Ġto Ġsuicide ĠB IB REF 1 Ġ. ĠMany Ġhigh Ġprofile Ġincidents Ġhave Ġemphasized Ġthe Ġprevalence Ġof Ġcyber bull ying Ġon Ġsocial Ġmedia . ĠMost Ġrecently Ġin ĠOctober Ġ2017 , Ġa ĠSwedish Ġmodel ĠAr v ida ĠBy str Ã¶ m Ġwas Ġcyber bull ied Ġto Ġthe Ġextent Ġof Ġreceiving Ġrape Ġthreats Ġafter Ġshe Ġappeared Ġin Ġan Ġadvertisement Ġwith Ġhairy Ġlegs . </s> Det ection Ġof Ġcyber bull ying Ġin Ġsocial Ġmedia Ġis Ġa Ġchallenging Ġtask . ĠDefinition Ġof Ġwhat Ġconstitutes Ġcyber bull ying Ġis Ġquite Ġsubjective . ĠFor Ġexample , Ġfrequent Ġuse Ġof Ġswear Ġwords Ġmight Ġbe Ġconsidered Ġas Ġbullying Ġby Ġthe Ġgeneral Ġpopulation . ĠHowever , Ġfor Ġteen Ġoriented Ġsocial Ġmedia Ġplatforms Ġsuch Ġas ĠForm spring , Ġthis Ġdoes Ġnot Ġnecessarily Ġmean Ġbullying Ġ( Table ĠT AB REF 9 Ġ). ĠAcross Ġmultiple ĠS MP s , Ġcyber bull ies Ġattack Ġvictims Ġon Ġdifferent Ġtopics Ġsuch Ġas Ġrace , Ġreligion , Ġand Ġgender . ĠDepending Ġon Ġthe Ġtopic Ġof Ġcyber bull ying , Ġvocabulary Ġand Ġperceived Ġmeaning Ġof Ġwords Ġvary Ġsignificantly Ġacross ĠS MP s . ĠFor Ġexample , Ġin Ġour Ġexperiments Ġwe Ġfound Ġthat Ġfor Ġword Ġ` fat ', Ġthe Ġmost Ġsimilar Ġwords Ġas Ġper ĠTwitter Ġdataset Ġare Ġ` female ' Ġand Ġ` woman ' Ġ( Table ĠT AB REF 23 Ġ). ĠHowever , Ġother Ġtwo Ġdatasets Ġdo Ġnot Ġshow Ġsuch Ġparticular Ġbias Ġagainst Ġwomen . ĠThis Ġplatform Ġspecific Ġsemantic Ġsimilarity Ġbetween Ġwords Ġis Ġa Ġkey Ġaspect Ġof Ġcyber bull ying Ġdetection Ġacross ĠS MP s . ĠStyle Ġof Ġcommunication Ġvaries Ġsignificantly Ġacross ĠS MP s . ĠFor Ġexample , ĠTwitter Ġposts Ġare Ġshort Ġand Ġlack Ġanonymity . ĠWhereas Ġposts Ġon ĠQ & A Ġoriented ĠS MP s Ġare Ġlong Ġand Ġhave Ġoption Ġof Ġanonymity Ġ( Table ĠT AB REF 7 Ġ). ĠFast Ġevolving Ġwords Ġand Ġhasht ags Ġin Ġsocial Ġmedia Ġmake Ġit Ġdifficult Ġto Ġdetect Ġcyber bull ying Ġusing Ġswear Ġword Ġlist Ġbased Ġsimple Ġfiltering Ġapproaches . ĠThe Ġoption Ġof Ġanonymity Ġin Ġcertain Ġsocial Ġnetworks Ġalso Ġmakes Ġit Ġharder Ġto Ġidentify Ġcyber bull ying Ġas Ġprofile Ġand Ġhistory Ġof Ġthe Ġbully Ġmight Ġnot Ġbe Ġavailable . </s> Past Ġworks Ġon Ġcyber bull ying Ġdetection Ġhave Ġat Ġleast Ġone Ġof Ġthe Ġfollowing Ġthree Ġbott len ec ks . ĠFirst Ġ( Bott leneck ĠB 1 ), Ġthey Ġtarget Ġonly Ġone Ġparticular Ġsocial Ġmedia Ġplatform . ĠHow Ġthese Ġmethods Ġperform Ġacross Ġother ĠS MP s Ġis Ġunknown . ĠSecond Ġ( Bott leneck ĠB 2 ), Ġthey Ġaddress Ġonly Ġone Ġtopic Ġof Ġcyber bull ying Ġsuch Ġas Ġracism , Ġand Ġsexism . ĠDepending Ġon Ġthe Ġtopic , Ġvocabulary Ġand Ġnature Ġof Ġcyber bull ying Ġchanges . ĠThese Ġmodels Ġare Ġnot Ġflexible Ġin Ġaccommodating Ġchanges Ġin Ġthe Ġdefinition Ġof Ġcyber bull ying . ĠThird Ġ( Bott leneck ĠB 3 ), Ġthey Ġrely Ġon Ġcarefully Ġhand crafted Ġfeatures Ġsuch Ġas Ġswear Ġword Ġlist Ġand ĠPOS Ġtagging . ĠHowever , Ġthese Ġhand crafted Ġfeatures Ġare Ġnot Ġrobust Ġagainst Ġvariations Ġin Ġwriting Ġstyle . ĠIn Ġcontrast Ġto Ġexisting Ġbott len ec ks , Ġthis Ġwork Ġtargets Ġthree Ġdifferent Ġtypes Ġof Ġsocial Ġnetworks Ġ( Form spring : Ġa ĠQ & A Ġforum , ĠTwitter : Ġmicro blog ging , Ġand ĠWikipedia : Ġcollaborative Ġknowledge Ġrepository ) Ġfor Ġthree Ġtopics Ġof Ġcyber bull ying Ġ( personal Ġattack , Ġracism , Ġand Ġsexism ) Ġwithout Ġdoing Ġany Ġexplicit Ġfeature Ġengineering Ġby Ġdeveloping Ġdeep Ġlearning Ġbased Ġmodels Ġalong Ġwith Ġtransfer Ġlearning . </s> We Ġexperimented Ġwith Ġdiverse Ġtraditional Ġmachine Ġlearning Ġmodels Ġ( log istic Ġregression , Ġsupport Ġvector Ġmachine , Ġrandom Ġforest , Ġnaive ĠBay es ) Ġand Ġdeep Ġneural Ġnetwork Ġmodels Ġ( CNN , ĠL ST M , ĠBL ST M , ĠBL ST M Ġwith ĠAttention ) Ġusing Ġvariety Ġof Ġrepresentation Ġmethods Ġfor Ġwords Ġ( bag Ġof Ġcharacter Ġn - gram , Ġbag Ġof Ġword Ġun ig ram , ĠGl o Ve Ġembed d ings , ĠSS WE Ġembed d ings ). ĠSummary Ġof Ġour Ġfindings Ġand Ġresearch Ġcontributions Ġis Ġas Ġfollows . </s> Dat as ets </s> Please Ġrefer Ġto ĠTable ĠT AB REF 7 Ġfor Ġsummary Ġof Ġdatasets Ġused . ĠWe Ġperformed Ġexperiments Ġusing Ġlarge , Ġdiverse , Ġmanually Ġannot ated , Ġand Ġpublicly Ġavailable Ġdatasets Ġfor Ġcyber bull ying Ġdetection Ġin Ġsocial Ġmedia . ĠWe Ġcover Ġthree Ġdifferent Ġtypes Ġof Ġsocial Ġnetworks : Ġteen Ġoriented ĠQ & A Ġforum Ġ( Form spring ), Ġlarge Ġmicro blog ging Ġplatform Ġ( Twitter ), Ġand Ġcollaborative Ġknowledge Ġrepository Ġ( Wikipedia Ġtalk Ġpages ). ĠEach Ġdataset Ġaddresses Ġa Ġdifferent Ġtopic Ġof Ġcyber bull ying . ĠTwitter Ġdataset Ġcontains Ġexamples Ġof Ġracism Ġand Ġsexism . ĠWikipedia Ġdataset Ġcontains Ġexamples Ġof Ġpersonal Ġattack . ĠHowever , ĠForm spring Ġdataset Ġis Ġnot Ġspecifically Ġabout Ġany Ġsingle Ġtopic . ĠAll Ġthree Ġdatasets Ġhave Ġthe Ġproblem Ġof Ġclass Ġimbalance Ġwhere Ġposts Ġlabeled Ġas Ġcyber bull ying Ġare Ġin Ġthe Ġminority Ġas Ġcompared Ġto Ġneutral Ġposts . ĠVari ation Ġin Ġthe Ġnumber Ġof Ġposts Ġacross Ġdatasets Ġalso Ġaffects Ġvocabulary Ġsize Ġthat Ġrepresents Ġthe Ġnumber Ġof Ġdistinct Ġwords Ġencountered Ġin Ġthe Ġdataset . ĠWe Ġmeasure Ġthe Ġsize Ġof Ġa Ġpost Ġin Ġterms Ġof Ġthe Ġnumber Ġof Ġwords Ġin Ġthe Ġpost . ĠFor Ġeach Ġdataset , Ġthere Ġare Ġonly Ġa Ġfew Ġposts Ġwith Ġlarge Ġsize . ĠWe Ġtrunc ate Ġsuch Ġlarge Ġposts Ġto Ġthe Ġsize Ġof Ġpost Ġranked Ġat Ġ95 Ġpercentile Ġin Ġthat Ġdataset . ĠFor Ġexample , Ġin ĠWikipedia Ġdataset , Ġthe Ġlargest Ġpost Ġhas Ġ28 46 Ġwords . ĠHowever , Ġsize Ġof Ġpost Ġranked Ġat Ġ95 Ġpercentile Ġin Ġthat Ġdataset Ġis Ġonly Ġ231 . ĠAny Ġpost Ġlarger Ġthan Ġsize Ġ231 Ġin ĠWikipedia Ġdataset Ġwill Ġbe Ġtrunc ated Ġby Ġconsidering Ġonly Ġfirst Ġ231 Ġwords . ĠThis Ġtrunc ation Ġaffects Ġonly Ġa Ġsmall Ġminority Ġof Ġposts Ġin Ġeach Ġdataset . ĠHowever , Ġit Ġis Ġrequired Ġfor Ġefficiently Ġtraining Ġvarious Ġmodels Ġin Ġour Ġexperiments . ĠDetails Ġof Ġeach Ġdataset Ġare Ġas Ġfollows . </s> Form spring ĠB IB REF 2 Ġ: ĠIt Ġwas Ġa Ġquestion Ġand Ġanswer Ġbased Ġwebsite Ġwhere Ġusers Ġcould Ġopenly Ġinvite Ġothers Ġto Ġask Ġand Ġanswer Ġquestions . ĠThe Ġdataset Ġincludes Ġ12 K Ġannot ated Ġquestion Ġand Ġanswer Ġpairs . ĠEach Ġpost Ġis Ġmanually Ġlabeled Ġby Ġthree Ġworkers . ĠAmong Ġthese Ġpairs , Ġ8 25 Ġwere Ġlabeled Ġas Ġcontaining Ġcyber bull ying Ġcontent Ġby Ġat Ġleast Ġtwo ĠAmazon ĠMechanical Ġtur k Ġworkers . </s> Twitter ĠB IB REF 3 Ġ: ĠThis Ġdataset Ġincludes Ġ16 K Ġannot ated Ġtweets . ĠThe Ġauthors Ġboot stra pped Ġthe Ġcorpus Ġcollection , Ġby Ġperforming Ġan Ġinitial Ġmanual Ġsearch Ġof Ġcommon Ġslurs Ġand Ġterms Ġused Ġpertaining Ġto Ġreligious , Ġsexual , Ġgender , Ġand Ġethnic Ġminorities . ĠOf Ġthe Ġ16 K Ġtweets , Ġ3 117 Ġare Ġlabeled Ġas Ġsexist , Ġ1937 Ġas Ġracist , Ġand Ġthe Ġremaining Ġare Ġmarked Ġas Ġneither Ġsexist Ġnor Ġracist . </s> Wikipedia ĠB IB REF 4 Ġ: ĠFor Ġeach Ġpage Ġin ĠWikipedia , Ġa Ġcorresponding Ġtalk Ġpage Ġmaintains Ġthe Ġhistory Ġof Ġdiscussion Ġamong Ġusers Ġwho Ġparticipated Ġin Ġits Ġediting . ĠThis Ġdata Ġset Ġincludes Ġover Ġ100 k Ġlabeled Ġdiscussion Ġcomments Ġfrom ĠEnglish ĠWikipedia 's Ġtalk Ġpages . ĠEach Ġcomment Ġwas Ġlabeled Ġby Ġ10 Ġannot ators Ġvia ĠCrowd flower Ġon Ġwhether Ġit Ġcontains Ġa Ġpersonal Ġattack . ĠThere Ġare Ġtotal Ġ13 590 Ġcomments Ġlabeled Ġas Ġpersonal Ġattack . </s> Use Ġof ĠSw ear ĠWords Ġand ĠAn onym ity </s> Please Ġrefer Ġto ĠTable ĠT AB REF 9 Ġ. ĠWe Ġuse Ġthe Ġfollowing Ġshort Ġforms Ġin Ġthis Ġsection : ĠB = Bull ying , ĠS = Sw earing , ĠA = Anonymous . ĠSome Ġof Ġthe Ġvalues Ġfor ĠTwitter Ġdataset Ġare Ġundefined Ġas ĠTwitter Ġdoes Ġnot Ġallow Ġanonymous Ġpostings . ĠUse Ġof Ġswear Ġwords Ġhas Ġbeen Ġrepeatedly Ġlinked Ġto Ġcyber bull ying . ĠHowever , Ġpreliminary Ġanalysis Ġof Ġdatasets Ġreveals Ġthat Ġdepending Ġon Ġswear Ġword Ġusage Ġcan Ġneither Ġlead Ġto Ġhigh Ġprecision Ġnor Ġhigh Ġrecall Ġfor Ġcyber bull ying Ġdetection . ĠSw ear Ġword Ġlist Ġbased Ġmethods Ġwill Ġhave Ġlow Ġprecision Ġas ĠP ( B ĠIN LINE FORM 0 ĠS ) Ġis Ġnot Ġclose Ġto Ġ1 . ĠIn Ġfact , Ġfor Ġteen Ġoriented Ġsocial Ġnetwork ĠForm spring , Ġ78 % Ġof Ġthe Ġswearing Ġposts Ġare Ġnon - bull ying . ĠSw ear Ġwords Ġbased Ġfiltering Ġwill Ġbe Ġirritating Ġto Ġthe Ġusers Ġin Ġsuch ĠS MP s Ġwhere Ġswear Ġwords Ġare Ġused Ġcasually . ĠSw ear Ġword Ġlist Ġbased Ġmethods Ġwill Ġalso Ġhave Ġa Ġlow Ġrecall Ġas ĠP ( S ĠIN LINE FORM 1 ĠB ) Ġis Ġnot Ġclose Ġto Ġ1 . ĠFor ĠTwitter Ġdataset , Ġ82 % Ġof Ġbullying Ġposts Ġdo Ġnot Ġuse Ġany Ġswear Ġwords . ĠSuch Ġpassive - aggressive Ġcyber bull ying Ġwill Ġgo Ġundet ected Ġwith Ġswear Ġword Ġlist Ġbased Ġmethods . ĠAn onym ity Ġis Ġanother Ġclue Ġthat Ġis Ġused Ġfor Ġdetecting Ġcyber bull ying Ġas Ġbully Ġmight Ġprefer Ġto Ġhide Ġits Ġidentity . ĠAn onym ity Ġdefinitely Ġleads Ġto Ġincreased Ġuse Ġof Ġswear Ġwords Ġ( P ( S ĠIN LINE FORM 2 ĠA ) ĠIN LINE FORM 3 ĠP ( S )) Ġand Ġcyber bull ying Ġ( P ( B ĠIN LINE FORM 4 ĠA ) ĠIN LINE FORM 5 ĠP ( B ), Ġand ĠP ( B ĠIN LINE FORM 6 ĠA & S )) ĠIN LINE FORM 7 ĠP ( B )). ĠHowever , Ġsignificant Ġfraction Ġof Ġanonymous Ġposts Ġare Ġnon - bull ying Ġ( P ( B ĠIN LINE FORM 8 ĠA ) Ġnot Ġclose Ġto Ġ1 ) Ġand Ġmany Ġof Ġbullying Ġposts Ġare Ġnot Ġanonymous Ġ( P ( A ĠIN LINE FORM 9 ĠB ) Ġnot Ġclose Ġto Ġ1 ). ĠFurther , Ġanonymity Ġmight Ġnot Ġbe Ġallowed Ġby Ġmany ĠS MP s Ġsuch Ġas ĠTwitter . </s> Related ĠWork </s> Cy ber bull ying Ġis Ġrecognized Ġas Ġa Ġphenomenon Ġat Ġleast Ġsince Ġ2003 ĠB IB REF 5 Ġ. ĠUse Ġof Ġsocial Ġmedia Ġexploded Ġwith Ġlaunching Ġof Ġmultiple Ġplatforms Ġsuch Ġas ĠWikipedia Ġ( 2001 ), ĠMy Space Ġ( 2003 ), ĠOr k ut Ġ( 2004 ), ĠFacebook Ġ( 2004 ), Ġand ĠTwitter Ġ( 2005 ). ĠBy Ġ2006 , Ġresearchers Ġhad Ġpointed Ġthat Ġcyber bull ying Ġwas Ġas Ġserious Ġphenomenon Ġas Ġoffline Ġbullying ĠB IB REF 6 Ġ. ĠHowever , Ġautomatic Ġdetection Ġof Ġcyber bull ying Ġwas Ġaddressed Ġonly Ġsince Ġ2009 ĠB IB REF 7 Ġ. ĠAs Ġa Ġresearch Ġtopic , Ġcyber bull ying Ġdetection Ġis Ġa Ġtext Ġclassification Ġproblem . ĠMost Ġof Ġthe Ġexisting Ġworks Ġfit Ġin Ġthe Ġfollowing Ġtemplate : Ġget Ġtraining Ġdataset Ġfrom Ġsingle ĠS MP , Ġengineer Ġvariety Ġof Ġfeatures Ġwith Ġcertain Ġstyle Ġof Ġcyber bull ying Ġas Ġthe Ġtarget , Ġapply Ġa Ġfew Ġtraditional Ġmachine Ġlearning Ġmethods , Ġand Ġevaluate Ġsuccess Ġin Ġterms Ġof Ġmeasures Ġsuch Ġas ĠF 1 Ġscore Ġand Ġaccuracy . ĠThese Ġworks Ġheavily Ġrely Ġon Ġhand crafted Ġfeatures Ġsuch Ġas Ġuse Ġof Ġswear Ġwords . ĠThese Ġmethods Ġtend Ġto Ġhave Ġlow Ġprecision Ġfor Ġcyber bull ying Ġdetection Ġas Ġhand crafted Ġfeatures Ġare Ġnot Ġrobust Ġagainst Ġvariations Ġin Ġbullying Ġstyle Ġacross ĠS MP s Ġand Ġbullying Ġtopics . ĠOnly Ġrecently , Ġdeep Ġlearning Ġhas Ġbeen Ġapplied Ġfor Ġcyber bull ying Ġdetection ĠB IB REF 8 Ġ. ĠTable ĠT AB REF 27 Ġsummarizes Ġimportant Ġrelated Ġwork . </s> Deep ĠNeural ĠNetwork Ġ( D NN ) ĠBased ĠModels </s> We Ġexperimented Ġwith Ġfour ĠD NN Ġbased Ġmodels Ġfor Ġcyber bull ying Ġdetection : ĠCNN , ĠL ST M , ĠBL ST M , Ġand ĠBL ST M Ġwith Ġattention . ĠThese Ġmodels Ġare Ġlisted Ġin Ġthe Ġincreasing Ġcomplexity Ġof Ġtheir Ġneural Ġarchitecture Ġand Ġamount Ġof Ġinformation Ġused Ġby Ġthese Ġmodels . ĠPlease Ġrefer Ġto ĠFigure Ġ1 Ġfor Ġgeneral Ġarchitecture Ġthat Ġwe Ġhave Ġused Ġacross Ġfour Ġmodels . ĠVarious Ġmodels Ġdiffer Ġonly Ġin Ġthe ĠNeural ĠArchitecture Ġlayer Ġwhile Ġhaving Ġidentical Ġrest Ġof Ġthe Ġlayers . ĠCNN s Ġare Ġproviding Ġstate - of - the - results Ġon Ġextracting Ġcontextual Ġfeature Ġfor Ġclassification Ġtasks Ġin Ġimages , Ġvideos , Ġaud ios , Ġand Ġtext . ĠRecently , ĠCNN s Ġwere Ġused Ġfor Ġsentiment Ġclassification ĠB IB REF 9 Ġ. ĠLong ĠShort ĠTerm ĠMemory Ġnetworks Ġare Ġa Ġspecial Ġkind Ġof ĠR NN , Ġcapable Ġof Ġlearning Ġlong - term Ġdependencies . ĠTheir Ġability Ġto Ġuse Ġtheir Ġinternal Ġmemory Ġto Ġprocess Ġarbitrary Ġsequences Ġof Ġinputs Ġhas Ġbeen Ġfound Ġto Ġbe Ġeffective Ġfor Ġtext Ġclassification ĠB IB REF 10 Ġ. ĠBid irection al ĠL ST Ms ĠB IB REF 11 Ġfurther Ġincrease Ġthe Ġamount Ġof Ġinput Ġinformation Ġavailable Ġto Ġthe Ġnetwork Ġby Ġencoding Ġinformation Ġin Ġboth Ġforward Ġand Ġbackward Ġdirection . ĠBy Ġusing Ġtwo Ġdirections , Ġinput Ġinformation Ġfrom Ġboth Ġthe Ġpast Ġand Ġfuture Ġof Ġthe Ġcurrent Ġtime Ġframe Ġcan Ġbe Ġused . ĠAttention Ġmechanisms Ġallow Ġfor Ġa Ġmore Ġdirect Ġdependence Ġbetween Ġthe Ġstate Ġof Ġthe Ġmodel Ġat Ġdifferent Ġpoints Ġin Ġtime . ĠImport antly , Ġattention Ġmechanism Ġlets Ġthe Ġmodel Ġlearn Ġwhat Ġto Ġattend Ġto Ġbased Ġon Ġthe Ġinput Ġsentence Ġand Ġwhat Ġit Ġhas Ġproduced Ġso Ġfar . </s> The Ġembed ding Ġlayer Ġprocesses Ġa Ġfixed Ġsize Ġsequence Ġof Ġwords . ĠEach Ġword Ġis Ġrepresented Ġas Ġa Ġreal - valued Ġvector , Ġalso Ġknown Ġas Ġword Ġembed d ings . ĠWe Ġhave Ġexperimented Ġwith Ġthree Ġmethods Ġfor Ġinitial izing Ġword Ġembed d ings : Ġrandom , ĠGl o Ve ĠB IB REF 12 Ġ, Ġand ĠSS WE ĠB IB REF 13 Ġ. ĠDuring Ġthe Ġtraining , Ġmodel Ġimproves Ġupon Ġthe Ġinitial Ġword Ġembed d ings Ġto Ġlearn Ġtask Ġspecific Ġword Ġembed d ings . ĠWe Ġhave Ġobserved Ġthat Ġthese Ġtask Ġspecific Ġword Ġembed d ings Ġcapture Ġthe ĠS MP Ġspecific Ġand Ġtopic Ġspecific Ġstyle Ġof Ġcyber bull ying . ĠUsing ĠGl o Ve Ġvectors Ġover Ġrandom Ġvector Ġinitialization Ġhas Ġbeen Ġreported Ġto Ġimprove Ġperformance Ġfor Ġsome ĠN LP Ġtasks . ĠMost Ġof Ġthe Ġword Ġembed ding Ġmethods Ġsuch Ġas ĠGl o Ve , Ġconsider Ġonly Ġsynt actic Ġcontext Ġof Ġthe Ġword Ġwhile Ġignoring Ġthe Ġsentiment Ġconveyed Ġby Ġthe Ġtext . ĠSS WE Ġmethod Ġover comes Ġthis Ġproblem Ġby Ġincorporating Ġthe Ġtext Ġsentiment Ġas Ġone Ġof Ġthe Ġparameters Ġfor Ġword Ġembed ding Ġgeneration . ĠWe Ġexperimented Ġwith Ġvarious Ġdimension Ġsize Ġfor Ġword Ġembed d ings . ĠExperimental Ġresults Ġreported Ġhere Ġare Ġwith Ġdimension Ġsize Ġas Ġ50 . ĠThere Ġwas Ġno Ġsignificant Ġvariation Ġin Ġresults Ġwith Ġdimension Ġsize Ġranging Ġfrom Ġ30 Ġto Ġ200 . </s> To Ġavoid Ġover fitting , Ġwe Ġused Ġtwo Ġdrop out Ġlayers , Ġone Ġbefore Ġthe Ġneural Ġarchitecture Ġlayer Ġand Ġone Ġafter , Ġwith Ġdrop out Ġrates Ġof Ġ0 . 25 Ġand Ġ0 . 5 Ġrespectively . ĠFully Ġconnected Ġlayer Ġis Ġa Ġdense Ġoutput Ġlayer Ġwith Ġthe Ġnumber Ġof Ġneurons Ġequal Ġto Ġthe Ġnumber Ġof Ġclasses , Ġfollowed Ġby Ġsoft max Ġlayer Ġthat Ġprovides Ġsoft max Ġactivation . ĠAll Ġour Ġmodels Ġare Ġtrained Ġusing Ġback prop ag ation . ĠThe Ġoptim izer Ġused Ġfor Ġtraining Ġis ĠAdam Ġand Ġthe Ġloss Ġfunction Ġis Ġcateg orical Ġcross - ent ropy . ĠBesides Ġlearning Ġthe Ġnetwork Ġweights , Ġthese Ġmethods Ġalso Ġlearn Ġtask - specific Ġword Ġembed d ings Ġtuned Ġtowards Ġthe Ġbullying Ġlabels Ġ( See ĠSection ĠSEC REF 21 Ġ). ĠOur Ġcode Ġis Ġavailable Ġat : Ġhttps :// github . com / sw eta 20 / Detect ing - Cy ber bull ying - Across - S MP s . </s> Exper iments </s> Ex isting Ġworks Ġhave Ġheavily Ġrelied Ġon Ġtraditional Ġmachine Ġlearning Ġmodels Ġfor Ġcyber bull ying Ġdetection . ĠHowever , Ġthey Ġdo Ġnot Ġstudy Ġthe Ġperformance Ġof Ġthese Ġmodels Ġacross Ġmultiple ĠS MP s . ĠWe Ġexperimented Ġwith Ġfour Ġmodels : Ġlog istic Ġregression Ġ( LR ), Ġsupport Ġvector Ġmachine Ġ( S VM ), Ġrandom Ġforest Ġ( RF ), Ġand Ġnaive ĠBay es Ġ( NB ), Ġas Ġthese Ġare Ġused Ġin Ġprevious Ġworks Ġ( Table ĠT AB REF 27 Ġ). ĠWe Ġused Ġtwo Ġdata Ġrepresentation Ġmethods : Ġcharacter Ġn - gram Ġand Ġword Ġun ig ram . ĠPast Ġwork Ġin Ġthe Ġdomain Ġof Ġdetecting Ġabusive Ġlanguage Ġhave Ġshowed Ġthat Ġsimple Ġn - gram Ġfeatures Ġare Ġmore Ġpowerful Ġthan Ġlinguistic Ġand Ġsynt actic Ġfeatures , Ġhand - engine ered Ġlex icons , Ġand Ġword Ġand Ġparagraph Ġembed d ings ĠB IB REF 14 Ġ. ĠAs Ġcompared Ġto ĠD NN Ġmodels , Ġperformance Ġof Ġall Ġfour Ġtraditional Ġmachine Ġlearning Ġmodels Ġwas Ġsignificantly Ġlower . ĠPlease Ġrefer Ġto ĠTable ĠT AB REF 11 Ġ. </s> All ĠD NN Ġmodels Ġreported Ġhere Ġwere Ġimplemented Ġusing ĠKer as . ĠWe Ġpre - process Ġthe Ġdata , Ġsubject ing Ġit Ġto Ġstandard Ġoperations Ġof Ġremoval Ġof Ġstop Ġwords , Ġpunct uation Ġmarks Ġand Ġlower c asing , Ġbefore Ġannot ating Ġit Ġto Ġassigning Ġrespective Ġlabels Ġto Ġeach Ġcomment . ĠFor Ġeach Ġtrained Ġmodel , Ġwe Ġreport Ġits Ġperformance Ġafter Ġdoing Ġfive - fold Ġcross - valid ation . ĠWe Ġuse Ġfollowing Ġshort Ġforms . </s> Effect Ġof ĠOvers am pling ĠBull ying ĠInst ances </s> The Ġtraining Ġdatasets Ġhad Ġa Ġmajor Ġproblem Ġof Ġclass Ġimbalance Ġwith Ġposts Ġmarked Ġas Ġbullying Ġin Ġthe Ġminority . ĠAs Ġa Ġresult , Ġall Ġmodels Ġwere Ġbiased Ġtowards Ġlabeling Ġthe Ġposts Ġas Ġnon - bull ying . ĠTo Ġremove Ġthis Ġbias , Ġwe Ġovers am pled Ġthe Ġdata Ġfrom Ġbullying Ġclass Ġthr ice . ĠThat Ġis , Ġwe Ġreplicated Ġbullying Ġposts Ġthr ice Ġin Ġthe Ġtraining Ġdata . ĠThis Ġsignificantly Ġimproved Ġthe Ġperformance Ġof Ġall ĠD NN Ġmodels Ġwith Ġmajor Ġleap Ġin Ġall Ġthree Ġevaluation Ġmeasures . ĠTable ĠT AB REF 17 Ġshows Ġthe Ġeffect Ġof Ġovers am pling Ġfor Ġa Ġvariety Ġof Ġword Ġembed ding Ġmethods Ġwith ĠBL ST M ĠAttention Ġas Ġthe Ġdetection Ġmodel . ĠResults Ġfor Ġother Ġmodels Ġare Ġsimilar ĠB IB REF 15 Ġ. ĠWe Ġcan Ġnotice Ġthat Ġovers am pled Ġdatasets Ġ( F +, ĠT +, ĠW +) Ġhave Ġfar Ġbetter Ġperformance Ġthan Ġtheir Ġcounterparts Ġ( F , ĠT , ĠW Ġrespectively ). ĠOvers am pling Ġparticularly Ġhelps Ġthe Ġsmallest Ġdataset ĠForm spring Ġwhere Ġnumber Ġof Ġtraining Ġinstances Ġfor Ġbullying Ġclass Ġis Ġquite Ġsmall Ġ( 825 ) Ġas Ġcompared Ġto Ġother Ġtwo Ġdatasets Ġ( about Ġ5 K Ġand Ġ13 K ). ĠWe Ġalso Ġexperimented Ġwith Ġvarying Ġthe Ġreplication Ġrate Ġfor Ġbullying Ġposts ĠB IB REF 15 Ġ. ĠHowever , Ġwe Ġobserved Ġthat Ġfor Ġbullying Ġposts , Ġreplication Ġrate Ġof Ġthree Ġis Ġgood Ġenough . </s> Choice Ġof ĠInitial ĠWord ĠEmb edd ings Ġand ĠModel </s> Initial Ġword Ġembed d ings Ġdecide Ġdata Ġrepresentation Ġfor ĠD NN Ġmodels . ĠHowever Ġduring Ġthe Ġtraining , ĠD NN Ġmodels Ġmodify Ġthese Ġinitial Ġword Ġembed d ings Ġto Ġlearn Ġtask Ġspecific Ġword Ġembed d ings . ĠWe Ġhave Ġexperimented Ġwith Ġthree Ġmethods Ġto Ġinitialize Ġword Ġembed d ings . ĠPlease Ġrefer Ġto ĠTable ĠT AB REF 19 Ġ. ĠThis Ġtable Ġshows Ġthe Ġeffect Ġof Ġvarying Ġinitial Ġword Ġembed d ings Ġfor Ġmultiple ĠD NN Ġmodels Ġacross Ġdatasets . ĠWe Ġcan Ġnotice Ġthat Ġinitial Ġword Ġembed d ings Ġdo Ġnot Ġhave Ġa Ġsignificant Ġeffect Ġon Ġcyber bull ying Ġdetection Ġwhen Ġovers am pling Ġof Ġbullying Ġposts Ġis Ġdone Ġ( rows Ġcorresponding Ġto ĠF +, ĠT +, ĠW + ). ĠIn Ġthe Ġabsence Ġof Ġovers am pling Ġ( rows Ġcorresponding Ġto ĠF , ĠT ĠW ), Ġthere Ġis Ġa Ġgap Ġin Ġperformance Ġof Ġsimplest Ġ( CNN ) Ġand Ġmost Ġcomplex Ġ( BL ST M Ġwith Ġattention ) Ġmodels . ĠHowever , Ġthis Ġgap Ġgoes Ġon Ġreducing Ġwith Ġthe Ġincrease Ġin Ġthe Ġsize Ġof Ġdatasets . </s> Table ĠT AB REF 20 Ġcompares Ġthe Ġperformance Ġof Ġfour ĠD NN Ġmodels Ġfor Ġthree Ġevaluation Ġmeasures Ġwhile Ġusing ĠSS WE Ġas Ġthe Ġinitial Ġword Ġembed d ings . ĠWe Ġhave Ġnoticed Ġthat Ġmost Ġof Ġthe Ġtime ĠL ST M Ġperforms Ġweaker Ġthan Ġother Ġthree Ġmodels . ĠHowever , Ġperformance Ġgap Ġin Ġthe Ġother Ġthree Ġmodels Ġis Ġnot Ġsignificant . </s> Task ĠSpecific ĠWord ĠEmb edd ings </s> D NN Ġmodels Ġlearn Ġword Ġembed d ings Ġover Ġthe Ġtraining Ġdata . ĠThese Ġlearned Ġembed d ings Ġacross Ġmultiple Ġdatasets Ġshow Ġthe Ġdifference Ġin Ġnature Ġand Ġstyle Ġof Ġbullying Ġacross Ġcyber bull ying Ġtopics Ġand ĠS MP s . ĠHere Ġwe Ġreport Ġresults Ġfor ĠBL ST M Ġwith Ġattention Ġmodel . ĠResults Ġfor Ġother Ġmodels Ġare Ġsimilar . ĠWe Ġfirst Ġverify Ġthat Ġimportant Ġwords Ġfor Ġeach Ġtopic Ġof Ġcyber bull ying Ġform Ġclusters Ġin Ġthe Ġlearned Ġembed d ings . ĠTo Ġenable Ġthe Ġvisualization Ġof Ġgrouping , Ġwe Ġreduced Ġdimension ality Ġwith Ġt - S NE ĠB IB REF 16 Ġ, Ġa Ġwell - known Ġtechnique Ġfor Ġdimension ality Ġreduction Ġparticularly Ġwell Ġsuited Ġfor Ġvisualization Ġof Ġhigh Ġdimensional Ġdatasets . ĠPlease Ġrefer Ġto ĠTable ĠT AB REF 22 Ġ. ĠThis Ġtable Ġshows Ġimportant Ġclusters Ġobserved Ġin Ġt - S NE Ġprojection Ġof Ġlearned Ġword Ġembed d ings . ĠEach Ġcluster Ġshows Ġthat Ġwords Ġmost Ġrelevant Ġto Ġa Ġparticular Ġtopic Ġof Ġbullying Ġform Ġcluster . </s> We Ġalso Ġobserved Ġchanges Ġin Ġthe Ġmeanings Ġof Ġthe Ġwords Ġacross Ġtopics Ġof Ġcyber bull ying . ĠTable ĠT AB REF 23 Ġshows Ġmost Ġsimilar Ġwords Ġfor Ġa Ġgiven Ġquery Ġword Ġfor Ġtwo Ġdatasets . ĠTwitter Ġdataset Ġwhich Ġis Ġheavy Ġon Ġsexism Ġand Ġracism , Ġconsiders Ġword Ġslave Ġas Ġsimilar Ġto Ġtargets Ġof Ġracism Ġand Ġsexism . ĠHowever , ĠWikipedia Ġdataset Ġthat Ġis Ġabout Ġpersonal Ġattacks Ġdoes Ġnot Ġshow Ġsuch Ġbias . </s> Transfer ĠLearning </s> We Ġused Ġtransfer Ġlearning Ġto Ġcheck Ġif Ġthe Ġknowledge Ġgained Ġby ĠD NN Ġmodels Ġon Ġone Ġdataset Ġcan Ġbe Ġused Ġto Ġimprove Ġcyber bull ying Ġdetection Ġperformance Ġon Ġother Ġdatasets . ĠWe Ġreport Ġresults Ġwhere ĠBL ST M Ġwith Ġattention Ġis Ġused Ġas Ġthe ĠD NN Ġmodel . ĠResults Ġfor Ġother Ġmodels Ġare Ġsimilar ĠB IB REF 15 Ġ. ĠWe Ġexperimented Ġwith Ġfollowing Ġthree Ġflavors Ġof Ġtransfer Ġlearning . </s> Complete ĠTransfer ĠLearning Ġ( TL 1 ): ĠIn Ġthis Ġflavor , Ġa Ġmodel Ġtrained Ġon Ġone Ġdataset Ġwas Ġdirectly Ġused Ġto Ġdetect Ġcyber bull ying Ġin Ġother Ġdatasets Ġwithout Ġany Ġextra Ġtraining . ĠTL 1 Ġresulted Ġin Ġsignificantly Ġlow Ġrecall Ġindicating Ġthat Ġthree Ġdatasets Ġhave Ġdifferent Ġnature Ġof Ġcyber bull ying Ġwith Ġlow Ġoverlap Ġ( Table ĠT AB REF 25 Ġ). ĠHowever Ġprecision Ġwas Ġrelatively Ġhigher Ġfor ĠTL 1 , Ġindicating Ġthat ĠD NN Ġmodels Ġare Ġcautious Ġin Ġlabeling Ġa Ġpost Ġas Ġbully Ġ( Table ĠT AB REF 25 Ġ). ĠTL 1 Ġalso Ġhelps Ġto Ġmeasure Ġsimilarity Ġin Ġnature Ġof Ġcyber bull ying Ġacross Ġthree Ġdatasets . ĠWe Ġcan Ġobserve Ġthat Ġbullying Ġnature Ġin ĠForm spring Ġand ĠWikipedia Ġdatasets Ġis Ġmore Ġsimilar Ġto Ġeach Ġother Ġthan Ġthe ĠTwitter Ġdataset . ĠThis Ġcan Ġbe Ġinferred Ġfrom Ġthe Ġfact Ġthat Ġwith ĠTL 1 , Ġcyber bull ying Ġdetection Ġperformance Ġfor ĠForm spring Ġdataset Ġis Ġhigher Ġwhen Ġbase Ġmodel Ġis ĠWikipedia Ġ( pre cision Ġ= 0 . 51 Ġand Ġrecall = 0 . 66 ) as Ġcompared Ġto ĠTwitter Ġas Ġthe Ġbase Ġmodel Ġ( pre cision = 0 . 38 Ġand Ġrecall = 0 . 04 ). ĠSimilarly , Ġfor ĠWikipedia Ġdataset , ĠForm spring Ġacts Ġas Ġa Ġbetter Ġbase Ġmodel Ġthan ĠTwitter Ġwhile Ġusing ĠTL 1 Ġflavor Ġof Ġtransfer Ġlearning . ĠNature Ġof ĠS MP Ġmight Ġbe Ġa Ġfactor Ġbehind Ġthis Ġsimilarity Ġin Ġnature Ġof Ġcyber bull ying . ĠBoth ĠForm spring Ġand ĠWikipedia Ġare Ġtask Ġoriented Ġsocial Ġnetworks Ġ( Q & A Ġand Ġcollaborative Ġknowledge Ġrepository Ġrespectively ) Ġthat Ġallow Ġanonymity Ġand Ġlarger Ġposts . ĠWhereas Ġcommunication Ġon ĠTwitter Ġis Ġshort , Ġfree Ġof Ġanonymity Ġand Ġnot Ġoriented Ġtowards Ġa Ġparticular Ġtask . </s> Feature ĠLevel ĠTransfer ĠLearning Ġ( TL 2 ): ĠIn Ġthis Ġflavor , Ġa Ġmodel Ġwas Ġtrained Ġon Ġone Ġdataset Ġand Ġonly Ġlearned Ġword Ġembed d ings Ġwere Ġtransferred Ġto Ġanother Ġdataset Ġfor Ġtraining Ġa Ġnew Ġmodel . ĠAs Ġcompared Ġto ĠTL 1 , Ġrecall Ġscore Ġimproved Ġdramatically Ġwith ĠTL 2 Ġ( Table ĠT AB REF 25 Ġ). ĠImprovement Ġin Ġprecision Ġwas Ġalso Ġsignificant Ġ( Table ĠT AB REF 25 Ġ). ĠThese Ġimprovements Ġindicate Ġthat Ġlearned Ġword Ġembed d ings Ġare Ġan Ġessential Ġpart Ġof Ġknowledge Ġtransfer Ġacross Ġdatasets Ġfor Ġcyber bull ying Ġdetection . </s> Model ĠLevel ĠTransfer ĠLearning Ġ( TL 3 ): ĠIn Ġthis Ġflavor , Ġa Ġmodel Ġwas Ġtrained Ġon Ġone Ġdataset Ġand Ġlearned Ġword Ġembed d ings , Ġas Ġwell Ġas Ġnetwork Ġweights , Ġwere Ġtransferred Ġto Ġanother Ġdataset Ġfor Ġtraining Ġa Ġnew Ġmodel . ĠTL 3 Ġdoes Ġnot Ġresult Ġin Ġany Ġsignificant Ġimprovement Ġover ĠTL 2 . ĠThis Ġlack Ġof Ġimprovement Ġindicates Ġthat Ġtransfer Ġof Ġnetwork Ġweights Ġis Ġnot Ġessential Ġfor Ġcyber bull ying Ġdetection Ġand Ġlearned Ġword Ġembed d ings Ġis Ġthe Ġkey Ġknowledge Ġgained Ġby Ġthe ĠD NN Ġmodels . </s> D NN Ġbased Ġmodels Ġcoupled Ġwith Ġtransfer Ġlearning Ġbeat Ġthe Ġbest - known Ġresults Ġfor Ġall Ġthree Ġdatasets . ĠPrevious Ġbest ĠF 1 Ġscores Ġfor ĠWikipedia ĠB IB REF 4 Ġand ĠTwitter ĠB IB REF 8 Ġdatasets Ġwere Ġ0 . 68 Ġand Ġ0 . 93 Ġrespectively . ĠWe Ġachieve ĠF 1 Ġscores Ġof Ġ0 . 94 Ġfor Ġboth Ġthese Ġdatasets Ġusing ĠBL ST M Ġwith Ġattention Ġand Ġfeature Ġlevel Ġtransfer Ġlearning Ġ( Table ĠT AB REF 25 Ġ). ĠFor ĠForm spring Ġdataset , Ġauthors Ġhave Ġnot Ġreported ĠF 1 Ġscore . ĠTheir Ġmethod Ġhas Ġaccuracy Ġscore Ġof Ġ78 . 5 % ĠB IB REF 2 Ġ. ĠWe Ġachieve ĠF 1 Ġscore Ġof Ġ0 . 95 Ġwith Ġaccuracy Ġscore Ġof Ġ98 % Ġfor Ġthe Ġsame Ġdataset . </s> Conclusion Ġand ĠFuture ĠWork </s> We Ġhave Ġshown Ġthat ĠD NN Ġmodels Ġcan Ġbe Ġused Ġfor Ġcyber bull ying Ġdetection Ġon Ġvarious Ġtopics Ġacross Ġmultiple ĠS MP s Ġusing Ġthree Ġdatasets Ġand Ġfour ĠD NN Ġmodels . ĠThese Ġmodels Ġcoupled Ġwith Ġtransfer Ġlearning Ġbeat Ġstate Ġof Ġthe Ġart Ġresults Ġfor Ġall Ġthree Ġdatasets . ĠThese Ġmodels Ġcan Ġbe Ġfurther Ġimproved Ġwith Ġextra Ġdata Ġsuch Ġas Ġinformation Ġabout Ġthe Ġprofile Ġand Ġsocial Ġgraph Ġof Ġusers . ĠMost Ġof Ġthe Ġcurrent Ġdatasets Ġdo Ġnot Ġprovide Ġany Ġinformation Ġabout Ġthe Ġseverity Ġof Ġbullying . ĠIf Ġsuch Ġfine - gr ained Ġinformation Ġis Ġmade Ġavailable , Ġthen Ġcyber bull ying Ġdetection Ġmodels Ġcan Ġbe Ġfurther Ġimproved Ġto Ġtake Ġa Ġvariety Ġof Ġactions Ġdepending Ġon Ġthe Ġperceived Ġseriousness Ġof Ġthe Ġposts .