10,000 Arabic tweet dataset 
EXTRACTIVE
10,000
EXTRACTIVE
Disclaimer: Due to the nature of the paper, some examples contain highly offensive language and hate speech. They don't reflect the views of the authors in any way, and the point of the paper is to help fight such speech. Much recent interest has focused on the detection of offensive language and hate speech in online social media. Such language is often associated with undesirable online behaviors such as trolling, cyberbullying, online extremism, political polarization, and propaganda. Thus, offensive language detection is instrumental for a variety of application such as: quantifying polarization BIBREF0, BIBREF1, trolls and propaganda account detection BIBREF2, detecting the likelihood of hate crimes BIBREF3; and predicting conflict BIBREF4. In this paper, we describe our methodology for building a large dataset of Arabic offensive tweets. Given that roughly 1-2% of all Arabic tweets are offensive BIBREF5, targeted annotation is essential for efficiently building a large dataset. Since our methodology does not use a seed list of offensive words, it is not biased by topic, target, or dialect. Using our methodology, we tagged 10,000 Arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. Further, we labeled tweets as vulgar or hate speech. To date, this is the largest available dataset, which we plan to make publicly available along with annotation guidelines. We use this dataset to characterize Arabic offensive language to ascertain the topics, dialects, and users' gender that are most associated with the use of offensive language. Though we suspect that there are common features that span different languages and cultures, some characteristics of Arabic offensive language is language and culture specific. Thus, we conduct a thorough analysis of how Arabic users use offensive language. Next, we use the dataset to train strong Arabic offensive language classifiers using state-of-the-art representations and classification techniques. Specifically, we experiment with static and contextualized embeddings for representation along with a variety of classifiers such as a deep neural network classifier and Support Vector Machine (SVM).
Disclaimer: Due to the nature of the paper, some examples contain highly offensive language and hate speech. They don't reflect the views of the authors in any way, and the point of the paper is to help fight such speech. Much recent interest has focused on the detection of offensive language and hate speech in online social media. Such language is often associated with undesirable online behaviors such as trolling, cyberbullying, online extremism, political polarization, and propaganda. Thus, offensive language detection is instrumental for a variety of application such as: quantifying polarization BIBREF0, BIBREF1, trolls and propaganda account detection BIBREF2, detecting the likelihood of hate crimes BIBREF3; and predicting conflict BIBREF4. In this paper, we describe our methodology for building a large dataset of Arabic offensive tweets. Given that roughly 1-2% of all Arabic tweets are offensive BIBREF5, targeted annotation is essential for efficiently building a large dataset. Since our methodology does not use a seed list of offensive words, it is not biased by topic, target, or dialect. Using our methodology, we tagged 10,000 Arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. Further, we labeled tweets as vulgar or hate speech. To date, this is the largest available dataset, which we plan to make publicly available along with annotation guidelines. We use this dataset to characterize Arabic offensive language to ascertain the topics, dialects, and users' gender that are most associated with the use of offensive language. Though we suspect that there are common features that span different languages and cultures, some characteristics of Arabic offensive language is language and culture specific. Thus, we conduct a thorough analysis of how Arabic users use offensive language. Next, we use the dataset to train strong Arabic offensive language classifiers using state-of-the-art representations and classification techniques. Specifically, we experiment with static and contextualized embeddings for representation along with a variety of classifiers such as a deep neural network classifier and Support Vector Machine (SVM).
2004.02192
Introduction </s> Disclaimer : ĠDue Ġto Ġthe Ġnature Ġof Ġthe Ġpaper , Ġsome Ġexamples Ġcontain Ġhighly Ġoffensive Ġlanguage Ġand Ġhate Ġspeech . ĠThey Ġdon 't Ġreflect Ġthe Ġviews Ġof Ġthe Ġauthors Ġin Ġany Ġway , Ġand Ġthe Ġpoint Ġof Ġthe Ġpaper Ġis Ġto Ġhelp Ġfight Ġsuch Ġspeech . ĠMuch Ġrecent Ġinterest Ġhas Ġfocused Ġon Ġthe Ġdetection Ġof Ġoffensive Ġlanguage Ġand Ġhate Ġspeech Ġin Ġonline Ġsocial Ġmedia . ĠSuch Ġlanguage Ġis Ġoften Ġassociated Ġwith Ġundesirable Ġonline Ġbehaviors Ġsuch Ġas Ġtrolling , Ġcyber bull ying , Ġonline Ġextremism , Ġpolitical Ġpolarization , Ġand Ġpropaganda . ĠThus , Ġoffensive Ġlanguage Ġdetection Ġis Ġinstrumental Ġfor Ġa Ġvariety Ġof Ġapplication Ġsuch Ġas : Ġquant ifying Ġpolarization ĠB IB REF 0 , ĠB IB REF 1 , Ġtrolls Ġand Ġpropaganda Ġaccount Ġdetection ĠB IB REF 2 , Ġdetecting Ġthe Ġlikelihood Ġof Ġhate Ġcrimes ĠB IB REF 3 ; Ġand Ġpredicting Ġconflict ĠB IB REF 4 . ĠIn Ġthis Ġpaper , Ġwe Ġdescribe Ġour Ġmethodology Ġfor Ġbuilding Ġa Ġlarge Ġdataset Ġof ĠArabic Ġoffensive Ġtweets . ĠGiven Ġthat Ġroughly Ġ1 - 2 % Ġof Ġall ĠArabic Ġtweets Ġare Ġoffensive ĠB IB REF 5 , Ġtargeted Ġannotation Ġis Ġessential Ġfor Ġefficiently Ġbuilding Ġa Ġlarge Ġdataset . ĠSince Ġour Ġmethodology Ġdoes Ġnot Ġuse Ġa Ġseed Ġlist Ġof Ġoffensive Ġwords , Ġit Ġis Ġnot Ġbiased Ġby Ġtopic , Ġtarget , Ġor Ġdialect . ĠUsing Ġour Ġmethodology , Ġwe Ġtagged Ġ10 , 000 ĠArabic Ġtweet Ġdataset Ġfor Ġoff ens iveness , Ġwhere Ġoffensive Ġtweets Ġaccount Ġfor Ġroughly Ġ19 % Ġof Ġthe Ġtweets . ĠFurther , Ġwe Ġlabeled Ġtweets Ġas Ġvulgar Ġor Ġhate Ġspeech . ĠTo Ġdate , Ġthis Ġis Ġthe Ġlargest Ġavailable Ġdataset , Ġwhich Ġwe Ġplan Ġto Ġmake Ġpublicly Ġavailable Ġalong Ġwith Ġannotation Ġguidelines . ĠWe Ġuse Ġthis Ġdataset Ġto Ġcharacterize ĠArabic Ġoffensive Ġlanguage Ġto Ġascertain Ġthe Ġtopics , Ġdialect s , Ġand Ġusers ' Ġgender Ġthat Ġare Ġmost Ġassociated Ġwith Ġthe Ġuse Ġof Ġoffensive Ġlanguage . ĠThough Ġwe Ġsuspect Ġthat Ġthere Ġare Ġcommon Ġfeatures Ġthat Ġspan Ġdifferent Ġlanguages Ġand Ġcultures , Ġsome Ġcharacteristics Ġof ĠArabic Ġoffensive Ġlanguage Ġis Ġlanguage Ġand Ġculture Ġspecific . ĠThus , Ġwe Ġconduct Ġa Ġthorough Ġanalysis Ġof Ġhow ĠArabic Ġusers Ġuse Ġoffensive Ġlanguage . ĠNext , Ġwe Ġuse Ġthe Ġdataset Ġto Ġtrain Ġstrong ĠArabic Ġoffensive Ġlanguage Ġclass ifiers Ġusing Ġstate - of - the - art Ġrepresentations Ġand Ġclassification Ġtechniques . ĠSpecifically , Ġwe Ġexperiment Ġwith Ġstatic Ġand Ġcontextual ized Ġembed d ings Ġfor Ġrepresentation Ġalong Ġwith Ġa Ġvariety Ġof Ġclass ifiers Ġsuch Ġas Ġa Ġdeep Ġneural Ġnetwork Ġclass ifier Ġand ĠSupport ĠVector ĠMachine Ġ( S VM ). </s> The Ġcontributions Ġof Ġthis Ġpaper Ġare Ġas Ġfollows : </s> We Ġbuilt Ġthe Ġlargest ĠArabic Ġoffensive Ġlanguage Ġdataset Ġto Ġdate Ġthat Ġincludes Ġspecial Ġtags Ġfor Ġvulgar Ġlanguage Ġand Ġhate Ġspeech . ĠWe Ġdescribe Ġthe Ġmethodology Ġfor Ġbuilding Ġit Ġalong Ġwith Ġannotation Ġguidelines . </s> We Ġperformed Ġthorough Ġanalysis Ġof Ġthe Ġdataset Ġand Ġdescribed Ġthe Ġpeculiar ities Ġof ĠArabic Ġoffensive Ġlanguage . </s> We Ġexperimented Ġwith ĠSupport ĠVector ĠMachine Ġclass ifiers Ġon Ġcharacter Ġand Ġword Ġn gram s Ġclassification Ġtechniques Ġto Ġprovide Ġstrong Ġresults Ġon ĠArabic Ġoffensive Ġlanguage Ġclassification . </s> Related ĠWork </s> Many Ġrecent Ġpapers Ġhave Ġfocused Ġon Ġthe Ġdetection Ġof Ġoffensive Ġlanguage , Ġincluding Ġhate Ġspeech ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 8 , ĠB IB REF 9 , ĠB IB REF 10 , ĠB IB REF 11 , ĠB IB REF 12 , ĠB IB REF 13 . ĠOffensive Ġlanguage Ġcan Ġbe Ġcategorized Ġas : ĠVul gar , Ġwhich Ġinclude Ġexplicit Ġand Ġrude Ġsexual Ġreferences , ĠPorn ographic , Ġand ĠH ateful , Ġwhich Ġincludes Ġoffensive Ġremarks Ġconcerning Ġpeople âĢ Ļ s Ġrace , Ġreligion , Ġcountry , Ġetc . ĠB IB REF 14 . ĠPrior Ġworks Ġhave Ġconcentrated Ġon Ġbuilding Ġannot ated Ġcorpor a Ġand Ġtraining Ġclassification Ġmodels . ĠCon cerning Ġcorpor a , Ġhates pe ech data . com Ġattempts Ġto Ġmaintain Ġan Ġupdated Ġlist Ġof Ġhate Ġspeech Ġcorpor a Ġfor Ġmultiple Ġlanguages Ġincluding ĠArabic Ġand ĠEnglish . ĠFurther , ĠSem E val Ġ2019 Ġran Ġan Ġevaluation Ġtask Ġtargeted Ġat Ġdetecting Ġoffensive Ġlanguage , Ġwhich Ġfocused Ġexclusively Ġon ĠEnglish ĠB IB REF 15 . ĠAs Ġfor Ġclassification Ġmodels , Ġmost Ġstudies Ġused Ġsupervised Ġclassification Ġat Ġeither Ġword Ġlevel ĠB IB REF 10 , Ġcharacter Ġsequence Ġlevel ĠB IB REF 11 , Ġand Ġword Ġembed d ings ĠB IB REF 9 . ĠThe Ġstudies Ġused Ġdifferent Ġclassification Ġtechniques Ġincluding Ġusing ĠNa Ã¯ve ĠBay es ĠB IB REF 10 , ĠS VM ĠB IB REF 11 , Ġand Ġdeep Ġlearning ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 12 Ġclassification . ĠThe Ġaccuracy Ġof Ġthe Ġaforementioned Ġsystem Ġranged Ġbetween Ġ76 % Ġand Ġ90 %. ĠEarlier Ġwork Ġlooked Ġat Ġthe Ġuse Ġof Ġsentiment Ġwords Ġas Ġfeatures Ġas Ġwell Ġas Ġcontextual Ġfeatures ĠB IB REF 13 . </s> The Ġwork Ġon ĠArabic Ġoffensive Ġlanguage Ġdetection Ġis Ġrelatively Ġnascent ĠB IB REF 16 , ĠB IB REF 17 , ĠB IB REF 18 , ĠB IB REF 19 , ĠB IB REF 5 . ĠMubarak Ġet Ġal . Ġm ub arak 2017 ab usive Ġsuggested Ġthat Ġcertain Ġusers Ġare Ġmore Ġlikely Ġto Ġuse Ġoffensive Ġlanguages Ġthan Ġothers , Ġand Ġthey Ġused Ġthis Ġinsight Ġto Ġbuild Ġa Ġlist Ġof Ġoffensive ĠArabic Ġwords Ġand Ġthey Ġconstructed Ġa Ġlabeled Ġset Ġof Ġ1 , 100 Ġtweets . ĠA bo zin ad ah Ġet Ġal . Ġab oz in ad ah 2017 det ect ing Ġused Ġsupervised Ġclassification Ġbased Ġon Ġa Ġvariety Ġof Ġfeatures Ġincluding Ġuser Ġprofile Ġfeatures , Ġtextual Ġfeatures , Ġand Ġnetwork Ġfeatures . ĠThey Ġreported Ġan Ġaccuracy Ġof Ġnearly Ġ90 %. ĠAl ak rot Ġet Ġal . Ġal ak rot 2018 t ow ards Ġused Ġsupervised Ġclassification Ġbased Ġon Ġword Ġun ig rams Ġand Ġn - gram s Ġto Ġdetect Ġoffensive Ġlanguage Ġin ĠYouTube Ġcomments . ĠThey Ġimproved Ġclassification Ġwith Ġstemming Ġand Ġachieved Ġa Ġprecision Ġof Ġ88 %. ĠAlb adi Ġet Ġal . Ġal b adi 2018 they Ġfocused Ġon Ġdetecting Ġreligious Ġhate Ġspeech Ġusing Ġa Ġrecurrent Ġneural Ġnetwork . ĠFurther ; ĠSchmidt Ġand ĠW ieg and Ġsch midt - w ieg and - 2017 - sur vey Ġsurveyed Ġmajor Ġworks Ġon Ġhate Ġspeech Ġdetection ; ĠF ortun a Ġand ĠNunes ĠF ortun a 2018 Sur vey Ġprovided Ġa Ġcomprehensive Ġsurvey Ġfor Ġtechniques Ġand Ġworks Ġdone Ġin Ġthe Ġarea Ġbetween Ġ2004 Ġand Ġ2017 . </s> Arab ic Ġis Ġa Ġmorph ologically Ġrich Ġlanguage Ġwith Ġa Ġstandard Ġvariety , Ġnamely ĠModern ĠStandard ĠArabic Ġ( M SA ) Ġand Ġis Ġtypically Ġused Ġin Ġformal Ġcommunication , Ġand Ġmany Ġdialect al Ġvarieties Ġthat Ġdiffer Ġfrom ĠM SA Ġin Ġlex ical Ġselection , Ġmorphology , Ġand Ġsynt actic Ġstructures . ĠFor ĠM SA , Ġwords Ġare Ġtypically Ġderived Ġfrom Ġa Ġset Ġof Ġthousands Ġof Ġroots Ġby Ġfitting Ġa Ġroot Ġinto Ġa Ġstem Ġtemplate Ġand Ġthe Ġresulting Ġstem Ġmay Ġaccept Ġa Ġvariety Ġof Ġprefix es Ġand Ġsuffix es Ġsuch Ġas Ġcoordinating Ġconj unctions Ġand Ġpronouns . ĠThough Ġword Ġsegment ation Ġ( or Ġstemming ) Ġis Ġquite Ġaccurate Ġfor ĠM SA ĠB IB REF 20 , Ġwith Ġaccuracy Ġapproaching Ġ99 %, Ġdialect al Ġsegment ation Ġis Ġnot Ġsufficiently Ġreliable , Ġwith Ġaccuracy Ġranging Ġbetween Ġ91 - 95 % Ġfor Ġdifferent Ġdialect s ĠB IB REF 21 . ĠSince Ġdialect al ĠArabic Ġis Ġubiquitous Ġin ĠArabic Ġtweets Ġand Ġmany Ġtweets Ġhave Ġcreative Ġspell ings Ġof Ġwords , Ġrecent Ġwork Ġon ĠArabic Ġoffensive Ġlanguage Ġdetection Ġused Ġcharacter - level Ġmodels ĠB IB REF 5 . </s> Data ĠCollection Ġ: :: ĠCollect ing ĠArabic ĠOffensive ĠTwe ets </s> Our Ġtarget Ġwas Ġto Ġbuild Ġa Ġlarge ĠArabic Ġoffensive Ġlanguage Ġdataset Ġthat Ġis Ġrepresentative Ġof Ġtheir Ġappearance Ġon ĠTwitter Ġand Ġis Ġhopefully Ġnot Ġbiased Ġto Ġspecific Ġdialect s , Ġtopics , Ġor Ġtargets . ĠOne Ġof Ġthe Ġmain Ġchallenges Ġis Ġthat Ġoffensive Ġtweets Ġconstitute Ġa Ġvery Ġsmall Ġportion Ġof Ġoverall Ġtweets . ĠTo Ġquantify Ġtheir Ġproportion , Ġwe Ġtook Ġ3 Ġrandom Ġsamples Ġof Ġtweets Ġfrom Ġdifferent Ġdays , Ġwith Ġeach Ġsample Ġcomposed Ġof Ġ1 , 000 Ġtweets , Ġand Ġwe Ġfound Ġthat Ġbetween Ġ1 % Ġand Ġ2 % Ġof Ġthem Ġwere Ġin Ġfact Ġoffensive Ġ( including Ġpornographic Ġadvertisement ). ĠThis Ġpercentage Ġis Ġconsistent Ġwith Ġpreviously Ġreported Ġpercentages ĠB IB REF 19 . ĠThus , Ġannot ating Ġrandom Ġtweets Ġis Ġgrossly Ġinefficient . ĠOne Ġway Ġto Ġovercome Ġthis Ġproblem Ġis Ġto Ġuse Ġa Ġseed Ġlist Ġof Ġoffensive Ġwords Ġto Ġfilter Ġtweets . ĠHowever , Ġdoing Ġso Ġis Ġproblematic Ġas Ġit Ġwould Ġskew Ġthe Ġdataset Ġto Ġparticular Ġtypes Ġof Ġoffensive Ġlanguage Ġor Ġto Ġspecific Ġdialect s . ĠOff ens iveness Ġis Ġoften Ġdialect Ġand Ġcountry Ġspecific . </s> After Ġinspecting Ġmany Ġtweets , Ġwe Ġobserved Ġthat Ġmany Ġoffensive Ġtweets Ġhave Ġthe Ġvoc ative Ġparticle ĠÙ Ĭ Ø§ > Ġ( âĢ ľ y A âĢ Ŀ ĠâĢĵ Ġmeaning ĠâĢ ľ O âĢ Ŀ ), Ġwhich Ġis Ġmainly Ġused Ġin Ġdirecting Ġthe Ġspeech Ġto Ġa Ġspecific Ġperson Ġor Ġgroup . ĠThe Ġratio Ġof Ġoffensive Ġtweets Ġincreases Ġto Ġ5 % Ġif Ġeach Ġtweet Ġcontains Ġone Ġvoc ative Ġparticle Ġand Ġto Ġ19 % Ġif Ġhas Ġat Ġleast Ġtwo Ġvoc ative Ġparticles . ĠUsers Ġoften Ġrepeat Ġthis Ġparticle Ġfor Ġemphasis , Ġas Ġin : ĠÙ Ĭ Ø§ ĠØ £ Ùħ ÙĬ ĠÙ Ĭ Ø§ ĠØ Ń ÙĨ ÙĪ ÙĨ Ø© > Ġ( âĢ ľ y A ĠAmy Ġy A ĠH n wn p âĢ Ŀ ĠâĢĵ ĠO Ġmy Ġmother , ĠO Ġkind Ġone ), Ġwhich Ġis Ġend earing Ġand Ġnon - offensive , Ġand ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ ĠÙ Ĭ Ø§ ĠÙ Ĥ Ø ° Ø± > Ġ( âĢ ľ y A Ġk lb Ġy A Ġq * r âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdog , ĠO Ġdirty Ġone âĢ Ŀ ), Ġwhich Ġis Ġoffensive . ĠWe Ġdecided Ġto Ġuse Ġthis Ġpattern Ġto Ġincrease Ġour Ġchances Ġof Ġfinding Ġoffensive Ġtweets . ĠOne Ġof Ġthe Ġmain Ġadvantages Ġof Ġthe Ġpattern ĠÙ Ĭ Ø§ Ġ... ĠÙ Ĭ Ø§ > Ġ( âĢ ľ y A Ġ... Ġy A âĢ Ŀ ) Ġis Ġthat Ġit Ġis Ġnot Ġassociated Ġwith Ġany Ġspecific Ġtopic Ġor Ġgenre , Ġand Ġit Ġappears Ġin Ġall ĠArabic Ġdialect s . ĠThough Ġthe Ġuse Ġof Ġoffensive Ġlanguage Ġdoes Ġnot Ġnecess itate Ġthe Ġappearance Ġof Ġthe Ġvoc ative Ġparticle , Ġthe Ġparticle Ġdoes Ġnot Ġfavor Ġany Ġspecific Ġoffensive Ġexpressions Ġand Ġgreatly Ġimproves Ġour Ġchances Ġof Ġfinding Ġoffensive Ġtweets . ĠIt Ġis Ġclear , Ġthe Ġdataset Ġis Ġmore Ġbiased Ġtoward Ġpositive Ġclass . ĠUsing Ġthe Ġdataset Ġfor Ġreal - life Ġapplication Ġmay Ġrequire Ġde - bi asing Ġit Ġby Ġboosting Ġnegative Ġclass Ġor Ġrandom Ġsampling Ġadditional Ġdata Ġfrom ĠTwitter ĠB IB REF 22 . Using Ġthe ĠTwitter ĠAPI , Ġwe Ġcollected Ġ660 k ĠArabic Ġtweets Ġhaving Ġthis Ġpattern Ġbetween ĠApril Ġ15 , Ġ2019 Ġand ĠMay Ġ6 , Ġ2019 . ĠTo Ġincrease Ġdiversity , Ġwe Ġsorted Ġthe Ġword Ġsequences Ġbetween Ġthe Ġvoc ative Ġparticles Ġand Ġtook Ġthe Ġmost Ġfrequent Ġ10 , 000 Ġunique Ġsequences . ĠFor Ġeach Ġword Ġsequence , Ġwe Ġtook Ġa Ġrandom Ġtweet Ġcontaining Ġeach Ġsequence . ĠThen Ġwe Ġannot ated Ġthose Ġtweets , Ġending Ġup Ġwith Ġ1 , 915 Ġoffensive Ġtweets Ġwhich Ġrepresent Ġroughly Ġ19 % Ġof Ġall Ġtweets . ĠEach Ġtweet Ġwas Ġlabeled Ġas : Ġoffensive , Ġwhich Ġcould Ġadditionally Ġbe Ġlabeled Ġas Ġvulgar Ġand / or Ġhate Ġspeech , Ġor ĠClean . ĠWe Ġdescribe Ġin Ġgreater Ġdetail Ġour Ġannotation Ġguidelines , Ġwhich Ġwe Ġmade Ġsure Ġthat Ġthey Ġare Ġcompatible Ġwith Ġthe ĠOff ens E val 2019 Ġannotation Ġguidelines ĠB IB REF 15 . ĠFor Ġexample , Ġif Ġa Ġtweet Ġhas Ġinsults Ġor Ġthreats Ġtargeting Ġa Ġgroup Ġbased Ġon Ġtheir Ġnationality , Ġethnicity , Ġgender , Ġpolitical Ġaffiliation , Ġreligious Ġbelief , Ġor Ġother Ġcommon Ġcharacteristics , Ġthis Ġis Ġconsidered Ġas Ġhate Ġspeech ĠB IB REF 15 . ĠIt Ġis Ġworth Ġmentioning Ġthat Ġwe Ġalso Ġconsidered Ġinsulting Ġgroups Ġbased Ġon Ġtheir Ġsport Ġaffiliation Ġas Ġa Ġform Ġof Ġhate Ġspeech . ĠIn Ġmost ĠArab Ġcountries , Ġbeing Ġa Ġfan Ġof Ġa Ġparticularly Ġsporting Ġclub Ġis Ġconsidered Ġas Ġpart Ġof Ġthe Ġpersonality Ġand Ġideology Ġwhich Ġrarely Ġchanges Ġover Ġtime Ġ( similar Ġto Ġpolitical Ġaffiliation ). ĠMany Ġincidents Ġof Ġviolence Ġhave Ġoccurred Ġamong Ġfans Ġof Ġrival Ġclubs . </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets </s> We Ġdeveloped Ġthe Ġannotation Ġguidelines Ġjointly Ġwith Ġan Ġexperienced Ġannot ator , Ġwho Ġis Ġa Ġnative ĠArabic Ġspeaker Ġwith Ġa Ġgood Ġknowledge Ġof Ġvarious ĠArabic Ġdialect s . ĠWe Ġmade Ġsure Ġthat Ġour Ġguidelines Ġwere Ġcompatible Ġwith Ġthose Ġof ĠOff ens E val 2019 . ĠThe Ġannot ator Ġcarried Ġout Ġall Ġannotation . ĠTwe ets Ġwere Ġgiven Ġone Ġor Ġmore Ġof Ġthe Ġfollowing Ġfour Ġlabels : Ġoffensive , Ġvulgar , Ġhate Ġspeech , Ġor Ġclean . ĠSince Ġthe Ġoffensive Ġlabel Ġcovers Ġboth Ġvulgar Ġand Ġhate Ġspeech Ġand Ġvulgar ity Ġand Ġhate Ġspeech Ġare Ġnot Ġmutually Ġexclusive , Ġa Ġtweet Ġcan Ġbe Ġjust Ġoffensive Ġor Ġoffensive Ġand Ġvulgar Ġand / or Ġhate Ġspeech . ĠThe Ġannotation Ġad hered Ġto Ġthe Ġfollowing Ġguidelines : </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠOFF ENS IVE Ġ( OFF ): </s> Off ensive Ġtweets Ġcontain Ġexplicit Ġor Ġimplicit Ġinsults Ġor Ġattacks Ġagainst Ġother Ġpeople , Ġor Ġinappropriate Ġlanguage , Ġsuch Ġas : </s> Direct Ġthreats Ġor Ġinc itement , Ġex : Ġ Ø§Ø Ń Ø± Ù Ĥ ÙĪ Ø§ > ĠÙħ Ù Ĥ Ø± Ø§Ø ª ĠØ§ÙĦ Ùħ Ø¹ Ø§ Ø± Ø ¶ Ø© > Ġ( âĢ ľ AH r q w A Ġm q r At ĠAlm E Ar D p âĢ Ŀ ĠâĢĵ ĠâĢ ľ burn Ġthe Ġheadquarters Ġof Ġthe Ġopposition âĢ Ŀ ) Ġand ĠÙ ĩ Ø ° Ø§ ĠØ§ÙĦ Ùħ ÙĨ Ø§ Ù ģ Ù Ĥ ĠÙ Ĭ Ø ¬ Ø¨ ĠÙ Ĥ Øª ÙĦ Ùĩ > Ġ( âĢ ľ h * A ĠAl mn Af q Ġy j b Ġq tl h âĢ Ŀ ĠâĢĵ ĠâĢ ľ this Ġhypocr ite Ġneeds Ġto Ġbe Ġkilled âĢ Ŀ ). </s> Ins ults Ġand Ġexpressions Ġof Ġcontempt , Ġwhich Ġinclude : ĠAnimal Ġanalogy , Ġex : ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ y A Ġk lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdog âĢ Ŀ ) Ġand ĠÙ ĥ ÙĦ ĠØ ª Ø¨ ÙĨ > Ġ( âĢ ľ kl Ġt bn âĢ Ŀ ĠâĢĵ ĠâĢ ľ eat Ġhay âĢ Ŀ ). ; ĠIns ult Ġto Ġfamily Ġmembers , Ġex : ĠÙ Ĭ Ø§ ĠØ ± ÙĪ Ø Ń ĠØ £ Ùħ Ù ĥ > Ġ( âĢ ľ y A Ġr w H ĠAm k âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġmother 's Ġsoul âĢ Ŀ ); ĠSex ually - related Ġinsults , Ġex : ĠÙ Ĭ Ø§ ĠØ ¯ ÙĬ ÙĪ Ø « > Ġ( âĢ ľ y A Ġdy w v âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġperson Ġwithout Ġenvy âĢ Ŀ ); ĠDam nation , Ġex : ĠØ§ÙĦ ÙĦ Ùĩ ĠÙ Ĭ ÙĦ Ø¹ ÙĨ Ù ĥ > Ġ( âĢ ľ All h Ġy l En k âĢ Ŀ ĠâĢĵ ĠâĢ ľ may ĠAllah / God Ġcurse Ġyou âĢ Ŀ ); Ġand ĠAttacks Ġon Ġmorals Ġand Ġethics , Ġex : ĠÙ Ĭ Ø§ ĠÙ ĥ Ø§Ø ° Ø¨ > Ġ( âĢ ľ y A Ġk A * b âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġliar âĢ Ŀ ) </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠV UL G AR Ġ( V LG ): </s> V ul gar Ġtweets Ġare Ġa Ġsubset Ġof Ġoffensive Ġtweets Ġand Ġcontain Ġprof anity , Ġsuch Ġas Ġmentions Ġof Ġprivate Ġparts Ġor Ġsexual - related Ġacts Ġor Ġreferences . </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠH ATE ĠSP E ECH Ġ( HS ): </s> H ate Ġspeech Ġtweets , Ġa Ġsubset Ġof Ġoffensive Ġtweets Ġcontaining Ġoffensive Ġlanguage Ġtargeting Ġgroup Ġbased Ġon Ġcommon Ġcharacteristics Ġsuch Ġas : ĠRace , Ġex : ĠÙ Ĭ Ø§ ĠØ ² ÙĨ Ø ¬ ÙĬ > Ġ( âĢ ľ y A Ġz n j y âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġnegro âĢ Ŀ ); ĠEthnic ity , Ġex . ĠØ§ÙĦ Ù ģ Ø± Ø³ ĠØ§ÙĦ Ø £ ÙĨ Ø ¬ Ø§Ø ³ > Ġ( âĢ ľ A lf rs ĠAl An j As âĢ Ŀ ĠâĢĵ ĠâĢ ľ Imp ure ĠPers ians âĢ Ŀ ); ĠGroup Ġor Ġparty , Ġex : ĠØ £ Ø¨ ÙĪ Ù ĥ ĠØ ´ ÙĬ ÙĪ Ø¹ ÙĬ > Ġ( âĢ ľ Ab wk Ġ$ y w Ey âĢ Ŀ ĠâĢĵ ĠâĢ ľ your Ġfather Ġis Ġcommunist âĢ Ŀ ); Ġand ĠReligion , Ġex : ĠØ ¯ ÙĬ ÙĨ Ù ĥ ĠØ§ÙĦ Ù Ĥ Ø ° Ø± > Ġ( âĢ ľ d yn k ĠAl q * r âĢ Ŀ ĠâĢĵ ĠâĢ ľ your Ġfilthy Ġreligion âĢ Ŀ ). </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠCLE AN Ġ( CL N ): </s> Clean Ġtweets Ġdo Ġnot Ġcontain Ġvulgar Ġor Ġoffensive Ġlanguage . ĠWe Ġnoticed Ġthat Ġsome Ġtweets Ġhave Ġsome Ġoffensive Ġwords , Ġbut Ġthe Ġwhole Ġtweet Ġshould Ġnot Ġbe Ġconsidered Ġas Ġoffensive Ġdue Ġto Ġthe Ġintention Ġof Ġusers . ĠThis Ġsuggests Ġthat Ġnormal Ġstring Ġmatch Ġwithout Ġconsidering Ġcontexts Ġwill Ġfail Ġin Ġsome Ġcases . ĠExamples Ġof Ġsuch Ġambiguous Ġcases Ġinclude : ĠHum or , Ġex : ĠÙ Ĭ Ø§ ĠØ ¹ Ø¯ ÙĪ Ø© ĠØ§ÙĦ Ù ģ Ø± Ø Ń Ø© ĠÙ ĩ Ùĩ Ùĩ > Ġ( âĢ ľ y A ĠEd wp ĠAl fr H p Ġh hh âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġenemy Ġof Ġhappiness Ġha h aha âĢ Ŀ ); ĠAdvice , Ġex : Ġ ÙĦ Ø§ ĠØ ª Ù Ĥ ÙĦ Ġ ÙĦ Ø µ Ø§Ø Ń Ø¨ Ù ĥ ĠÙ Ĭ Ø§ ĠØ ® ÙĨ Ø ² ÙĬ Ø± > Ġ( âĢ ľ l A Ġt ql Ġl SA H b k Ġy A Ġx nz yr âĢ Ŀ ĠâĢĵ ĠâĢ ľ don 't Ġsay Ġto Ġyour Ġfriend : ĠYou Ġare Ġa Ġpig âĢ Ŀ ); ĠCondition , Ġex : ĠØ ¥ Ø ° Ø§ ĠØ ¹ Ø§ Ø± Ø ¶ Øª Ùĩ Ùħ ĠÙ Ĭ Ù Ĥ ÙĪ ÙĦ ÙĪ ÙĨ ĠÙ Ĭ Ø§ ĠØ ¹ Ùħ ÙĬ ÙĦ > Ġ( âĢ ľ A * A ĠE Ar D th m Ġy q wl wn Ġy A ĠE my l âĢ Ŀ ĠâĢĵ ĠâĢ ľ if Ġyou Ġdisagree Ġwith Ġthem Ġthey Ġwill Ġsay : ĠYou Ġare Ġan Ġagent âĢ Ŀ ); ĠCond em nation , Ġex : Ġ ÙĦ Ùħ Ø§Ø ° Ø§ ĠÙ Ĩ Ø³ Ø¨ ĠØ ¨ Ù Ĥ ÙĪ ÙĦ : ĠÙ Ĭ Ø§ ĠØ ¨ Ù Ĥ Ø± Ø© Ø Ł > Ġ( âĢ ľ l mA * A Ġn sb Ġb q wl : Ġy A Ġb q r p ? âĢ Ŀ ĠâĢĵ ĠâĢ ľ Why Ġdo Ġwe Ġinsult Ġothers Ġby Ġsaying : ĠO Ġcow ? âĢ Ŀ ); ĠSelf Ġoffense , Ġex : ĠØ ª Ø¹ Ø¨ Øª ĠÙħ ÙĨ Ġ ÙĦ Ø³ Ø§ ÙĨ ÙĬ ĠØ§ÙĦ Ù Ĥ Ø ° Ø± > Ġ( âĢ ľ t E bt Ġm n Ġls Any ĠAl q * r âĢ Ŀ ĠâĢĵ ĠâĢ ľ I Ġam Ġtired Ġof Ġmy Ġdirty Ġtongue âĢ Ŀ ); ĠNon - human Ġtarget , Ġex : ĠÙ Ĭ Ø§ ĠØ ¨ ÙĨ Øª ĠØ§ÙĦ Ùħ Ø ¬ ÙĨ ÙĪ ÙĨ Ø© ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĪ Ø± Ø© > Ġ( âĢ ľ y A Ġb nt ĠAlm j n wn p Ġy A Ġk wr p âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdaughter Ġof Ġthe Ġcrazy Ġone ĠO Ġfootball âĢ Ŀ ); Ġand ĠQu otation Ġfrom Ġa Ġmovies Ġor Ġa Ġstory , Ġex : ĠØ ª Ø§ ÙĨ ÙĬ ĠÙ Ĭ Ø§ ĠØ ² Ù ĥ ÙĬ ! ĠØ ª Ø§ ÙĨ ÙĬ ĠÙ Ĭ Ø§ ĠÙ ģ Ø§Ø ´ ÙĦ > Ġ( âĢ ľ t Any Ġy A Ġz ky ! Ġt Any Ġy A Ġf A $ l âĢ Ŀ ĠâĢĵ ĠâĢ ľ again ĠO ĠZ aky ! Ġagain ĠO Ġloser âĢ Ŀ ). ĠFor Ġother Ġambiguous Ġcases , Ġthe Ġannot ator Ġsearched ĠTwitter Ġto Ġfind Ġhow Ġactual Ġusers Ġused Ġexpressions . </s> Table ĠT AB REF 11 Ġshows Ġthe Ġdistribution Ġof Ġthe Ġannot ated Ġtweets . ĠThere Ġare Ġ1 , 915 Ġoffensive Ġtweets , Ġincluding Ġ225 Ġvulgar Ġtweet Ġand Ġ50 6 Ġhate Ġspeech Ġtweets , Ġand Ġ8 , 08 5 Ġclean Ġtweets . ĠTo Ġvalidate Ġthe Ġquality Ġof Ġannotation , Ġa Ġrandom Ġsample Ġof Ġ100 Ġtweets Ġfrom Ġthe Ġdata , Ġcontaining Ġ50 Ġoffensive Ġand Ġ50 Ġclean Ġtweets , Ġwas Ġgiven Ġto Ġadditional Ġthree Ġannot ators . ĠWe Ġcalculated Ġthe ĠInter - An not ator ĠAgreement Ġbetween Ġthe Ġannot ators Ġusing ĠFle iss âĢ Ļ s ĠKappa Ġcoefficient ĠB IB REF 23 . ĠThe ĠKappa Ġscore Ġwas Ġ0 . 92 Ġindicating Ġhigh Ġquality Ġannotation Ġand Ġagreement . </s> Data ĠCollection Ġ: :: ĠStatistics Ġand ĠUser ĠDem ographics </s> Given Ġthe Ġannot ated Ġtweets , Ġwe Ġwanted Ġto Ġascertain Ġthe Ġdistribution Ġof : Ġtypes Ġof Ġoffensive Ġlanguage , Ġgenres Ġwhere Ġit Ġis Ġused , Ġthe Ġdialect s Ġused , Ġand Ġthe Ġgender Ġof Ġusers Ġusing Ġsuch Ġlanguage . </s> Figure ĠFIG REF 13 Ġshows Ġthe Ġdistribution Ġof Ġtopics Ġassociated Ġwith Ġoffensive Ġtweets . ĠAs Ġthe Ġfigure Ġshows , Ġsports Ġand Ġpolitics Ġare Ġmost Ġdominant Ġfor Ġoffensive Ġlanguage Ġincluding Ġvulgar Ġand Ġhate Ġspeech . ĠAs Ġfor Ġdialect , Ġwe Ġlooked Ġat ĠM SA Ġand Ġfour Ġmajor Ġdialect s , Ġnamely ĠEgyptian Ġ( EG Y ), ĠLe vent ine Ġ( LE V ), ĠMag h re bi Ġ( M GR ), Ġand ĠGulf Ġ( GL F ). ĠFigure ĠFIG REF 14 Ġshows Ġthat Ġ71 % Ġof Ġvulgar Ġtweets Ġwere Ġwritten Ġin ĠE GY Ġfollowed Ġby ĠGL F , Ġwhich Ġaccounted Ġfor Ġ13 % Ġof Ġvulgar Ġtweets . ĠM SA Ġwas Ġnot Ġused Ġin Ġany Ġof Ġthe Ġvulgar Ġtweets . ĠAs Ġfor Ġoffensive Ġtweets Ġin Ġgeneral , ĠE GY Ġand ĠGL F Ġwere Ġused Ġin Ġ36 % Ġand Ġ35 % Ġof Ġthe Ġoffensive Ġtweets Ġrespectively . ĠUnlike Ġthe Ġcase Ġof Ġvulgar Ġlanguage Ġwhere ĠM SA Ġwas Ġnon - existent , Ġ15 % Ġof Ġthe Ġoffensive Ġtweets Ġwere Ġin Ġfact Ġwritten Ġin ĠM SA . ĠFor Ġhate Ġspeech , ĠGL F Ġand ĠE GY Ġwere Ġagain Ġdominant Ġand ĠM SA Ġconsist uted Ġ21 % Ġof Ġthe Ġtweets . ĠThis Ġis Ġconsistent Ġwith Ġfindings Ġfor Ġother Ġlanguages Ġsuch Ġas ĠEnglish Ġand ĠItalian Ġwhere Ġvulgar Ġlanguage Ġwas Ġmore Ġfrequently Ġassociated Ġwith Ġcoll oqu ial Ġlanguage ĠB IB REF 24 , ĠB IB REF 25 . ĠRegarding Ġthe Ġgender , ĠFigure ĠFIG REF 15 Ġshows Ġthat Ġthe Ġvast Ġmajority Ġof Ġoffensive Ġtweets , Ġincluding Ġvulgar Ġand Ġhate Ġspeech , Ġwere Ġauthored Ġby Ġmales . ĠFemale ĠTwitter Ġusers Ġaccounted Ġfor Ġ14 % Ġof Ġoffensive Ġtweets Ġin Ġgeneral Ġand Ġ6 % Ġand Ġ9 % Ġof Ġvulgar Ġand Ġhate Ġspeech Ġrespectively . ĠFigure ĠFIG REF 16 Ġshows Ġa Ġdetailed Ġcategor ization Ġof Ġhate Ġspeech Ġtypes , Ġwhere Ġthe Ġtop Ġthree Ġinclude Ġinsulting Ġgroups Ġbased Ġon Ġtheir Ġpolitical Ġideology , Ġorigin , Ġand Ġsport Ġaffiliation . ĠReligious Ġhate Ġspeech Ġappeared Ġin Ġonly Ġ15 % Ġof Ġall Ġhate Ġspeech Ġtweets . </s> Next , Ġwe Ġanalyzed Ġall Ġtweets Ġlabeled Ġas Ġoffensive Ġto Ġbetter Ġunderstand Ġhow ĠArabic Ġspeakers Ġuse Ġoffensive Ġlanguage . ĠHere Ġis Ġa Ġbreakdown Ġof Ġusage : </s> Direct Ġname Ġcalling : ĠThe Ġmost Ġfrequent Ġattack Ġis Ġto Ġcall Ġa Ġperson Ġan Ġanimal Ġname , Ġand Ġthe Ġmost Ġused Ġanimals Ġwere ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ k lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ dog âĢ Ŀ ), ĠØ Ń Ùħ Ø§ Ø± > Ġ( âĢ ľ H m Ar âĢ Ŀ ĠâĢĵ ĠâĢ ľ don key âĢ Ŀ ), Ġand ĠØ ¨ Ùĩ ÙĬ Ùħ > Ġ( âĢ ľ bh ym âĢ Ŀ ĠâĢĵ ĠâĢ ľ be ast âĢ Ŀ ). ĠThe Ġsecond Ġmost Ġcommon Ġwas Ġinsulting Ġmental Ġabilities Ġusing Ġwords Ġsuch Ġas ĠØ º Ø¨ ÙĬ > Ġ( âĢ ľ g by âĢ Ŀ ĠâĢĵ ĠâĢ ľ st upid âĢ Ŀ ) Ġand ĠØ ¹ Ø¨ ÙĬ Ø · > Ġ( âĢ ľ E by T âĢ Ŀ ĠâĢĵ âĢ ľ id iot âĢ Ŀ ). ĠSome Ġculture - specific Ġdifferences Ġshould Ġbe Ġconsidered . ĠNot Ġall Ġanimal Ġnames Ġare Ġused Ġas Ġinsults . ĠFor Ġexample , Ġanimals Ġsuch Ġas ĠØ £ Ø³ Ø¯ > Ġ( âĢ ľ As d âĢ Ŀ ĠâĢĵ ĠâĢ ľ l ion âĢ Ŀ ), ĠØ µ Ù Ĥ Ø± > Ġ( âĢ ľ S q r âĢ Ŀ ĠâĢĵ ĠâĢ ľ fal con âĢ Ŀ ), Ġand ĠØ º Ø ² Ø§ÙĦ > Ġ( âĢ ľ gz Al âĢ Ŀ ĠâĢĵ ĠâĢ ľ g az elle âĢ Ŀ ) Ġare Ġtypically Ġused Ġfor Ġpraise . ĠFor Ġother Ġinsults , Ġpeople Ġuse : Ġsome Ġbird Ġnames Ġsuch Ġas ĠØ ¯ Ø ¬ Ø§Ø ¬ Ø© > Ġ( âĢ ľ dj A jp âĢ Ŀ ĠâĢĵ ĠâĢ ľ ch icken âĢ Ŀ ), ĠØ ¨ ÙĪ Ùħ Ø© > Ġ( âĢ ľ b w mp âĢ Ŀ ĠâĢĵ ĠâĢ ľ owl âĢ Ŀ ), Ġand ĠØ º Ø± Ø§Ø ¨ > Ġ( âĢ ľ gr Ab âĢ Ŀ ĠâĢĵ ĠâĢ ľ crow âĢ Ŀ ); Ġinsects Ġsuch Ġas ĠØ ° Ø¨ Ø§Ø ¨ Ø© > Ġ( âĢ ľ * b Ab p âĢ Ŀ ĠâĢĵ ĠâĢ ľ fly âĢ Ŀ ), ĠØ µ Ø± Ø µ ÙĪ Ø± > Ġ( âĢ ľ S r Sw r âĢ Ŀ ĠâĢĵ ĠâĢ ľ cock roach âĢ Ŀ ), Ġand ĠØ Ń Ø ´ Ø± Ø© > Ġ( âĢ ľ H $ r p âĢ Ŀ ĠâĢĵ ĠâĢ ľ in sect âĢ Ŀ ); Ġmicro organisms Ġsuch Ġas ĠØ ¬ Ø± Ø « ÙĪ Ùħ Ø© > Ġ( âĢ ľ j r v w mp âĢ Ŀ ĠâĢĵ ĠâĢ ľ micro be âĢ Ŀ ) Ġand ĠØ · Ø Ń Ø§ÙĦ Ø¨ > Ġ( âĢ ľ TH Al b âĢ Ŀ ĠâĢĵ ĠâĢ ľ alg ae âĢ Ŀ ); Ġin animate Ġobjects Ġsuch Ġas ĠØ ¬ Ø ² Ùħ Ø© > Ġ( âĢ ľ j z mp âĢ Ŀ ĠâĢĵ ĠâĢ ľ sh oes âĢ Ŀ ) Ġand ĠØ ³ Ø · ÙĦ > Ġ( âĢ ľ s T l âĢ Ŀ ĠâĢĵ ĠâĢ ľ buck et âĢ Ŀ ) Ġamong Ġother Ġus ages . </s> Sim ile Ġand Ġmetaphor : ĠUsers Ġuse Ġsim ile Ġand Ġmetaphor Ġwere Ġthey Ġwould Ġcompare Ġa Ġperson Ġto : Ġan Ġanimal Ġas Ġin ĠØ ² ÙĬ ĠØ§ÙĦ Ø « ÙĪ Ø± > Ġ( âĢ ľ zy ĠAlv wr âĢ Ŀ ĠâĢĵ ĠâĢ ľ like Ġa Ġbull âĢ Ŀ ), ĠØ ³ Ùħ Ø¹ ÙĨ ÙĬ ĠÙ Ĩ Ùĩ ÙĬ Ù Ĥ Ù ĥ > Ġ( âĢ ľ sm E ny Ġn hy q k âĢ Ŀ ĠâĢĵ ĠâĢ ľ let Ġme Ġhear Ġyour Ġbr aying âĢ Ŀ ), Ġand ĠÙ ĩ Ø ² ĠØ ¯ ÙĬ ÙĦ Ù ĥ > Ġ( âĢ ľ hz Ġd yl k âĢ Ŀ ĠâĢĵ ĠâĢ ľ w ag Ġyour Ġtail âĢ Ŀ ); Ġa Ġperson Ġwith Ġmental Ġor Ġphysical Ġdisability Ġsuch Ġas ĠÙħ ÙĨ Ø º ÙĪ ÙĦ ÙĬ > Ġ( âĢ ľ m ng w ly âĢ Ŀ ĠâĢĵ ĠâĢ ľ M ong ol ian Ġ( down - sy nd rome ) âĢ Ŀ ), ĠÙħ Ø¹ ÙĪ Ù Ĥ > Ġ( âĢ ľ m E w q âĢ Ŀ ĠâĢĵ ĠâĢ ľ disabled âĢ Ŀ ), Ġand ĠÙ Ĥ Ø ² Ùħ > Ġ( âĢ ľ q z m âĢ Ŀ ĠâĢĵ ĠâĢ ľ d war f âĢ Ŀ ); Ġand Ġto Ġthe Ġopposite Ġgender Ġsuch Ġas ĠØ ¬ ÙĬ Ø ´ ĠÙ Ĩ ÙĪ Ø§ÙĦ > Ġ( âĢ ľ j y $ Ġn w Al âĢ Ŀ ĠâĢĵ ĠâĢ ľ N aw al 's Ġarmy Ġ( N aw al Ġis Ġfemale Ġname ) âĢ Ŀ ) Ġand ĠÙ Ĩ Ø§Ø ¯ ÙĬ ĠØ ² ÙĬ Ø ² ÙĬ > Ġ( âĢ ľ n Ad y Ġz y zy âĢ Ŀ ĠâĢĵ ĠâĢ ľ Z iz i 's Ġclub Ġ( Z iz i Ġis Ġa Ġfemale Ġpet Ġname ) âĢ Ŀ ). </s> Ind irect Ġspeech : ĠThis Ġtype Ġof Ġoffensive Ġlanguage Ġincludes : Ġsarc asm Ġsuch Ġas ĠØ £ Ø ° Ù ĥ Ù ī ĠØ ¥ Ø ® ÙĪ Ø§Ø ª Ù ĥ > Ġ( âĢ ľ A * k Y ĠAx w At k âĢ Ŀ ĠâĢĵ ĠâĢ ľ smart est Ġone Ġof Ġyour Ġsiblings âĢ Ŀ ) Ġand ĠÙ ģ ÙĬ ÙĦ Ø³ ÙĪ Ù ģ ĠØ§ÙĦ Ø Ń Ùħ ÙĬ Ø± > Ġ( âĢ ľ f yl sw f ĠAl H my r âĢ Ŀ ĠâĢĵ ĠâĢ ľ the Ġdon keys ' Ġphilosopher âĢ Ŀ ); Ġquestions Ġsuch Ġas Ġ Ø§ ÙĬ Ùĩ ĠÙ ĥ ÙĦ ĠØ§ÙĦ Ø º Ø¨ Ø§Ø ¡ ĠØ ¯ Ùĩ > Ġ( âĢ ľ Ay h Ġk l ĠAl gb A Ġdh âĢ Ŀ ĠâĢĵ ĠâĢ ľ what Ġis Ġall Ġthis Ġstupidity âĢ Ŀ ); Ġand Ġindirect Ġspeech Ġsuch Ġas ĠØ§ÙĦ ÙĨ Ù Ĥ Ø§Ø ´ ĠÙħ Ø¹ ĠØ§ÙĦ Ø¨ Ùĩ Ø§ ÙĬ Ùħ ĠØ º ÙĬ Ø± ĠÙħ Ø « Ùħ Ø± > Ġ( âĢ ľ Al n q A $ Ġm E ĠAlb h A ym Ġg yr Ġm vm r âĢ Ŀ ĠâĢĵ ĠâĢ ľ no Ġuse Ġtalking Ġto Ġcattle âĢ Ŀ ). </s> W ishing ĠEvil : ĠThis Ġentails Ġwishing Ġdeath Ġor Ġmajor Ġharm Ġto Ġbe fall Ġsomeone Ġsuch Ġas ĠØ ± Ø¨ ÙĨ Ø§ ĠÙ Ĭ Ø§Ø ® Ø¯ Ù ĥ > Ġ( âĢ ľ r bn A Ġy A xd k âĢ Ŀ ĠâĢĵ ĠâĢ ľ May ĠGod Ġtake Ġ( kill ) Ġyou âĢ Ŀ ), ĠØ§ÙĦ ÙĦ Ùĩ ĠÙ Ĭ ÙĦ Ø¹ ÙĨ Ù ĥ > Ġ( âĢ ľ All h Ġy l En k âĢ Ŀ ĠâĢĵ ĠâĢ ľ may ĠAllah / God Ġcurse Ġyou âĢ Ŀ ), Ġand ĠØ ± ÙĪ Ø Ń ĠÙ ģ ÙĬ ĠØ ¯ Ø§ Ùĩ ÙĬ Ø© > Ġ( âĢ ľ rw H Ġf y Ġd Ah yp âĢ Ŀ ĠâĢĵ Ġequivalent Ġto ĠâĢ ľ go Ġto Ġhell âĢ Ŀ ). </s> Name Ġalteration : ĠOne Ġcommon Ġway Ġto Ġinsult Ġothers Ġis Ġto Ġchange Ġa Ġletter Ġor Ġtwo Ġin Ġtheir Ġnames Ġto Ġproduce Ġnew Ġoffensive Ġwords Ġthat Ġrhy me Ġwith Ġthe Ġoriginal Ġnames . ĠSome Ġexamples Ġof Ġsuch Ġinclude Ġchanging ĠØ§ÙĦ Ø ¬ Ø ² ÙĬ Ø± Ø© > Ġ( âĢ ľ Al j z yr p âĢ Ŀ ĠâĢĵ ĠâĢ ľ Al j azeera Ġ( channel ) âĢ Ŀ ) Ġto ĠØ§ÙĦ Ø ® ÙĨ Ø ² ÙĬ Ø± Ø© > Ġ( âĢ ľ Al x nz yr p âĢ Ŀ ĠâĢĵ ĠâĢ ľ the Ġpig âĢ Ŀ ) Ġand ĠØ ® ÙĦ Ù ģ Ø§ ÙĨ > Ġ( âĢ ľ x lf An âĢ Ŀ ĠâĢĵ ĠâĢ ľ K half an Ġ( person Ġname ) âĢ Ŀ ) Ġto ĠØ ® Ø± Ù ģ Ø§ ÙĨ > Ġ( âĢ ľ x rf An âĢ Ŀ ĠâĢĵ ĠâĢ ľ c raz ed âĢ Ŀ ). </s> Soc ietal Ġstrat ification : ĠSome Ġinsults Ġare Ġassociated Ġwith : Ġcertain Ġjobs Ġsuch Ġas ĠØ ¨ ÙĪ Ø§Ø ¨ > Ġ( âĢ ľ b w Ab âĢ Ŀ ĠâĢĵ ĠâĢ ľ do orman âĢ Ŀ ) Ġor ĠØ ® Ø§Ø ¯ Ùħ > Ġ( âĢ ľ x Ad m âĢ Ŀ ĠâĢĵ ĠâĢ ľ serv ant âĢ Ŀ ); Ġand Ġspecific Ġsocietal Ġcomponents Ġsuch ĠØ ¨ Ø¯ ÙĪ ÙĬ > Ġ( âĢ ľ bd wy âĢ Ŀ ĠâĢĵ ĠâĢ ľ bed ou in âĢ Ŀ ) Ġand ĠÙ ģ ÙĦ Ø§Ø Ń > Ġ( âĢ ľ fl AH âĢ Ŀ ĠâĢĵ ĠâĢ ľ far mer âĢ Ŀ ). </s> Imm oral Ġbehavior : ĠThese Ġinsults Ġare Ġassociated Ġwith Ġnegative Ġmoral Ġtraits Ġor Ġbehaviors Ġsuch Ġas ĠØ Ń Ù Ĥ ÙĬ Ø± > Ġ( âĢ ľ H q yr âĢ Ŀ ĠâĢĵ ĠâĢ ľ v ile âĢ Ŀ ), ĠØ ® Ø§ ÙĬ ÙĨ > Ġ( âĢ ľ x A yn âĢ Ŀ ĠâĢĵ ĠâĢ ľ tra itor âĢ Ŀ ), Ġand ĠÙħ ÙĨ Ø§ Ù ģ Ù Ĥ > Ġ( âĢ ľ mn Af q âĢ Ŀ ĠâĢĵ ĠâĢ ľ hyp ocr ite âĢ Ŀ ). </s> Sex ually Ġrelated : ĠThey Ġinclude Ġexpressions Ġsuch Ġas ĠØ ® ÙĪ ÙĦ > Ġ( âĢ ľ x wl âĢ Ŀ ĠâĢĵ ĠâĢ ľ gay âĢ Ŀ ), ĠÙĪ Ø³ Ø ® Ø© > Ġ( âĢ ľ ws xp âĢ Ŀ ĠâĢĵ ĠâĢ ľ pro st itute âĢ Ŀ ), Ġand ĠØ ¹ Ø± Ø µ > Ġ( âĢ ľ Er S âĢ Ŀ ĠâĢĵ ĠâĢ ľ p imp âĢ Ŀ ). </s> Figure ĠFIG REF 17 Ġshows Ġtop Ġwords Ġwith Ġthe Ġhighest Ġval ance Ġscore Ġfor Ġindividual Ġwords Ġin Ġthe Ġoffensive Ġtweets . ĠL arger Ġfonts Ġare Ġused Ġto Ġhighlight Ġwords Ġwith Ġhighest Ġscore Ġand Ġalign Ġas Ġwell Ġwith Ġthe Ġcategories Ġmentioned Ġahead Ġin Ġthe Ġbreakdown Ġfor Ġthe Ġoffensive Ġlanguages . ĠWe Ġmodified Ġthe Ġval ence Ġscore Ġdescribed Ġby ĠB IB REF 1 Ġcon over 2011 political Ġto Ġmagn ify Ġits Ġvalue Ġbased Ġon Ġfrequency Ġof Ġoccurrence . ĠThe Ġscore Ġis Ġcomputed Ġas Ġfollows : </s> $ V ( I ) Ġ= Ġ2 Ġ\ frac { Ġ\ frac { tf ( I , ĠC _ off ) }{ total ( C _ off ) }} {\ frac { tf ( I , ĠC _ off ) }{ total ( C _ off )} Ġ+ Ġ\ frac { tf ( I , ĠC _ cl n ) }{ total ( C _ cl n )} Ġ} Ġ- Ġ1 $ </s> where </s> $ tf ( I , ĠC _ i ) Ġ= Ġ\ sum Ġ_ { a Ġ\ in ĠI Ġ\ big cap ĠC _ i } Ġ[ ln ( C nt ( a , ĠC _ i )) Ġ+ Ġ1 ] $ </s> $ total ( C _ i ) Ġ= Ġ\ sum Ġ_ { I } Ġtf ( I , ĠC _ i ) $ </s> $ C nt ( a , ĠC _ i ) $ Ġis Ġthe Ġnumber Ġof Ġtimes Ġword Ġ$ a $ Ġwas Ġused Ġin Ġoffensive Ġor Ġclean Ġtweets Ġtweets Ġ$ C _ i $. ĠIn Ġessence , Ġwe Ġare Ġreplacing Ġterm Ġfrequencies Ġwith Ġthe Ġnatural Ġlog Ġof Ġthe Ġterm Ġfrequencies . </s> Exper iments </s> We Ġconducted Ġan Ġextensive Ġbattery Ġof Ġexperiments Ġon Ġthe Ġdataset Ġto Ġestablish Ġstrong ĠArabic Ġoffensive Ġlanguage Ġclassification Ġresults . ĠThough Ġthe Ġoffensive Ġtweets Ġhave Ġfiner - gr ained Ġlabels Ġwhere Ġoffensive Ġtweet Ġcould Ġalso Ġbe Ġvulgar Ġor Ġconstitute Ġhate Ġspeech , Ġwe Ġconducted Ġco arser - gr ained Ġclassification Ġto Ġdetermine Ġif Ġa Ġtweet Ġwas Ġoffensive Ġor Ġnot . ĠFor Ġclassification , Ġwe Ġexperimented Ġwith Ġseveral Ġtweet Ġrepresentation Ġand Ġclassification Ġmodels . ĠFor Ġtweet Ġrepresentations , Ġwe Ġused : Ġthe Ġcount Ġof Ġpositive Ġand Ġnegative Ġterms , Ġbased Ġon Ġa Ġpol arity Ġlex icon ; Ġstatic Ġembed d ings , Ġnamely Ġfast Text Ġand ĠSkip - G ram ; Ġand Ġdeep Ġcontextual Ġembed d ings , Ġnamely ĠB ERT base - mult ilingual . </s> Exper iments Ġ: :: ĠData ĠPre - processing </s> We Ġperformed Ġseveral Ġtext Ġpre - processing Ġsteps . ĠFirst , Ġwe Ġtoken ized Ġthe Ġtext Ġusing Ġthe ĠFar asa ĠArabic ĠN LP Ġtool kit ĠB IB REF 20 . ĠSecond , Ġwe Ġremoved ĠURLs , Ġnumbers , Ġand Ġall Ġtweet Ġspecific Ġtokens , Ġnamely Ġmentions , Ġret we ets , Ġand Ġhasht ags Ġas Ġthey Ġare Ġnot Ġpart Ġof Ġthe Ġlanguage Ġsemantic Ġstructure , Ġand Ġtherefore , Ġnot Ġusable Ġin Ġpre - trained Ġembed d ings . ĠThird , Ġwe Ġperformed Ġbasic ĠArabic Ġletter Ġnormal ization , Ġnamely Ġvariants Ġof Ġthe Ġletter Ġale f Ġto Ġbare Ġale f , Ġta Ġmar b out a Ġto Ġha , Ġand Ġale f Ġma qs ou ra Ġto Ġya . ĠWe Ġalso Ġseparated Ġwords Ġthat Ġare Ġcommonly Ġincorrectly Ġattached Ġsuch Ġas ĠÙ Ĭ Ø§ Ù ĥ ÙĦ Ø¨ > Ġ( âĢ ľ y Ak lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdog âĢ Ŀ ), Ġis Ġsplit Ġto ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ y A Ġk lb âĢ Ŀ ). ĠLastly , Ġwe Ġnormalized Ġletter Ġrepet itions Ġto Ġallow Ġfor Ġa Ġmaximum Ġof Ġ2 Ġrepeated Ġletters . ĠFor Ġexample , Ġthe Ġtoken ĠÙ ĩ Ùĩ Ùĩ Ùĩ Ùĩ > Ġ( âĢ ľ hhhh h âĢ Ŀ ĠâĢĵ ĠâĢ ľ ha Ġha Ġha Ġ.. âĢ Ŀ ) Ġis Ġnormalized Ġto ĠÙ ĩ Ùĩ > Ġ( âĢ ľ hh âĢ Ŀ ). ĠWe Ġalso Ġremoved Ġthe ĠArabic Ġshort - d iac rit ics Ġ( t ash ke el ) Ġand Ġword Ġelong ation Ġ( k ash ida ). </s> Exper iments Ġ: :: ĠRepresent ations Ġ: :: ĠLex ical ĠFeatures </s> Since Ġoffensive Ġwords Ġtypically Ġhave Ġa Ġnegative Ġpol arity , Ġwe Ġwanted Ġto Ġtest Ġthe Ġeffectiveness Ġof Ġusing Ġa Ġpol arity Ġlex icon Ġin Ġdetecting Ġoffensive Ġtweets . ĠFor Ġthe Ġlex icon , Ġwe Ġused ĠNile UL ex ĠB IB REF 26 , Ġwhich Ġis Ġan ĠArabic Ġpol arity Ġlex icon Ġcontaining Ġ3 , 279 ĠM SA Ġand Ġ2 , 674 ĠEgyptian Ġterms , Ġout Ġof Ġwhich Ġ4 , 256 Ġare Ġnegative Ġand Ġ1 , 697 Ġare Ġpositive . ĠWe Ġused Ġthe Ġcounts Ġof Ġterms Ġwith Ġpositive Ġpol arity Ġand Ġterms Ġwith Ġnegative Ġpol arity Ġin Ġtweets Ġas Ġfeatures . </s> Exper iments Ġ: :: ĠRepresent ations Ġ: :: ĠStatic ĠEmb edd ings </s> We Ġexperimented Ġwith Ġvarious Ġstatic Ġembed d ings Ġthat Ġwere Ġpre - trained Ġon Ġdifferent Ġcorpor a Ġwith Ġdifferent Ġvector Ġdimension ality . ĠWe Ġcompared Ġpre - trained Ġembed d ings Ġto Ġembed d ings Ġthat Ġwere Ġtrained Ġon Ġour Ġdataset . ĠFor Ġpre - trained Ġembed d ings , Ġwe Ġused : Ġfast Text ĠEgyptian ĠArabic Ġpre - trained Ġembed d ings ĠB IB REF 27 Ġwith Ġvector Ġdimension ality Ġof Ġ300 ; ĠAra V ec Ġskip - gram Ġembed d ings ĠB IB REF 28 , Ġtrained Ġon Ġ66 . 9 M ĠArabic Ġtweets Ġwith Ġ100 - dimensional Ġvectors ; Ġand ĠMaz aj ak Ġskip - gram Ġembed d ings ĠB IB REF 29 , Ġtrained Ġon Ġ250 M ĠArabic Ġtweets Ġwith Ġ300 - dimensional Ġvectors . </s> Sent ence Ġembed d ings Ġwere Ġcalculated Ġby Ġtaking Ġthe Ġmean Ġof Ġthe Ġembed d ings Ġof Ġtheir Ġtokens . ĠThe Ġimportance Ġof Ġtesting Ġa Ġcharacter Ġlevel Ġn - gram Ġmodel Ġlike Ġfast Text Ġlies Ġin Ġthe Ġagg l ut in ative Ġnature Ġof Ġthe ĠArabic Ġlanguage . ĠWe Ġtrained Ġa Ġnew Ġfast Text Ġtext Ġclassification Ġmodel ĠB IB REF 30 Ġon Ġour Ġdataset Ġwith Ġvectors Ġof Ġ40 Ġdimensions , Ġ0 . 5 Ġlearning Ġrate , Ġ2 $ -$ 10 Ġcharacter Ġn - gram s Ġas Ġfeatures , Ġfor Ġ30 Ġepoch s . ĠThese Ġhyper - param eters Ġwere Ġtuned Ġusing Ġa Ġ5 - fold Ġcross - valid ated Ġgrid - search . </s> Exper iments Ġ: :: ĠRepresent ations Ġ: :: ĠDeep ĠContext ual ized ĠEmb edd ings </s> We Ġalso Ġexperimented Ġwith Ġpre - trained Ġcontextual ized Ġembed d ings Ġwith Ġfine - tun ing Ġfor Ġdown - stream Ġtasks . ĠRecently , Ġdeep Ġcontextual ized Ġlanguage Ġmodels Ġsuch Ġas ĠB ERT Ġ( B id irection al ĠEnc oder ĠRepresent ations Ġfrom ĠTransformers ) ĠB IB REF 31 , ĠU ML F IT ĠB IB REF 32 , Ġand ĠOpen AI ĠG PT ĠB IB REF 33 , Ġto Ġname Ġbut Ġa Ġfew , Ġhave Ġachieved Ġground - breaking Ġresults Ġin Ġmany ĠN LP Ġclassification Ġand Ġlanguage Ġunderstanding Ġtasks . ĠIn Ġthis Ġpaper , Ġwe Ġused ĠB ERT base - mult ilingual Ġ( here after Ġas Ġsimply ĠB ERT ) Ġfine - tun ing Ġmethod Ġto Ġclassify ĠArabic Ġoffensive Ġlanguage Ġon ĠTwitter Ġas Ġit Ġeliminates Ġthe Ġneed Ġof Ġheavily Ġengineered Ġtask - specific Ġarchitectures . ĠAlthough ĠRob ust ly ĠOptim ized ĠB ERT Ġ( Ro BER Ta ) Ġembed d ings Ġperform Ġbetter Ġthan Ġ( BER T large ) Ġon ĠGL UE ĠB IB REF 34 , ĠR ACE ĠB IB REF 35 , Ġand ĠS Qu AD ĠB IB REF 36 Ġtasks , Ġpre - trained Ġmult ilingual ĠRo BER Ta Ġmodels Ġare Ġnot Ġavailable . ĠB ERT Ġis Ġpre - trained Ġon ĠWikipedia Ġtext Ġfrom Ġ104 Ġlanguages Ġand Ġcomes Ġwith Ġhundreds Ġof Ġmillions Ġof Ġparameters . ĠIt Ġcontains Ġan Ġenc oder Ġwith Ġ12 ĠTrans former Ġblocks , Ġhidden Ġsize Ġof Ġ768 , Ġand Ġ12 Ġself - att ention Ġheads . ĠThough Ġthe Ġtraining Ġdata Ġfor Ġthe ĠB ERT Ġembed d ings Ġdon 't Ġmatch Ġour Ġgenre , Ġthese Ġembed ding Ġuse ĠBP Ġsub - word Ġsegments . ĠFollowing Ġdev lin - 2019 - bert , Ġthe Ġclassification Ġconsists Ġof Ġintroducing Ġa Ġdense Ġlayer Ġover Ġthe Ġfinal Ġhidden Ġstate Ġ$ h $ Ġcorresponding Ġto Ġfirst Ġtoken Ġof Ġthe Ġsequence , Ġ[ CL S ], Ġadding Ġa Ġsoft max Ġactivation Ġon Ġthe Ġtop Ġof ĠB ERT Ġto Ġpredict Ġthe Ġprobability Ġof Ġthe Ġ$ l $ Ġlabel : </s> where Ġ$ W $ Ġis Ġthe Ġtask - specific Ġweight Ġmatrix . ĠDuring Ġfine - tun ing , Ġall ĠB ERT Ġparameters Ġtogether Ġwith Ġ$ W $ Ġare Ġoptimized Ġend - to - end Ġto Ġmaximize Ġthe Ġlog - pro b ability Ġof Ġthe Ġcorrect Ġlabels . </s> Exper iments Ġ: :: ĠClassification ĠModels </s> We Ġexplored Ġdifferent Ġclass ifiers . ĠWhen Ġusing Ġlex ical Ġfeatures Ġand Ġpre - trained Ġstatic Ġembed d ings , Ġwe Ġprimarily Ġused Ġan ĠS VM Ġclass ifier Ġwith Ġa Ġradial Ġbasis Ġfunction Ġkernel . ĠOnly Ġwhen Ġusing Ġthe ĠMaz aj ak Ġembed d ings , Ġwe Ġexperimented Ġwith Ġother Ġclass ifiers Ġsuch Ġas ĠAda Boost Ġand ĠLog istic Ġregression . ĠWe Ġdid Ġso Ġto Ġshow Ġthat Ġthe ĠS VM Ġclass ifier Ġwas Ġindeed Ġthe Ġbest Ġof Ġthe Ġbunch , Ġand Ġwe Ġpicked Ġthe ĠMaz aj ak Ġembed d ings Ġbecause Ġthey Ġyielded Ġthe Ġbest Ġresults Ġamong Ġall Ġstatic Ġembed d ings . ĠWe Ġused Ġthe ĠSci kit ĠLearn Ġimplementations Ġof Ġall Ġthe Ġclass ifiers Ġsuch Ġas Ġlib s vm Ġfor Ġthe ĠS VM Ġclass ifier . ĠWe Ġalso Ġexperimented Ġwith Ġfast Text , Ġwhich Ġtrained Ġembed d ings Ġon Ġour Ġdata . ĠWhen Ġusing Ġcontextual ized Ġembed d ings , Ġwe Ġfine - tun ed ĠB ERT Ġby Ġadding Ġa Ġfully - connected Ġdense Ġlayer Ġfollowed Ġby Ġa Ġsoft max Ġclass ifier , Ġminimizing Ġthe Ġbinary Ġcross - ent ropy Ġloss Ġfunction Ġfor Ġthe Ġtraining Ġdata . ĠFor Ġall Ġexperiments , Ġwe Ġused Ġthe ĠPy Tor ch Ġimplementation Ġby ĠHug ging Face Ġas Ġit Ġprovides Ġpre - trained Ġweights Ġand Ġvocabulary . </s> Exper iments Ġ: :: ĠEvaluation </s> For Ġall Ġof Ġour Ġexperiments , Ġwe Ġused Ġ5 - fold Ġcross Ġvalidation Ġwith Ġidentical Ġfolds Ġfor Ġall Ġexperiments . ĠTable ĠT AB REF 27 Ġreports Ġon Ġthe Ġresults Ġof Ġusing Ġlex ical Ġfeatures , Ġstatic Ġpre - trained Ġembed d ings Ġwith Ġan ĠS VM Ġclass ifier , Ġembed d ings Ġtrained Ġon Ġour Ġdata Ġwith Ġfast Text Ġclass ifier , Ġand ĠB ERT Ġover Ġa Ġdense Ġlayer Ġwith Ġsoft max Ġactivation . ĠAs Ġthe Ġresults Ġshow , Ġusing ĠMaz aj ak / S VM Ġyielded Ġthe Ġbest Ġresults Ġoverall Ġwith Ġlarge Ġimprovements Ġin Ġprecision Ġover Ġusing ĠB ERT . ĠWe Ġsuspect Ġthat Ġthe ĠMaz aj ak / S VM Ġcombination Ġperformed Ġbetter Ġthan Ġthe ĠB ERT Ġsetup Ġdue Ġto Ġthe Ġfact Ġthat Ġthe ĠMaz aj ak Ġembed d ings , Ġthough Ġstatic , Ġwere Ġtrained Ġon Ġin - domain Ġdata , Ġas Ġopposed Ġto ĠB ERT . ĠPerhaps Ġif ĠB ERT Ġembed d ings Ġwere Ġtrained Ġon Ġtweets , Ġthey Ġmight Ġhave Ġoutper formed Ġall Ġother Ġsetups . ĠFor Ġcomple teness , Ġwe Ġcompared Ġ7 Ġother Ġclass ifiers Ġwith ĠS VM Ġusing ĠMaz aj ak Ġembed d ings . ĠAs Ġresults Ġin ĠTable ĠT AB REF 28 Ġshow , Ġusing ĠS VM Ġyielded Ġthe Ġbest Ġresults . </s> Exper iments Ġ: :: ĠError ĠAnalysis </s> We Ġinspected Ġthe Ġtweets Ġof Ġone Ġfold Ġthat Ġwere Ġmis classified Ġby Ġthe ĠMaz aj ak / S VM Ġmodel Ġ( 36 Ġfalse Ġpositives / 121 Ġfalse Ġnegatives ) Ġto Ġdetermine Ġthe Ġmost Ġcommon Ġerrors . </s> Exper iments Ġ: :: ĠError ĠAnalysis Ġ: :: ĠFalse ĠP osit ives </s> had Ġfour Ġmain Ġtypes : </s> [ left margin = * ] </s> G lo ating : Ġex . ĠÙ Ĭ Ø§ ĠÙ ĩ Ø¨ ÙĬ Ø¯ Ùĩ > Ġ( âĢ ľ y A Ġh by dp âĢ Ŀ Ġ- ĠâĢ ľ O Ġyou Ġdelusional âĢ Ŀ ) Ġreferring Ġto Ġfans Ġof Ġrival Ġsports Ġteam Ġfor Ġthinking Ġthey Ġcould Ġwin . </s> Qu oting : Ġex . Ġ ÙĦ Ùħ Ø§ ĠØ Ń Ø¯ ĠÙ Ĭ Ø³ Ø¨ ĠÙĪ ÙĬ Ù Ĥ ÙĪ ÙĦ ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ l mA ĠH d Ġy sb Ġw y q wl Ġy A Ġk lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ when Ġsomeone Ġsw ears Ġand Ġsays : ĠO Ġdog âĢ Ŀ ). </s> Id i oms : Ġex . ĠÙ Ĭ Ø§ ĠÙ ģ Ø§Ø · Ø± ĠØ ± Ùħ Ø ¶ Ø§ ÙĨ ĠÙ Ĭ Ø§ ĠØ ® Ø§Ø ³ Ø± ĠØ ¯ ÙĬ ÙĨ Ù ĥ > Ġ( âĢ ľ y A Ġf AT r Ġrm D An Ġy A Ġx As r Ġdyn k âĢ Ŀ ĠâĢĵ ĠâĢ ľ o Ġyou Ġwho Ġdoes Ġnot Ġfast ĠRamadan , Ġyou Ġwho Ġhave Ġlost Ġyour Ġreligion âĢ Ŀ ), Ġwhich Ġis Ġa Ġcoll oqu ial Ġid iom . </s> Impl icit ĠS arc asm : Ġex . ĠÙ Ĭ Ø§ ĠØ ® Ø§ ÙĬ ÙĨ Ġ Ø§ ÙĨ Øª ĠØ ¹ Ø§ ÙĬ Ø ² ĠØ ª Ø ´ Ù ĥ Ù ĥ > </s> Ù ģ ÙĬ ĠØ Ń Ø¨ ĠØ§ÙĦ Ø ´ Ø¹ Ø¨ Ġ ÙĦ ÙĦ Ø± ÙĬ Ø³ > Ġ( âĢ ľ y A ĠK A yn ĠAnt ĠEA w z Ġt $ kk Ġf y ĠH b ĠAl $ E b Ġll rys âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġtraitor , Ġ( you ) Ġwant Ġto Ġquestion Ġthe Ġlove Ġof Ġthe Ġpeople Ġto Ġthe Ġpresident ĠâĢ Ŀ ) Ġwhere Ġthe Ġauthor Ġis Ġmocking Ġthe Ġpresident 's Ġpopularity . </s> Exper iments Ġ: :: ĠError ĠAnalysis Ġ: :: ĠFalse ĠNeg atives </s> had Ġtwo Ġtypes : </s> [ left margin = * ] </s> M ixture Ġof Ġoff ens iveness Ġand Ġadmiration : Ġex . Ġcalling Ġa Ġgirl Ġa Ġpuppy ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ ÙĪ Ø¨ Ø© > Ġ( âĢ ľ y A Ġk lb wb p âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġpuppy âĢ Ŀ ) Ġin Ġa Ġflirt at ious Ġmanner . </s> Impl icit Ġoff ens iveness : Ġex . Ġcalling Ġfor Ġcure Ġwhile Ġimplying Ġsanity Ġin : ĠÙĪ Øª Ø ´ Ù ģ ÙĬ ĠØ Ń Ù ĥ Ø§ Ùħ ĠÙ Ĥ Ø · Ø± ĠÙħ ÙĨ ĠØ§ÙĦ Ùħ Ø± Ø ¶ > Ġ( âĢ ľ wt $ fy ĠH k Am Ġq Tr Ġm n ĠAlm r D âĢ Ŀ ĠâĢĵ ĠâĢ ľ and Ġcure ĠQatar Ġrulers Ġfrom Ġillness âĢ Ŀ ). </s> Many Ġerrors Ġstem Ġfrom Ġheavy Ġuse Ġof Ġdialect al ĠArabic Ġas Ġwell Ġas Ġambiguity . ĠSince ĠB ERT Ġwas Ġtrained Ġon ĠWikipedia Ġ( M SA ) Ġand ĠGoogle Ġbooks , Ġthe Ġmodel Ġfailed Ġto Ġclassify Ġtweets Ġwith Ġdialect al Ġcues . ĠConversely , ĠMaz aj ak / S VM Ġis Ġmore Ġbiased Ġtowards Ġdialect s , Ġoften Ġfailing Ġto Ġclassify ĠM SA Ġtweets . </s> Conclusion Ġand ĠFuture ĠWork </s> In Ġthis Ġpaper Ġwe Ġpresented Ġa Ġsystematic Ġmethod Ġfor Ġbuilding Ġan ĠArabic Ġoffensive Ġlanguage Ġtweet Ġdataset Ġthat Ġdoes Ġnot Ġfavor Ġspecific Ġdialect s , Ġtopics , Ġor Ġgenres . ĠWe Ġdeveloped Ġdetailed Ġguidelines Ġfor Ġtagging Ġthe Ġtweets Ġas Ġclean Ġor Ġoffensive , Ġincluding Ġspecial Ġtags Ġfor Ġvulgar Ġtweets Ġand Ġhate Ġspeech . ĠWe Ġtagged Ġ10 , 000 Ġtweets , Ġwhich Ġwe Ġplan Ġto Ġrelease Ġpublicly Ġand Ġwould Ġconstitute Ġthe Ġlargest Ġavailable ĠArabic Ġoffensive Ġlanguage Ġdataset . ĠWe Ġcharacterized Ġthe Ġoffensive Ġtweets Ġin Ġthe Ġdataset Ġto Ġdetermine Ġthe Ġtopics Ġthat Ġillicit Ġsuch Ġlanguage , Ġthe Ġdialect s Ġthat Ġare Ġmost Ġoften Ġused , Ġthe Ġcommon Ġmodes Ġof Ġoff ens iveness , Ġand Ġthe Ġgender Ġdistribution Ġof Ġtheir Ġauthors . ĠWe Ġperformed Ġthis Ġbreakdown Ġfor Ġoffensive Ġtweets Ġin Ġgeneral Ġand Ġfor Ġvulgar Ġand Ġhate Ġspeech Ġtweets Ġseparately . ĠWe Ġbelieve Ġthat Ġthis Ġis Ġthe Ġfirst Ġdetailed Ġanalysis Ġof Ġits Ġkind . ĠLastly , Ġwe Ġconducted Ġa Ġlarge Ġbattery Ġof Ġexperiments Ġon Ġthe Ġdataset , Ġusing Ġcross - valid ation , Ġto Ġestablish Ġa Ġstrong Ġsystem Ġfor ĠArabic Ġoffensive Ġlanguage Ġdetection . ĠWe Ġshowed Ġthat Ġusing Ġstatic Ġembed d ings Ġproduced Ġa Ġcompetitive Ġresults Ġon Ġthe Ġdataset . </s> For Ġfuture Ġwork , Ġwe Ġplan Ġto Ġpursue Ġseveral Ġdirections . ĠFirst , Ġwe Ġwant Ġexplore Ġtarget Ġspecific Ġoffensive Ġlanguage , Ġwhere Ġattacks Ġagainst Ġan Ġentity Ġor Ġa Ġgroup Ġmay Ġemploy Ġcertain Ġexpressions Ġthat Ġare Ġonly Ġoffensive Ġwithin Ġthe Ġcontext Ġof Ġthat Ġtarget Ġand Ġcompletely Ġinnocuous Ġotherwise . ĠSecond , Ġwe Ġplan Ġto Ġexamine Ġthe Ġeffectiveness Ġof Ġcross Ġdialect al Ġand Ġcross Ġling ual Ġlearning Ġof Ġoffensive Ġlanguage .
Introduction Disclaimer: Due to the nature of the paper, some examples contain highly offensive language and hate speech. They don't reflect the views of the authors in any way, and the point of the paper is to help fight such speech. Much recent interest has focused on the detection of offensive language and hate speech in online social media. Such language is often associated with undesirable online behaviors such as trolling, cyberbullying, online extremism, political polarization, and propaganda. Thus, offensive language detection is instrumental for a variety of application such as: quantifying polarization BIBREF0, BIBREF1, trolls and propaganda account detection BIBREF2, detecting the likelihood of hate crimes BIBREF3; and predicting conflict BIBREF4. In this paper, we describe our methodology for building a large dataset of Arabic offensive tweets. Given that roughly 1-2% of all Arabic tweets are offensive BIBREF5, targeted annotation is essential for efficiently building a large dataset. Since our methodology does not use a seed list of offensive words, it is not biased by topic, target, or dialect. Using our methodology, we tagged 10,000 Arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. Further, we labeled tweets as vulgar or hate speech. To date, this is the largest available dataset, which we plan to make publicly available along with annotation guidelines. We use this dataset to characterize Arabic offensive language to ascertain the topics, dialects, and users' gender that are most associated with the use of offensive language. Though we suspect that there are common features that span different languages and cultures, some characteristics of Arabic offensive language is language and culture specific. Thus, we conduct a thorough analysis of how Arabic users use offensive language. Next, we use the dataset to train strong Arabic offensive language classifiers using state-of-the-art representations and classification techniques. Specifically, we experiment with static and contextualized embeddings for representation along with a variety of classifiers such as a deep neural network classifier and Support Vector Machine (SVM). The contributions of this paper are as follows: We built the largest Arabic offensive language dataset to date that includes special tags for vulgar language and hate speech. We describe the methodology for building it along with annotation guidelines. We performed thorough analysis of the dataset and described the peculiarities of Arabic offensive language. We experimented with Support Vector Machine classifiers on character and word ngrams classification techniques to provide strong results on Arabic offensive language classification. Related Work Many recent papers have focused on the detection of offensive language, including hate speech BIBREF6, BIBREF7, BIBREF8, BIBREF9, BIBREF10, BIBREF11, BIBREF12, BIBREF13. Offensive language can be categorized as: Vulgar, which include explicit and rude sexual references, Pornographic, and Hateful, which includes offensive remarks concerning people’s race, religion, country, etc. BIBREF14. Prior works have concentrated on building annotated corpora and training classification models. Concerning corpora, hatespeechdata.com attempts to maintain an updated list of hate speech corpora for multiple languages including Arabic and English. Further, SemEval 2019 ran an evaluation task targeted at detecting offensive language, which focused exclusively on English BIBREF15. As for classification models, most studies used supervised classification at either word level BIBREF10, character sequence level BIBREF11, and word embeddings BIBREF9. The studies used different classification techniques including using Naïve Bayes BIBREF10, SVM BIBREF11, and deep learning BIBREF6, BIBREF7, BIBREF12 classification. The accuracy of the aforementioned system ranged between 76% and 90%. Earlier work looked at the use of sentiment words as features as well as contextual features BIBREF13. The work on Arabic offensive language detection is relatively nascent BIBREF16, BIBREF17, BIBREF18, BIBREF19, BIBREF5. Mubarak et al. mubarak2017abusive suggested that certain users are more likely to use offensive languages than others, and they used this insight to build a list of offensive Arabic words and they constructed a labeled set of 1,100 tweets. Abozinadah et al. abozinadah2017detecting used supervised classification based on a variety of features including user profile features, textual features, and network features. They reported an accuracy of nearly 90%. Alakrot et al. alakrot2018towards used supervised classification based on word unigrams and n-grams to detect offensive language in YouTube comments. They improved classification with stemming and achieved a precision of 88%. Albadi et al. albadi2018they focused on detecting religious hate speech using a recurrent neural network. Further; Schmidt and Wiegand schmidt-wiegand-2017-survey surveyed major works on hate speech detection; Fortuna and Nunes Fortuna2018Survey provided a comprehensive survey for techniques and works done in the area between 2004 and 2017. Arabic is a morphologically rich language with a standard variety, namely Modern Standard Arabic (MSA) and is typically used in formal communication, and many dialectal varieties that differ from MSA in lexical selection, morphology, and syntactic structures. For MSA, words are typically derived from a set of thousands of roots by fitting a root into a stem template and the resulting stem may accept a variety of prefixes and suffixes such as coordinating conjunctions and pronouns. Though word segmentation (or stemming) is quite accurate for MSA BIBREF20, with accuracy approaching 99%, dialectal segmentation is not sufficiently reliable, with accuracy ranging between 91-95% for different dialects BIBREF21. Since dialectal Arabic is ubiquitous in Arabic tweets and many tweets have creative spellings of words, recent work on Arabic offensive language detection used character-level models BIBREF5. Data Collection ::: Collecting Arabic Offensive Tweets Our target was to build a large Arabic offensive language dataset that is representative of their appearance on Twitter and is hopefully not biased to specific dialects, topics, or targets. One of the main challenges is that offensive tweets constitute a very small portion of overall tweets. To quantify their proportion, we took 3 random samples of tweets from different days, with each sample composed of 1,000 tweets, and we found that between 1% and 2% of them were in fact offensive (including pornographic advertisement). This percentage is consistent with previously reported percentages BIBREF19. Thus, annotating random tweets is grossly inefficient. One way to overcome this problem is to use a seed list of offensive words to filter tweets. However, doing so is problematic as it would skew the dataset to particular types of offensive language or to specific dialects. Offensiveness is often dialect and country specific. After inspecting many tweets, we observed that many offensive tweets have the vocative particle يا> (“yA” – meaning “O”), which is mainly used in directing the speech to a specific person or group. The ratio of offensive tweets increases to 5% if each tweet contains one vocative particle and to 19% if has at least two vocative particles. Users often repeat this particle for emphasis, as in: يا أمي يا حنونة> (“yA Amy yA Hnwnp” – O my mother, O kind one), which is endearing and non-offensive, and يا كلب يا قذر> (“yA klb yA q*r” – “O dog, O dirty one”), which is offensive. We decided to use this pattern to increase our chances of finding offensive tweets. One of the main advantages of the pattern يا ... يا> (“yA ... yA”) is that it is not associated with any specific topic or genre, and it appears in all Arabic dialects. Though the use of offensive language does not necessitate the appearance of the vocative particle, the particle does not favor any specific offensive expressions and greatly improves our chances of finding offensive tweets. It is clear, the dataset is more biased toward positive class. Using the dataset for real-life application may require de-biasing it by boosting negative class or random sampling additional data from Twitter BIBREF22.Using the Twitter API, we collected 660k Arabic tweets having this pattern between April 15, 2019 and May 6, 2019. To increase diversity, we sorted the word sequences between the vocative particles and took the most frequent 10,000 unique sequences. For each word sequence, we took a random tweet containing each sequence. Then we annotated those tweets, ending up with 1,915 offensive tweets which represent roughly 19% of all tweets. Each tweet was labeled as: offensive, which could additionally be labeled as vulgar and/or hate speech, or Clean. We describe in greater detail our annotation guidelines, which we made sure that they are compatible with the OffensEval2019 annotation guidelines BIBREF15. For example, if a tweet has insults or threats targeting a group based on their nationality, ethnicity, gender, political affiliation, religious belief, or other common characteristics, this is considered as hate speech BIBREF15. It is worth mentioning that we also considered insulting groups based on their sport affiliation as a form of hate speech. In most Arab countries, being a fan of a particularly sporting club is considered as part of the personality and ideology which rarely changes over time (similar to political affiliation). Many incidents of violence have occurred among fans of rival clubs. Data Collection ::: Annotating Tweets We developed the annotation guidelines jointly with an experienced annotator, who is a native Arabic speaker with a good knowledge of various Arabic dialects. We made sure that our guidelines were compatible with those of OffensEval2019. The annotator carried out all annotation. Tweets were given one or more of the following four labels: offensive, vulgar, hate speech, or clean. Since the offensive label covers both vulgar and hate speech and vulgarity and hate speech are not mutually exclusive, a tweet can be just offensive or offensive and vulgar and/or hate speech. The annotation adhered to the following guidelines: Data Collection ::: Annotating Tweets ::: OFFENSIVE (OFF): Offensive tweets contain explicit or implicit insults or attacks against other people, or inappropriate language, such as: Direct threats or incitement, ex: احرقوا> مقرات المعارضة> (“AHrqwA mqrAt AlmEArDp” – “burn the headquarters of the opposition”) and هذا المنافق يجب قتله> (“h*A AlmnAfq yjb qtlh” – “this hypocrite needs to be killed”). Insults and expressions of contempt, which include: Animal analogy, ex: يا كلب> (“yA klb” – “O dog”) and كل تبن> (“kl tbn” – “eat hay”).; Insult to family members, ex: يا روح أمك> (“yA rwH Amk” – “O mother's soul”); Sexually-related insults, ex: يا ديوث> (“yA dywv” – “O person without envy”); Damnation, ex: الله يلعنك> (“Allh ylEnk” – “may Allah/God curse you”); and Attacks on morals and ethics, ex: يا كاذب> (“yA kA*b” – “O liar”) Data Collection ::: Annotating Tweets ::: VULGAR (VLG): Vulgar tweets are a subset of offensive tweets and contain profanity, such as mentions of private parts or sexual-related acts or references. Data Collection ::: Annotating Tweets ::: HATE SPEECH (HS): Hate speech tweets, a subset of offensive tweets containing offensive language targeting group based on common characteristics such as: Race, ex: يا زنجي> (“yA znjy” – “O negro”); Ethnicity, ex. الفرس الأنجاس> (“Alfrs AlAnjAs” – “Impure Persians”); Group or party, ex: أبوك شيوعي> (“Abwk $ywEy” – “your father is communist”); and Religion, ex: دينك القذر> (“dynk Alq*r” – “your filthy religion”). Data Collection ::: Annotating Tweets ::: CLEAN (CLN): Clean tweets do not contain vulgar or offensive language. We noticed that some tweets have some offensive words, but the whole tweet should not be considered as offensive due to the intention of users. This suggests that normal string match without considering contexts will fail in some cases. Examples of such ambiguous cases include: Humor, ex: يا عدوة الفرحة ههه> (“yA Edwp AlfrHp hhh” – “O enemy of happiness hahaha”); Advice, ex: لا تقل لصاحبك يا خنزير> (“lA tql lSAHbk yA xnzyr” – “don't say to your friend: You are a pig”); Condition, ex: إذا عارضتهم يقولون يا عميل> (“A*A EArDthm yqwlwn yA Emyl” – “if you disagree with them they will say: You are an agent”); Condemnation, ex: لماذا نسب بقول: يا بقرة؟> (“lmA*A nsb bqwl: yA bqrp?” – “Why do we insult others by saying: O cow?”); Self offense, ex: تعبت من لساني القذر> (“tEbt mn lsAny Alq*r” – “I am tired of my dirty tongue”); Non-human target, ex: يا بنت المجنونة يا كورة> (“yA bnt Almjnwnp yA kwrp” – “O daughter of the crazy one O football”); and Quotation from a movies or a story, ex: تاني يا زكي! تاني يا فاشل> (“tAny yA zky! tAny yA fA$l” – “again O Zaky! again O loser”). For other ambiguous cases, the annotator searched Twitter to find how actual users used expressions. Table TABREF11 shows the distribution of the annotated tweets. There are 1,915 offensive tweets, including 225 vulgar tweet and 506 hate speech tweets, and 8,085 clean tweets. To validate the quality of annotation, a random sample of 100 tweets from the data, containing 50 offensive and 50 clean tweets, was given to additional three annotators. We calculated the Inter-Annotator Agreement between the annotators using Fleiss’s Kappa coefficient BIBREF23. The Kappa score was 0.92 indicating high quality annotation and agreement. Data Collection ::: Statistics and User Demographics Given the annotated tweets, we wanted to ascertain the distribution of: types of offensive language, genres where it is used, the dialects used, and the gender of users using such language. Figure FIGREF13 shows the distribution of topics associated with offensive tweets. As the figure shows, sports and politics are most dominant for offensive language including vulgar and hate speech. As for dialect, we looked at MSA and four major dialects, namely Egyptian (EGY), Leventine (LEV), Maghrebi (MGR), and Gulf (GLF). Figure FIGREF14 shows that 71% of vulgar tweets were written in EGY followed by GLF, which accounted for 13% of vulgar tweets. MSA was not used in any of the vulgar tweets. As for offensive tweets in general, EGY and GLF were used in 36% and 35% of the offensive tweets respectively. Unlike the case of vulgar language where MSA was non-existent, 15% of the offensive tweets were in fact written in MSA. For hate speech, GLF and EGY were again dominant and MSA consistuted 21% of the tweets. This is consistent with findings for other languages such as English and Italian where vulgar language was more frequently associated with colloquial language BIBREF24, BIBREF25. Regarding the gender, Figure FIGREF15 shows that the vast majority of offensive tweets, including vulgar and hate speech, were authored by males. Female Twitter users accounted for 14% of offensive tweets in general and 6% and 9% of vulgar and hate speech respectively. Figure FIGREF16 shows a detailed categorization of hate speech types, where the top three include insulting groups based on their political ideology, origin, and sport affiliation. Religious hate speech appeared in only 15% of all hate speech tweets. Next, we analyzed all tweets labeled as offensive to better understand how Arabic speakers use offensive language. Here is a breakdown of usage: Direct name calling: The most frequent attack is to call a person an animal name, and the most used animals were كلب> (“klb” – “dog”), حمار> (“HmAr” – “donkey”), and بهيم> (“bhym” – “beast”). The second most common was insulting mental abilities using words such as غبي> (“gby” – “stupid”) and عبيط> (“EbyT” –“idiot”). Some culture-specific differences should be considered. Not all animal names are used as insults. For example, animals such as أسد> (“Asd” – “lion”), صقر> (“Sqr” – “falcon”), and غزال> (“gzAl” – “gazelle”) are typically used for praise. For other insults, people use: some bird names such as دجاجة> (“djAjp” – “chicken”), بومة> (“bwmp” – “owl”), and غراب> (“grAb” – “crow”); insects such as ذبابة> (“*bAbp” – “fly”), صرصور> (“SrSwr” – “cockroach”), and حشرة> (“H$rp” – “insect”); microorganisms such as جرثومة> (“jrvwmp” – “microbe”) and طحالب> (“THAlb” – “algae”); inanimate objects such as جزمة> (“jzmp” – “shoes”) and سطل> (“sTl” – “bucket”) among other usages. Simile and metaphor: Users use simile and metaphor were they would compare a person to: an animal as in زي الثور> (“zy Alvwr” – “like a bull”), سمعني نهيقك> (“smEny nhyqk” – “let me hear your braying”), and هز ديلك> (“hz dylk” – “wag your tail”); a person with mental or physical disability such as منغولي> (“mngwly” – “Mongolian (down-syndrome)”), معوق> (“mEwq” – “disabled”), and قزم> (“qzm” – “dwarf”); and to the opposite gender such as جيش نوال> (“jy$ nwAl” – “Nawal's army (Nawal is female name)”) and نادي زيزي> (“nAdy zyzy” – “Zizi's club (Zizi is a female pet name)”). Indirect speech: This type of offensive language includes: sarcasm such as أذكى إخواتك> (“A*kY AxwAtk” – “smartest one of your siblings”) and فيلسوف الحمير> (“fylswf AlHmyr” – “the donkeys' philosopher”); questions such as ايه كل الغباء ده> (“Ayh kl AlgbA dh” – “what is all this stupidity”); and indirect speech such as النقاش مع البهايم غير مثمر> (“AlnqA$ mE AlbhAym gyr mvmr” – “no use talking to cattle”). Wishing Evil: This entails wishing death or major harm to befall someone such as ربنا ياخدك> (“rbnA yAxdk” – “May God take (kill) you”), الله يلعنك> (“Allh ylEnk” – “may Allah/God curse you”), and روح في داهية> (“rwH fy dAhyp” – equivalent to “go to hell”). Name alteration: One common way to insult others is to change a letter or two in their names to produce new offensive words that rhyme with the original names. Some examples of such include changing الجزيرة> (“Aljzyrp” – “Aljazeera (channel)”) to الخنزيرة> (“Alxnzyrp” – “the pig”) and خلفان> (“xlfAn” – “Khalfan (person name)”) to خرفان> (“xrfAn” – “crazed”). Societal stratification: Some insults are associated with: certain jobs such as بواب> (“bwAb” – “doorman”) or خادم> (“xAdm” – “servant”); and specific societal components such بدوي> (“bdwy” – “bedouin”) and فلاح> (“flAH” – “farmer”). Immoral behavior: These insults are associated with negative moral traits or behaviors such as حقير> (“Hqyr” – “vile”), خاين> (“xAyn” – “traitor”), and منافق> (“mnAfq” – “hypocrite”). Sexually related: They include expressions such as خول> (“xwl” – “gay”), وسخة> (“wsxp” – “prostitute”), and عرص> (“ErS” – “pimp”). Figure FIGREF17 shows top words with the highest valance score for individual words in the offensive tweets. Larger fonts are used to highlight words with highest score and align as well with the categories mentioned ahead in the breakdown for the offensive languages. We modified the valence score described by BIBREF1 conover2011political to magnify its value based on frequency of occurrence. The score is computed as follows: $V(I) = 2 \frac{ \frac{tf(I, C_off)}{total(C_off)}}{\frac{tf(I, C_off)}{total(C_off)} + \frac{tf(I, C_cln)}{total(C_cln)} } - 1$ where $tf(I, C_i) = \sum _{a \in I \bigcap C_i} [ln(Cnt(a, C_i)) + 1]$ $total(C_i) = \sum _{I} tf(I, C_i)$ $Cnt(a, C_i)$ is the number of times word $a$ was used in offensive or clean tweets tweets $C_i$. In essence, we are replacing term frequencies with the natural log of the term frequencies. Experiments We conducted an extensive battery of experiments on the dataset to establish strong Arabic offensive language classification results. Though the offensive tweets have finer-grained labels where offensive tweet could also be vulgar or constitute hate speech, we conducted coarser-grained classification to determine if a tweet was offensive or not. For classification, we experimented with several tweet representation and classification models. For tweet representations, we used: the count of positive and negative terms, based on a polarity lexicon; static embeddings, namely fastText and Skip-Gram; and deep contextual embeddings, namely BERTbase-multilingual. Experiments ::: Data Pre-processing We performed several text pre-processing steps. First, we tokenized the text using the Farasa Arabic NLP toolkit BIBREF20. Second, we removed URLs, numbers, and all tweet specific tokens, namely mentions, retweets, and hashtags as they are not part of the language semantic structure, and therefore, not usable in pre-trained embeddings. Third, we performed basic Arabic letter normalization, namely variants of the letter alef to bare alef, ta marbouta to ha, and alef maqsoura to ya. We also separated words that are commonly incorrectly attached such as ياكلب> (“yAklb” – “O dog”), is split to يا كلب> (“yA klb”). Lastly, we normalized letter repetitions to allow for a maximum of 2 repeated letters. For example, the token ههههه> (“hhhhh” – “ha ha ha ..”) is normalized to هه> (“hh”). We also removed the Arabic short-diacritics (tashkeel) and word elongation (kashida). Experiments ::: Representations ::: Lexical Features Since offensive words typically have a negative polarity, we wanted to test the effectiveness of using a polarity lexicon in detecting offensive tweets. For the lexicon, we used NileULex BIBREF26, which is an Arabic polarity lexicon containing 3,279 MSA and 2,674 Egyptian terms, out of which 4,256 are negative and 1,697 are positive. We used the counts of terms with positive polarity and terms with negative polarity in tweets as features. Experiments ::: Representations ::: Static Embeddings We experimented with various static embeddings that were pre-trained on different corpora with different vector dimensionality. We compared pre-trained embeddings to embeddings that were trained on our dataset. For pre-trained embeddings, we used: fastText Egyptian Arabic pre-trained embeddings BIBREF27 with vector dimensionality of 300; AraVec skip-gram embeddings BIBREF28, trained on 66.9M Arabic tweets with 100-dimensional vectors; and Mazajak skip-gram embeddings BIBREF29, trained on 250M Arabic tweets with 300-dimensional vectors. Sentence embeddings were calculated by taking the mean of the embeddings of their tokens. The importance of testing a character level n-gram model like fastText lies in the agglutinative nature of the Arabic language. We trained a new fastText text classification model BIBREF30 on our dataset with vectors of 40 dimensions, 0.5 learning rate, 2$-$10 character n-grams as features, for 30 epochs. These hyper-parameters were tuned using a 5-fold cross-validated grid-search. Experiments ::: Representations ::: Deep Contextualized Embeddings We also experimented with pre-trained contextualized embeddings with fine-tuning for down-stream tasks. Recently, deep contextualized language models such as BERT (Bidirectional Encoder Representations from Transformers) BIBREF31, UMLFIT BIBREF32, and OpenAI GPT BIBREF33, to name but a few, have achieved ground-breaking results in many NLP classification and language understanding tasks. In this paper, we used BERTbase-multilingual (hereafter as simply BERT) fine-tuning method to classify Arabic offensive language on Twitter as it eliminates the need of heavily engineered task-specific architectures. Although Robustly Optimized BERT (RoBERTa) embeddings perform better than (BERTlarge) on GLUE BIBREF34, RACE BIBREF35, and SQuAD BIBREF36 tasks, pre-trained multilingual RoBERTa models are not available. BERT is pre-trained on Wikipedia text from 104 languages and comes with hundreds of millions of parameters. It contains an encoder with 12 Transformer blocks, hidden size of 768, and 12 self-attention heads. Though the training data for the BERT embeddings don't match our genre, these embedding use BP sub-word segments. Following devlin-2019-bert, the classification consists of introducing a dense layer over the final hidden state $h$ corresponding to first token of the sequence, [CLS], adding a softmax activation on the top of BERT to predict the probability of the $l$ label: where $W$ is the task-specific weight matrix. During fine-tuning, all BERT parameters together with $W$ are optimized end-to-end to maximize the log-probability of the correct labels. Experiments ::: Classification Models We explored different classifiers. When using lexical features and pre-trained static embeddings, we primarily used an SVM classifier with a radial basis function kernel. Only when using the Mazajak embeddings, we experimented with other classifiers such as AdaBoost and Logistic regression. We did so to show that the SVM classifier was indeed the best of the bunch, and we picked the Mazajak embeddings because they yielded the best results among all static embeddings. We used the Scikit Learn implementations of all the classifiers such as libsvm for the SVM classifier. We also experimented with fastText, which trained embeddings on our data. When using contextualized embeddings, we fine-tuned BERT by adding a fully-connected dense layer followed by a softmax classifier, minimizing the binary cross-entropy loss function for the training data. For all experiments, we used the PyTorch implementation by HuggingFace as it provides pre-trained weights and vocabulary. Experiments ::: Evaluation For all of our experiments, we used 5-fold cross validation with identical folds for all experiments. Table TABREF27 reports on the results of using lexical features, static pre-trained embeddings with an SVM classifier, embeddings trained on our data with fastText classifier, and BERT over a dense layer with softmax activation. As the results show, using Mazajak/SVM yielded the best results overall with large improvements in precision over using BERT. We suspect that the Mazajak/SVM combination performed better than the BERT setup due to the fact that the Mazajak embeddings, though static, were trained on in-domain data, as opposed to BERT. Perhaps if BERT embeddings were trained on tweets, they might have outperformed all other setups. For completeness, we compared 7 other classifiers with SVM using Mazajak embeddings. As results in Table TABREF28 show, using SVM yielded the best results. Experiments ::: Error Analysis We inspected the tweets of one fold that were misclassified by the Mazajak/SVM model (36 false positives/121 false negatives) to determine the most common errors. Experiments ::: Error Analysis ::: False Positives had four main types: [leftmargin=*] Gloating: ex. يا هبيده> (“yA hbydp” - “O you delusional”) referring to fans of rival sports team for thinking they could win. Quoting: ex. لما حد يسب ويقول يا كلب> (“lmA Hd ysb wyqwl yA klb” – “when someone swears and says: O dog”). Idioms: ex. يا فاطر رمضان يا خاسر دينك> (“yA fATr rmDAn yA xAsr dynk” – “o you who does not fast Ramadan, you who have lost your religion”), which is a colloquial idiom. Implicit Sarcasm: ex. يا خاين انت عايز تشكك> في حب الشعب للريس> (“yA KAyn Ant EAwz t$kk fy Hb Al$Eb llrys” – “O traitor, (you) want to question the love of the people to the president ”) where the author is mocking the president's popularity. Experiments ::: Error Analysis ::: False Negatives had two types: [leftmargin=*] Mixture of offensiveness and admiration: ex. calling a girl a puppy يا كلبوبة> (“yA klbwbp” – “O puppy”) in a flirtatious manner. Implicit offensiveness: ex. calling for cure while implying sanity in: وتشفي حكام قطر من المرض> (“wt$fy HkAm qTr mn AlmrD” – “and cure Qatar rulers from illness”). Many errors stem from heavy use of dialectal Arabic as well as ambiguity. Since BERT was trained on Wikipedia (MSA) and Google books, the model failed to classify tweets with dialectal cues. Conversely, Mazajak/SVM is more biased towards dialects, often failing to classify MSA tweets. Conclusion and Future Work In this paper we presented a systematic method for building an Arabic offensive language tweet dataset that does not favor specific dialects, topics, or genres. We developed detailed guidelines for tagging the tweets as clean or offensive, including special tags for vulgar tweets and hate speech. We tagged 10,000 tweets, which we plan to release publicly and would constitute the largest available Arabic offensive language dataset. We characterized the offensive tweets in the dataset to determine the topics that illicit such language, the dialects that are most often used, the common modes of offensiveness, and the gender distribution of their authors. We performed this breakdown for offensive tweets in general and for vulgar and hate speech tweets separately. We believe that this is the first detailed analysis of its kind. Lastly, we conducted a large battery of experiments on the dataset, using cross-validation, to establish a strong system for Arabic offensive language detection. We showed that using static embeddings produced a competitive results on the dataset. For future work, we plan to pursue several directions. First, we want explore target specific offensive language, where attacks against an entity or a group may employ certain expressions that are only offensive within the context of that target and completely innocuous otherwise. Second, we plan to examine the effectiveness of cross dialectal and cross lingual learning of offensive language.
How many tweets are in the dataset?
b3de9357c569fb1454be8f2ac5fcecaea295b967
How Ġmany Ġtweets Ġare Ġin Ġthe Ġdataset ?
10 , 000 ĠArabic Ġtweet Ġdataset Ġ
<s> How Ġmany Ġtweets Ġare Ġin Ġthe Ġdataset ? </s> Introduction </s> Disclaimer : ĠDue Ġto Ġthe Ġnature Ġof Ġthe Ġpaper , Ġsome Ġexamples Ġcontain Ġhighly Ġoffensive Ġlanguage Ġand Ġhate Ġspeech . ĠThey Ġdon 't Ġreflect Ġthe Ġviews Ġof Ġthe Ġauthors Ġin Ġany Ġway , Ġand Ġthe Ġpoint Ġof Ġthe Ġpaper Ġis Ġto Ġhelp Ġfight Ġsuch Ġspeech . ĠMuch Ġrecent Ġinterest Ġhas Ġfocused Ġon Ġthe Ġdetection Ġof Ġoffensive Ġlanguage Ġand Ġhate Ġspeech Ġin Ġonline Ġsocial Ġmedia . ĠSuch Ġlanguage Ġis Ġoften Ġassociated Ġwith Ġundesirable Ġonline Ġbehaviors Ġsuch Ġas Ġtrolling , Ġcyber bull ying , Ġonline Ġextremism , Ġpolitical Ġpolarization , Ġand Ġpropaganda . ĠThus , Ġoffensive Ġlanguage Ġdetection Ġis Ġinstrumental Ġfor Ġa Ġvariety Ġof Ġapplication Ġsuch Ġas : Ġquant ifying Ġpolarization ĠB IB REF 0 , ĠB IB REF 1 , Ġtrolls Ġand Ġpropaganda Ġaccount Ġdetection ĠB IB REF 2 , Ġdetecting Ġthe Ġlikelihood Ġof Ġhate Ġcrimes ĠB IB REF 3 ; Ġand Ġpredicting Ġconflict ĠB IB REF 4 . ĠIn Ġthis Ġpaper , Ġwe Ġdescribe Ġour Ġmethodology Ġfor Ġbuilding Ġa Ġlarge Ġdataset Ġof ĠArabic Ġoffensive Ġtweets . ĠGiven Ġthat Ġroughly Ġ1 - 2 % Ġof Ġall ĠArabic Ġtweets Ġare Ġoffensive ĠB IB REF 5 , Ġtargeted Ġannotation Ġis Ġessential Ġfor Ġefficiently Ġbuilding Ġa Ġlarge Ġdataset . ĠSince Ġour Ġmethodology Ġdoes Ġnot Ġuse Ġa Ġseed Ġlist Ġof Ġoffensive Ġwords , Ġit Ġis Ġnot Ġbiased Ġby Ġtopic , Ġtarget , Ġor Ġdialect . ĠUsing Ġour Ġmethodology , Ġwe Ġtagged Ġ10 , 000 ĠArabic Ġtweet Ġdataset Ġfor Ġoff ens iveness , Ġwhere Ġoffensive Ġtweets Ġaccount Ġfor Ġroughly Ġ19 % Ġof Ġthe Ġtweets . ĠFurther , Ġwe Ġlabeled Ġtweets Ġas Ġvulgar Ġor Ġhate Ġspeech . ĠTo Ġdate , Ġthis Ġis Ġthe Ġlargest Ġavailable Ġdataset , Ġwhich Ġwe Ġplan Ġto Ġmake Ġpublicly Ġavailable Ġalong Ġwith Ġannotation Ġguidelines . ĠWe Ġuse Ġthis Ġdataset Ġto Ġcharacterize ĠArabic Ġoffensive Ġlanguage Ġto Ġascertain Ġthe Ġtopics , Ġdialect s , Ġand Ġusers ' Ġgender Ġthat Ġare Ġmost Ġassociated Ġwith Ġthe Ġuse Ġof Ġoffensive Ġlanguage . ĠThough Ġwe Ġsuspect Ġthat Ġthere Ġare Ġcommon Ġfeatures Ġthat Ġspan Ġdifferent Ġlanguages Ġand Ġcultures , Ġsome Ġcharacteristics Ġof ĠArabic Ġoffensive Ġlanguage Ġis Ġlanguage Ġand Ġculture Ġspecific . ĠThus , Ġwe Ġconduct Ġa Ġthorough Ġanalysis Ġof Ġhow ĠArabic Ġusers Ġuse Ġoffensive Ġlanguage . ĠNext , Ġwe Ġuse Ġthe Ġdataset Ġto Ġtrain Ġstrong ĠArabic Ġoffensive Ġlanguage Ġclass ifiers Ġusing Ġstate - of - the - art Ġrepresentations Ġand Ġclassification Ġtechniques . ĠSpecifically , Ġwe Ġexperiment Ġwith Ġstatic Ġand Ġcontextual ized Ġembed d ings Ġfor Ġrepresentation Ġalong Ġwith Ġa Ġvariety Ġof Ġclass ifiers Ġsuch Ġas Ġa Ġdeep Ġneural Ġnetwork Ġclass ifier Ġand ĠSupport ĠVector ĠMachine Ġ( S VM ). </s> The Ġcontributions Ġof Ġthis Ġpaper Ġare Ġas Ġfollows : </s> We Ġbuilt Ġthe Ġlargest ĠArabic Ġoffensive Ġlanguage Ġdataset Ġto Ġdate Ġthat Ġincludes Ġspecial Ġtags Ġfor Ġvulgar Ġlanguage Ġand Ġhate Ġspeech . ĠWe Ġdescribe Ġthe Ġmethodology Ġfor Ġbuilding Ġit Ġalong Ġwith Ġannotation Ġguidelines . </s> We Ġperformed Ġthorough Ġanalysis Ġof Ġthe Ġdataset Ġand Ġdescribed Ġthe Ġpeculiar ities Ġof ĠArabic Ġoffensive Ġlanguage . </s> We Ġexperimented Ġwith ĠSupport ĠVector ĠMachine Ġclass ifiers Ġon Ġcharacter Ġand Ġword Ġn gram s Ġclassification Ġtechniques Ġto Ġprovide Ġstrong Ġresults Ġon ĠArabic Ġoffensive Ġlanguage Ġclassification . </s> Related ĠWork </s> Many Ġrecent Ġpapers Ġhave Ġfocused Ġon Ġthe Ġdetection Ġof Ġoffensive Ġlanguage , Ġincluding Ġhate Ġspeech ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 8 , ĠB IB REF 9 , ĠB IB REF 10 , ĠB IB REF 11 , ĠB IB REF 12 , ĠB IB REF 13 . ĠOffensive Ġlanguage Ġcan Ġbe Ġcategorized Ġas : ĠVul gar , Ġwhich Ġinclude Ġexplicit Ġand Ġrude Ġsexual Ġreferences , ĠPorn ographic , Ġand ĠH ateful , Ġwhich Ġincludes Ġoffensive Ġremarks Ġconcerning Ġpeople âĢ Ļ s Ġrace , Ġreligion , Ġcountry , Ġetc . ĠB IB REF 14 . ĠPrior Ġworks Ġhave Ġconcentrated Ġon Ġbuilding Ġannot ated Ġcorpor a Ġand Ġtraining Ġclassification Ġmodels . ĠCon cerning Ġcorpor a , Ġhates pe ech data . com Ġattempts Ġto Ġmaintain Ġan Ġupdated Ġlist Ġof Ġhate Ġspeech Ġcorpor a Ġfor Ġmultiple Ġlanguages Ġincluding ĠArabic Ġand ĠEnglish . ĠFurther , ĠSem E val Ġ2019 Ġran Ġan Ġevaluation Ġtask Ġtargeted Ġat Ġdetecting Ġoffensive Ġlanguage , Ġwhich Ġfocused Ġexclusively Ġon ĠEnglish ĠB IB REF 15 . ĠAs Ġfor Ġclassification Ġmodels , Ġmost Ġstudies Ġused Ġsupervised Ġclassification Ġat Ġeither Ġword Ġlevel ĠB IB REF 10 , Ġcharacter Ġsequence Ġlevel ĠB IB REF 11 , Ġand Ġword Ġembed d ings ĠB IB REF 9 . ĠThe Ġstudies Ġused Ġdifferent Ġclassification Ġtechniques Ġincluding Ġusing ĠNa Ã¯ve ĠBay es ĠB IB REF 10 , ĠS VM ĠB IB REF 11 , Ġand Ġdeep Ġlearning ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 12 Ġclassification . ĠThe Ġaccuracy Ġof Ġthe Ġaforementioned Ġsystem Ġranged Ġbetween Ġ76 % Ġand Ġ90 %. ĠEarlier Ġwork Ġlooked Ġat Ġthe Ġuse Ġof Ġsentiment Ġwords Ġas Ġfeatures Ġas Ġwell Ġas Ġcontextual Ġfeatures ĠB IB REF 13 . </s> The Ġwork Ġon ĠArabic Ġoffensive Ġlanguage Ġdetection Ġis Ġrelatively Ġnascent ĠB IB REF 16 , ĠB IB REF 17 , ĠB IB REF 18 , ĠB IB REF 19 , ĠB IB REF 5 . ĠMubarak Ġet Ġal . Ġm ub arak 2017 ab usive Ġsuggested Ġthat Ġcertain Ġusers Ġare Ġmore Ġlikely Ġto Ġuse Ġoffensive Ġlanguages Ġthan Ġothers , Ġand Ġthey Ġused Ġthis Ġinsight Ġto Ġbuild Ġa Ġlist Ġof Ġoffensive ĠArabic Ġwords Ġand Ġthey Ġconstructed Ġa Ġlabeled Ġset Ġof Ġ1 , 100 Ġtweets . ĠA bo zin ad ah Ġet Ġal . Ġab oz in ad ah 2017 det ect ing Ġused Ġsupervised Ġclassification Ġbased Ġon Ġa Ġvariety Ġof Ġfeatures Ġincluding Ġuser Ġprofile Ġfeatures , Ġtextual Ġfeatures , Ġand Ġnetwork Ġfeatures . ĠThey Ġreported Ġan Ġaccuracy Ġof Ġnearly Ġ90 %. ĠAl ak rot Ġet Ġal . Ġal ak rot 2018 t ow ards Ġused Ġsupervised Ġclassification Ġbased Ġon Ġword Ġun ig rams Ġand Ġn - gram s Ġto Ġdetect Ġoffensive Ġlanguage Ġin ĠYouTube Ġcomments . ĠThey Ġimproved Ġclassification Ġwith Ġstemming Ġand Ġachieved Ġa Ġprecision Ġof Ġ88 %. ĠAlb adi Ġet Ġal . Ġal b adi 2018 they Ġfocused Ġon Ġdetecting Ġreligious Ġhate Ġspeech Ġusing Ġa Ġrecurrent Ġneural Ġnetwork . ĠFurther ; ĠSchmidt Ġand ĠW ieg and Ġsch midt - w ieg and - 2017 - sur vey Ġsurveyed Ġmajor Ġworks Ġon Ġhate Ġspeech Ġdetection ; ĠF ortun a Ġand ĠNunes ĠF ortun a 2018 Sur vey Ġprovided Ġa Ġcomprehensive Ġsurvey Ġfor Ġtechniques Ġand Ġworks Ġdone Ġin Ġthe Ġarea Ġbetween Ġ2004 Ġand Ġ2017 . </s> Arab ic Ġis Ġa Ġmorph ologically Ġrich Ġlanguage Ġwith Ġa Ġstandard Ġvariety , Ġnamely ĠModern ĠStandard ĠArabic Ġ( M SA ) Ġand Ġis Ġtypically Ġused Ġin Ġformal Ġcommunication , Ġand Ġmany Ġdialect al Ġvarieties Ġthat Ġdiffer Ġfrom ĠM SA Ġin Ġlex ical Ġselection , Ġmorphology , Ġand Ġsynt actic Ġstructures . ĠFor ĠM SA , Ġwords Ġare Ġtypically Ġderived Ġfrom Ġa Ġset Ġof Ġthousands Ġof Ġroots Ġby Ġfitting Ġa Ġroot Ġinto Ġa Ġstem Ġtemplate Ġand Ġthe Ġresulting Ġstem Ġmay Ġaccept Ġa Ġvariety Ġof Ġprefix es Ġand Ġsuffix es Ġsuch Ġas Ġcoordinating Ġconj unctions Ġand Ġpronouns . ĠThough Ġword Ġsegment ation Ġ( or Ġstemming ) Ġis Ġquite Ġaccurate Ġfor ĠM SA ĠB IB REF 20 , Ġwith Ġaccuracy Ġapproaching Ġ99 %, Ġdialect al Ġsegment ation Ġis Ġnot Ġsufficiently Ġreliable , Ġwith Ġaccuracy Ġranging Ġbetween Ġ91 - 95 % Ġfor Ġdifferent Ġdialect s ĠB IB REF 21 . ĠSince Ġdialect al ĠArabic Ġis Ġubiquitous Ġin ĠArabic Ġtweets Ġand Ġmany Ġtweets Ġhave Ġcreative Ġspell ings Ġof Ġwords , Ġrecent Ġwork Ġon ĠArabic Ġoffensive Ġlanguage Ġdetection Ġused Ġcharacter - level Ġmodels ĠB IB REF 5 . </s> Data ĠCollection Ġ: :: ĠCollect ing ĠArabic ĠOffensive ĠTwe ets </s> Our Ġtarget Ġwas Ġto Ġbuild Ġa Ġlarge ĠArabic Ġoffensive Ġlanguage Ġdataset Ġthat Ġis Ġrepresentative Ġof Ġtheir Ġappearance Ġon ĠTwitter Ġand Ġis Ġhopefully Ġnot Ġbiased Ġto Ġspecific Ġdialect s , Ġtopics , Ġor Ġtargets . ĠOne Ġof Ġthe Ġmain Ġchallenges Ġis Ġthat Ġoffensive Ġtweets Ġconstitute Ġa Ġvery Ġsmall Ġportion Ġof Ġoverall Ġtweets . ĠTo Ġquantify Ġtheir Ġproportion , Ġwe Ġtook Ġ3 Ġrandom Ġsamples Ġof Ġtweets Ġfrom Ġdifferent Ġdays , Ġwith Ġeach Ġsample Ġcomposed Ġof Ġ1 , 000 Ġtweets , Ġand Ġwe Ġfound Ġthat Ġbetween Ġ1 % Ġand Ġ2 % Ġof Ġthem Ġwere Ġin Ġfact Ġoffensive Ġ( including Ġpornographic Ġadvertisement ). ĠThis Ġpercentage Ġis Ġconsistent Ġwith Ġpreviously Ġreported Ġpercentages ĠB IB REF 19 . ĠThus , Ġannot ating Ġrandom Ġtweets Ġis Ġgrossly Ġinefficient . ĠOne Ġway Ġto Ġovercome Ġthis Ġproblem Ġis Ġto Ġuse Ġa Ġseed Ġlist Ġof Ġoffensive Ġwords Ġto Ġfilter Ġtweets . ĠHowever , Ġdoing Ġso Ġis Ġproblematic Ġas Ġit Ġwould Ġskew Ġthe Ġdataset Ġto Ġparticular Ġtypes Ġof Ġoffensive Ġlanguage Ġor Ġto Ġspecific Ġdialect s . ĠOff ens iveness Ġis Ġoften Ġdialect Ġand Ġcountry Ġspecific . </s> After Ġinspecting Ġmany Ġtweets , Ġwe Ġobserved Ġthat Ġmany Ġoffensive Ġtweets Ġhave Ġthe Ġvoc ative Ġparticle ĠÙ Ĭ Ø§ > Ġ( âĢ ľ y A âĢ Ŀ ĠâĢĵ Ġmeaning ĠâĢ ľ O âĢ Ŀ ), Ġwhich Ġis Ġmainly Ġused Ġin Ġdirecting Ġthe Ġspeech Ġto Ġa Ġspecific Ġperson Ġor Ġgroup . ĠThe Ġratio Ġof Ġoffensive Ġtweets Ġincreases Ġto Ġ5 % Ġif Ġeach Ġtweet Ġcontains Ġone Ġvoc ative Ġparticle Ġand Ġto Ġ19 % Ġif Ġhas Ġat Ġleast Ġtwo Ġvoc ative Ġparticles . ĠUsers Ġoften Ġrepeat Ġthis Ġparticle Ġfor Ġemphasis , Ġas Ġin : ĠÙ Ĭ Ø§ ĠØ £ Ùħ ÙĬ ĠÙ Ĭ Ø§ ĠØ Ń ÙĨ ÙĪ ÙĨ Ø© > Ġ( âĢ ľ y A ĠAmy Ġy A ĠH n wn p âĢ Ŀ ĠâĢĵ ĠO Ġmy Ġmother , ĠO Ġkind Ġone ), Ġwhich Ġis Ġend earing Ġand Ġnon - offensive , Ġand ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ ĠÙ Ĭ Ø§ ĠÙ Ĥ Ø ° Ø± > Ġ( âĢ ľ y A Ġk lb Ġy A Ġq * r âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdog , ĠO Ġdirty Ġone âĢ Ŀ ), Ġwhich Ġis Ġoffensive . ĠWe Ġdecided Ġto Ġuse Ġthis Ġpattern Ġto Ġincrease Ġour Ġchances Ġof Ġfinding Ġoffensive Ġtweets . ĠOne Ġof Ġthe Ġmain Ġadvantages Ġof Ġthe Ġpattern ĠÙ Ĭ Ø§ Ġ... ĠÙ Ĭ Ø§ > Ġ( âĢ ľ y A Ġ... Ġy A âĢ Ŀ ) Ġis Ġthat Ġit Ġis Ġnot Ġassociated Ġwith Ġany Ġspecific Ġtopic Ġor Ġgenre , Ġand Ġit Ġappears Ġin Ġall ĠArabic Ġdialect s . ĠThough Ġthe Ġuse Ġof Ġoffensive Ġlanguage Ġdoes Ġnot Ġnecess itate Ġthe Ġappearance Ġof Ġthe Ġvoc ative Ġparticle , Ġthe Ġparticle Ġdoes Ġnot Ġfavor Ġany Ġspecific Ġoffensive Ġexpressions Ġand Ġgreatly Ġimproves Ġour Ġchances Ġof Ġfinding Ġoffensive Ġtweets . ĠIt Ġis Ġclear , Ġthe Ġdataset Ġis Ġmore Ġbiased Ġtoward Ġpositive Ġclass . ĠUsing Ġthe Ġdataset Ġfor Ġreal - life Ġapplication Ġmay Ġrequire Ġde - bi asing Ġit Ġby Ġboosting Ġnegative Ġclass Ġor Ġrandom Ġsampling Ġadditional Ġdata Ġfrom ĠTwitter ĠB IB REF 22 . Using Ġthe ĠTwitter ĠAPI , Ġwe Ġcollected Ġ660 k ĠArabic Ġtweets Ġhaving Ġthis Ġpattern Ġbetween ĠApril Ġ15 , Ġ2019 Ġand ĠMay Ġ6 , Ġ2019 . ĠTo Ġincrease Ġdiversity , Ġwe Ġsorted Ġthe Ġword Ġsequences Ġbetween Ġthe Ġvoc ative Ġparticles Ġand Ġtook Ġthe Ġmost Ġfrequent Ġ10 , 000 Ġunique Ġsequences . ĠFor Ġeach Ġword Ġsequence , Ġwe Ġtook Ġa Ġrandom Ġtweet Ġcontaining Ġeach Ġsequence . ĠThen Ġwe Ġannot ated Ġthose Ġtweets , Ġending Ġup Ġwith Ġ1 , 915 Ġoffensive Ġtweets Ġwhich Ġrepresent Ġroughly Ġ19 % Ġof Ġall Ġtweets . ĠEach Ġtweet Ġwas Ġlabeled Ġas : Ġoffensive , Ġwhich Ġcould Ġadditionally Ġbe Ġlabeled Ġas Ġvulgar Ġand / or Ġhate Ġspeech , Ġor ĠClean . ĠWe Ġdescribe Ġin Ġgreater Ġdetail Ġour Ġannotation Ġguidelines , Ġwhich Ġwe Ġmade Ġsure Ġthat Ġthey Ġare Ġcompatible Ġwith Ġthe ĠOff ens E val 2019 Ġannotation Ġguidelines ĠB IB REF 15 . ĠFor Ġexample , Ġif Ġa Ġtweet Ġhas Ġinsults Ġor Ġthreats Ġtargeting Ġa Ġgroup Ġbased Ġon Ġtheir Ġnationality , Ġethnicity , Ġgender , Ġpolitical Ġaffiliation , Ġreligious Ġbelief , Ġor Ġother Ġcommon Ġcharacteristics , Ġthis Ġis Ġconsidered Ġas Ġhate Ġspeech ĠB IB REF 15 . ĠIt Ġis Ġworth Ġmentioning Ġthat Ġwe Ġalso Ġconsidered Ġinsulting Ġgroups Ġbased Ġon Ġtheir Ġsport Ġaffiliation Ġas Ġa Ġform Ġof Ġhate Ġspeech . ĠIn Ġmost ĠArab Ġcountries , Ġbeing Ġa Ġfan Ġof Ġa Ġparticularly Ġsporting Ġclub Ġis Ġconsidered Ġas Ġpart Ġof Ġthe Ġpersonality Ġand Ġideology Ġwhich Ġrarely Ġchanges Ġover Ġtime Ġ( similar Ġto Ġpolitical Ġaffiliation ). ĠMany Ġincidents Ġof Ġviolence Ġhave Ġoccurred Ġamong Ġfans Ġof Ġrival Ġclubs . </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets </s> We Ġdeveloped Ġthe Ġannotation Ġguidelines Ġjointly Ġwith Ġan Ġexperienced Ġannot ator , Ġwho Ġis Ġa Ġnative ĠArabic Ġspeaker Ġwith Ġa Ġgood Ġknowledge Ġof Ġvarious ĠArabic Ġdialect s . ĠWe Ġmade Ġsure Ġthat Ġour Ġguidelines Ġwere Ġcompatible Ġwith Ġthose Ġof ĠOff ens E val 2019 . ĠThe Ġannot ator Ġcarried Ġout Ġall Ġannotation . ĠTwe ets Ġwere Ġgiven Ġone Ġor Ġmore Ġof Ġthe Ġfollowing Ġfour Ġlabels : Ġoffensive , Ġvulgar , Ġhate Ġspeech , Ġor Ġclean . ĠSince Ġthe Ġoffensive Ġlabel Ġcovers Ġboth Ġvulgar Ġand Ġhate Ġspeech Ġand Ġvulgar ity Ġand Ġhate Ġspeech Ġare Ġnot Ġmutually Ġexclusive , Ġa Ġtweet Ġcan Ġbe Ġjust Ġoffensive Ġor Ġoffensive Ġand Ġvulgar Ġand / or Ġhate Ġspeech . ĠThe Ġannotation Ġad hered Ġto Ġthe Ġfollowing Ġguidelines : </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠOFF ENS IVE Ġ( OFF ): </s> Off ensive Ġtweets Ġcontain Ġexplicit Ġor Ġimplicit Ġinsults Ġor Ġattacks Ġagainst Ġother Ġpeople , Ġor Ġinappropriate Ġlanguage , Ġsuch Ġas : </s> Direct Ġthreats Ġor Ġinc itement , Ġex : Ġ Ø§Ø Ń Ø± Ù Ĥ ÙĪ Ø§ > ĠÙħ Ù Ĥ Ø± Ø§Ø ª ĠØ§ÙĦ Ùħ Ø¹ Ø§ Ø± Ø ¶ Ø© > Ġ( âĢ ľ AH r q w A Ġm q r At ĠAlm E Ar D p âĢ Ŀ ĠâĢĵ ĠâĢ ľ burn Ġthe Ġheadquarters Ġof Ġthe Ġopposition âĢ Ŀ ) Ġand ĠÙ ĩ Ø ° Ø§ ĠØ§ÙĦ Ùħ ÙĨ Ø§ Ù ģ Ù Ĥ ĠÙ Ĭ Ø ¬ Ø¨ ĠÙ Ĥ Øª ÙĦ Ùĩ > Ġ( âĢ ľ h * A ĠAl mn Af q Ġy j b Ġq tl h âĢ Ŀ ĠâĢĵ ĠâĢ ľ this Ġhypocr ite Ġneeds Ġto Ġbe Ġkilled âĢ Ŀ ). </s> Ins ults Ġand Ġexpressions Ġof Ġcontempt , Ġwhich Ġinclude : ĠAnimal Ġanalogy , Ġex : ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ y A Ġk lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdog âĢ Ŀ ) Ġand ĠÙ ĥ ÙĦ ĠØ ª Ø¨ ÙĨ > Ġ( âĢ ľ kl Ġt bn âĢ Ŀ ĠâĢĵ ĠâĢ ľ eat Ġhay âĢ Ŀ ). ; ĠIns ult Ġto Ġfamily Ġmembers , Ġex : ĠÙ Ĭ Ø§ ĠØ ± ÙĪ Ø Ń ĠØ £ Ùħ Ù ĥ > Ġ( âĢ ľ y A Ġr w H ĠAm k âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġmother 's Ġsoul âĢ Ŀ ); ĠSex ually - related Ġinsults , Ġex : ĠÙ Ĭ Ø§ ĠØ ¯ ÙĬ ÙĪ Ø « > Ġ( âĢ ľ y A Ġdy w v âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġperson Ġwithout Ġenvy âĢ Ŀ ); ĠDam nation , Ġex : ĠØ§ÙĦ ÙĦ Ùĩ ĠÙ Ĭ ÙĦ Ø¹ ÙĨ Ù ĥ > Ġ( âĢ ľ All h Ġy l En k âĢ Ŀ ĠâĢĵ ĠâĢ ľ may ĠAllah / God Ġcurse Ġyou âĢ Ŀ ); Ġand ĠAttacks Ġon Ġmorals Ġand Ġethics , Ġex : ĠÙ Ĭ Ø§ ĠÙ ĥ Ø§Ø ° Ø¨ > Ġ( âĢ ľ y A Ġk A * b âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġliar âĢ Ŀ ) </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠV UL G AR Ġ( V LG ): </s> V ul gar Ġtweets Ġare Ġa Ġsubset Ġof Ġoffensive Ġtweets Ġand Ġcontain Ġprof anity , Ġsuch Ġas Ġmentions Ġof Ġprivate Ġparts Ġor Ġsexual - related Ġacts Ġor Ġreferences . </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠH ATE ĠSP E ECH Ġ( HS ): </s> H ate Ġspeech Ġtweets , Ġa Ġsubset Ġof Ġoffensive Ġtweets Ġcontaining Ġoffensive Ġlanguage Ġtargeting Ġgroup Ġbased Ġon Ġcommon Ġcharacteristics Ġsuch Ġas : ĠRace , Ġex : ĠÙ Ĭ Ø§ ĠØ ² ÙĨ Ø ¬ ÙĬ > Ġ( âĢ ľ y A Ġz n j y âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġnegro âĢ Ŀ ); ĠEthnic ity , Ġex . ĠØ§ÙĦ Ù ģ Ø± Ø³ ĠØ§ÙĦ Ø £ ÙĨ Ø ¬ Ø§Ø ³ > Ġ( âĢ ľ A lf rs ĠAl An j As âĢ Ŀ ĠâĢĵ ĠâĢ ľ Imp ure ĠPers ians âĢ Ŀ ); ĠGroup Ġor Ġparty , Ġex : ĠØ £ Ø¨ ÙĪ Ù ĥ ĠØ ´ ÙĬ ÙĪ Ø¹ ÙĬ > Ġ( âĢ ľ Ab wk Ġ$ y w Ey âĢ Ŀ ĠâĢĵ ĠâĢ ľ your Ġfather Ġis Ġcommunist âĢ Ŀ ); Ġand ĠReligion , Ġex : ĠØ ¯ ÙĬ ÙĨ Ù ĥ ĠØ§ÙĦ Ù Ĥ Ø ° Ø± > Ġ( âĢ ľ d yn k ĠAl q * r âĢ Ŀ ĠâĢĵ ĠâĢ ľ your Ġfilthy Ġreligion âĢ Ŀ ). </s> Data ĠCollection Ġ: :: ĠAn not ating ĠTwe ets Ġ: :: ĠCLE AN Ġ( CL N ): </s> Clean Ġtweets Ġdo Ġnot Ġcontain Ġvulgar Ġor Ġoffensive Ġlanguage . ĠWe Ġnoticed Ġthat Ġsome Ġtweets Ġhave Ġsome Ġoffensive Ġwords , Ġbut Ġthe Ġwhole Ġtweet Ġshould Ġnot Ġbe Ġconsidered Ġas Ġoffensive Ġdue Ġto Ġthe Ġintention Ġof Ġusers . ĠThis Ġsuggests Ġthat Ġnormal Ġstring Ġmatch Ġwithout Ġconsidering Ġcontexts Ġwill Ġfail Ġin Ġsome Ġcases . ĠExamples Ġof Ġsuch Ġambiguous Ġcases Ġinclude : ĠHum or , Ġex : ĠÙ Ĭ Ø§ ĠØ ¹ Ø¯ ÙĪ Ø© ĠØ§ÙĦ Ù ģ Ø± Ø Ń Ø© ĠÙ ĩ Ùĩ Ùĩ > Ġ( âĢ ľ y A ĠEd wp ĠAl fr H p Ġh hh âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġenemy Ġof Ġhappiness Ġha h aha âĢ Ŀ ); ĠAdvice , Ġex : Ġ ÙĦ Ø§ ĠØ ª Ù Ĥ ÙĦ Ġ ÙĦ Ø µ Ø§Ø Ń Ø¨ Ù ĥ ĠÙ Ĭ Ø§ ĠØ ® ÙĨ Ø ² ÙĬ Ø± > Ġ( âĢ ľ l A Ġt ql Ġl SA H b k Ġy A Ġx nz yr âĢ Ŀ ĠâĢĵ ĠâĢ ľ don 't Ġsay Ġto Ġyour Ġfriend : ĠYou Ġare Ġa Ġpig âĢ Ŀ ); ĠCondition , Ġex : ĠØ ¥ Ø ° Ø§ ĠØ ¹ Ø§ Ø± Ø ¶ Øª Ùĩ Ùħ ĠÙ Ĭ Ù Ĥ ÙĪ ÙĦ ÙĪ ÙĨ ĠÙ Ĭ Ø§ ĠØ ¹ Ùħ ÙĬ ÙĦ > Ġ( âĢ ľ A * A ĠE Ar D th m Ġy q wl wn Ġy A ĠE my l âĢ Ŀ ĠâĢĵ ĠâĢ ľ if Ġyou Ġdisagree Ġwith Ġthem Ġthey Ġwill Ġsay : ĠYou Ġare Ġan Ġagent âĢ Ŀ ); ĠCond em nation , Ġex : Ġ ÙĦ Ùħ Ø§Ø ° Ø§ ĠÙ Ĩ Ø³ Ø¨ ĠØ ¨ Ù Ĥ ÙĪ ÙĦ : ĠÙ Ĭ Ø§ ĠØ ¨ Ù Ĥ Ø± Ø© Ø Ł > Ġ( âĢ ľ l mA * A Ġn sb Ġb q wl : Ġy A Ġb q r p ? âĢ Ŀ ĠâĢĵ ĠâĢ ľ Why Ġdo Ġwe Ġinsult Ġothers Ġby Ġsaying : ĠO Ġcow ? âĢ Ŀ ); ĠSelf Ġoffense , Ġex : ĠØ ª Ø¹ Ø¨ Øª ĠÙħ ÙĨ Ġ ÙĦ Ø³ Ø§ ÙĨ ÙĬ ĠØ§ÙĦ Ù Ĥ Ø ° Ø± > Ġ( âĢ ľ t E bt Ġm n Ġls Any ĠAl q * r âĢ Ŀ ĠâĢĵ ĠâĢ ľ I Ġam Ġtired Ġof Ġmy Ġdirty Ġtongue âĢ Ŀ ); ĠNon - human Ġtarget , Ġex : ĠÙ Ĭ Ø§ ĠØ ¨ ÙĨ Øª ĠØ§ÙĦ Ùħ Ø ¬ ÙĨ ÙĪ ÙĨ Ø© ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĪ Ø± Ø© > Ġ( âĢ ľ y A Ġb nt ĠAlm j n wn p Ġy A Ġk wr p âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdaughter Ġof Ġthe Ġcrazy Ġone ĠO Ġfootball âĢ Ŀ ); Ġand ĠQu otation Ġfrom Ġa Ġmovies Ġor Ġa Ġstory , Ġex : ĠØ ª Ø§ ÙĨ ÙĬ ĠÙ Ĭ Ø§ ĠØ ² Ù ĥ ÙĬ ! ĠØ ª Ø§ ÙĨ ÙĬ ĠÙ Ĭ Ø§ ĠÙ ģ Ø§Ø ´ ÙĦ > Ġ( âĢ ľ t Any Ġy A Ġz ky ! Ġt Any Ġy A Ġf A $ l âĢ Ŀ ĠâĢĵ ĠâĢ ľ again ĠO ĠZ aky ! Ġagain ĠO Ġloser âĢ Ŀ ). ĠFor Ġother Ġambiguous Ġcases , Ġthe Ġannot ator Ġsearched ĠTwitter Ġto Ġfind Ġhow Ġactual Ġusers Ġused Ġexpressions . </s> Table ĠT AB REF 11 Ġshows Ġthe Ġdistribution Ġof Ġthe Ġannot ated Ġtweets . ĠThere Ġare Ġ1 , 915 Ġoffensive Ġtweets , Ġincluding Ġ225 Ġvulgar Ġtweet Ġand Ġ50 6 Ġhate Ġspeech Ġtweets , Ġand Ġ8 , 08 5 Ġclean Ġtweets . ĠTo Ġvalidate Ġthe Ġquality Ġof Ġannotation , Ġa Ġrandom Ġsample Ġof Ġ100 Ġtweets Ġfrom Ġthe Ġdata , Ġcontaining Ġ50 Ġoffensive Ġand Ġ50 Ġclean Ġtweets , Ġwas Ġgiven Ġto Ġadditional Ġthree Ġannot ators . ĠWe Ġcalculated Ġthe ĠInter - An not ator ĠAgreement Ġbetween Ġthe Ġannot ators Ġusing ĠFle iss âĢ Ļ s ĠKappa Ġcoefficient ĠB IB REF 23 . ĠThe ĠKappa Ġscore Ġwas Ġ0 . 92 Ġindicating Ġhigh Ġquality Ġannotation Ġand Ġagreement . </s> Data ĠCollection Ġ: :: ĠStatistics Ġand ĠUser ĠDem ographics </s> Given Ġthe Ġannot ated Ġtweets , Ġwe Ġwanted Ġto Ġascertain Ġthe Ġdistribution Ġof : Ġtypes Ġof Ġoffensive Ġlanguage , Ġgenres Ġwhere Ġit Ġis Ġused , Ġthe Ġdialect s Ġused , Ġand Ġthe Ġgender Ġof Ġusers Ġusing Ġsuch Ġlanguage . </s> Figure ĠFIG REF 13 Ġshows Ġthe Ġdistribution Ġof Ġtopics Ġassociated Ġwith Ġoffensive Ġtweets . ĠAs Ġthe Ġfigure Ġshows , Ġsports Ġand Ġpolitics Ġare Ġmost Ġdominant Ġfor Ġoffensive Ġlanguage Ġincluding Ġvulgar Ġand Ġhate Ġspeech . ĠAs Ġfor Ġdialect , Ġwe Ġlooked Ġat ĠM SA Ġand Ġfour Ġmajor Ġdialect s , Ġnamely ĠEgyptian Ġ( EG Y ), ĠLe vent ine Ġ( LE V ), ĠMag h re bi Ġ( M GR ), Ġand ĠGulf Ġ( GL F ). ĠFigure ĠFIG REF 14 Ġshows Ġthat Ġ71 % Ġof Ġvulgar Ġtweets Ġwere Ġwritten Ġin ĠE GY Ġfollowed Ġby ĠGL F , Ġwhich Ġaccounted Ġfor Ġ13 % Ġof Ġvulgar Ġtweets . ĠM SA Ġwas Ġnot Ġused Ġin Ġany Ġof Ġthe Ġvulgar Ġtweets . ĠAs Ġfor Ġoffensive Ġtweets Ġin Ġgeneral , ĠE GY Ġand ĠGL F Ġwere Ġused Ġin Ġ36 % Ġand Ġ35 % Ġof Ġthe Ġoffensive Ġtweets Ġrespectively . ĠUnlike Ġthe Ġcase Ġof Ġvulgar Ġlanguage Ġwhere ĠM SA Ġwas Ġnon - existent , Ġ15 % Ġof Ġthe Ġoffensive Ġtweets Ġwere Ġin Ġfact Ġwritten Ġin ĠM SA . ĠFor Ġhate Ġspeech , ĠGL F Ġand ĠE GY Ġwere Ġagain Ġdominant Ġand ĠM SA Ġconsist uted Ġ21 % Ġof Ġthe Ġtweets . ĠThis Ġis Ġconsistent Ġwith Ġfindings Ġfor Ġother Ġlanguages Ġsuch Ġas ĠEnglish Ġand ĠItalian Ġwhere Ġvulgar Ġlanguage Ġwas Ġmore Ġfrequently Ġassociated Ġwith Ġcoll oqu ial Ġlanguage ĠB IB REF 24 , ĠB IB REF 25 . ĠRegarding Ġthe Ġgender , ĠFigure ĠFIG REF 15 Ġshows Ġthat Ġthe Ġvast Ġmajority Ġof Ġoffensive Ġtweets , Ġincluding Ġvulgar Ġand Ġhate Ġspeech , Ġwere Ġauthored Ġby Ġmales . ĠFemale ĠTwitter Ġusers Ġaccounted Ġfor Ġ14 % Ġof Ġoffensive Ġtweets Ġin Ġgeneral Ġand Ġ6 % Ġand Ġ9 % Ġof Ġvulgar Ġand Ġhate Ġspeech Ġrespectively . ĠFigure ĠFIG REF 16 Ġshows Ġa Ġdetailed Ġcategor ization Ġof Ġhate Ġspeech Ġtypes , Ġwhere Ġthe Ġtop Ġthree Ġinclude Ġinsulting Ġgroups Ġbased Ġon Ġtheir Ġpolitical Ġideology , Ġorigin , Ġand Ġsport Ġaffiliation . ĠReligious Ġhate Ġspeech Ġappeared Ġin Ġonly Ġ15 % Ġof Ġall Ġhate Ġspeech Ġtweets . </s> Next , Ġwe Ġanalyzed Ġall Ġtweets Ġlabeled Ġas Ġoffensive Ġto Ġbetter Ġunderstand Ġhow ĠArabic Ġspeakers Ġuse Ġoffensive Ġlanguage . ĠHere Ġis Ġa Ġbreakdown Ġof Ġusage : </s> Direct Ġname Ġcalling : ĠThe Ġmost Ġfrequent Ġattack Ġis Ġto Ġcall Ġa Ġperson Ġan Ġanimal Ġname , Ġand Ġthe Ġmost Ġused Ġanimals Ġwere ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ k lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ dog âĢ Ŀ ), ĠØ Ń Ùħ Ø§ Ø± > Ġ( âĢ ľ H m Ar âĢ Ŀ ĠâĢĵ ĠâĢ ľ don key âĢ Ŀ ), Ġand ĠØ ¨ Ùĩ ÙĬ Ùħ > Ġ( âĢ ľ bh ym âĢ Ŀ ĠâĢĵ ĠâĢ ľ be ast âĢ Ŀ ). ĠThe Ġsecond Ġmost Ġcommon Ġwas Ġinsulting Ġmental Ġabilities Ġusing Ġwords Ġsuch Ġas ĠØ º Ø¨ ÙĬ > Ġ( âĢ ľ g by âĢ Ŀ ĠâĢĵ ĠâĢ ľ st upid âĢ Ŀ ) Ġand ĠØ ¹ Ø¨ ÙĬ Ø · > Ġ( âĢ ľ E by T âĢ Ŀ ĠâĢĵ âĢ ľ id iot âĢ Ŀ ). ĠSome Ġculture - specific Ġdifferences Ġshould Ġbe Ġconsidered . ĠNot Ġall Ġanimal Ġnames Ġare Ġused Ġas Ġinsults . ĠFor Ġexample , Ġanimals Ġsuch Ġas ĠØ £ Ø³ Ø¯ > Ġ( âĢ ľ As d âĢ Ŀ ĠâĢĵ ĠâĢ ľ l ion âĢ Ŀ ), ĠØ µ Ù Ĥ Ø± > Ġ( âĢ ľ S q r âĢ Ŀ ĠâĢĵ ĠâĢ ľ fal con âĢ Ŀ ), Ġand ĠØ º Ø ² Ø§ÙĦ > Ġ( âĢ ľ gz Al âĢ Ŀ ĠâĢĵ ĠâĢ ľ g az elle âĢ Ŀ ) Ġare Ġtypically Ġused Ġfor Ġpraise . ĠFor Ġother Ġinsults , Ġpeople Ġuse : Ġsome Ġbird Ġnames Ġsuch Ġas ĠØ ¯ Ø ¬ Ø§Ø ¬ Ø© > Ġ( âĢ ľ dj A jp âĢ Ŀ ĠâĢĵ ĠâĢ ľ ch icken âĢ Ŀ ), ĠØ ¨ ÙĪ Ùħ Ø© > Ġ( âĢ ľ b w mp âĢ Ŀ ĠâĢĵ ĠâĢ ľ owl âĢ Ŀ ), Ġand ĠØ º Ø± Ø§Ø ¨ > Ġ( âĢ ľ gr Ab âĢ Ŀ ĠâĢĵ ĠâĢ ľ crow âĢ Ŀ ); Ġinsects Ġsuch Ġas ĠØ ° Ø¨ Ø§Ø ¨ Ø© > Ġ( âĢ ľ * b Ab p âĢ Ŀ ĠâĢĵ ĠâĢ ľ fly âĢ Ŀ ), ĠØ µ Ø± Ø µ ÙĪ Ø± > Ġ( âĢ ľ S r Sw r âĢ Ŀ ĠâĢĵ ĠâĢ ľ cock roach âĢ Ŀ ), Ġand ĠØ Ń Ø ´ Ø± Ø© > Ġ( âĢ ľ H $ r p âĢ Ŀ ĠâĢĵ ĠâĢ ľ in sect âĢ Ŀ ); Ġmicro organisms Ġsuch Ġas ĠØ ¬ Ø± Ø « ÙĪ Ùħ Ø© > Ġ( âĢ ľ j r v w mp âĢ Ŀ ĠâĢĵ ĠâĢ ľ micro be âĢ Ŀ ) Ġand ĠØ · Ø Ń Ø§ÙĦ Ø¨ > Ġ( âĢ ľ TH Al b âĢ Ŀ ĠâĢĵ ĠâĢ ľ alg ae âĢ Ŀ ); Ġin animate Ġobjects Ġsuch Ġas ĠØ ¬ Ø ² Ùħ Ø© > Ġ( âĢ ľ j z mp âĢ Ŀ ĠâĢĵ ĠâĢ ľ sh oes âĢ Ŀ ) Ġand ĠØ ³ Ø · ÙĦ > Ġ( âĢ ľ s T l âĢ Ŀ ĠâĢĵ ĠâĢ ľ buck et âĢ Ŀ ) Ġamong Ġother Ġus ages . </s> Sim ile Ġand Ġmetaphor : ĠUsers Ġuse Ġsim ile Ġand Ġmetaphor Ġwere Ġthey Ġwould Ġcompare Ġa Ġperson Ġto : Ġan Ġanimal Ġas Ġin ĠØ ² ÙĬ ĠØ§ÙĦ Ø « ÙĪ Ø± > Ġ( âĢ ľ zy ĠAlv wr âĢ Ŀ ĠâĢĵ ĠâĢ ľ like Ġa Ġbull âĢ Ŀ ), ĠØ ³ Ùħ Ø¹ ÙĨ ÙĬ ĠÙ Ĩ Ùĩ ÙĬ Ù Ĥ Ù ĥ > Ġ( âĢ ľ sm E ny Ġn hy q k âĢ Ŀ ĠâĢĵ ĠâĢ ľ let Ġme Ġhear Ġyour Ġbr aying âĢ Ŀ ), Ġand ĠÙ ĩ Ø ² ĠØ ¯ ÙĬ ÙĦ Ù ĥ > Ġ( âĢ ľ hz Ġd yl k âĢ Ŀ ĠâĢĵ ĠâĢ ľ w ag Ġyour Ġtail âĢ Ŀ ); Ġa Ġperson Ġwith Ġmental Ġor Ġphysical Ġdisability Ġsuch Ġas ĠÙħ ÙĨ Ø º ÙĪ ÙĦ ÙĬ > Ġ( âĢ ľ m ng w ly âĢ Ŀ ĠâĢĵ ĠâĢ ľ M ong ol ian Ġ( down - sy nd rome ) âĢ Ŀ ), ĠÙħ Ø¹ ÙĪ Ù Ĥ > Ġ( âĢ ľ m E w q âĢ Ŀ ĠâĢĵ ĠâĢ ľ disabled âĢ Ŀ ), Ġand ĠÙ Ĥ Ø ² Ùħ > Ġ( âĢ ľ q z m âĢ Ŀ ĠâĢĵ ĠâĢ ľ d war f âĢ Ŀ ); Ġand Ġto Ġthe Ġopposite Ġgender Ġsuch Ġas ĠØ ¬ ÙĬ Ø ´ ĠÙ Ĩ ÙĪ Ø§ÙĦ > Ġ( âĢ ľ j y $ Ġn w Al âĢ Ŀ ĠâĢĵ ĠâĢ ľ N aw al 's Ġarmy Ġ( N aw al Ġis Ġfemale Ġname ) âĢ Ŀ ) Ġand ĠÙ Ĩ Ø§Ø ¯ ÙĬ ĠØ ² ÙĬ Ø ² ÙĬ > Ġ( âĢ ľ n Ad y Ġz y zy âĢ Ŀ ĠâĢĵ ĠâĢ ľ Z iz i 's Ġclub Ġ( Z iz i Ġis Ġa Ġfemale Ġpet Ġname ) âĢ Ŀ ). </s> Ind irect Ġspeech : ĠThis Ġtype Ġof Ġoffensive Ġlanguage Ġincludes : Ġsarc asm Ġsuch Ġas ĠØ £ Ø ° Ù ĥ Ù ī ĠØ ¥ Ø ® ÙĪ Ø§Ø ª Ù ĥ > Ġ( âĢ ľ A * k Y ĠAx w At k âĢ Ŀ ĠâĢĵ ĠâĢ ľ smart est Ġone Ġof Ġyour Ġsiblings âĢ Ŀ ) Ġand ĠÙ ģ ÙĬ ÙĦ Ø³ ÙĪ Ù ģ ĠØ§ÙĦ Ø Ń Ùħ ÙĬ Ø± > Ġ( âĢ ľ f yl sw f ĠAl H my r âĢ Ŀ ĠâĢĵ ĠâĢ ľ the Ġdon keys ' Ġphilosopher âĢ Ŀ ); Ġquestions Ġsuch Ġas Ġ Ø§ ÙĬ Ùĩ ĠÙ ĥ ÙĦ ĠØ§ÙĦ Ø º Ø¨ Ø§Ø ¡ ĠØ ¯ Ùĩ > Ġ( âĢ ľ Ay h Ġk l ĠAl gb A Ġdh âĢ Ŀ ĠâĢĵ ĠâĢ ľ what Ġis Ġall Ġthis Ġstupidity âĢ Ŀ ); Ġand Ġindirect Ġspeech Ġsuch Ġas ĠØ§ÙĦ ÙĨ Ù Ĥ Ø§Ø ´ ĠÙħ Ø¹ ĠØ§ÙĦ Ø¨ Ùĩ Ø§ ÙĬ Ùħ ĠØ º ÙĬ Ø± ĠÙħ Ø « Ùħ Ø± > Ġ( âĢ ľ Al n q A $ Ġm E ĠAlb h A ym Ġg yr Ġm vm r âĢ Ŀ ĠâĢĵ ĠâĢ ľ no Ġuse Ġtalking Ġto Ġcattle âĢ Ŀ ). </s> W ishing ĠEvil : ĠThis Ġentails Ġwishing Ġdeath Ġor Ġmajor Ġharm Ġto Ġbe fall Ġsomeone Ġsuch Ġas ĠØ ± Ø¨ ÙĨ Ø§ ĠÙ Ĭ Ø§Ø ® Ø¯ Ù ĥ > Ġ( âĢ ľ r bn A Ġy A xd k âĢ Ŀ ĠâĢĵ ĠâĢ ľ May ĠGod Ġtake Ġ( kill ) Ġyou âĢ Ŀ ), ĠØ§ÙĦ ÙĦ Ùĩ ĠÙ Ĭ ÙĦ Ø¹ ÙĨ Ù ĥ > Ġ( âĢ ľ All h Ġy l En k âĢ Ŀ ĠâĢĵ ĠâĢ ľ may ĠAllah / God Ġcurse Ġyou âĢ Ŀ ), Ġand ĠØ ± ÙĪ Ø Ń ĠÙ ģ ÙĬ ĠØ ¯ Ø§ Ùĩ ÙĬ Ø© > Ġ( âĢ ľ rw H Ġf y Ġd Ah yp âĢ Ŀ ĠâĢĵ Ġequivalent Ġto ĠâĢ ľ go Ġto Ġhell âĢ Ŀ ). </s> Name Ġalteration : ĠOne Ġcommon Ġway Ġto Ġinsult Ġothers Ġis Ġto Ġchange Ġa Ġletter Ġor Ġtwo Ġin Ġtheir Ġnames Ġto Ġproduce Ġnew Ġoffensive Ġwords Ġthat Ġrhy me Ġwith Ġthe Ġoriginal Ġnames . ĠSome Ġexamples Ġof Ġsuch Ġinclude Ġchanging ĠØ§ÙĦ Ø ¬ Ø ² ÙĬ Ø± Ø© > Ġ( âĢ ľ Al j z yr p âĢ Ŀ ĠâĢĵ ĠâĢ ľ Al j azeera Ġ( channel ) âĢ Ŀ ) Ġto ĠØ§ÙĦ Ø ® ÙĨ Ø ² ÙĬ Ø± Ø© > Ġ( âĢ ľ Al x nz yr p âĢ Ŀ ĠâĢĵ ĠâĢ ľ the Ġpig âĢ Ŀ ) Ġand ĠØ ® ÙĦ Ù ģ Ø§ ÙĨ > Ġ( âĢ ľ x lf An âĢ Ŀ ĠâĢĵ ĠâĢ ľ K half an Ġ( person Ġname ) âĢ Ŀ ) Ġto ĠØ ® Ø± Ù ģ Ø§ ÙĨ > Ġ( âĢ ľ x rf An âĢ Ŀ ĠâĢĵ ĠâĢ ľ c raz ed âĢ Ŀ ). </s> Soc ietal Ġstrat ification : ĠSome Ġinsults Ġare Ġassociated Ġwith : Ġcertain Ġjobs Ġsuch Ġas ĠØ ¨ ÙĪ Ø§Ø ¨ > Ġ( âĢ ľ b w Ab âĢ Ŀ ĠâĢĵ ĠâĢ ľ do orman âĢ Ŀ ) Ġor ĠØ ® Ø§Ø ¯ Ùħ > Ġ( âĢ ľ x Ad m âĢ Ŀ ĠâĢĵ ĠâĢ ľ serv ant âĢ Ŀ ); Ġand Ġspecific Ġsocietal Ġcomponents Ġsuch ĠØ ¨ Ø¯ ÙĪ ÙĬ > Ġ( âĢ ľ bd wy âĢ Ŀ ĠâĢĵ ĠâĢ ľ bed ou in âĢ Ŀ ) Ġand ĠÙ ģ ÙĦ Ø§Ø Ń > Ġ( âĢ ľ fl AH âĢ Ŀ ĠâĢĵ ĠâĢ ľ far mer âĢ Ŀ ). </s> Imm oral Ġbehavior : ĠThese Ġinsults Ġare Ġassociated Ġwith Ġnegative Ġmoral Ġtraits Ġor Ġbehaviors Ġsuch Ġas ĠØ Ń Ù Ĥ ÙĬ Ø± > Ġ( âĢ ľ H q yr âĢ Ŀ ĠâĢĵ ĠâĢ ľ v ile âĢ Ŀ ), ĠØ ® Ø§ ÙĬ ÙĨ > Ġ( âĢ ľ x A yn âĢ Ŀ ĠâĢĵ ĠâĢ ľ tra itor âĢ Ŀ ), Ġand ĠÙħ ÙĨ Ø§ Ù ģ Ù Ĥ > Ġ( âĢ ľ mn Af q âĢ Ŀ ĠâĢĵ ĠâĢ ľ hyp ocr ite âĢ Ŀ ). </s> Sex ually Ġrelated : ĠThey Ġinclude Ġexpressions Ġsuch Ġas ĠØ ® ÙĪ ÙĦ > Ġ( âĢ ľ x wl âĢ Ŀ ĠâĢĵ ĠâĢ ľ gay âĢ Ŀ ), ĠÙĪ Ø³ Ø ® Ø© > Ġ( âĢ ľ ws xp âĢ Ŀ ĠâĢĵ ĠâĢ ľ pro st itute âĢ Ŀ ), Ġand ĠØ ¹ Ø± Ø µ > Ġ( âĢ ľ Er S âĢ Ŀ ĠâĢĵ ĠâĢ ľ p imp âĢ Ŀ ). </s> Figure ĠFIG REF 17 Ġshows Ġtop Ġwords Ġwith Ġthe Ġhighest Ġval ance Ġscore Ġfor Ġindividual Ġwords Ġin Ġthe Ġoffensive Ġtweets . ĠL arger Ġfonts Ġare Ġused Ġto Ġhighlight Ġwords Ġwith Ġhighest Ġscore Ġand Ġalign Ġas Ġwell Ġwith Ġthe Ġcategories Ġmentioned Ġahead Ġin Ġthe Ġbreakdown Ġfor Ġthe Ġoffensive Ġlanguages . ĠWe Ġmodified Ġthe Ġval ence Ġscore Ġdescribed Ġby ĠB IB REF 1 Ġcon over 2011 political Ġto Ġmagn ify Ġits Ġvalue Ġbased Ġon Ġfrequency Ġof Ġoccurrence . ĠThe Ġscore Ġis Ġcomputed Ġas Ġfollows : </s> $ V ( I ) Ġ= Ġ2 Ġ\ frac { Ġ\ frac { tf ( I , ĠC _ off ) }{ total ( C _ off ) }} {\ frac { tf ( I , ĠC _ off ) }{ total ( C _ off )} Ġ+ Ġ\ frac { tf ( I , ĠC _ cl n ) }{ total ( C _ cl n )} Ġ} Ġ- Ġ1 $ </s> where </s> $ tf ( I , ĠC _ i ) Ġ= Ġ\ sum Ġ_ { a Ġ\ in ĠI Ġ\ big cap ĠC _ i } Ġ[ ln ( C nt ( a , ĠC _ i )) Ġ+ Ġ1 ] $ </s> $ total ( C _ i ) Ġ= Ġ\ sum Ġ_ { I } Ġtf ( I , ĠC _ i ) $ </s> $ C nt ( a , ĠC _ i ) $ Ġis Ġthe Ġnumber Ġof Ġtimes Ġword Ġ$ a $ Ġwas Ġused Ġin Ġoffensive Ġor Ġclean Ġtweets Ġtweets Ġ$ C _ i $. ĠIn Ġessence , Ġwe Ġare Ġreplacing Ġterm Ġfrequencies Ġwith Ġthe Ġnatural Ġlog Ġof Ġthe Ġterm Ġfrequencies . </s> Exper iments </s> We Ġconducted Ġan Ġextensive Ġbattery Ġof Ġexperiments Ġon Ġthe Ġdataset Ġto Ġestablish Ġstrong ĠArabic Ġoffensive Ġlanguage Ġclassification Ġresults . ĠThough Ġthe Ġoffensive Ġtweets Ġhave Ġfiner - gr ained Ġlabels Ġwhere Ġoffensive Ġtweet Ġcould Ġalso Ġbe Ġvulgar Ġor Ġconstitute Ġhate Ġspeech , Ġwe Ġconducted Ġco arser - gr ained Ġclassification Ġto Ġdetermine Ġif Ġa Ġtweet Ġwas Ġoffensive Ġor Ġnot . ĠFor Ġclassification , Ġwe Ġexperimented Ġwith Ġseveral Ġtweet Ġrepresentation Ġand Ġclassification Ġmodels . ĠFor Ġtweet Ġrepresentations , Ġwe Ġused : Ġthe Ġcount Ġof Ġpositive Ġand Ġnegative Ġterms , Ġbased Ġon Ġa Ġpol arity Ġlex icon ; Ġstatic Ġembed d ings , Ġnamely Ġfast Text Ġand ĠSkip - G ram ; Ġand Ġdeep Ġcontextual Ġembed d ings , Ġnamely ĠB ERT base - mult ilingual . </s> Exper iments Ġ: :: ĠData ĠPre - processing </s> We Ġperformed Ġseveral Ġtext Ġpre - processing Ġsteps . ĠFirst , Ġwe Ġtoken ized Ġthe Ġtext Ġusing Ġthe ĠFar asa ĠArabic ĠN LP Ġtool kit ĠB IB REF 20 . ĠSecond , Ġwe Ġremoved ĠURLs , Ġnumbers , Ġand Ġall Ġtweet Ġspecific Ġtokens , Ġnamely Ġmentions , Ġret we ets , Ġand Ġhasht ags Ġas Ġthey Ġare Ġnot Ġpart Ġof Ġthe Ġlanguage Ġsemantic Ġstructure , Ġand Ġtherefore , Ġnot Ġusable Ġin Ġpre - trained Ġembed d ings . ĠThird , Ġwe Ġperformed Ġbasic ĠArabic Ġletter Ġnormal ization , Ġnamely Ġvariants Ġof Ġthe Ġletter Ġale f Ġto Ġbare Ġale f , Ġta Ġmar b out a Ġto Ġha , Ġand Ġale f Ġma qs ou ra Ġto Ġya . ĠWe Ġalso Ġseparated Ġwords Ġthat Ġare Ġcommonly Ġincorrectly Ġattached Ġsuch Ġas ĠÙ Ĭ Ø§ Ù ĥ ÙĦ Ø¨ > Ġ( âĢ ľ y Ak lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġdog âĢ Ŀ ), Ġis Ġsplit Ġto ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ y A Ġk lb âĢ Ŀ ). ĠLastly , Ġwe Ġnormalized Ġletter Ġrepet itions Ġto Ġallow Ġfor Ġa Ġmaximum Ġof Ġ2 Ġrepeated Ġletters . ĠFor Ġexample , Ġthe Ġtoken ĠÙ ĩ Ùĩ Ùĩ Ùĩ Ùĩ > Ġ( âĢ ľ hhhh h âĢ Ŀ ĠâĢĵ ĠâĢ ľ ha Ġha Ġha Ġ.. âĢ Ŀ ) Ġis Ġnormalized Ġto ĠÙ ĩ Ùĩ > Ġ( âĢ ľ hh âĢ Ŀ ). ĠWe Ġalso Ġremoved Ġthe ĠArabic Ġshort - d iac rit ics Ġ( t ash ke el ) Ġand Ġword Ġelong ation Ġ( k ash ida ). </s> Exper iments Ġ: :: ĠRepresent ations Ġ: :: ĠLex ical ĠFeatures </s> Since Ġoffensive Ġwords Ġtypically Ġhave Ġa Ġnegative Ġpol arity , Ġwe Ġwanted Ġto Ġtest Ġthe Ġeffectiveness Ġof Ġusing Ġa Ġpol arity Ġlex icon Ġin Ġdetecting Ġoffensive Ġtweets . ĠFor Ġthe Ġlex icon , Ġwe Ġused ĠNile UL ex ĠB IB REF 26 , Ġwhich Ġis Ġan ĠArabic Ġpol arity Ġlex icon Ġcontaining Ġ3 , 279 ĠM SA Ġand Ġ2 , 674 ĠEgyptian Ġterms , Ġout Ġof Ġwhich Ġ4 , 256 Ġare Ġnegative Ġand Ġ1 , 697 Ġare Ġpositive . ĠWe Ġused Ġthe Ġcounts Ġof Ġterms Ġwith Ġpositive Ġpol arity Ġand Ġterms Ġwith Ġnegative Ġpol arity Ġin Ġtweets Ġas Ġfeatures . </s> Exper iments Ġ: :: ĠRepresent ations Ġ: :: ĠStatic ĠEmb edd ings </s> We Ġexperimented Ġwith Ġvarious Ġstatic Ġembed d ings Ġthat Ġwere Ġpre - trained Ġon Ġdifferent Ġcorpor a Ġwith Ġdifferent Ġvector Ġdimension ality . ĠWe Ġcompared Ġpre - trained Ġembed d ings Ġto Ġembed d ings Ġthat Ġwere Ġtrained Ġon Ġour Ġdataset . ĠFor Ġpre - trained Ġembed d ings , Ġwe Ġused : Ġfast Text ĠEgyptian ĠArabic Ġpre - trained Ġembed d ings ĠB IB REF 27 Ġwith Ġvector Ġdimension ality Ġof Ġ300 ; ĠAra V ec Ġskip - gram Ġembed d ings ĠB IB REF 28 , Ġtrained Ġon Ġ66 . 9 M ĠArabic Ġtweets Ġwith Ġ100 - dimensional Ġvectors ; Ġand ĠMaz aj ak Ġskip - gram Ġembed d ings ĠB IB REF 29 , Ġtrained Ġon Ġ250 M ĠArabic Ġtweets Ġwith Ġ300 - dimensional Ġvectors . </s> Sent ence Ġembed d ings Ġwere Ġcalculated Ġby Ġtaking Ġthe Ġmean Ġof Ġthe Ġembed d ings Ġof Ġtheir Ġtokens . ĠThe Ġimportance Ġof Ġtesting Ġa Ġcharacter Ġlevel Ġn - gram Ġmodel Ġlike Ġfast Text Ġlies Ġin Ġthe Ġagg l ut in ative Ġnature Ġof Ġthe ĠArabic Ġlanguage . ĠWe Ġtrained Ġa Ġnew Ġfast Text Ġtext Ġclassification Ġmodel ĠB IB REF 30 Ġon Ġour Ġdataset Ġwith Ġvectors Ġof Ġ40 Ġdimensions , Ġ0 . 5 Ġlearning Ġrate , Ġ2 $ -$ 10 Ġcharacter Ġn - gram s Ġas Ġfeatures , Ġfor Ġ30 Ġepoch s . ĠThese Ġhyper - param eters Ġwere Ġtuned Ġusing Ġa Ġ5 - fold Ġcross - valid ated Ġgrid - search . </s> Exper iments Ġ: :: ĠRepresent ations Ġ: :: ĠDeep ĠContext ual ized ĠEmb edd ings </s> We Ġalso Ġexperimented Ġwith Ġpre - trained Ġcontextual ized Ġembed d ings Ġwith Ġfine - tun ing Ġfor Ġdown - stream Ġtasks . ĠRecently , Ġdeep Ġcontextual ized Ġlanguage Ġmodels Ġsuch Ġas ĠB ERT Ġ( B id irection al ĠEnc oder ĠRepresent ations Ġfrom ĠTransformers ) ĠB IB REF 31 , ĠU ML F IT ĠB IB REF 32 , Ġand ĠOpen AI ĠG PT ĠB IB REF 33 , Ġto Ġname Ġbut Ġa Ġfew , Ġhave Ġachieved Ġground - breaking Ġresults Ġin Ġmany ĠN LP Ġclassification Ġand Ġlanguage Ġunderstanding Ġtasks . ĠIn Ġthis Ġpaper , Ġwe Ġused ĠB ERT base - mult ilingual Ġ( here after Ġas Ġsimply ĠB ERT ) Ġfine - tun ing Ġmethod Ġto Ġclassify ĠArabic Ġoffensive Ġlanguage Ġon ĠTwitter Ġas Ġit Ġeliminates Ġthe Ġneed Ġof Ġheavily Ġengineered Ġtask - specific Ġarchitectures . ĠAlthough ĠRob ust ly ĠOptim ized ĠB ERT Ġ( Ro BER Ta ) Ġembed d ings Ġperform Ġbetter Ġthan Ġ( BER T large ) Ġon ĠGL UE ĠB IB REF 34 , ĠR ACE ĠB IB REF 35 , Ġand ĠS Qu AD ĠB IB REF 36 Ġtasks , Ġpre - trained Ġmult ilingual ĠRo BER Ta Ġmodels Ġare Ġnot Ġavailable . ĠB ERT Ġis Ġpre - trained Ġon ĠWikipedia Ġtext Ġfrom Ġ104 Ġlanguages Ġand Ġcomes Ġwith Ġhundreds Ġof Ġmillions Ġof Ġparameters . ĠIt Ġcontains Ġan Ġenc oder Ġwith Ġ12 ĠTrans former Ġblocks , Ġhidden Ġsize Ġof Ġ768 , Ġand Ġ12 Ġself - att ention Ġheads . ĠThough Ġthe Ġtraining Ġdata Ġfor Ġthe ĠB ERT Ġembed d ings Ġdon 't Ġmatch Ġour Ġgenre , Ġthese Ġembed ding Ġuse ĠBP Ġsub - word Ġsegments . ĠFollowing Ġdev lin - 2019 - bert , Ġthe Ġclassification Ġconsists Ġof Ġintroducing Ġa Ġdense Ġlayer Ġover Ġthe Ġfinal Ġhidden Ġstate Ġ$ h $ Ġcorresponding Ġto Ġfirst Ġtoken Ġof Ġthe Ġsequence , Ġ[ CL S ], Ġadding Ġa Ġsoft max Ġactivation Ġon Ġthe Ġtop Ġof ĠB ERT Ġto Ġpredict Ġthe Ġprobability Ġof Ġthe Ġ$ l $ Ġlabel : </s> where Ġ$ W $ Ġis Ġthe Ġtask - specific Ġweight Ġmatrix . ĠDuring Ġfine - tun ing , Ġall ĠB ERT Ġparameters Ġtogether Ġwith Ġ$ W $ Ġare Ġoptimized Ġend - to - end Ġto Ġmaximize Ġthe Ġlog - pro b ability Ġof Ġthe Ġcorrect Ġlabels . </s> Exper iments Ġ: :: ĠClassification ĠModels </s> We Ġexplored Ġdifferent Ġclass ifiers . ĠWhen Ġusing Ġlex ical Ġfeatures Ġand Ġpre - trained Ġstatic Ġembed d ings , Ġwe Ġprimarily Ġused Ġan ĠS VM Ġclass ifier Ġwith Ġa Ġradial Ġbasis Ġfunction Ġkernel . ĠOnly Ġwhen Ġusing Ġthe ĠMaz aj ak Ġembed d ings , Ġwe Ġexperimented Ġwith Ġother Ġclass ifiers Ġsuch Ġas ĠAda Boost Ġand ĠLog istic Ġregression . ĠWe Ġdid Ġso Ġto Ġshow Ġthat Ġthe ĠS VM Ġclass ifier Ġwas Ġindeed Ġthe Ġbest Ġof Ġthe Ġbunch , Ġand Ġwe Ġpicked Ġthe ĠMaz aj ak Ġembed d ings Ġbecause Ġthey Ġyielded Ġthe Ġbest Ġresults Ġamong Ġall Ġstatic Ġembed d ings . ĠWe Ġused Ġthe ĠSci kit ĠLearn Ġimplementations Ġof Ġall Ġthe Ġclass ifiers Ġsuch Ġas Ġlib s vm Ġfor Ġthe ĠS VM Ġclass ifier . ĠWe Ġalso Ġexperimented Ġwith Ġfast Text , Ġwhich Ġtrained Ġembed d ings Ġon Ġour Ġdata . ĠWhen Ġusing Ġcontextual ized Ġembed d ings , Ġwe Ġfine - tun ed ĠB ERT Ġby Ġadding Ġa Ġfully - connected Ġdense Ġlayer Ġfollowed Ġby Ġa Ġsoft max Ġclass ifier , Ġminimizing Ġthe Ġbinary Ġcross - ent ropy Ġloss Ġfunction Ġfor Ġthe Ġtraining Ġdata . ĠFor Ġall Ġexperiments , Ġwe Ġused Ġthe ĠPy Tor ch Ġimplementation Ġby ĠHug ging Face Ġas Ġit Ġprovides Ġpre - trained Ġweights Ġand Ġvocabulary . </s> Exper iments Ġ: :: ĠEvaluation </s> For Ġall Ġof Ġour Ġexperiments , Ġwe Ġused Ġ5 - fold Ġcross Ġvalidation Ġwith Ġidentical Ġfolds Ġfor Ġall Ġexperiments . ĠTable ĠT AB REF 27 Ġreports Ġon Ġthe Ġresults Ġof Ġusing Ġlex ical Ġfeatures , Ġstatic Ġpre - trained Ġembed d ings Ġwith Ġan ĠS VM Ġclass ifier , Ġembed d ings Ġtrained Ġon Ġour Ġdata Ġwith Ġfast Text Ġclass ifier , Ġand ĠB ERT Ġover Ġa Ġdense Ġlayer Ġwith Ġsoft max Ġactivation . ĠAs Ġthe Ġresults Ġshow , Ġusing ĠMaz aj ak / S VM Ġyielded Ġthe Ġbest Ġresults Ġoverall Ġwith Ġlarge Ġimprovements Ġin Ġprecision Ġover Ġusing ĠB ERT . ĠWe Ġsuspect Ġthat Ġthe ĠMaz aj ak / S VM Ġcombination Ġperformed Ġbetter Ġthan Ġthe ĠB ERT Ġsetup Ġdue Ġto Ġthe Ġfact Ġthat Ġthe ĠMaz aj ak Ġembed d ings , Ġthough Ġstatic , Ġwere Ġtrained Ġon Ġin - domain Ġdata , Ġas Ġopposed Ġto ĠB ERT . ĠPerhaps Ġif ĠB ERT Ġembed d ings Ġwere Ġtrained Ġon Ġtweets , Ġthey Ġmight Ġhave Ġoutper formed Ġall Ġother Ġsetups . ĠFor Ġcomple teness , Ġwe Ġcompared Ġ7 Ġother Ġclass ifiers Ġwith ĠS VM Ġusing ĠMaz aj ak Ġembed d ings . ĠAs Ġresults Ġin ĠTable ĠT AB REF 28 Ġshow , Ġusing ĠS VM Ġyielded Ġthe Ġbest Ġresults . </s> Exper iments Ġ: :: ĠError ĠAnalysis </s> We Ġinspected Ġthe Ġtweets Ġof Ġone Ġfold Ġthat Ġwere Ġmis classified Ġby Ġthe ĠMaz aj ak / S VM Ġmodel Ġ( 36 Ġfalse Ġpositives / 121 Ġfalse Ġnegatives ) Ġto Ġdetermine Ġthe Ġmost Ġcommon Ġerrors . </s> Exper iments Ġ: :: ĠError ĠAnalysis Ġ: :: ĠFalse ĠP osit ives </s> had Ġfour Ġmain Ġtypes : </s> [ left margin = * ] </s> G lo ating : Ġex . ĠÙ Ĭ Ø§ ĠÙ ĩ Ø¨ ÙĬ Ø¯ Ùĩ > Ġ( âĢ ľ y A Ġh by dp âĢ Ŀ Ġ- ĠâĢ ľ O Ġyou Ġdelusional âĢ Ŀ ) Ġreferring Ġto Ġfans Ġof Ġrival Ġsports Ġteam Ġfor Ġthinking Ġthey Ġcould Ġwin . </s> Qu oting : Ġex . Ġ ÙĦ Ùħ Ø§ ĠØ Ń Ø¯ ĠÙ Ĭ Ø³ Ø¨ ĠÙĪ ÙĬ Ù Ĥ ÙĪ ÙĦ ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ > Ġ( âĢ ľ l mA ĠH d Ġy sb Ġw y q wl Ġy A Ġk lb âĢ Ŀ ĠâĢĵ ĠâĢ ľ when Ġsomeone Ġsw ears Ġand Ġsays : ĠO Ġdog âĢ Ŀ ). </s> Id i oms : Ġex . ĠÙ Ĭ Ø§ ĠÙ ģ Ø§Ø · Ø± ĠØ ± Ùħ Ø ¶ Ø§ ÙĨ ĠÙ Ĭ Ø§ ĠØ ® Ø§Ø ³ Ø± ĠØ ¯ ÙĬ ÙĨ Ù ĥ > Ġ( âĢ ľ y A Ġf AT r Ġrm D An Ġy A Ġx As r Ġdyn k âĢ Ŀ ĠâĢĵ ĠâĢ ľ o Ġyou Ġwho Ġdoes Ġnot Ġfast ĠRamadan , Ġyou Ġwho Ġhave Ġlost Ġyour Ġreligion âĢ Ŀ ), Ġwhich Ġis Ġa Ġcoll oqu ial Ġid iom . </s> Impl icit ĠS arc asm : Ġex . ĠÙ Ĭ Ø§ ĠØ ® Ø§ ÙĬ ÙĨ Ġ Ø§ ÙĨ Øª ĠØ ¹ Ø§ ÙĬ Ø ² ĠØ ª Ø ´ Ù ĥ Ù ĥ > </s> Ù ģ ÙĬ ĠØ Ń Ø¨ ĠØ§ÙĦ Ø ´ Ø¹ Ø¨ Ġ ÙĦ ÙĦ Ø± ÙĬ Ø³ > Ġ( âĢ ľ y A ĠK A yn ĠAnt ĠEA w z Ġt $ kk Ġf y ĠH b ĠAl $ E b Ġll rys âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġtraitor , Ġ( you ) Ġwant Ġto Ġquestion Ġthe Ġlove Ġof Ġthe Ġpeople Ġto Ġthe Ġpresident ĠâĢ Ŀ ) Ġwhere Ġthe Ġauthor Ġis Ġmocking Ġthe Ġpresident 's Ġpopularity . </s> Exper iments Ġ: :: ĠError ĠAnalysis Ġ: :: ĠFalse ĠNeg atives </s> had Ġtwo Ġtypes : </s> [ left margin = * ] </s> M ixture Ġof Ġoff ens iveness Ġand Ġadmiration : Ġex . Ġcalling Ġa Ġgirl Ġa Ġpuppy ĠÙ Ĭ Ø§ ĠÙ ĥ ÙĦ Ø¨ ÙĪ Ø¨ Ø© > Ġ( âĢ ľ y A Ġk lb wb p âĢ Ŀ ĠâĢĵ ĠâĢ ľ O Ġpuppy âĢ Ŀ ) Ġin Ġa Ġflirt at ious Ġmanner . </s> Impl icit Ġoff ens iveness : Ġex . Ġcalling Ġfor Ġcure Ġwhile Ġimplying Ġsanity Ġin : ĠÙĪ Øª Ø ´ Ù ģ ÙĬ ĠØ Ń Ù ĥ Ø§ Ùħ ĠÙ Ĥ Ø · Ø± ĠÙħ ÙĨ ĠØ§ÙĦ Ùħ Ø± Ø ¶ > Ġ( âĢ ľ wt $ fy ĠH k Am Ġq Tr Ġm n ĠAlm r D âĢ Ŀ ĠâĢĵ ĠâĢ ľ and Ġcure ĠQatar Ġrulers Ġfrom Ġillness âĢ Ŀ ). </s> Many Ġerrors Ġstem Ġfrom Ġheavy Ġuse Ġof Ġdialect al ĠArabic Ġas Ġwell Ġas Ġambiguity . ĠSince ĠB ERT Ġwas Ġtrained Ġon ĠWikipedia Ġ( M SA ) Ġand ĠGoogle Ġbooks , Ġthe Ġmodel Ġfailed Ġto Ġclassify Ġtweets Ġwith Ġdialect al Ġcues . ĠConversely , ĠMaz aj ak / S VM Ġis Ġmore Ġbiased Ġtowards Ġdialect s , Ġoften Ġfailing Ġto Ġclassify ĠM SA Ġtweets . </s> Conclusion Ġand ĠFuture ĠWork </s> In Ġthis Ġpaper Ġwe Ġpresented Ġa Ġsystematic Ġmethod Ġfor Ġbuilding Ġan ĠArabic Ġoffensive Ġlanguage Ġtweet Ġdataset Ġthat Ġdoes Ġnot Ġfavor Ġspecific Ġdialect s , Ġtopics , Ġor Ġgenres . ĠWe Ġdeveloped Ġdetailed Ġguidelines Ġfor Ġtagging Ġthe Ġtweets Ġas Ġclean Ġor Ġoffensive , Ġincluding Ġspecial Ġtags Ġfor Ġvulgar Ġtweets Ġand Ġhate Ġspeech . ĠWe Ġtagged Ġ10 , 000 Ġtweets , Ġwhich Ġwe Ġplan Ġto Ġrelease Ġpublicly Ġand Ġwould Ġconstitute Ġthe Ġlargest Ġavailable ĠArabic Ġoffensive Ġlanguage Ġdataset . ĠWe Ġcharacterized Ġthe Ġoffensive Ġtweets Ġin Ġthe Ġdataset Ġto Ġdetermine Ġthe Ġtopics Ġthat Ġillicit Ġsuch Ġlanguage , Ġthe Ġdialect s Ġthat Ġare Ġmost Ġoften Ġused , Ġthe Ġcommon Ġmodes Ġof Ġoff ens iveness , Ġand Ġthe Ġgender Ġdistribution Ġof Ġtheir Ġauthors . ĠWe Ġperformed Ġthis Ġbreakdown Ġfor Ġoffensive Ġtweets Ġin Ġgeneral Ġand Ġfor Ġvulgar Ġand Ġhate Ġspeech Ġtweets Ġseparately . ĠWe Ġbelieve Ġthat Ġthis Ġis Ġthe Ġfirst Ġdetailed Ġanalysis Ġof Ġits Ġkind . ĠLastly , Ġwe Ġconducted Ġa Ġlarge Ġbattery Ġof Ġexperiments Ġon Ġthe Ġdataset , Ġusing Ġcross - valid ation , Ġto Ġestablish Ġa Ġstrong Ġsystem Ġfor ĠArabic Ġoffensive Ġlanguage Ġdetection . ĠWe Ġshowed Ġthat Ġusing Ġstatic Ġembed d ings Ġproduced Ġa Ġcompetitive Ġresults Ġon Ġthe Ġdataset . </s> For Ġfuture Ġwork , Ġwe Ġplan Ġto Ġpursue Ġseveral Ġdirections . ĠFirst , Ġwe Ġwant Ġexplore Ġtarget Ġspecific Ġoffensive Ġlanguage , Ġwhere Ġattacks Ġagainst Ġan Ġentity Ġor Ġa Ġgroup Ġmay Ġemploy Ġcertain Ġexpressions Ġthat Ġare Ġonly Ġoffensive Ġwithin Ġthe Ġcontext Ġof Ġthat Ġtarget Ġand Ġcompletely Ġinnocuous Ġotherwise . ĠSecond , Ġwe Ġplan Ġto Ġexamine Ġthe Ġeffectiveness Ġof Ġcross Ġdialect al Ġand Ġcross Ġling ual Ġlearning Ġof Ġoffensive Ġlanguage .