tweets from the past two years from domains like `sports', `politics', `entertainment'
EXTRACTIVE
twitter
EXTRACTIVE
Python package twitterscraper is used to scrap tweets from twitter. 10,478 tweets from the past two years from domains like `sports', `politics', `entertainment' were extracted. Among those tweets, we manually removed the tweets which were written either in English or Hindi entirely. There were 4161 tweets written in English and 2774 written in Hindi. Finally, a total of 3543 English-Hindi code-mixed tweets were collected. Table 1 describes the number of tweets and words in each category.
Python package twitterscraper is used to scrap tweets from twitter. 10,478 tweets from the past two years from domains like `sports', `politics', `entertainment' were extracted. Among those tweets, we manually removed the tweets which were written either in English or Hindi entirely. There were 4161 tweets written in English and 2774 written in Hindi. Finally, a total of 3543 English-Hindi code-mixed tweets were collected. Table 1 describes the number of tweets and words in each category.
1806.05513
Introduction </s> âĢ ľ L aughter Ġis Ġthe Ġbest ĠMedicine âĢ Ŀ Ġis Ġa Ġsaying Ġwhich Ġis Ġpopular Ġwith Ġmost Ġof Ġthe Ġpeople . ĠHum or Ġis Ġa Ġform Ġof Ġcommunication Ġthat Ġbridges Ġthe Ġgap Ġbetween Ġvarious Ġlanguages , Ġcultures , Ġages Ġand Ġdemographics . ĠThat 's Ġwhy Ġhumorous Ġcontent Ġwith Ġfunny Ġand Ġwitty Ġhasht ags Ġare Ġso Ġmuch Ġpopular Ġon Ġsocial Ġmedia . ĠIt Ġis Ġa Ġvery Ġpowerful Ġtool Ġto Ġconnect Ġwith Ġthe Ġaudience . ĠAutomatic ĠHum or ĠRecogn ition Ġis Ġthe Ġtask Ġof Ġdetermining Ġwhether Ġa Ġtext Ġcontains Ġsome Ġlevel Ġof Ġhumorous Ġcontent Ġor Ġnot . ĠFirst Ġconference Ġon ĠComput ational Ġhumor Ġwas Ġorganized Ġin Ġ1996 , Ġsince Ġthen Ġmany Ġresearch Ġhave Ġbeen Ġdone Ġin Ġthis Ġfield . Ġk ao 2016 com put ational Ġdoes Ġpun Ġdetection Ġin Ġone - liners Ġand Ġde hum or Ġdetects Ġhumor Ġin ĠYelp Ġreviews . ĠBecause Ġof Ġthe Ġcomplex Ġand Ġinteresting Ġaspects Ġinvolved Ġin Ġdetecting Ġhumor Ġin Ġtexts , Ġit Ġis Ġone Ġof Ġthe Ġchallenging Ġresearch Ġfield Ġin ĠNatural ĠLanguage ĠProcessing ĠB IB REF 3 Ġ. ĠIdent ifying Ġhumor Ġin Ġa Ġsentence Ġsometimes Ġrequire Ġa Ġgreat Ġamount Ġof Ġexternal Ġknowledge Ġto Ġcompletely Ġunderstand Ġit . ĠThere Ġare Ġmany Ġtypes Ġof Ġhumor , Ġnamely Ġanecdotes , Ġfantasy , Ġinsult , Ġirony , Ġjokes , Ġquote , Ġself Ġdep rec ation Ġetc ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ. ĠMost Ġof Ġthe Ġtimes Ġthere Ġare Ġdifferent Ġmeanings Ġhidden Ġinside Ġa Ġsentence Ġwhich Ġis Ġgrasped Ġdifferently Ġby Ġindividuals , Ġmaking Ġthe Ġtask Ġof Ġhumor Ġidentification Ġdifficult , Ġwhich Ġis Ġwhy Ġthe Ġdevelopment Ġof Ġa Ġgeneralized Ġalgorithm Ġto Ġclassify Ġdifferent Ġtype Ġof Ġhumor Ġis Ġa Ġchallenging Ġtask . </s> Major ity Ġof Ġthe Ġresear ches Ġon Ġsocial Ġmedia Ġtexts Ġis Ġfocused Ġon ĠEnglish . ĠA Ġstudy Ġby Ġsch ro eder 2010 half Ġshows Ġthat , Ġa Ġhigh Ġpercentage Ġof Ġthese Ġtexts Ġare Ġin Ġnon - English Ġlanguages . Ġf ischer 2011 language Ġgives Ġsome Ġinteresting Ġinformation Ġabout Ġthe Ġlanguages Ġused Ġon ĠTwitter Ġbased Ġon Ġthe Ġgeographical Ġlocations . ĠWith Ġa Ġhuge Ġamount Ġof Ġsuch Ġuser Ġgenerated Ġdata Ġavailable Ġon Ġsocial Ġmedia , Ġthere Ġis Ġa Ġneed Ġto Ġdevelop Ġtechnologies Ġfor Ġnon - English Ġlanguages . ĠIn Ġmult ilingual Ġregions Ġlike ĠSouth ĠAsia , Ġmajority Ġof Ġthe Ġsocial Ġmedia Ġusers Ġspeak Ġmore Ġthan Ġtwo Ġlanguages . ĠIn ĠIndia , ĠHindi Ġis Ġthe Ġmost Ġspoken Ġlanguage Ġ( spoken Ġby Ġ41 % Ġof Ġthe Ġpopulation ) Ġand ĠEnglish Ġis Ġthe Ġofficial Ġlanguage Ġof Ġthe Ġcountry . ĠTwitter Ġhas Ġaround Ġ23 . 2 Ġmillion Ġmonthly Ġactive Ġusers Ġin ĠIndia . ĠNative Ġspeakers Ġof ĠHindi Ġoften Ġput ĠEnglish Ġwords Ġin Ġthe Ġsentences Ġand Ġtransl iter ate Ġthe Ġwhole Ġsentence Ġto ĠLatin Ġscript Ġwhile Ġposting Ġon Ġsocial Ġmedia , Ġthereby Ġmaking Ġthe Ġtask Ġof Ġautomatic Ġtext Ġclassification Ġa Ġvery Ġchallenging Ġproblem . ĠL ingu ists Ġcame Ġup Ġwith Ġa Ġterm Ġfor Ġany Ġtype Ġof Ġlanguage Ġmixing , Ġknown Ġas Ġ` code - mix ing ' Ġor Ġ` code - sw itching ' ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġ. ĠBoth Ġthe Ġterms Ġare Ġused Ġinterchange ably , Ġbut Ġthere Ġis Ġa Ġslight Ġdifference Ġbetween Ġthe Ġtwo Ġterms . ĠCode - mix ing Ġrefers Ġto Ġthe Ġinsertion Ġof Ġwords , Ġphrases , Ġand Ġmorp hem es Ġof Ġone Ġlanguage Ġinto Ġa Ġstatement Ġor Ġan Ġexpression Ġof Ġanother Ġlanguage , Ġwhereas Ġtransl iter ation Ġof Ġevery Ġword Ġin Ġa Ġsentence Ġto Ġanother Ġscript Ġ( Ġhere ĠDev an ag ari Ġto ĠLatin ) Ġis Ġcoined Ġcode - sw itching ĠB IB REF 10 Ġ. ĠThe Ġfirst Ġtweet Ġin ĠFigure Ġ1 Ġis Ġan Ġexample Ġof Ġcode - mix ing Ġand Ġsecond Ġis Ġan Ġexample Ġof Ġcode - sw itching . ĠIn Ġthis Ġpaper , Ġwe Ġuse Ġcode - mix ing Ġto Ġdenote Ġboth Ġcases . </s> In Ġthis Ġpaper , Ġwe Ġpresent Ġa Ġfreely Ġavailable Ġcorpus Ġcontaining Ġcode - m ixed Ġtweets Ġin ĠHindi Ġand ĠEnglish Ġlanguage Ġwith Ġtweets Ġwritten Ġin ĠLatin Ġscript . ĠTwe ets Ġare Ġmanually Ġclassified Ġinto Ġhumorous Ġand Ġnon - hum orous Ġclasses . ĠMoreover , Ġeach Ġtoken Ġin Ġthe Ġtweets Ġis Ġalso Ġgiven Ġa Ġlanguage Ġtag Ġwhich Ġdetermines Ġthe Ġsource Ġor Ġorigin Ġlanguage Ġof Ġthe Ġtoken Ġ( English Ġor ĠHindi ). ĠThe Ġpaper Ġis Ġdivided Ġin Ġsections Ġas Ġfollows , Ġwe Ġstart Ġby Ġdescribing Ġthe Ġcorpus Ġand Ġthe Ġannotation Ġscheme Ġin ĠSection Ġ2 . ĠSection Ġ3 Ġsummarizes Ġour Ġsupervised Ġclassification Ġsystem Ġwhich Ġincludes Ġpre - processing Ġof Ġthe Ġtweets Ġin Ġthe Ġdataset Ġand Ġthe Ġfeature Ġextraction Ġfollowed Ġby Ġthe Ġmethod Ġused Ġto Ġidentify Ġhumor Ġin Ġtweets . ĠIn Ġthe Ġnext Ġsubsection , Ġwe Ġdescribe Ġthe Ġclassification Ġmodel Ġand Ġthe Ġresults Ġof Ġthe Ġexperiments Ġconducted Ġusing Ġcharacter Ġand Ġword Ġlevel Ġfeatures . ĠIn Ġthe Ġlast Ġsection , Ġwe Ġconclude Ġthe Ġpaper Ġfollowed Ġby Ġfuture Ġwork Ġand Ġreferences . </s> Corp us ĠCreation Ġand ĠAn notation </s> In Ġthis Ġsection , Ġwe Ġexplain Ġthe Ġtechniques Ġused Ġin Ġthe Ġcreation Ġand Ġannotation Ġof Ġthe Ġcorpus . </s> Data ĠCollection </s> Python Ġpackage Ġtw itters c raper Ġis Ġused Ġto Ġscrap Ġtweets Ġfrom Ġtwitter . Ġ10 , 478 Ġtweets Ġfrom Ġthe Ġpast Ġtwo Ġyears Ġfrom Ġdomains Ġlike Ġ` sports ', Ġ` politics ', Ġ` ent ertain ment ' Ġwere Ġextracted . ĠAmong Ġthose Ġtweets , Ġwe Ġmanually Ġremoved Ġthe Ġtweets Ġwhich Ġwere Ġwritten Ġeither Ġin ĠEnglish Ġor ĠHindi Ġentirely . ĠThere Ġwere Ġ4 161 Ġtweets Ġwritten Ġin ĠEnglish Ġand Ġ27 74 Ġwritten Ġin ĠHindi . ĠFinally , Ġa Ġtotal Ġof Ġ35 43 ĠEnglish - H ind i Ġcode - m ixed Ġtweets Ġwere Ġcollected . ĠTable Ġ1 Ġdescribes Ġthe Ġnumber Ġof Ġtweets Ġand Ġwords Ġin Ġeach Ġcategory . </s> We Ġtried Ġto Ġmake Ġthe Ġcorpus Ġbalanced Ġi . e . Ġuniform Ġdistribution Ġof Ġtweets Ġin Ġeach Ġcategory Ġto Ġyield Ġbetter Ġsupervised Ġclassification Ġresults Ġas Ġdescribed Ġby ĠB IB REF 11 Ġ. </s> Hum or ĠAn notation </s> The Ġfinal Ġcode - m ixed Ġtweets Ġwere Ġforwarded Ġto Ġa Ġgroup Ġof Ġthree Ġannot ators Ġwho Ġwere Ġuniversity Ġstudents Ġand Ġfluent Ġin Ġboth ĠEnglish Ġand ĠHindi . ĠApproximately Ġ60 Ġhours Ġwere Ġspent Ġin Ġtagging Ġtweets Ġfor Ġthe Ġpresence Ġof Ġhumor . ĠTwe ets Ġwhich Ġconsisted Ġof Ġany Ġanecdotes , Ġfantasy , Ġirony , Ġjokes , Ġinsults Ġwere Ġannot ated Ġas Ġhumorous Ġwhereas Ġtweets Ġstating Ġany Ġfacts , Ġdialog ues Ġor Ġspeech Ġwhich Ġdid Ġnot Ġcontain Ġamusement Ġwere Ġput Ġin Ġnon - hum orous Ġclass . ĠFollowing Ġare Ġsome Ġexamples Ġof Ġcode - m ixed Ġtweets Ġin Ġthe Ġcorpus : </s> âĢ ľ For Ġ# W ont Give It Back Ġto Ġwork , ĠDh oni Ġneeds Ġto Ġsay Ġ` Tro phy Ġto h Ġg har Ġpar Ġhi Ġb h ul Ġa aye ' ĠâĢ Ŀ </s> ( For Ġ# W ont Give It Back Ġto Ġwork , ĠDh oni Ġneeds Ġto Ġsay ĠâĢ ľ We Ġforgot Ġthe Ġtrophy Ġat Ġhome âĢ Ŀ ) Ġ. </s> Na Ġsad ak Ġna Ġn au k ri Ġbas Ġbad h Ġr ahi Ġgu nd ag ardi Ġ# F ailed CM Nit ish </s> ( No Ġroads Ġno Ġn au k ri , Ġonly Ġh ool igan ism Ġincreasing Ġ# F ailed CM Nit ish Ġ) </s> Sub ha Ġka Ġb h ula Ġag ar Ġsham Ġko Ġw ap as Ġg har Ġa a Ġj aye Ġthen Ġwe Ġmust Ġthank ĠGPS Ġtechnology . </s> ( If Ġsomeone Ġlost Ġin Ġthe Ġmorning Ġreturn Ġhome Ġin Ġthe Ġevening Ġthen Ġwe Ġmust Ġthank ĠGPS Ġtechnology .) </s> She Ġ: ĠG ori Ġha i Ġk al ai yan Ġpe h na Ġde Ġm uj he Ġh ari Ġh ari Ġch ud iya ĠHe Ġ* gets Ġgreen Ġbang les * ĠShe Ġ: ĠNot Ġthis Ġgreen Ġya , Ġbottle Ġgreen Ġcolor . </s> ( She Ġ: Ġ( s ings Ġa Ġsong ) ĠMy Ġwrists Ġare Ġso Ġbeautiful , Ġgive Ġme Ġgreen Ġgreen Ġbang les ĠHe Ġ* gets Ġgreen Ġbang les * ĠShe Ġ: ĠNot Ġthis Ġgreen Ġya , Ġbottle Ġgreen Ġcolor . Ġ) </s> D ard Ġdil on Ġke Ġk am Ġho Ġja ate ... Ġtwitter Ġpar Ġag ar Ġpoet ries Ġk am Ġho Ġja ate </s> ( pain Ġin Ġthe Ġheart Ġwill Ġreduce ... if Ġnumber Ġof Ġpoet ries Ġdecreases Ġon Ġtwitter ). </s> An not ators Ġwere Ġgiven Ġcertain Ġguidelines Ġto Ġdecide Ġwhether Ġa Ġtweet Ġwas Ġhumorous Ġor Ġnot . ĠThe Ġcontext Ġof Ġthe Ġtweet Ġcould Ġbe Ġfound Ġby Ġsearching Ġabout Ġhashtag Ġor Ġkeywords Ġused Ġin Ġthe Ġtweet . ĠExample Ġ( 1 ) Ġuses Ġa Ġhashtag Ġ` # W ont Give It Back ' Ġwhich Ġwas Ġtrending Ġduring Ġthe ĠICC Ġcricket Ġworld Ġcup Ġ2015 . ĠSearch ing Ġit Ġon ĠGoogle Ġgave Ġ435 k Ġresults Ġand Ġthe Ġtime Ġof Ġthe Ġtweet Ġwas Ġafter Ġthe Ġfinal Ġmatch Ġof Ġthe Ġtournament . ĠSo Ġthere Ġis Ġan Ġobservational Ġhumor Ġin Ġ( 1 ) Ġas ĠIndia Ġwon Ġthe Ġworld Ġcup Ġin Ġ2011 Ġand Ġlost Ġin Ġ2015 Ġ, Ġhence Ġthe Ġtweet Ġwas Ġclassified Ġas Ġhumorous . ĠAny Ġtweets Ġstating Ġany Ġfacts , Ġnews Ġor Ġreality Ġwere Ġclassified Ġas Ġnon - hum orous . ĠThere Ġwere Ġmany Ġtweets Ġwhich Ġdid Ġnot Ġcontain Ġany Ġhasht ags , Ġto Ġunderstand Ġthe Ġcontext Ġof Ġsuch Ġtweets Ġannot ators Ġselected Ġsome Ġkeywords Ġfrom Ġthe Ġtweet Ġand Ġsearched Ġthem Ġonline . ĠExample Ġ( 2 ) Ġcontains Ġa Ġcomment Ġtowards Ġa Ġpolitical Ġleader Ġtowards Ġdevelopment Ġand Ġwas Ġcategorized Ġas Ġnon - hum orous . ĠTwe ets Ġcontaining Ġnormal Ġjokes Ġand Ġfunny Ġquotes Ġlike Ġin Ġ( 3 ) Ġand Ġ( 4 ) Ġwere Ġput Ġin Ġhumorous Ġcategory . ĠThere Ġwere Ġsome Ġtweets Ġlike Ġ( 5 ) Ġwhich Ġconsists Ġof Ġpoem Ġor Ġlines Ġof Ġa Ġsong Ġbut Ġmodified . ĠAn not ators Ġwere Ġguided Ġthat Ġif Ġsuch Ġtweets Ġcontains Ġsatire Ġor Ġany Ġhumor istic Ġfeatures , Ġthen Ġit Ġcould Ġbe Ġcategorized Ġas Ġhumorous Ġotherwise Ġnot . ĠThere Ġwere Ġsome Ġtweets Ġwhich Ġwere Ġtypical Ġto Ġcategor ize Ġlike Ġ( 5 ), Ġhence Ġit Ġwas Ġleft Ġto Ġthe Ġannot ators Ġto Ġthe Ġbest Ġof Ġtheir Ġunderstanding . ĠBased Ġon Ġthe Ġabove Ġguidelines Ġannot ators Ġcategorized Ġthe Ġtweets . ĠTo Ġmeasure Ġinter Ġannot ator Ġagreement Ġwe Ġopted Ġfor ĠFle iss ' ĠKappa ĠB IB REF 12 Ġobtaining Ġan Ġagreement Ġof Ġ0 . 8 21 Ġ. ĠBoth Ġhumorous Ġand Ġnon - hum orous Ġtweets Ġin Ġnearly Ġbalanced Ġamount Ġwere Ġselected Ġto Ġprepare Ġthe Ġcorpus . ĠIf Ġwe Ġhad Ġincluded Ġhumorous Ġtweets Ġfrom Ġone Ġdomain Ġlike Ġsports Ġand Ġnon Ġhumorous Ġtweets Ġfrom Ġanother Ġdomain Ġlike Ġnews Ġthen , Ġit Ġwould Ġhave Ġgiven Ġhigh Ġperformance Ġof Ġclassification ĠB IB REF 13 Ġ. ĠTo Ġclassify Ġbased Ġon Ġthe Ġsemantics Ġand Ġnot Ġon Ġthe Ġdomain Ġdifferences , Ġwe Ġincluded Ġboth Ġtypes Ġof Ġtweets Ġfrom Ġdifferent Ġdomains . ĠMany Ġtweets Ġcontains Ġa Ġpicture Ġalong Ġwith Ġa Ġcaption . ĠSometimes Ġa Ġcaption Ġmay Ġnot Ġcontain Ġhumor Ġbut Ġcombined Ġwith Ġthe Ġpicture , Ġit Ġcan Ġprovide Ġsome Ġdegree Ġof Ġhumor . ĠSuch Ġtweets Ġwere Ġremoved Ġfrom Ġthe Ġcorpus Ġto Ġmake Ġthe Ġcorpus Ġunim od al . ĠIn ĠFigure Ġ1 , Ġthe Ġfirst Ġtweet , ĠâĢ ľ An ur ag ĠKash y ap Ġcan Ġnever Ġjoin ĠAAP Ġbecause Ġministers Ġtook Ġoath Ġ` main Ġk isi ĠAn ur ag Ġaur Ġdw esh Ġke Ġb ina Ġka am Ġk ar ung a ' ĠâĢ Ŀ Ġ( An ur ag ĠKash y ap Ġcan Ġnever Ġjoin ĠAAP Ġbecause Ġministers Ġtook Ġoath Ġ` I Ġwill Ġwork Ġwithout Ġany Ġaffection Ġ( An ur ag Ġin ĠHindi ) Ġand Ġwithout Ġhesitation Ġ( d w esh Ġin ĠHindi )' ), Ġwas Ġclassified Ġas Ġhumorous . ĠThe Ġsecond Ġtweet , ĠâĢ ľ # S ak shi Mal ik Ġtake Ġa Ġbow ! Ġ# p roud Indian Ġ# R io Ġ# O lymp ics Ġ# Bron ze Med al Ġ# girl power ĠH ama ara Ġk ha ata Ġkh ul ĠG aya ! âĢ Ŀ Ġ(# S ak shi Mal ik Ġtake Ġa Ġbow ! Ġ# p roud Indian Ġ# R io Ġ# O lymp ics Ġ# Bron ze Med al Ġ# girl power ĠOur Ġaccount Ġopened !) Ġwas Ġclassified Ġas Ġnon - hum orous Ġas Ġit Ġcontains Ġa Ġpride Ġstatement . </s> Language ĠAn notation </s> Code - mix ing Ġprovides Ġsome Ġchallenges Ġfor Ġlanguage Ġidentification Ġlike Ġspelling Ġvariations Ġand Ġnon - gram matical Ġstructures Ġin Ġa Ġsentence ĠB IB REF 0 Ġ. ĠTherefore , Ġwe Ġannot ated Ġthe Ġtweets Ġwith Ġthe Ġlanguage Ġat Ġthe Ġword Ġlevel . ĠNative Ġspeakers Ġof ĠHindi Ġand Ġproficient Ġin ĠEnglish , Ġlabelled Ġthe Ġlanguage Ġof Ġthe Ġtokens Ġin Ġthe Ġtweets . ĠThree Ġtypes Ġof Ġtags Ġwere Ġassigned Ġto Ġthe Ġtokens Ġ, ĠEn Ġis Ġassigned Ġto Ġthe Ġtokens Ġpresent Ġin ĠEnglish Ġvocabulary Ġlike ĠâĢ ľ family âĢ Ŀ , ĠâĢ ľ Children âĢ Ŀ Ġetc . ĠSimilarly , ĠHi Ġis Ġassigned Ġto Ġthe Ġtokens Ġpresent Ġin ĠHindi Ġvocabulary Ġbut Ġtransl iter ated Ġto ĠLatin Ġscript Ġlike ĠâĢ ľ sam ay âĢ Ŀ Ġ( time ), ĠâĢ ľ a ama ad mi âĢ Ŀ Ġ( common Ġman ). ĠRest Ġof Ġthe Ġtokens Ġconsists Ġof Ġproper Ġnoun s , Ġnumbers , Ġdates , Ġur ls , Ġhasht ags , Ġmentions , Ġem oj is Ġand Ġpunct uations Ġwhich Ġare Ġlabelled Ġas ĠOt ( other s ). ĠMajor Ġconcern Ġin Ġlanguage Ġannotation Ġwas Ġto Ġannot ate Ġthe Ġwords Ġpresent Ġin Ġboth Ġlanguages Ġ( amb iguous Ġwords ). ĠFor Ġexample , Ġ` to ' Ġ( ` but ' Ġin ĠHindi ) Ġand Ġ` is ' Ġ( ` this ' Ġin ĠHindi ) Ġ, Ġfor Ġthis Ġscenario Ġannot ators Ġunderstood Ġthe Ġcontext Ġof Ġthe Ġtweet Ġand Ġbased Ġon Ġthat Ġthe Ġwords Ġwere Ġbeing Ġannot ated . ĠFor Ġexample , Ġconsider Ġtwo Ġsentences Ġ` This Ġplace Ġis Ġ500 Ġyears Ġold ' Ġand Ġ` Is Ġbar ĠModi ĠSark ar ' ( This Ġtime ĠModi ĠGovernment ). ĠSo Ġ` is ' Ġin Ġfirst Ġsentence Ġshould Ġbe Ġtagged Ġas ĠEnglish Ġbecause Ġit Ġis Ġused Ġas Ġa Ġverb . ĠIt Ġis Ġtransl iter ated Ġto ĠEnglish Ġin Ġlatter Ġsentence Ġand Ġused Ġas Ġdetermin er Ġ( ` this ') Ġso Ġit Ġwas Ġtagged Ġas ĠHindi . ĠSo Ġthese Ġkind Ġof Ġwords Ġwere Ġtagged Ġbased Ġon Ġtheir Ġmeanings Ġand Ġusage Ġin Ġboth Ġlanguages . ĠFigure Ġ1 Ġdescribes Ġthe Ġlanguage Ġannotation Ġof Ġtweets Ġin Ġthe Ġcorpus . </s> An notation ĠScheme </s> Two Ġexample Ġannotations Ġare Ġillustrated Ġin ĠFigure Ġ1 . ĠFirst Ġline Ġin Ġevery Ġannotation Ġconsists Ġof Ġtweet Ġid . ĠEach Ġtweet Ġis Ġenclosed Ġwithin Ġ< t weet ></ t weet > Ġtags Ġand Ġeach Ġword Ġis Ġenclosed Ġwithin Ġ< word Ġlang = âĢ ľ ĠâĢ Ŀ ></ word > Ġcontaining Ġthe Ġlanguage Ġannotation Ġfor Ġeach Ġword . ĠLast Ġline Ġdetermines Ġthe Ġcategory Ġin Ġwhich Ġthe Ġtweet Ġbelongs Ġi . e . Ġhumorous Ġor Ġnon - hum orous , Ġenclosed Ġby Ġ< class ></ class > Ġtags . ĠThe Ġannot ated Ġdataset Ġwith Ġthe Ġclassification Ġsystem Ġis Ġmade Ġavailable Ġonline Ġ. </s> Error ĠAnalysis </s> Social Ġmedia Ġusers Ġoften Ġmake Ġspelling Ġmistakes Ġor Ġuse Ġmultiple Ġvariations Ġof Ġa Ġword Ġwhile Ġposting Ġa Ġtext . ĠWe Ġreplaced Ġall Ġsuch Ġerrors Ġwith Ġtheir Ġcorrect Ġversion . ĠFor Ġwords Ġin ĠHindi Ġthat Ġwere Ġtransl iter ated Ġto ĠLatin Ġscript , Ġwe Ġadopted Ġa Ġcommon Ġspelling Ġfor Ġall Ġthose Ġwords Ġacross Ġthe Ġcorpus . ĠFor Ġexample , Ġ` dis ' Ġis Ġoften Ġused Ġas Ġa Ġshort Ġform Ġfor Ġ` this ', Ġso Ġwe Ġreplaced Ġevery Ġoccurrence Ġof Ġ` dis ' Ġto Ġ` this ' Ġin Ġthe Ġcorpus . ĠSome Ġexamples Ġof Ġspelling Ġvariations Ġare , Ġ` p ese ' Ġfor Ġ` pa isa ' Ġ( unit Ġof Ġcur ren y ), Ġ` h ' Ġfor Ġ` hai ' Ġ( is ), Ġ` ad ' Ġfor Ġ` add ' Ġ( add ition ). </s> System ĠArchitecture </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġour Ġmachine Ġlearning Ġmodel Ġwhich Ġis Ġtrained Ġand Ġtested Ġon Ġthe Ġcorpus Ġdescribed Ġin Ġthe Ġprevious Ġsection . </s> Pre - processing Ġof ĠCorpus </s> Pre processing Ġstarts Ġwith Ġtoken ization , Ġwhich Ġinvolves Ġseparation Ġof Ġwords Ġusing Ġspace Ġas Ġthe Ġdelim iter Ġand Ġthen Ġconverting Ġthe Ġwords Ġto Ġlower Ġcases Ġwhich Ġis Ġfollowed Ġby Ġthe Ġremoval Ġof Ġpunct uation Ġmarks Ġfrom Ġthe Ġtoken ized Ġtweet . ĠAll Ġthe Ġhasht ags , Ġmentions Ġand Ġur ls , Ġare Ġstored Ġand Ġconverted Ġto Ġ` hash tag ', Ġ` ment ion ' Ġand Ġ` url ' Ġrespectively . ĠSometimes Ġhasht ags Ġprovide Ġsome Ġdegree Ġof Ġhumor Ġin Ġtweets , Ġhence Ġwe Ġsegregated Ġhasht ags Ġon Ġthe Ġbasis Ġof Ġcamel Ġcases Ġand Ġincluded Ġthe Ġtokens Ġin Ġthe Ġtoken ized Ġtweets Ġ( hash tag Ġdecom position ) ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ. Ġfor Ġexample , Ġ# A ad ab Ar z H ai Ġcan Ġbe Ġdecomp osed Ġinto Ġ3 Ġwords , Ġ` A ad ab ', Ġ` Ar z ' Ġand Ġ` H ai '. ĠFinally Ġthe Ġtoken ized Ġtweets Ġare Ġstored Ġalong Ġwith Ġthe Ġpresence Ġof Ġhumor Ġas Ġthe Ġtarget Ġclass . </s> Class ification ĠFeatures </s> The Ġfeatures Ġused Ġto Ġbuild Ġattribute Ġvectors Ġfor Ġtraining Ġour Ġclassification Ġmodel Ġare Ġdescribed Ġbelow . ĠWe Ġuse Ġcharacter Ġlevel Ġand Ġword Ġlevel Ġfeatures Ġfor Ġthe Ġclassification ĠB IB REF 15 Ġ. ĠFor Ġall Ġthe Ġfeatures , Ġwe Ġseparated Ġthe Ġwords Ġin Ġthe Ġtweets Ġbased Ġon Ġthe Ġlanguage Ġannotation Ġ( Section Ġ2 . 3 ) Ġand Ġprepared Ġthe Ġfeature Ġvector Ġfor Ġeach Ġtweet Ġby Ġcombining Ġthe Ġvectors Ġfor Ġboth Ġthe Ġlanguages Ġ. </s> Previous Ġresear ches Ġshows Ġthat Ġletter Ġn - gram s Ġare Ġvery Ġefficient Ġfor Ġclass ifying Ġtext . ĠThey Ġare Ġlanguage Ġindependent Ġand Ġdoes Ġnot Ġrequire Ġexpensive Ġtext Ġpre - processing Ġtechniques Ġlike Ġtoken ization , Ġstemming Ġand Ġstop Ġwords Ġremoval , Ġhence Ġin Ġthe Ġcase Ġof Ġcode - mix Ġtexts , Ġthis Ġcould Ġyield Ġgood Ġresults ĠB IB REF 16 Ġ, ĠB IB REF 17 Ġ. ĠSince Ġthe Ġnumber Ġof Ġn - gram s Ġcan Ġbe Ġvery Ġlarge Ġwe Ġtook Ġtrig rams Ġwhich Ġoccur Ġmore Ġthan Ġten Ġtimes Ġin Ġthe Ġcorpus . </s> For Ġclass ifying Ġhumor Ġin Ġtexts , Ġit Ġis Ġimportant Ġto Ġunderstand Ġthe Ġsemantics Ġof Ġthe Ġsentence . ĠThus , Ġwe Ġtook Ġa Ġthree Ġword Ġwindow Ġas Ġa Ġfeature Ġto Ġtrain Ġour Ġclassification Ġmodels Ġto Ġincorporate Ġthe Ġcontextual Ġinformation . </s> Many Ġjokes Ġand Ġidi oms Ġsometimes Ġhave Ġcommon Ġwords . ĠWe Ġidentified Ġthose Ġwords Ġand Ġtook Ġthem Ġas Ġas Ġa Ġfeature Ġfor Ġclassification . ĠIn Ġthe Ġpre processing Ġstep , Ġwe Ġdecomp osed Ġhasht ags Ġusing Ġcamel Ġcases Ġand Ġadded Ġthem Ġalong Ġwith Ġthe Ġwords . ĠHence , Ġcommon Ġwords Ġin Ġthe Ġhasht ags Ġwere Ġalso Ġincluded Ġin Ġthe Ġfeature Ġvector . </s> Class ification ĠApproach Ġand ĠResults </s> We Ġexperimented Ġwith Ġfour Ġdifferent Ġclass ifiers , Ġnamely , Ġsupport Ġvector Ġmachine ĠB IB REF 18 Ġ, Ġrandom Ġforest , Ġextra Ġtree Ġand Ġnaive Ġbay es Ġclass ifier ĠB IB REF 19 Ġ. ĠChi Ġsquare Ġfeature Ġselection Ġalgorithm Ġis Ġapplied Ġto Ġreduces Ġthe Ġsize Ġof Ġour Ġfeature Ġvector . ĠFor Ġtraining Ġour Ġsystem Ġclass ifier , Ġwe Ġused ĠSci kit - learn ĠB IB REF 19 Ġ. </s> 10 - fold Ġcross Ġvalidation Ġon Ġ35 43 Ġcode - m ixed Ġtweets Ġwas Ġcarried Ġout Ġby Ġdividing Ġthe Ġcorpus Ġinto Ġ10 Ġequal Ġparts Ġwith Ġnine Ġparts Ġas Ġtraining Ġcorpus Ġand Ġrest Ġone Ġfor Ġtesting . ĠMean Ġaccuracy Ġis Ġcalculated Ġby Ġtaking Ġthe Ġaverage Ġof Ġthe Ġaccuracy Ġobtained Ġin Ġeach Ġiteration Ġof Ġthe Ġtesting Ġprocess . ĠTable Ġ2 Ġshows Ġthe Ġaccuracy Ġfor Ġeach Ġfeature Ġwhen Ġtrained Ġusing Ġmentioned Ġclass ifiers Ġalong Ġwith Ġthe Ġaccuracy Ġwhen Ġall Ġthe Ġfeatures Ġare Ġused Ġalong Ġwith Ġthe Ġoverall Ġaccuracy . ĠSupport Ġvector Ġmachine Ġwith Ġradial Ġbasis Ġfunction Ġkernel Ġand Ġextra Ġtree Ġclass ifier Ġperforms Ġbetter Ġthan Ġother Ġclass ifiers Ġand Ġyields Ġ69 . 3 % Ġand Ġ67 . 8 % Ġaccuracy Ġrespectively . ĠThe Ġreason Ġkernel ĠS VM Ġyields Ġthe Ġbest Ġresult Ġis Ġbecause Ġthe Ġnumber Ġof Ġobservations Ġis Ġgreat or Ġthan Ġthe Ġnumber Ġof Ġfeatures ĠB IB REF 20 Ġ. ĠN - gram s Ġproved Ġto Ġbe Ġthe Ġmost Ġefficient Ġin Ġall Ġclassification Ġmodels Ġfollowed Ġby Ġcommon Ġwords Ġand Ġhast ags . ĠBag - of - words Ġfeature Ġperformed Ġthe Ġworst Ġin ĠS VM , Ġrandom Ġforest Ġand Ġextra Ġtree Ġclass ifier Ġbut Ġyielded Ġbetter Ġresult Ġin Ġnaive Ġbay es Ġclass ifiers . ĠAcc ur acies Ġmentioned Ġin Ġtable Ġ2 Ġwere Ġcalculated Ġusing Ġfine Ġtuning Ġof Ġmodel Ġparameters Ġusing Ġgrid Ġsearch . </s> Conclusion Ġand ĠFuture ĠWork </s> In Ġthis Ġpaper , Ġwe Ġdescribe Ġa Ġfreely Ġavailable Ġcorpus Ġof Ġ3 453 ĠEnglish - H ind i Ġcode - m ixed Ġtweets . ĠThe Ġtweets Ġare Ġannot ated Ġwith Ġhumorous ( H ) Ġand Ġnon - hum orous ( N ) Ġtags Ġalong Ġwith Ġthe Ġlanguage Ġtags Ġat Ġthe Ġword Ġlevel . ĠThe Ġtask Ġof Ġhumor Ġidentification Ġin Ġsocial Ġmedia Ġtexts Ġis Ġanalyzed Ġas Ġa Ġclassification Ġproblem Ġand Ġseveral Ġmachine Ġlearning Ġclassification Ġmodels Ġare Ġused . ĠThe Ġfeatures Ġused Ġin Ġour Ġclassification Ġsystem Ġare Ġn - gram s , Ġbag - of - words , Ġcommon Ġwords Ġand Ġhasht ags . ĠN - gram s Ġwhen Ġtrained Ġwith Ġsupport Ġvector Ġmachines Ġwith Ġradial Ġbasis Ġfunction Ġkernel Ġperformed Ġbetter Ġthan Ġother Ġfeatures Ġand Ġyielded Ġan Ġaccuracy Ġof Ġ68 . 5 %. ĠThe Ġbest Ġaccuracy Ġ( 69 . 3 %) Ġwas Ġgiven Ġby Ġsupport Ġvector Ġmachines Ġwith Ġradial Ġbasis Ġfunction Ġkernel . </s> This Ġpaper Ġdescribes Ġthe Ġinitial Ġefforts Ġin Ġautomatic Ġhumor Ġdetection Ġin Ġcode - m ixed Ġsocial Ġmedia Ġtexts . ĠCorpus Ġcan Ġbe Ġannot ated Ġwith Ġpart - of - speech Ġtags Ġat Ġthe Ġword Ġlevel Ġwhich Ġmay Ġyield Ġbetter Ġresults Ġin Ġlanguage Ġdetection . ĠMoreover , Ġthe Ġdataset Ġcan Ġbe Ġfurther Ġextended Ġto Ġinclude Ġtweets Ġfrom Ġother Ġdomains . ĠCode - mix ing Ġis Ġvery Ġcommon Ġphenomenon Ġon Ġsocial Ġmedia Ġand Ġit Ġis Ġprevalent Ġmostly Ġin Ġmult ilingual Ġregions . ĠIt Ġwould Ġbe Ġinteresting Ġto Ġexperiment Ġwith Ġcode - m ixed Ġtexts Ġconsisting Ġof Ġmore Ġthan Ġtwo Ġlanguages Ġin Ġwhich Ġthe Ġissue Ġof Ġtransl iter ation Ġexists Ġlike ĠArabic , ĠGreek Ġand ĠSouth ĠAsian Ġlanguages . ĠComp aring Ġtraining Ġwith Ġcode - m ixed Ġtweets Ġwith Ġtraining Ġwith Ġa Ġmerged Ġcorpus Ġof Ġmon oling ual Ġtweets Ġin ĠEnglish Ġand ĠHindi Ġcould Ġbe Ġan Ġinteresting Ġfuture Ġwork .
Introduction “Laughter is the best Medicine” is a saying which is popular with most of the people. Humor is a form of communication that bridges the gap between various languages, cultures, ages and demographics. That's why humorous content with funny and witty hashtags are so much popular on social media. It is a very powerful tool to connect with the audience. Automatic Humor Recognition is the task of determining whether a text contains some level of humorous content or not. First conference on Computational humor was organized in 1996, since then many research have been done in this field. kao2016computational does pun detection in one-liners and dehumor detects humor in Yelp reviews. Because of the complex and interesting aspects involved in detecting humor in texts, it is one of the challenging research field in Natural Language Processing BIBREF3 . Identifying humor in a sentence sometimes require a great amount of external knowledge to completely understand it. There are many types of humor, namely anecdotes, fantasy, insult, irony, jokes, quote, self deprecation etc BIBREF4 , BIBREF5 . Most of the times there are different meanings hidden inside a sentence which is grasped differently by individuals, making the task of humor identification difficult, which is why the development of a generalized algorithm to classify different type of humor is a challenging task. Majority of the researches on social media texts is focused on English. A study by schroeder2010half shows that, a high percentage of these texts are in non-English languages. fischer2011language gives some interesting information about the languages used on Twitter based on the geographical locations. With a huge amount of such user generated data available on social media, there is a need to develop technologies for non-English languages. In multilingual regions like South Asia, majority of the social media users speak more than two languages. In India, Hindi is the most spoken language (spoken by 41% of the population) and English is the official language of the country. Twitter has around 23.2 million monthly active users in India. Native speakers of Hindi often put English words in the sentences and transliterate the whole sentence to Latin script while posting on social media, thereby making the task of automatic text classification a very challenging problem. Linguists came up with a term for any type of language mixing, known as `code-mixing' or `code-switching' BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 . Both the terms are used interchangeably, but there is a slight difference between the two terms. Code-mixing refers to the insertion of words, phrases, and morphemes of one language into a statement or an expression of another language, whereas transliteration of every word in a sentence to another script ( here Devanagari to Latin) is coined code-switching BIBREF10 . The first tweet in Figure 1 is an example of code-mixing and second is an example of code-switching. In this paper, we use code-mixing to denote both cases. In this paper, we present a freely available corpus containing code-mixed tweets in Hindi and English language with tweets written in Latin script. Tweets are manually classified into humorous and non-humorous classes. Moreover, each token in the tweets is also given a language tag which determines the source or origin language of the token (English or Hindi). The paper is divided in sections as follows, we start by describing the corpus and the annotation scheme in Section 2. Section 3 summarizes our supervised classification system which includes pre-processing of the tweets in the dataset and the feature extraction followed by the method used to identify humor in tweets. In the next subsection, we describe the classification model and the results of the experiments conducted using character and word level features. In the last section, we conclude the paper followed by future work and references. Corpus Creation and Annotation In this section, we explain the techniques used in the creation and annotation of the corpus. Data Collection Python package twitterscraper is used to scrap tweets from twitter. 10,478 tweets from the past two years from domains like `sports', `politics', `entertainment' were extracted. Among those tweets, we manually removed the tweets which were written either in English or Hindi entirely. There were 4161 tweets written in English and 2774 written in Hindi. Finally, a total of 3543 English-Hindi code-mixed tweets were collected. Table 1 describes the number of tweets and words in each category. We tried to make the corpus balanced i.e. uniform distribution of tweets in each category to yield better supervised classification results as described by BIBREF11 . Humor Annotation The final code-mixed tweets were forwarded to a group of three annotators who were university students and fluent in both English and Hindi. Approximately 60 hours were spent in tagging tweets for the presence of humor. Tweets which consisted of any anecdotes, fantasy, irony, jokes, insults were annotated as humorous whereas tweets stating any facts, dialogues or speech which did not contain amusement were put in non-humorous class. Following are some examples of code-mixed tweets in the corpus: “For #WontGiveItBack to work, Dhoni needs to say `Trophy toh ghar par hi bhul aaye' ” (For #WontGiveItBack to work, Dhoni needs to say “We forgot the trophy at home”) . Na sadak na naukri bas badh rahi gundagardi #FailedCMNitish (No roads no naukri, only hooliganism increasing #FailedCMNitish ) Subha ka bhula agar sham ko wapas ghar aa jaye then we must thank GPS technology. (If someone lost in the morning return home in the evening then we must thank GPS technology.) She : Gori hai kalaiyan pehna de mujhe hari hari chudiya He *gets green bangles* She : Not this green ya, bottle green color. (She : (sings a song) My wrists are so beautiful, give me green green bangles He *gets green bangles* She : Not this green ya, bottle green color. ) Dard dilon ke kam ho jaate... twitter par agar poetries kam ho jaate (pain in the heart will reduce...if number of poetries decreases on twitter). Annotators were given certain guidelines to decide whether a tweet was humorous or not. The context of the tweet could be found by searching about hashtag or keywords used in the tweet. Example (1) uses a hashtag `#WontGiveItBack' which was trending during the ICC cricket world cup 2015. Searching it on Google gave 435k results and the time of the tweet was after the final match of the tournament. So there is an observational humor in (1) as India won the world cup in 2011 and lost in 2015 , hence the tweet was classified as humorous. Any tweets stating any facts, news or reality were classified as non-humorous. There were many tweets which did not contain any hashtags, to understand the context of such tweets annotators selected some keywords from the tweet and searched them online. Example (2) contains a comment towards a political leader towards development and was categorized as non-humorous. Tweets containing normal jokes and funny quotes like in (3) and (4) were put in humorous category. There were some tweets like (5) which consists of poem or lines of a song but modified. Annotators were guided that if such tweets contains satire or any humoristic features, then it could be categorized as humorous otherwise not. There were some tweets which were typical to categorize like (5), hence it was left to the annotators to the best of their understanding. Based on the above guidelines annotators categorized the tweets. To measure inter annotator agreement we opted for Fleiss' Kappa BIBREF12 obtaining an agreement of 0.821 . Both humorous and non-humorous tweets in nearly balanced amount were selected to prepare the corpus. If we had included humorous tweets from one domain like sports and non humorous tweets from another domain like news then, it would have given high performance of classification BIBREF13 . To classify based on the semantics and not on the domain differences, we included both types of tweets from different domains. Many tweets contains a picture along with a caption. Sometimes a caption may not contain humor but combined with the picture, it can provide some degree of humor. Such tweets were removed from the corpus to make the corpus unimodal. In Figure 1, the first tweet, “Anurag Kashyap can never join AAP because ministers took oath `main kisi Anurag aur dwesh ke bina kaam karunga' ” (Anurag Kashyap can never join AAP because ministers took oath `I will work without any affection (Anurag in Hindi) and without hesitation (dwesh in Hindi)'), was classified as humorous. The second tweet, “#SakshiMalik take a bow! #proudIndian #Rio #Olympics #BronzeMedal #girlpower Hamaara khaata khul Gaya!” (#SakshiMalik take a bow! #proudIndian #Rio #Olympics #BronzeMedal #girlpower Our account opened!) was classified as non-humorous as it contains a pride statement. Language Annotation Code-mixing provides some challenges for language identification like spelling variations and non-grammatical structures in a sentence BIBREF0 . Therefore, we annotated the tweets with the language at the word level. Native speakers of Hindi and proficient in English, labelled the language of the tokens in the tweets. Three types of tags were assigned to the tokens , En is assigned to the tokens present in English vocabulary like “family”, “Children” etc. Similarly, Hi is assigned to the tokens present in Hindi vocabulary but transliterated to Latin script like “samay” (time), “aamaadmi” (common man). Rest of the tokens consists of proper nouns, numbers, dates, urls, hashtags, mentions, emojis and punctuations which are labelled as Ot(others). Major concern in language annotation was to annotate the words present in both languages (ambiguous words). For example, `to' (`but' in Hindi) and `is' (`this' in Hindi) , for this scenario annotators understood the context of the tweet and based on that the words were being annotated. For example, consider two sentences `This place is 500 years old' and `Is bar Modi Sarkar'(This time Modi Government). So `is' in first sentence should be tagged as English because it is used as a verb. It is transliterated to English in latter sentence and used as determiner (`this') so it was tagged as Hindi. So these kind of words were tagged based on their meanings and usage in both languages. Figure 1 describes the language annotation of tweets in the corpus. Annotation Scheme Two example annotations are illustrated in Figure 1. First line in every annotation consists of tweet id. Each tweet is enclosed within <tweet></tweet> tags and each word is enclosed within <word lang=“ ”></word> containing the language annotation for each word. Last line determines the category in which the tweet belongs i.e. humorous or non-humorous, enclosed by <class></class> tags. The annotated dataset with the classification system is made available online . Error Analysis Social media users often make spelling mistakes or use multiple variations of a word while posting a text. We replaced all such errors with their correct version. For words in Hindi that were transliterated to Latin script, we adopted a common spelling for all those words across the corpus. For example, `dis' is often used as a short form for `this', so we replaced every occurrence of `dis' to `this' in the corpus. Some examples of spelling variations are, `pese' for `paisa' (unit of curreny), `h' for `hai' (is), `ad' for `add' (addition). System Architecture In this section, we describe our machine learning model which is trained and tested on the corpus described in the previous section. Pre-processing of Corpus Preprocessing starts with tokenization, which involves separation of words using space as the delimiter and then converting the words to lower cases which is followed by the removal of punctuation marks from the tokenized tweet. All the hashtags, mentions and urls, are stored and converted to `hashtag', `mention' and `url' respectively. Sometimes hashtags provide some degree of humor in tweets, hence we segregated hashtags on the basis of camel cases and included the tokens in the tokenized tweets (hashtag decomposition) BIBREF14 , BIBREF15 . for example, #AadabArzHai can be decomposed into 3 words, `Aadab', `Arz' and `Hai'. Finally the tokenized tweets are stored along with the presence of humor as the target class. Classification Features The features used to build attribute vectors for training our classification model are described below. We use character level and word level features for the classification BIBREF15 . For all the features, we separated the words in the tweets based on the language annotation (Section 2.3) and prepared the feature vector for each tweet by combining the vectors for both the languages . Previous researches shows that letter n-grams are very efficient for classifying text. They are language independent and does not require expensive text pre-processing techniques like tokenization, stemming and stop words removal, hence in the case of code-mix texts, this could yield good results BIBREF16 , BIBREF17 . Since the number of n-grams can be very large we took trigrams which occur more than ten times in the corpus. For classifying humor in texts, it is important to understand the semantics of the sentence. Thus, we took a three word window as a feature to train our classification models to incorporate the contextual information. Many jokes and idioms sometimes have common words. We identified those words and took them as as a feature for classification. In the preprocessing step, we decomposed hashtags using camel cases and added them along with the words. Hence, common words in the hashtags were also included in the feature vector. Classification Approach and Results We experimented with four different classifiers, namely, support vector machine BIBREF18 , random forest, extra tree and naive bayes classifier BIBREF19 . Chi square feature selection algorithm is applied to reduces the size of our feature vector. For training our system classifier, we used Scikit-learn BIBREF19 . 10-fold cross validation on 3543 code-mixed tweets was carried out by dividing the corpus into 10 equal parts with nine parts as training corpus and rest one for testing. Mean accuracy is calculated by taking the average of the accuracy obtained in each iteration of the testing process. Table 2 shows the accuracy for each feature when trained using mentioned classifiers along with the accuracy when all the features are used along with the overall accuracy. Support vector machine with radial basis function kernel and extra tree classifier performs better than other classifiers and yields 69.3% and 67.8% accuracy respectively. The reason kernel SVM yields the best result is because the number of observations is greator than the number of features BIBREF20 . N-grams proved to be the most efficient in all classification models followed by common words and hastags. Bag-of-words feature performed the worst in SVM, random forest and extra tree classifier but yielded better result in naive bayes classifiers. Accuracies mentioned in table 2 were calculated using fine tuning of model parameters using grid search. Conclusion and Future Work In this paper, we describe a freely available corpus of 3453 English-Hindi code-mixed tweets. The tweets are annotated with humorous(H) and non-humorous(N) tags along with the language tags at the word level. The task of humor identification in social media texts is analyzed as a classification problem and several machine learning classification models are used. The features used in our classification system are n-grams, bag-of-words, common words and hashtags. N-grams when trained with support vector machines with radial basis function kernel performed better than other features and yielded an accuracy of 68.5%. The best accuracy (69.3%) was given by support vector machines with radial basis function kernel. This paper describes the initial efforts in automatic humor detection in code-mixed social media texts. Corpus can be annotated with part-of-speech tags at the word level which may yield better results in language detection. Moreover, the dataset can be further extended to include tweets from other domains. Code-mixing is very common phenomenon on social media and it is prevalent mostly in multilingual regions. It would be interesting to experiment with code-mixed texts consisting of more than two languages in which the issue of transliteration exists like Arabic, Greek and South Asian languages. Comparing training with code-mixed tweets with training with a merged corpus of monolingual tweets in English and Hindi could be an interesting future work.
Where did the texts in the corpus come from?
15cdd9ea4bae8891c1652da2ed34c87bbbd0edb8
Where Ġdid Ġthe Ġtexts Ġin Ġthe Ġcorpus Ġcome Ġfrom ?
t we ets Ġfrom Ġthe Ġpast Ġtwo Ġyears Ġfrom Ġdomains Ġlike Ġ` sports ', Ġ` politics ', Ġ` ent ertain ment '
<s> Where Ġdid Ġthe Ġtexts Ġin Ġthe Ġcorpus Ġcome Ġfrom ? </s> Introduction </s> âĢ ľ L aughter Ġis Ġthe Ġbest ĠMedicine âĢ Ŀ Ġis Ġa Ġsaying Ġwhich Ġis Ġpopular Ġwith Ġmost Ġof Ġthe Ġpeople . ĠHum or Ġis Ġa Ġform Ġof Ġcommunication Ġthat Ġbridges Ġthe Ġgap Ġbetween Ġvarious Ġlanguages , Ġcultures , Ġages Ġand Ġdemographics . ĠThat 's Ġwhy Ġhumorous Ġcontent Ġwith Ġfunny Ġand Ġwitty Ġhasht ags Ġare Ġso Ġmuch Ġpopular Ġon Ġsocial Ġmedia . ĠIt Ġis Ġa Ġvery Ġpowerful Ġtool Ġto Ġconnect Ġwith Ġthe Ġaudience . ĠAutomatic ĠHum or ĠRecogn ition Ġis Ġthe Ġtask Ġof Ġdetermining Ġwhether Ġa Ġtext Ġcontains Ġsome Ġlevel Ġof Ġhumorous Ġcontent Ġor Ġnot . ĠFirst Ġconference Ġon ĠComput ational Ġhumor Ġwas Ġorganized Ġin Ġ1996 , Ġsince Ġthen Ġmany Ġresearch Ġhave Ġbeen Ġdone Ġin Ġthis Ġfield . Ġk ao 2016 com put ational Ġdoes Ġpun Ġdetection Ġin Ġone - liners Ġand Ġde hum or Ġdetects Ġhumor Ġin ĠYelp Ġreviews . ĠBecause Ġof Ġthe Ġcomplex Ġand Ġinteresting Ġaspects Ġinvolved Ġin Ġdetecting Ġhumor Ġin Ġtexts , Ġit Ġis Ġone Ġof Ġthe Ġchallenging Ġresearch Ġfield Ġin ĠNatural ĠLanguage ĠProcessing ĠB IB REF 3 Ġ. ĠIdent ifying Ġhumor Ġin Ġa Ġsentence Ġsometimes Ġrequire Ġa Ġgreat Ġamount Ġof Ġexternal Ġknowledge Ġto Ġcompletely Ġunderstand Ġit . ĠThere Ġare Ġmany Ġtypes Ġof Ġhumor , Ġnamely Ġanecdotes , Ġfantasy , Ġinsult , Ġirony , Ġjokes , Ġquote , Ġself Ġdep rec ation Ġetc ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ. ĠMost Ġof Ġthe Ġtimes Ġthere Ġare Ġdifferent Ġmeanings Ġhidden Ġinside Ġa Ġsentence Ġwhich Ġis Ġgrasped Ġdifferently Ġby Ġindividuals , Ġmaking Ġthe Ġtask Ġof Ġhumor Ġidentification Ġdifficult , Ġwhich Ġis Ġwhy Ġthe Ġdevelopment Ġof Ġa Ġgeneralized Ġalgorithm Ġto Ġclassify Ġdifferent Ġtype Ġof Ġhumor Ġis Ġa Ġchallenging Ġtask . </s> Major ity Ġof Ġthe Ġresear ches Ġon Ġsocial Ġmedia Ġtexts Ġis Ġfocused Ġon ĠEnglish . ĠA Ġstudy Ġby Ġsch ro eder 2010 half Ġshows Ġthat , Ġa Ġhigh Ġpercentage Ġof Ġthese Ġtexts Ġare Ġin Ġnon - English Ġlanguages . Ġf ischer 2011 language Ġgives Ġsome Ġinteresting Ġinformation Ġabout Ġthe Ġlanguages Ġused Ġon ĠTwitter Ġbased Ġon Ġthe Ġgeographical Ġlocations . ĠWith Ġa Ġhuge Ġamount Ġof Ġsuch Ġuser Ġgenerated Ġdata Ġavailable Ġon Ġsocial Ġmedia , Ġthere Ġis Ġa Ġneed Ġto Ġdevelop Ġtechnologies Ġfor Ġnon - English Ġlanguages . ĠIn Ġmult ilingual Ġregions Ġlike ĠSouth ĠAsia , Ġmajority Ġof Ġthe Ġsocial Ġmedia Ġusers Ġspeak Ġmore Ġthan Ġtwo Ġlanguages . ĠIn ĠIndia , ĠHindi Ġis Ġthe Ġmost Ġspoken Ġlanguage Ġ( spoken Ġby Ġ41 % Ġof Ġthe Ġpopulation ) Ġand ĠEnglish Ġis Ġthe Ġofficial Ġlanguage Ġof Ġthe Ġcountry . ĠTwitter Ġhas Ġaround Ġ23 . 2 Ġmillion Ġmonthly Ġactive Ġusers Ġin ĠIndia . ĠNative Ġspeakers Ġof ĠHindi Ġoften Ġput ĠEnglish Ġwords Ġin Ġthe Ġsentences Ġand Ġtransl iter ate Ġthe Ġwhole Ġsentence Ġto ĠLatin Ġscript Ġwhile Ġposting Ġon Ġsocial Ġmedia , Ġthereby Ġmaking Ġthe Ġtask Ġof Ġautomatic Ġtext Ġclassification Ġa Ġvery Ġchallenging Ġproblem . ĠL ingu ists Ġcame Ġup Ġwith Ġa Ġterm Ġfor Ġany Ġtype Ġof Ġlanguage Ġmixing , Ġknown Ġas Ġ` code - mix ing ' Ġor Ġ` code - sw itching ' ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ, ĠB IB REF 9 Ġ. ĠBoth Ġthe Ġterms Ġare Ġused Ġinterchange ably , Ġbut Ġthere Ġis Ġa Ġslight Ġdifference Ġbetween Ġthe Ġtwo Ġterms . ĠCode - mix ing Ġrefers Ġto Ġthe Ġinsertion Ġof Ġwords , Ġphrases , Ġand Ġmorp hem es Ġof Ġone Ġlanguage Ġinto Ġa Ġstatement Ġor Ġan Ġexpression Ġof Ġanother Ġlanguage , Ġwhereas Ġtransl iter ation Ġof Ġevery Ġword Ġin Ġa Ġsentence Ġto Ġanother Ġscript Ġ( Ġhere ĠDev an ag ari Ġto ĠLatin ) Ġis Ġcoined Ġcode - sw itching ĠB IB REF 10 Ġ. ĠThe Ġfirst Ġtweet Ġin ĠFigure Ġ1 Ġis Ġan Ġexample Ġof Ġcode - mix ing Ġand Ġsecond Ġis Ġan Ġexample Ġof Ġcode - sw itching . ĠIn Ġthis Ġpaper , Ġwe Ġuse Ġcode - mix ing Ġto Ġdenote Ġboth Ġcases . </s> In Ġthis Ġpaper , Ġwe Ġpresent Ġa Ġfreely Ġavailable Ġcorpus Ġcontaining Ġcode - m ixed Ġtweets Ġin ĠHindi Ġand ĠEnglish Ġlanguage Ġwith Ġtweets Ġwritten Ġin ĠLatin Ġscript . ĠTwe ets Ġare Ġmanually Ġclassified Ġinto Ġhumorous Ġand Ġnon - hum orous Ġclasses . ĠMoreover , Ġeach Ġtoken Ġin Ġthe Ġtweets Ġis Ġalso Ġgiven Ġa Ġlanguage Ġtag Ġwhich Ġdetermines Ġthe Ġsource Ġor Ġorigin Ġlanguage Ġof Ġthe Ġtoken Ġ( English Ġor ĠHindi ). ĠThe Ġpaper Ġis Ġdivided Ġin Ġsections Ġas Ġfollows , Ġwe Ġstart Ġby Ġdescribing Ġthe Ġcorpus Ġand Ġthe Ġannotation Ġscheme Ġin ĠSection Ġ2 . ĠSection Ġ3 Ġsummarizes Ġour Ġsupervised Ġclassification Ġsystem Ġwhich Ġincludes Ġpre - processing Ġof Ġthe Ġtweets Ġin Ġthe Ġdataset Ġand Ġthe Ġfeature Ġextraction Ġfollowed Ġby Ġthe Ġmethod Ġused Ġto Ġidentify Ġhumor Ġin Ġtweets . ĠIn Ġthe Ġnext Ġsubsection , Ġwe Ġdescribe Ġthe Ġclassification Ġmodel Ġand Ġthe Ġresults Ġof Ġthe Ġexperiments Ġconducted Ġusing Ġcharacter Ġand Ġword Ġlevel Ġfeatures . ĠIn Ġthe Ġlast Ġsection , Ġwe Ġconclude Ġthe Ġpaper Ġfollowed Ġby Ġfuture Ġwork Ġand Ġreferences . </s> Corp us ĠCreation Ġand ĠAn notation </s> In Ġthis Ġsection , Ġwe Ġexplain Ġthe Ġtechniques Ġused Ġin Ġthe Ġcreation Ġand Ġannotation Ġof Ġthe Ġcorpus . </s> Data ĠCollection </s> Python Ġpackage Ġtw itters c raper Ġis Ġused Ġto Ġscrap Ġtweets Ġfrom Ġtwitter . Ġ10 , 478 Ġtweets Ġfrom Ġthe Ġpast Ġtwo Ġyears Ġfrom Ġdomains Ġlike Ġ` sports ', Ġ` politics ', Ġ` ent ertain ment ' Ġwere Ġextracted . ĠAmong Ġthose Ġtweets , Ġwe Ġmanually Ġremoved Ġthe Ġtweets Ġwhich Ġwere Ġwritten Ġeither Ġin ĠEnglish Ġor ĠHindi Ġentirely . ĠThere Ġwere Ġ4 161 Ġtweets Ġwritten Ġin ĠEnglish Ġand Ġ27 74 Ġwritten Ġin ĠHindi . ĠFinally , Ġa Ġtotal Ġof Ġ35 43 ĠEnglish - H ind i Ġcode - m ixed Ġtweets Ġwere Ġcollected . ĠTable Ġ1 Ġdescribes Ġthe Ġnumber Ġof Ġtweets Ġand Ġwords Ġin Ġeach Ġcategory . </s> We Ġtried Ġto Ġmake Ġthe Ġcorpus Ġbalanced Ġi . e . Ġuniform Ġdistribution Ġof Ġtweets Ġin Ġeach Ġcategory Ġto Ġyield Ġbetter Ġsupervised Ġclassification Ġresults Ġas Ġdescribed Ġby ĠB IB REF 11 Ġ. </s> Hum or ĠAn notation </s> The Ġfinal Ġcode - m ixed Ġtweets Ġwere Ġforwarded Ġto Ġa Ġgroup Ġof Ġthree Ġannot ators Ġwho Ġwere Ġuniversity Ġstudents Ġand Ġfluent Ġin Ġboth ĠEnglish Ġand ĠHindi . ĠApproximately Ġ60 Ġhours Ġwere Ġspent Ġin Ġtagging Ġtweets Ġfor Ġthe Ġpresence Ġof Ġhumor . ĠTwe ets Ġwhich Ġconsisted Ġof Ġany Ġanecdotes , Ġfantasy , Ġirony , Ġjokes , Ġinsults Ġwere Ġannot ated Ġas Ġhumorous Ġwhereas Ġtweets Ġstating Ġany Ġfacts , Ġdialog ues Ġor Ġspeech Ġwhich Ġdid Ġnot Ġcontain Ġamusement Ġwere Ġput Ġin Ġnon - hum orous Ġclass . ĠFollowing Ġare Ġsome Ġexamples Ġof Ġcode - m ixed Ġtweets Ġin Ġthe Ġcorpus : </s> âĢ ľ For Ġ# W ont Give It Back Ġto Ġwork , ĠDh oni Ġneeds Ġto Ġsay Ġ` Tro phy Ġto h Ġg har Ġpar Ġhi Ġb h ul Ġa aye ' ĠâĢ Ŀ </s> ( For Ġ# W ont Give It Back Ġto Ġwork , ĠDh oni Ġneeds Ġto Ġsay ĠâĢ ľ We Ġforgot Ġthe Ġtrophy Ġat Ġhome âĢ Ŀ ) Ġ. </s> Na Ġsad ak Ġna Ġn au k ri Ġbas Ġbad h Ġr ahi Ġgu nd ag ardi Ġ# F ailed CM Nit ish </s> ( No Ġroads Ġno Ġn au k ri , Ġonly Ġh ool igan ism Ġincreasing Ġ# F ailed CM Nit ish Ġ) </s> Sub ha Ġka Ġb h ula Ġag ar Ġsham Ġko Ġw ap as Ġg har Ġa a Ġj aye Ġthen Ġwe Ġmust Ġthank ĠGPS Ġtechnology . </s> ( If Ġsomeone Ġlost Ġin Ġthe Ġmorning Ġreturn Ġhome Ġin Ġthe Ġevening Ġthen Ġwe Ġmust Ġthank ĠGPS Ġtechnology .) </s> She Ġ: ĠG ori Ġha i Ġk al ai yan Ġpe h na Ġde Ġm uj he Ġh ari Ġh ari Ġch ud iya ĠHe Ġ* gets Ġgreen Ġbang les * ĠShe Ġ: ĠNot Ġthis Ġgreen Ġya , Ġbottle Ġgreen Ġcolor . </s> ( She Ġ: Ġ( s ings Ġa Ġsong ) ĠMy Ġwrists Ġare Ġso Ġbeautiful , Ġgive Ġme Ġgreen Ġgreen Ġbang les ĠHe Ġ* gets Ġgreen Ġbang les * ĠShe Ġ: ĠNot Ġthis Ġgreen Ġya , Ġbottle Ġgreen Ġcolor . Ġ) </s> D ard Ġdil on Ġke Ġk am Ġho Ġja ate ... Ġtwitter Ġpar Ġag ar Ġpoet ries Ġk am Ġho Ġja ate </s> ( pain Ġin Ġthe Ġheart Ġwill Ġreduce ... if Ġnumber Ġof Ġpoet ries Ġdecreases Ġon Ġtwitter ). </s> An not ators Ġwere Ġgiven Ġcertain Ġguidelines Ġto Ġdecide Ġwhether Ġa Ġtweet Ġwas Ġhumorous Ġor Ġnot . ĠThe Ġcontext Ġof Ġthe Ġtweet Ġcould Ġbe Ġfound Ġby Ġsearching Ġabout Ġhashtag Ġor Ġkeywords Ġused Ġin Ġthe Ġtweet . ĠExample Ġ( 1 ) Ġuses Ġa Ġhashtag Ġ` # W ont Give It Back ' Ġwhich Ġwas Ġtrending Ġduring Ġthe ĠICC Ġcricket Ġworld Ġcup Ġ2015 . ĠSearch ing Ġit Ġon ĠGoogle Ġgave Ġ435 k Ġresults Ġand Ġthe Ġtime Ġof Ġthe Ġtweet Ġwas Ġafter Ġthe Ġfinal Ġmatch Ġof Ġthe Ġtournament . ĠSo Ġthere Ġis Ġan Ġobservational Ġhumor Ġin Ġ( 1 ) Ġas ĠIndia Ġwon Ġthe Ġworld Ġcup Ġin Ġ2011 Ġand Ġlost Ġin Ġ2015 Ġ, Ġhence Ġthe Ġtweet Ġwas Ġclassified Ġas Ġhumorous . ĠAny Ġtweets Ġstating Ġany Ġfacts , Ġnews Ġor Ġreality Ġwere Ġclassified Ġas Ġnon - hum orous . ĠThere Ġwere Ġmany Ġtweets Ġwhich Ġdid Ġnot Ġcontain Ġany Ġhasht ags , Ġto Ġunderstand Ġthe Ġcontext Ġof Ġsuch Ġtweets Ġannot ators Ġselected Ġsome Ġkeywords Ġfrom Ġthe Ġtweet Ġand Ġsearched Ġthem Ġonline . ĠExample Ġ( 2 ) Ġcontains Ġa Ġcomment Ġtowards Ġa Ġpolitical Ġleader Ġtowards Ġdevelopment Ġand Ġwas Ġcategorized Ġas Ġnon - hum orous . ĠTwe ets Ġcontaining Ġnormal Ġjokes Ġand Ġfunny Ġquotes Ġlike Ġin Ġ( 3 ) Ġand Ġ( 4 ) Ġwere Ġput Ġin Ġhumorous Ġcategory . ĠThere Ġwere Ġsome Ġtweets Ġlike Ġ( 5 ) Ġwhich Ġconsists Ġof Ġpoem Ġor Ġlines Ġof Ġa Ġsong Ġbut Ġmodified . ĠAn not ators Ġwere Ġguided Ġthat Ġif Ġsuch Ġtweets Ġcontains Ġsatire Ġor Ġany Ġhumor istic Ġfeatures , Ġthen Ġit Ġcould Ġbe Ġcategorized Ġas Ġhumorous Ġotherwise Ġnot . ĠThere Ġwere Ġsome Ġtweets Ġwhich Ġwere Ġtypical Ġto Ġcategor ize Ġlike Ġ( 5 ), Ġhence Ġit Ġwas Ġleft Ġto Ġthe Ġannot ators Ġto Ġthe Ġbest Ġof Ġtheir Ġunderstanding . ĠBased Ġon Ġthe Ġabove Ġguidelines Ġannot ators Ġcategorized Ġthe Ġtweets . ĠTo Ġmeasure Ġinter Ġannot ator Ġagreement Ġwe Ġopted Ġfor ĠFle iss ' ĠKappa ĠB IB REF 12 Ġobtaining Ġan Ġagreement Ġof Ġ0 . 8 21 Ġ. ĠBoth Ġhumorous Ġand Ġnon - hum orous Ġtweets Ġin Ġnearly Ġbalanced Ġamount Ġwere Ġselected Ġto Ġprepare Ġthe Ġcorpus . ĠIf Ġwe Ġhad Ġincluded Ġhumorous Ġtweets Ġfrom Ġone Ġdomain Ġlike Ġsports Ġand Ġnon Ġhumorous Ġtweets Ġfrom Ġanother Ġdomain Ġlike Ġnews Ġthen , Ġit Ġwould Ġhave Ġgiven Ġhigh Ġperformance Ġof Ġclassification ĠB IB REF 13 Ġ. ĠTo Ġclassify Ġbased Ġon Ġthe Ġsemantics Ġand Ġnot Ġon Ġthe Ġdomain Ġdifferences , Ġwe Ġincluded Ġboth Ġtypes Ġof Ġtweets Ġfrom Ġdifferent Ġdomains . ĠMany Ġtweets Ġcontains Ġa Ġpicture Ġalong Ġwith Ġa Ġcaption . ĠSometimes Ġa Ġcaption Ġmay Ġnot Ġcontain Ġhumor Ġbut Ġcombined Ġwith Ġthe Ġpicture , Ġit Ġcan Ġprovide Ġsome Ġdegree Ġof Ġhumor . ĠSuch Ġtweets Ġwere Ġremoved Ġfrom Ġthe Ġcorpus Ġto Ġmake Ġthe Ġcorpus Ġunim od al . ĠIn ĠFigure Ġ1 , Ġthe Ġfirst Ġtweet , ĠâĢ ľ An ur ag ĠKash y ap Ġcan Ġnever Ġjoin ĠAAP Ġbecause Ġministers Ġtook Ġoath Ġ` main Ġk isi ĠAn ur ag Ġaur Ġdw esh Ġke Ġb ina Ġka am Ġk ar ung a ' ĠâĢ Ŀ Ġ( An ur ag ĠKash y ap Ġcan Ġnever Ġjoin ĠAAP Ġbecause Ġministers Ġtook Ġoath Ġ` I Ġwill Ġwork Ġwithout Ġany Ġaffection Ġ( An ur ag Ġin ĠHindi ) Ġand Ġwithout Ġhesitation Ġ( d w esh Ġin ĠHindi )' ), Ġwas Ġclassified Ġas Ġhumorous . ĠThe Ġsecond Ġtweet , ĠâĢ ľ # S ak shi Mal ik Ġtake Ġa Ġbow ! Ġ# p roud Indian Ġ# R io Ġ# O lymp ics Ġ# Bron ze Med al Ġ# girl power ĠH ama ara Ġk ha ata Ġkh ul ĠG aya ! âĢ Ŀ Ġ(# S ak shi Mal ik Ġtake Ġa Ġbow ! Ġ# p roud Indian Ġ# R io Ġ# O lymp ics Ġ# Bron ze Med al Ġ# girl power ĠOur Ġaccount Ġopened !) Ġwas Ġclassified Ġas Ġnon - hum orous Ġas Ġit Ġcontains Ġa Ġpride Ġstatement . </s> Language ĠAn notation </s> Code - mix ing Ġprovides Ġsome Ġchallenges Ġfor Ġlanguage Ġidentification Ġlike Ġspelling Ġvariations Ġand Ġnon - gram matical Ġstructures Ġin Ġa Ġsentence ĠB IB REF 0 Ġ. ĠTherefore , Ġwe Ġannot ated Ġthe Ġtweets Ġwith Ġthe Ġlanguage Ġat Ġthe Ġword Ġlevel . ĠNative Ġspeakers Ġof ĠHindi Ġand Ġproficient Ġin ĠEnglish , Ġlabelled Ġthe Ġlanguage Ġof Ġthe Ġtokens Ġin Ġthe Ġtweets . ĠThree Ġtypes Ġof Ġtags Ġwere Ġassigned Ġto Ġthe Ġtokens Ġ, ĠEn Ġis Ġassigned Ġto Ġthe Ġtokens Ġpresent Ġin ĠEnglish Ġvocabulary Ġlike ĠâĢ ľ family âĢ Ŀ , ĠâĢ ľ Children âĢ Ŀ Ġetc . ĠSimilarly , ĠHi Ġis Ġassigned Ġto Ġthe Ġtokens Ġpresent Ġin ĠHindi Ġvocabulary Ġbut Ġtransl iter ated Ġto ĠLatin Ġscript Ġlike ĠâĢ ľ sam ay âĢ Ŀ Ġ( time ), ĠâĢ ľ a ama ad mi âĢ Ŀ Ġ( common Ġman ). ĠRest Ġof Ġthe Ġtokens Ġconsists Ġof Ġproper Ġnoun s , Ġnumbers , Ġdates , Ġur ls , Ġhasht ags , Ġmentions , Ġem oj is Ġand Ġpunct uations Ġwhich Ġare Ġlabelled Ġas ĠOt ( other s ). ĠMajor Ġconcern Ġin Ġlanguage Ġannotation Ġwas Ġto Ġannot ate Ġthe Ġwords Ġpresent Ġin Ġboth Ġlanguages Ġ( amb iguous Ġwords ). ĠFor Ġexample , Ġ` to ' Ġ( ` but ' Ġin ĠHindi ) Ġand Ġ` is ' Ġ( ` this ' Ġin ĠHindi ) Ġ, Ġfor Ġthis Ġscenario Ġannot ators Ġunderstood Ġthe Ġcontext Ġof Ġthe Ġtweet Ġand Ġbased Ġon Ġthat Ġthe Ġwords Ġwere Ġbeing Ġannot ated . ĠFor Ġexample , Ġconsider Ġtwo Ġsentences Ġ` This Ġplace Ġis Ġ500 Ġyears Ġold ' Ġand Ġ` Is Ġbar ĠModi ĠSark ar ' ( This Ġtime ĠModi ĠGovernment ). ĠSo Ġ` is ' Ġin Ġfirst Ġsentence Ġshould Ġbe Ġtagged Ġas ĠEnglish Ġbecause Ġit Ġis Ġused Ġas Ġa Ġverb . ĠIt Ġis Ġtransl iter ated Ġto ĠEnglish Ġin Ġlatter Ġsentence Ġand Ġused Ġas Ġdetermin er Ġ( ` this ') Ġso Ġit Ġwas Ġtagged Ġas ĠHindi . ĠSo Ġthese Ġkind Ġof Ġwords Ġwere Ġtagged Ġbased Ġon Ġtheir Ġmeanings Ġand Ġusage Ġin Ġboth Ġlanguages . ĠFigure Ġ1 Ġdescribes Ġthe Ġlanguage Ġannotation Ġof Ġtweets Ġin Ġthe Ġcorpus . </s> An notation ĠScheme </s> Two Ġexample Ġannotations Ġare Ġillustrated Ġin ĠFigure Ġ1 . ĠFirst Ġline Ġin Ġevery Ġannotation Ġconsists Ġof Ġtweet Ġid . ĠEach Ġtweet Ġis Ġenclosed Ġwithin Ġ< t weet ></ t weet > Ġtags Ġand Ġeach Ġword Ġis Ġenclosed Ġwithin Ġ< word Ġlang = âĢ ľ ĠâĢ Ŀ ></ word > Ġcontaining Ġthe Ġlanguage Ġannotation Ġfor Ġeach Ġword . ĠLast Ġline Ġdetermines Ġthe Ġcategory Ġin Ġwhich Ġthe Ġtweet Ġbelongs Ġi . e . Ġhumorous Ġor Ġnon - hum orous , Ġenclosed Ġby Ġ< class ></ class > Ġtags . ĠThe Ġannot ated Ġdataset Ġwith Ġthe Ġclassification Ġsystem Ġis Ġmade Ġavailable Ġonline Ġ. </s> Error ĠAnalysis </s> Social Ġmedia Ġusers Ġoften Ġmake Ġspelling Ġmistakes Ġor Ġuse Ġmultiple Ġvariations Ġof Ġa Ġword Ġwhile Ġposting Ġa Ġtext . ĠWe Ġreplaced Ġall Ġsuch Ġerrors Ġwith Ġtheir Ġcorrect Ġversion . ĠFor Ġwords Ġin ĠHindi Ġthat Ġwere Ġtransl iter ated Ġto ĠLatin Ġscript , Ġwe Ġadopted Ġa Ġcommon Ġspelling Ġfor Ġall Ġthose Ġwords Ġacross Ġthe Ġcorpus . ĠFor Ġexample , Ġ` dis ' Ġis Ġoften Ġused Ġas Ġa Ġshort Ġform Ġfor Ġ` this ', Ġso Ġwe Ġreplaced Ġevery Ġoccurrence Ġof Ġ` dis ' Ġto Ġ` this ' Ġin Ġthe Ġcorpus . ĠSome Ġexamples Ġof Ġspelling Ġvariations Ġare , Ġ` p ese ' Ġfor Ġ` pa isa ' Ġ( unit Ġof Ġcur ren y ), Ġ` h ' Ġfor Ġ` hai ' Ġ( is ), Ġ` ad ' Ġfor Ġ` add ' Ġ( add ition ). </s> System ĠArchitecture </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġour Ġmachine Ġlearning Ġmodel Ġwhich Ġis Ġtrained Ġand Ġtested Ġon Ġthe Ġcorpus Ġdescribed Ġin Ġthe Ġprevious Ġsection . </s> Pre - processing Ġof ĠCorpus </s> Pre processing Ġstarts Ġwith Ġtoken ization , Ġwhich Ġinvolves Ġseparation Ġof Ġwords Ġusing Ġspace Ġas Ġthe Ġdelim iter Ġand Ġthen Ġconverting Ġthe Ġwords Ġto Ġlower Ġcases Ġwhich Ġis Ġfollowed Ġby Ġthe Ġremoval Ġof Ġpunct uation Ġmarks Ġfrom Ġthe Ġtoken ized Ġtweet . ĠAll Ġthe Ġhasht ags , Ġmentions Ġand Ġur ls , Ġare Ġstored Ġand Ġconverted Ġto Ġ` hash tag ', Ġ` ment ion ' Ġand Ġ` url ' Ġrespectively . ĠSometimes Ġhasht ags Ġprovide Ġsome Ġdegree Ġof Ġhumor Ġin Ġtweets , Ġhence Ġwe Ġsegregated Ġhasht ags Ġon Ġthe Ġbasis Ġof Ġcamel Ġcases Ġand Ġincluded Ġthe Ġtokens Ġin Ġthe Ġtoken ized Ġtweets Ġ( hash tag Ġdecom position ) ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ. Ġfor Ġexample , Ġ# A ad ab Ar z H ai Ġcan Ġbe Ġdecomp osed Ġinto Ġ3 Ġwords , Ġ` A ad ab ', Ġ` Ar z ' Ġand Ġ` H ai '. ĠFinally Ġthe Ġtoken ized Ġtweets Ġare Ġstored Ġalong Ġwith Ġthe Ġpresence Ġof Ġhumor Ġas Ġthe Ġtarget Ġclass . </s> Class ification ĠFeatures </s> The Ġfeatures Ġused Ġto Ġbuild Ġattribute Ġvectors Ġfor Ġtraining Ġour Ġclassification Ġmodel Ġare Ġdescribed Ġbelow . ĠWe Ġuse Ġcharacter Ġlevel Ġand Ġword Ġlevel Ġfeatures Ġfor Ġthe Ġclassification ĠB IB REF 15 Ġ. ĠFor Ġall Ġthe Ġfeatures , Ġwe Ġseparated Ġthe Ġwords Ġin Ġthe Ġtweets Ġbased Ġon Ġthe Ġlanguage Ġannotation Ġ( Section Ġ2 . 3 ) Ġand Ġprepared Ġthe Ġfeature Ġvector Ġfor Ġeach Ġtweet Ġby Ġcombining Ġthe Ġvectors Ġfor Ġboth Ġthe Ġlanguages Ġ. </s> Previous Ġresear ches Ġshows Ġthat Ġletter Ġn - gram s Ġare Ġvery Ġefficient Ġfor Ġclass ifying Ġtext . ĠThey Ġare Ġlanguage Ġindependent Ġand Ġdoes Ġnot Ġrequire Ġexpensive Ġtext Ġpre - processing Ġtechniques Ġlike Ġtoken ization , Ġstemming Ġand Ġstop Ġwords Ġremoval , Ġhence Ġin Ġthe Ġcase Ġof Ġcode - mix Ġtexts , Ġthis Ġcould Ġyield Ġgood Ġresults ĠB IB REF 16 Ġ, ĠB IB REF 17 Ġ. ĠSince Ġthe Ġnumber Ġof Ġn - gram s Ġcan Ġbe Ġvery Ġlarge Ġwe Ġtook Ġtrig rams Ġwhich Ġoccur Ġmore Ġthan Ġten Ġtimes Ġin Ġthe Ġcorpus . </s> For Ġclass ifying Ġhumor Ġin Ġtexts , Ġit Ġis Ġimportant Ġto Ġunderstand Ġthe Ġsemantics Ġof Ġthe Ġsentence . ĠThus , Ġwe Ġtook Ġa Ġthree Ġword Ġwindow Ġas Ġa Ġfeature Ġto Ġtrain Ġour Ġclassification Ġmodels Ġto Ġincorporate Ġthe Ġcontextual Ġinformation . </s> Many Ġjokes Ġand Ġidi oms Ġsometimes Ġhave Ġcommon Ġwords . ĠWe Ġidentified Ġthose Ġwords Ġand Ġtook Ġthem Ġas Ġas Ġa Ġfeature Ġfor Ġclassification . ĠIn Ġthe Ġpre processing Ġstep , Ġwe Ġdecomp osed Ġhasht ags Ġusing Ġcamel Ġcases Ġand Ġadded Ġthem Ġalong Ġwith Ġthe Ġwords . ĠHence , Ġcommon Ġwords Ġin Ġthe Ġhasht ags Ġwere Ġalso Ġincluded Ġin Ġthe Ġfeature Ġvector . </s> Class ification ĠApproach Ġand ĠResults </s> We Ġexperimented Ġwith Ġfour Ġdifferent Ġclass ifiers , Ġnamely , Ġsupport Ġvector Ġmachine ĠB IB REF 18 Ġ, Ġrandom Ġforest , Ġextra Ġtree Ġand Ġnaive Ġbay es Ġclass ifier ĠB IB REF 19 Ġ. ĠChi Ġsquare Ġfeature Ġselection Ġalgorithm Ġis Ġapplied Ġto Ġreduces Ġthe Ġsize Ġof Ġour Ġfeature Ġvector . ĠFor Ġtraining Ġour Ġsystem Ġclass ifier , Ġwe Ġused ĠSci kit - learn ĠB IB REF 19 Ġ. </s> 10 - fold Ġcross Ġvalidation Ġon Ġ35 43 Ġcode - m ixed Ġtweets Ġwas Ġcarried Ġout Ġby Ġdividing Ġthe Ġcorpus Ġinto Ġ10 Ġequal Ġparts Ġwith Ġnine Ġparts Ġas Ġtraining Ġcorpus Ġand Ġrest Ġone Ġfor Ġtesting . ĠMean Ġaccuracy Ġis Ġcalculated Ġby Ġtaking Ġthe Ġaverage Ġof Ġthe Ġaccuracy Ġobtained Ġin Ġeach Ġiteration Ġof Ġthe Ġtesting Ġprocess . ĠTable Ġ2 Ġshows Ġthe Ġaccuracy Ġfor Ġeach Ġfeature Ġwhen Ġtrained Ġusing Ġmentioned Ġclass ifiers Ġalong Ġwith Ġthe Ġaccuracy Ġwhen Ġall Ġthe Ġfeatures Ġare Ġused Ġalong Ġwith Ġthe Ġoverall Ġaccuracy . ĠSupport Ġvector Ġmachine Ġwith Ġradial Ġbasis Ġfunction Ġkernel Ġand Ġextra Ġtree Ġclass ifier Ġperforms Ġbetter Ġthan Ġother Ġclass ifiers Ġand Ġyields Ġ69 . 3 % Ġand Ġ67 . 8 % Ġaccuracy Ġrespectively . ĠThe Ġreason Ġkernel ĠS VM Ġyields Ġthe Ġbest Ġresult Ġis Ġbecause Ġthe Ġnumber Ġof Ġobservations Ġis Ġgreat or Ġthan Ġthe Ġnumber Ġof Ġfeatures ĠB IB REF 20 Ġ. ĠN - gram s Ġproved Ġto Ġbe Ġthe Ġmost Ġefficient Ġin Ġall Ġclassification Ġmodels Ġfollowed Ġby Ġcommon Ġwords Ġand Ġhast ags . ĠBag - of - words Ġfeature Ġperformed Ġthe Ġworst Ġin ĠS VM , Ġrandom Ġforest Ġand Ġextra Ġtree Ġclass ifier Ġbut Ġyielded Ġbetter Ġresult Ġin Ġnaive Ġbay es Ġclass ifiers . ĠAcc ur acies Ġmentioned Ġin Ġtable Ġ2 Ġwere Ġcalculated Ġusing Ġfine Ġtuning Ġof Ġmodel Ġparameters Ġusing Ġgrid Ġsearch . </s> Conclusion Ġand ĠFuture ĠWork </s> In Ġthis Ġpaper , Ġwe Ġdescribe Ġa Ġfreely Ġavailable Ġcorpus Ġof Ġ3 453 ĠEnglish - H ind i Ġcode - m ixed Ġtweets . ĠThe Ġtweets Ġare Ġannot ated Ġwith Ġhumorous ( H ) Ġand Ġnon - hum orous ( N ) Ġtags Ġalong Ġwith Ġthe Ġlanguage Ġtags Ġat Ġthe Ġword Ġlevel . ĠThe Ġtask Ġof Ġhumor Ġidentification Ġin Ġsocial Ġmedia Ġtexts Ġis Ġanalyzed Ġas Ġa Ġclassification Ġproblem Ġand Ġseveral Ġmachine Ġlearning Ġclassification Ġmodels Ġare Ġused . ĠThe Ġfeatures Ġused Ġin Ġour Ġclassification Ġsystem Ġare Ġn - gram s , Ġbag - of - words , Ġcommon Ġwords Ġand Ġhasht ags . ĠN - gram s Ġwhen Ġtrained Ġwith Ġsupport Ġvector Ġmachines Ġwith Ġradial Ġbasis Ġfunction Ġkernel Ġperformed Ġbetter Ġthan Ġother Ġfeatures Ġand Ġyielded Ġan Ġaccuracy Ġof Ġ68 . 5 %. ĠThe Ġbest Ġaccuracy Ġ( 69 . 3 %) Ġwas Ġgiven Ġby Ġsupport Ġvector Ġmachines Ġwith Ġradial Ġbasis Ġfunction Ġkernel . </s> This Ġpaper Ġdescribes Ġthe Ġinitial Ġefforts Ġin Ġautomatic Ġhumor Ġdetection Ġin Ġcode - m ixed Ġsocial Ġmedia Ġtexts . ĠCorpus Ġcan Ġbe Ġannot ated Ġwith Ġpart - of - speech Ġtags Ġat Ġthe Ġword Ġlevel Ġwhich Ġmay Ġyield Ġbetter Ġresults Ġin Ġlanguage Ġdetection . ĠMoreover , Ġthe Ġdataset Ġcan Ġbe Ġfurther Ġextended Ġto Ġinclude Ġtweets Ġfrom Ġother Ġdomains . ĠCode - mix ing Ġis Ġvery Ġcommon Ġphenomenon Ġon Ġsocial Ġmedia Ġand Ġit Ġis Ġprevalent Ġmostly Ġin Ġmult ilingual Ġregions . ĠIt Ġwould Ġbe Ġinteresting Ġto Ġexperiment Ġwith Ġcode - m ixed Ġtexts Ġconsisting Ġof Ġmore Ġthan Ġtwo Ġlanguages Ġin Ġwhich Ġthe Ġissue Ġof Ġtransl iter ation Ġexists Ġlike ĠArabic , ĠGreek Ġand ĠSouth ĠAsian Ġlanguages . ĠComp aring Ġtraining Ġwith Ġcode - m ixed Ġtweets Ġwith Ġtraining Ġwith Ġa Ġmerged Ġcorpus Ġof Ġmon oling ual Ġtweets Ġin ĠEnglish Ġand ĠHindi Ġcould Ġbe Ġan Ġinteresting Ġfuture Ġwork .