22,880 users
EXTRACTIVE
20,000
EXTRACTIVE
The final set of categories is shown in Table TABREF1 , along with the number of users in each category. The resulting dataset consists of 22,880 users, 41,094 blogs, and 561,003 posts. Table TABREF2 presents additional statistics of our dataset.
Specifically, this paper makes four main contributions. First, we build a large, industry-annotated dataset that contains over 20,000 blog users. In addition to their posted text, we also link a number of user metadata including their gender, location, occupation, introduction and interests.
1612.08205
Introduction </s> Over Ġthe Ġpast Ġtwo Ġdecades , Ġthe Ġemergence Ġof Ġsocial Ġmedia Ġhas Ġenabled Ġthe Ġproliferation Ġof Ġtrace able Ġhuman Ġbehavior . ĠThe Ġcontent Ġposted Ġby Ġusers Ġcan Ġreflect Ġwho Ġtheir Ġfriends Ġare , Ġwhat Ġtopics Ġthey Ġare Ġinterested Ġin , Ġor Ġwhich Ġcompany Ġthey Ġare Ġworking Ġfor . ĠAt Ġthe Ġsame Ġtime , Ġusers Ġare Ġlisting Ġa Ġnumber Ġof Ġprofile Ġfields Ġto Ġdefine Ġthemselves Ġto Ġothers . ĠThe Ġutilization Ġof Ġsuch Ġmetadata Ġhas Ġproven Ġimportant Ġin Ġfacilitating Ġfurther Ġdevelopments Ġof Ġapplications Ġin Ġadvertising ĠB IB REF 0 Ġ, Ġpersonal ization ĠB IB REF 1 Ġ, Ġand Ġrecomm ender Ġsystems ĠB IB REF 2 Ġ. ĠHowever , Ġprofile Ġinformation Ġcan Ġbe Ġlimited , Ġdepending Ġon Ġthe Ġplatform , Ġor Ġit Ġis Ġoften Ġdeliberately Ġomitted ĠB IB REF 3 Ġ. ĠTo Ġun cl oak Ġthis Ġinformation , Ġa Ġnumber Ġof Ġstudies Ġhave Ġutilized Ġsocial Ġmedia Ġusers ' Ġfootprints Ġto Ġapproximate Ġtheir Ġprofiles . </s> This Ġpaper Ġexplores Ġthe Ġpotential Ġof Ġpredicting Ġa Ġuser 's Ġindustry ĠâĢĵ the Ġaggregate Ġof Ġenterprises Ġin Ġa Ġparticular Ġfield âĢĵ Ġby Ġidentifying Ġindustry Ġindicative Ġtext Ġin Ġsocial Ġmedia . ĠThe Ġaccurate Ġprediction Ġof Ġusers ' Ġindustry Ġcan Ġhave Ġa Ġbig Ġimpact Ġon Ġtargeted Ġadvertising Ġby Ġminimizing Ġwasted Ġadvertising ĠB IB REF 4 Ġand Ġimproved Ġpersonalized Ġuser Ġexperience . ĠA Ġnumber Ġof Ġstudies Ġin Ġthe Ġsocial Ġsciences Ġhave Ġassociated Ġlanguage Ġuse Ġwith Ġsocial Ġfactors Ġsuch Ġas Ġoccupation , Ġsocial Ġclass , Ġeducation , Ġand Ġincome ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ. ĠAn Ġadditional Ġgoal Ġof Ġthis Ġpaper Ġis Ġto Ġexamine Ġsuch Ġfindings , Ġand Ġin Ġparticular Ġthe Ġlink Ġbetween Ġlanguage Ġand Ġoccupational Ġclass , Ġthrough Ġa Ġdata - driven Ġapproach . </s> In Ġaddition , Ġwe Ġexplore Ġhow Ġmeaning Ġchanges Ġdepending Ġon Ġthe Ġoccupational Ġcontext . ĠBy Ġleveraging Ġword Ġembed d ings , Ġwe Ġseek Ġto Ġquantify Ġhow , Ġfor Ġexample , Ġcloud Ġmight Ġmean Ġa Ġseparate Ġconcept Ġ( e . g ., Ġcondensed Ġwater Ġvapor ) Ġin Ġthe Ġtext Ġwritten Ġby Ġusers Ġthat Ġwork Ġin Ġenvironmental Ġjobs Ġwhile Ġit Ġmight Ġbe Ġused Ġdifferently Ġby Ġusers Ġin Ġtechnology Ġoccupations Ġ( e . g ., ĠInternet - based Ġcomputing ). </s> Specifically , Ġthis Ġpaper Ġmakes Ġfour Ġmain Ġcontributions . ĠFirst , Ġwe Ġbuild Ġa Ġlarge , Ġindustry - annot ated Ġdataset Ġthat Ġcontains Ġover Ġ20 , 000 Ġblog Ġusers . ĠIn Ġaddition Ġto Ġtheir Ġposted Ġtext , Ġwe Ġalso Ġlink Ġa Ġnumber Ġof Ġuser Ġmetadata Ġincluding Ġtheir Ġgender , Ġlocation , Ġoccupation , Ġintroduction Ġand Ġinterests . </s> Second , Ġwe Ġbuild Ġcontent - based Ġclass ifiers Ġfor Ġthe Ġindustry Ġprediction Ġtask Ġand Ġstudy Ġthe Ġeffect Ġof Ġincorporating Ġtextual Ġfeatures Ġfrom Ġthe Ġusers ' Ġprofile Ġmetadata Ġusing Ġvarious Ġmeta - class ification Ġtechniques , Ġsignificantly Ġimproving Ġboth Ġthe Ġoverall Ġaccuracy Ġand Ġthe Ġaverage Ġper Ġindustry Ġaccuracy . </s> Next , Ġafter Ġexamining Ġwhich Ġwords Ġare Ġindicative Ġfor Ġeach Ġindustry , Ġwe Ġbuild Ġvector - space Ġrepresentations Ġof Ġword Ġmeanings Ġand Ġcalculate Ġone Ġdeviation Ġfor Ġeach Ġindustry , Ġillustrating Ġhow Ġmeaning Ġis Ġdifferentiated Ġbased Ġon Ġthe Ġusers ' Ġindustries . ĠWe Ġqual itatively Ġexamine Ġthe Ġresulting Ġindustry - informed Ġsemantic Ġrepresentations Ġof Ġwords Ġby Ġlisting Ġthe Ġwords Ġper Ġindustry Ġthat Ġare Ġmost Ġsimilar Ġto Ġjob Ġrelated Ġand Ġgeneral Ġinterest Ġterms . </s> Finally , Ġwe Ġrank Ġthe Ġdifferent Ġindustries Ġbased Ġon Ġthe Ġnormalized Ġrelative Ġfrequencies Ġof Ġemotionally Ġcharged Ġwords Ġ( positive Ġand Ġnegative ) Ġand , Ġin Ġaddition , Ġdiscover Ġthat , Ġfor Ġboth Ġgenders , Ġthese Ġfrequencies Ġdo Ġnot Ġstatistically Ġsignificantly Ġcorrelate Ġwith Ġan Ġindustry 's Ġgender Ġdominance Ġratio . </s> After Ġdiscussing Ġrelated Ġwork Ġin ĠSection ĠSEC REF 2 Ġ, Ġwe Ġpresent Ġthe Ġdataset Ġused Ġin Ġthis Ġstudy Ġin ĠSection ĠSEC REF 3 Ġ. ĠIn ĠSection ĠSEC REF 4 Ġwe Ġevaluate Ġtwo Ġfeature Ġselection Ġmethods Ġand Ġexamine Ġthe Ġindustry Ġinference Ġproblem Ġusing Ġthe Ġtext Ġof Ġthe Ġusers ' Ġpostings . ĠWe Ġthen Ġaugment Ġour Ġcontent - based Ġclass ifier Ġby Ġbuilding Ġan Ġensemble Ġthat Ġincorporates Ġseveral Ġmetadata Ġclass ifiers . ĠWe Ġlist Ġthe Ġmost Ġindustry Ġindicative Ġwords Ġand Ġexpose Ġhow Ġeach Ġindustrial Ġsemantic Ġfield Ġvaries Ġwith Ġrespect Ġto Ġa Ġvariety Ġof Ġterms Ġin ĠSection ĠSEC REF 5 Ġ. ĠWe Ġexplore Ġhow Ġthe Ġfrequencies Ġof Ġemotionally Ġcharged Ġwords Ġin Ġeach Ġgender Ġcorrelate Ġwith Ġthe Ġindustries Ġand Ġtheir Ġrespective Ġgender Ġdominance Ġratio Ġand , Ġfinally , Ġconclude Ġin ĠSection ĠSEC REF 6 Ġ. </s> Related ĠWork </s> Along side Ġthe Ġwide Ġadoption Ġof Ġsocial Ġmedia Ġby Ġthe Ġpublic , Ġresearchers Ġhave Ġbeen Ġleveraging Ġthe Ġnewly Ġavailable Ġdata Ġto Ġcreate Ġand Ġrefine Ġmodels Ġof Ġusers ' Ġbehavior Ġand Ġprofiling . ĠThere Ġexists Ġa Ġmyriad Ġresearch Ġthat Ġanaly zes Ġlanguage Ġin Ġorder Ġto Ġprofile Ġsocial Ġmedia Ġusers . ĠSome Ġstudies Ġsought Ġto Ġcharacterize Ġusers ' Ġpersonality ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ, Ġwhile Ġothers Ġsequ enced Ġthe Ġexpressed Ġemotions ĠB IB REF 11 Ġ, Ġstudied Ġmental Ġdisorders ĠB IB REF 12 Ġ, Ġand Ġthe Ġprogression Ġof Ġhealth Ġconditions ĠB IB REF 13 Ġ. ĠAt Ġthe Ġsame Ġtime , Ġa Ġnumber Ġof Ġresearchers Ġsought Ġto Ġpredict Ġthe Ġsocial Ġmedia Ġusers ' Ġage Ġand / or Ġgender ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 16 Ġ, Ġwhile Ġothers Ġtargeted Ġand Ġanalyzed Ġthe Ġethnicity , Ġnationality , Ġand Ġrace Ġof Ġthe Ġusers ĠB IB REF 17 Ġ, ĠB IB REF 18 Ġ, ĠB IB REF 19 Ġ. ĠOne Ġof Ġthe Ġprofile Ġfields Ġthat Ġhas Ġdrawn Ġa Ġgreat Ġdeal Ġof Ġattention Ġis Ġthe Ġlocation Ġof Ġa Ġuser . ĠAmong Ġothers , ĠHe cht Ġet Ġal . ĠHe cht 11 Ġpredicted ĠTwitter Ġusers ' Ġlocations Ġusing Ġmachine Ġlearning Ġon Ġnationwide Ġand Ġstate Ġlevels . ĠLater , ĠHan Ġet Ġal . ĠHan 14 Ġidentified Ġlocation Ġindicative Ġwords Ġto Ġpredict Ġthe Ġlocation Ġof ĠTwitter Ġusers Ġdown Ġto Ġthe Ġcity Ġlevel . </s> As Ġa Ġseparate Ġline Ġof Ġresearch , Ġa Ġnumber Ġof Ġstudies Ġhave Ġfocused Ġon Ġdiscovering Ġthe Ġpolitical Ġorientation Ġof Ġusers ĠB IB REF 15 Ġ, ĠB IB REF 20 Ġ, ĠB IB REF 21 Ġ. ĠFinally , ĠLi Ġet Ġal . ĠLi 14 a Ġproposed Ġa Ġway Ġto Ġmodel Ġmajor Ġlife Ġevents Ġsuch Ġas Ġgetting Ġmarried , Ġmoving Ġto Ġa Ġnew Ġplace , Ġor Ġgraduating . ĠIn Ġa Ġsubsequent Ġstudy , ĠB IB REF 22 Ġdescribed Ġa Ġweak ly Ġsupervised Ġinformation Ġextraction Ġmethod Ġthat Ġwas Ġused Ġin Ġconjunction Ġwith Ġsocial Ġnetwork Ġinformation Ġto Ġidentify Ġthe Ġname Ġof Ġa Ġuser 's Ġspouse , Ġthe Ġcollege Ġthey Ġattended , Ġand Ġthe Ġcompany Ġwhere Ġthey Ġare Ġemployed . </s> The Ġline Ġof Ġwork Ġthat Ġis Ġmost Ġclosely Ġrelated Ġto Ġour Ġresearch Ġis Ġthe Ġone Ġconcerned Ġwith Ġunderstanding Ġthe Ġrelation Ġbetween Ġpeople 's Ġlanguage Ġand Ġtheir Ġindustry . ĠPrevious Ġresearch Ġfrom Ġthe Ġfields Ġof Ġpsychology Ġand Ġeconomics Ġhave Ġexplored Ġthe Ġpotential Ġfor Ġpredicting Ġone 's Ġoccupation Ġfrom Ġtheir Ġability Ġto Ġuse Ġmath Ġand Ġverbal Ġsymbols ĠB IB REF 23 Ġand Ġthe Ġrelationship Ġbetween Ġjob - types Ġand Ġdemographics ĠB IB REF 24 Ġ. ĠMore Ġrecently , ĠHuang Ġet Ġal . ĠHuang 15 Ġused Ġmachine Ġlearning Ġto Ġclassify ĠSina ĠWe ibo Ġusers Ġto Ġtwelve Ġdifferent Ġplatform - defined Ġoccupational Ġclasses Ġhighlighting Ġthe Ġeffect Ġof Ġhom oph ily Ġin Ġuser Ġinteractions . ĠThis Ġwork Ġexamined Ġonly Ġusers Ġthat Ġhave Ġbeen Ġverified Ġby Ġthe ĠSina ĠWe ibo Ġplatform , Ġintroducing Ġa Ġpotential Ġbias Ġin Ġthe Ġresulting Ġdataset . ĠFinally , ĠPre oti uc - P iet ro Ġet Ġal . ĠPre oct i uc 15 Ġpredicted Ġthe Ġoccupational Ġclass Ġof ĠTwitter Ġusers Ġusing Ġthe ĠStandard ĠOccup ational ĠClassification Ġ( S OC ) Ġsystem , Ġwhich Ġgroups Ġthe Ġdifferent Ġjobs Ġbased Ġon Ġskill Ġrequirements . ĠIn Ġthat Ġwork , Ġthe Ġdata Ġcollection Ġprocess Ġwas Ġlimited Ġto Ġonly Ġusers Ġthat Ġspecifically Ġmentioned Ġtheir Ġoccupation Ġin Ġtheir Ġself - description Ġin Ġa Ġway Ġthat Ġcould Ġbe Ġdirectly Ġmapped Ġto Ġa ĠSOC Ġoccupational Ġclass . ĠThe Ġmapping Ġbetween Ġa Ġsubst ring Ġof Ġtheir Ġself - description Ġand Ġa ĠSOC Ġoccupational Ġclass Ġwas Ġdone Ġmanually . ĠBecause Ġof Ġthe Ġmanual Ġannotation Ġstep , Ġtheir Ġmethod Ġwas Ġnot Ġscalable ; Ġmoreover , Ġbecause Ġthey Ġidentified Ġthe Ġoccupation Ġclass Ġinside Ġa Ġuser Ġself - description , Ġonly Ġa Ġvery Ġsmall Ġfraction Ġof Ġthe ĠTwitter Ġusers Ġcould Ġbe Ġincluded Ġ( in Ġtheir Ġcase , Ġ5 , 191 Ġusers ). </s> Both Ġof Ġthese Ġrecent Ġstudies Ġare Ġbased Ġon Ġmicro - blog ging Ġplatforms , Ġwhich Ġinherently Ġrestrict Ġthe Ġnumber Ġof Ġcharacters Ġthat Ġa Ġpost Ġcan Ġhave , Ġand Ġconsequently Ġthe Ġway Ġthat Ġusers Ġcan Ġexpress Ġthemselves . </s> Moreover , Ġboth Ġstudies Ġused Ġoff - the - she lf Ġoccupational Ġtax onom ies Ġ( rather Ġthan Ġself - decl ared Ġoccupation Ġcategories ), Ġresulting Ġin Ġclasses Ġthat Ġare Ġeither Ġtoo Ġgeneric Ġ( e . g ., Ġmedia , Ġwelfare Ġand Ġelectronic Ġare Ġthree Ġof Ġthe Ġtwelve ĠSina ĠWe ibo Ġcategories ), Ġor Ġtoo Ġinter m ixed Ġ( e . g ., Ġan Ġassistant Ġaccountant Ġis Ġin Ġa Ġdifferent Ġclass Ġfrom Ġan Ġaccountant Ġin ĠSOC ). ĠTo Ġaddress Ġthese Ġlimitations , Ġwe Ġinvestigate Ġthe Ġindustry Ġprediction Ġtask Ġin Ġa Ġlarge Ġblog Ġcorpus Ġconsisting Ġof Ġover Ġ20 K ĠAmerican Ġusers , Ġ40 K Ġweb - blogs , Ġand Ġ560 K Ġblog Ġposts . </s> Dat as et </s> We Ġcompile Ġour Ġindustry - annot ated Ġdataset Ġby Ġidentifying Ġblogger Ġprofiles Ġlocated Ġin Ġthe ĠU . S . Ġon Ġthe Ġprofile Ġfind er Ġon Ġhttp :// www . blog ger . com , Ġand Ġscraping Ġonly Ġthose Ġusers Ġthat Ġhad Ġthe Ġindustry Ġprofile Ġelement Ġcompleted . </s> For Ġeach Ġof Ġthese Ġbloggers , Ġwe Ġretrieve Ġall Ġtheir Ġblogs , Ġand Ġfor Ġeach Ġof Ġthese Ġblogs Ġwe Ġdownload Ġthe Ġ21 Ġmost Ġrecent Ġblog Ġpostings . ĠWe Ġthen Ġclean Ġthese Ġblog Ġposts Ġof ĠHTML Ġtags Ġand Ġtoken ize Ġthem , Ġand Ġdrop Ġthose Ġbloggers Ġwhose Ġcumulative Ġtextual Ġcontent Ġin Ġtheir Ġposts Ġis Ġless Ġthan Ġ600 Ġcharacters . ĠFollowing Ġthese Ġguidelines , Ġwe Ġidentified Ġall Ġthe ĠU . S . Ġbloggers Ġwith Ġcompleted Ġindustry Ġinformation . </s> Tr ad itionally , Ġstandardized Ġindustry Ġtax onom ies Ġorganize Ġeconomic Ġactivities Ġinto Ġgroups Ġbased Ġon Ġsimilar Ġproduction Ġprocesses , Ġproducts Ġor Ġservices , Ġdelivery Ġsystems Ġor Ġbehavior Ġin Ġfinancial Ġmarkets . ĠFollowing Ġsuch Ġassumptions Ġand Ġregardless Ġof Ġtheir Ġmany Ġsimilarities , Ġa Ġtomato Ġfarmer Ġwould Ġbe Ġcategorized Ġinto Ġa Ġdistinct Ġindustry Ġfrom Ġa Ġtobacco Ġfarmer . ĠAs Ġdemonstrated Ġin ĠPre oti uc - P iet ro Ġet Ġal . ĠPre oct i uc 15 Ġsuch Ġgroup ings Ġcan Ġcause Ġunw arranted Ġmis class ifications . </s> The ĠBlog ger Ġplatform Ġprovides Ġa Ġtotal Ġof Ġ39 Ġdifferent Ġindustry Ġoptions . ĠEven Ġthough Ġa Ġcompleted Ġindustry Ġvalue Ġis Ġan Ġimplicit Ġtext Ġannotation , Ġwe Ġacknowledge Ġthe Ġsame Ġproblem Ġnoted Ġin Ġprevious Ġstudies : Ġsome Ġcategories Ġare Ġtoo Ġbroad , Ġwhile Ġothers Ġare Ġvery Ġsimilar . ĠTo Ġremedy Ġthis Ġand Ġfollowing ĠGu iber t Ġet Ġal . ĠGu iber t 71 , Ġwho Ġargued Ġthat Ġthe Ġdenominations Ġused Ġin Ġa Ġclassification Ġmust Ġreflect Ġthe Ġpurpose Ġof Ġthe Ġstudy , Ġwe Ġgroup Ġthe Ġdifferent ĠBlog ger Ġindustries Ġbased Ġon Ġsimilar Ġeducational Ġbackground Ġand Ġsimilar Ġtechnical Ġterminology . ĠTo Ġdo Ġthat , Ġwe Ġexclude Ġvery Ġgeneral Ġcategories Ġand Ġmerge Ġconcept ually Ġsimilar Ġones . ĠExamples Ġof Ġbroad Ġcategories Ġare Ġthe ĠEducation Ġand Ġthe ĠStudent Ġoptions : Ġa Ġteacher Ġcould Ġbe Ġteaching Ġin Ġany Ġconcentration , Ġwhile Ġa Ġstudent Ġcould Ġbe Ġenrolled Ġin Ġany Ġdiscipline . ĠExamples Ġof Ġconcept ually Ġsimilar Ġcategories Ġare Ġthe ĠInvestment ĠBanking Ġand Ġthe ĠBanking Ġoptions . </s> The Ġfinal Ġset Ġof Ġcategories Ġis Ġshown Ġin ĠTable ĠT AB REF 1 Ġ, Ġalong Ġwith Ġthe Ġnumber Ġof Ġusers Ġin Ġeach Ġcategory . ĠThe Ġresulting Ġdataset Ġconsists Ġof Ġ22 , 880 Ġusers , Ġ41 , 09 4 Ġblogs , Ġand Ġ5 61 , 003 Ġposts . ĠTable ĠT AB REF 2 Ġpresents Ġadditional Ġstatistics Ġof Ġour Ġdataset . </s> Text - based ĠIndustry ĠModel ing </s> After Ġcollecting Ġour Ġdataset , Ġwe Ġsplit Ġit Ġinto Ġthree Ġsets : Ġa Ġtrain Ġset , Ġa Ġdevelopment Ġset , Ġand Ġa Ġtest Ġset . ĠThe Ġsizes Ġof Ġthese Ġsets Ġare Ġ17 , 880 , Ġ2 , 500 , Ġand Ġ2 , 500 Ġusers , Ġrespectively , Ġwith Ġusers Ġrandomly Ġassigned Ġto Ġthese Ġsets . ĠIn Ġall Ġthe Ġexperiments Ġthat Ġfollow , Ġwe Ġevaluate Ġour Ġclass ifiers Ġby Ġtraining Ġthem Ġon Ġthe Ġtrain Ġset , Ġconfigure Ġthe Ġparameters Ġand Ġmeasure Ġperformance Ġon Ġthe Ġdevelopment Ġset , Ġand Ġfinally Ġreport Ġthe Ġprediction Ġaccuracy Ġand Ġresults Ġon Ġthe Ġtest Ġset . ĠNote Ġthat Ġall Ġthe Ġexperiments Ġare Ġperformed Ġat Ġuser Ġlevel , Ġi . e ., Ġall Ġthe Ġdata Ġfor Ġone Ġuser Ġis Ġcompiled Ġinto Ġone Ġinstance Ġin Ġour Ġdata Ġsets . </s> To Ġmeasure Ġthe Ġperformance Ġof Ġour Ġclass ifiers , Ġwe Ġuse Ġthe Ġprediction Ġaccuracy . ĠHowever , Ġas Ġshown Ġin ĠTable ĠT AB REF 1 Ġ, Ġthe Ġavailable Ġdata Ġis Ġskewed Ġacross Ġcategories , Ġwhich Ġcould Ġlead Ġto Ġsomewhat Ġdistorted Ġaccuracy Ġnumbers Ġdepending Ġon Ġhow Ġwell Ġa Ġmodel Ġlearns Ġto Ġpredict Ġthe Ġmost Ġpopulous Ġclasses . ĠMoreover , Ġaccuracy Ġalone Ġdoes Ġnot Ġprovide Ġa Ġgreat Ġdeal Ġof Ġinsight Ġinto Ġthe Ġindividual Ġperformance Ġper Ġindustry , Ġwhich Ġis Ġone Ġof Ġthe Ġmain Ġobjectives Ġin Ġthis Ġstudy . ĠTherefore , Ġin Ġour Ġresults Ġbelow , Ġwe Ġreport : Ġ( 1 ) Ġmicro - acc uracy Ġ( ĠIN LINE FORM 0 Ġ), Ġcalculated Ġas Ġthe Ġpercentage Ġof Ġcorrectly Ġclassified Ġinstances Ġout Ġof Ġall Ġthe Ġinstances Ġin Ġthe Ġdevelopment Ġ( test ) Ġdata ; Ġand Ġ( 2 ) Ġmacro - acc uracy Ġ( ĠIN LINE FORM 1 Ġ), Ġcalculated Ġas Ġthe Ġaverage Ġof Ġthe Ġper - category Ġaccur acies , Ġwhere Ġthe Ġper - category Ġaccuracy Ġis Ġthe Ġpercentage Ġof Ġcorrectly Ġclassified Ġinstances Ġout Ġof Ġthe Ġinstances Ġbelonging Ġto Ġone Ġcategory Ġin Ġthe Ġdevelopment Ġ( test ) Ġdata . </s> L ever aging ĠBlog ĠContent </s> In Ġthis Ġsection , Ġwe Ġseek Ġthe Ġeffectiveness Ġof Ġusing Ġsolely Ġtextual Ġfeatures Ġobtained Ġfrom Ġthe Ġusers ' Ġpostings Ġto Ġpredict Ġtheir Ġindustry . </s> The Ġindustry Ġprediction Ġbaseline ĠMajority Ġis Ġset Ġby Ġdiscovering Ġthe Ġmost Ġfrequently Ġfeatured Ġclass Ġin Ġour Ġtraining Ġset Ġand Ġpicking Ġthat Ġclass Ġin Ġall Ġpredictions Ġin Ġthe Ġrespective Ġdevelopment Ġor Ġtesting Ġset . </s> After Ġexcluding Ġall Ġthe Ġwords Ġthat Ġare Ġnot Ġused Ġby Ġat Ġleast Ġthree Ġseparate Ġusers Ġin Ġour Ġtraining Ġset , Ġwe Ġbuild Ġour ĠAll Words Ġmodel Ġby Ġcounting Ġthe Ġfrequencies Ġof Ġall Ġthe Ġremaining Ġwords Ġand Ġtraining Ġa Ġmult in omial ĠNa ive ĠBay es Ġclass ifier . ĠAs Ġseen Ġin ĠFigure ĠFIG REF 3 Ġ, Ġwe Ġcan Ġfar Ġexceed Ġthe ĠMajority Ġbaseline Ġperformance Ġby Ġincorporating Ġbasic Ġlanguage Ġsignals Ġinto Ġmachine Ġlearning Ġalgorithms Ġ( 173 % ĠIN LINE FORM 0 Ġimprovement ). </s> We Ġadditionally Ġexplore Ġthe Ġpotential Ġof Ġimproving Ġour Ġtext Ġclassification Ġtask Ġby Ġapplying Ġa Ġnumber Ġof Ġfeature Ġranking Ġmethods Ġand Ġselecting Ġvarying Ġproportions Ġof Ġtop Ġranked Ġfeatures Ġin Ġan Ġattempt Ġto Ġexclude Ġnoisy Ġfeatures . ĠWe Ġstart Ġby Ġranking Ġthe Ġdifferent Ġfeatures , Ġw , Ġaccording Ġto Ġtheir ĠInformation ĠGain ĠRatio Ġscore Ġ( IG R ) Ġwith Ġrespect Ġto Ġevery Ġindustry , Ġi , Ġand Ġtraining Ġour Ġclass ifier Ġusing Ġdifferent Ġproportions Ġof Ġthe Ġtop Ġfeatures . ĠIN LINE FORM 0 ĠIN LINE FORM 1 </s> Even Ġthough Ġwe Ġfind Ġthat Ġusing Ġthe Ġtop Ġ95 % Ġof Ġall Ġthe Ġfeatures Ġalready Ġexceeds Ġthe Ġperformance Ġof Ġthe ĠAll ĠWords Ġmodel Ġon Ġthe Ġdevelopment Ġdata , Ġwe Ġfurther Ġexperiment Ġwith Ġranking Ġour Ġfeatures Ġwith Ġa Ġmore Ġaggressive Ġformula Ġthat Ġheavily Ġpromotes Ġthe Ġfeatures Ġthat Ġare Ġtightly Ġassociated Ġwith Ġany Ġindustry Ġcategory . ĠTherefore , Ġfor Ġevery Ġword Ġin Ġour Ġtraining Ġset , Ġwe Ġdefine Ġour Ġnewly Ġintroduced Ġranking Ġmethod , Ġthe ĠAgg ressive ĠFeature ĠRanking Ġ( AF R ), Ġas : ĠIN LINE FORM 0 </s> In ĠFigure ĠFIG REF 3 Ġwe Ġillustrate Ġthe Ġperformance Ġof Ġall Ġfour Ġmethods Ġin Ġour Ġindustry Ġprediction Ġtask Ġon Ġthe Ġdevelopment Ġdata . ĠNote Ġthat Ġfor Ġeach Ġmethod , Ġwe Ġprovide Ġboth Ġthe Ġaccuracy Ġ( ĠIN LINE FORM 0 Ġ) Ġand Ġthe Ġaverage Ġper - class Ġaccuracy Ġ( ĠIN LINE FORM 1 Ġ). ĠThe ĠMajority Ġand ĠAll ĠWords Ġmethods Ġapply Ġto Ġall Ġthe Ġfeatures ; Ġtherefore , Ġthey Ġare Ġrepresented Ġas Ġa Ġstraight Ġline Ġin Ġthe Ġfigure . ĠThe ĠI GR Ġand ĠA FR Ġmethods Ġare Ġapplied Ġto Ġvarying Ġsubs ets Ġof Ġthe Ġfeatures Ġusing Ġa Ġ5 % Ġstep . </s> Our Ġexperiments Ġdemonstrate Ġthat Ġthe Ġword Ġchoice Ġthat Ġthe Ġusers Ġmake Ġin Ġtheir Ġposts Ġcorrelates Ġwith Ġtheir Ġindustry . ĠThe Ġfirst Ġobservation Ġin ĠFigure ĠFIG REF 3 Ġis Ġthat Ġthe ĠIN LINE FORM 0 Ġis Ġproportional Ġto ĠIN LINE FORM 1 Ġ; Ġas ĠIN LINE FORM 2 Ġincreases , Ġso Ġdoes ĠIN LINE FORM 3 Ġ. ĠSecondly , Ġthe Ġbest Ġresult Ġon Ġthe Ġdevelopment Ġset Ġis Ġachieved Ġby Ġusing Ġthe Ġtop Ġ90 % Ġof Ġthe Ġfeatures Ġusing Ġthe ĠA FR Ġmethod . ĠLastly , Ġthe Ġimprovements Ġof Ġthe ĠI GR Ġand ĠA FR Ġfeature Ġselections Ġare Ġnot Ġsubstantially Ġbetter Ġin Ġcomparison Ġto ĠAll ĠWords Ġ( at Ġmost Ġ5 % Ġimprovement Ġbetween ĠAll ĠWords Ġand ĠA FR ), Ġwhich Ġsuggest Ġthat Ġonly Ġa Ġfew Ġnoisy Ġfeatures Ġexist Ġand Ġmost Ġof Ġthe Ġwords Ġplay Ġsome Ġrole Ġin Ġshaping Ġthe ĠâĢ ľ language " Ġof Ġan Ġindustry . </s> As Ġa Ġfinal Ġevaluation , Ġwe Ġapply Ġon Ġthe Ġtest Ġdata Ġthe Ġclass ifier Ġfound Ġto Ġwork Ġbest Ġon Ġthe Ġdevelopment Ġdata Ġ( AF R Ġfeature Ġselection , Ġtop Ġ90 % Ġfeatures ), Ġfor Ġan ĠIN LINE FORM 0 Ġof Ġ0 . 5 34 Ġand ĠIN LINE FORM 1 Ġof Ġ0 . 477 . </s> L ever aging ĠUser ĠMet adata </s> Together Ġwith Ġthe Ġindustry Ġinformation Ġand Ġthe Ġmost Ġrecent Ġpostings Ġof Ġeach Ġblogger , Ġwe Ġalso Ġdownload Ġa Ġnumber Ġof Ġaccompanying Ġprofile Ġelements . ĠUsing Ġthese Ġadditional Ġelements , Ġwe Ġexplore Ġthe Ġpotential Ġof Ġincorporating Ġusers ' Ġmetadata Ġin Ġour Ġclass ifiers . </s> Table ĠT AB REF 7 Ġshows Ġthe Ġdifferent Ġuser Ġmetadata Ġwe Ġconsider Ġtogether Ġwith Ġtheir Ġcoverage Ġpercentage Ġ( not Ġall Ġusers Ġprovide Ġa Ġvalue Ġfor Ġall Ġof Ġthe Ġprofile Ġelements ). ĠWith Ġthe Ġexception Ġof Ġthe Ġgender Ġfield , Ġthe Ġremaining Ġmetadata Ġelements Ġshown Ġin ĠTable ĠT AB REF 7 Ġare Ġcompleted Ġby Ġthe Ġusers Ġas Ġa Ġfreely Ġedit able Ġtext Ġfield . ĠThis Ġintroduces Ġa Ġconsiderable Ġamount Ġof Ġnoise Ġin Ġthe Ġset Ġof Ġpossible Ġmetadata Ġvalues . ĠExamples Ġof Ġnoise Ġin Ġthe Ġoccupation Ġfield Ġinclude Ġvalues Ġsuch Ġas ĠâĢ ľ Ret ired âĢ Ŀ , ĠâĢ ľ I Ġwork . âĢ Ŀ , Ġor ĠâĢ ľ m omm a âĢ Ŀ Ġwhich Ġare Ġnot Ġnecessarily Ġinformative Ġfor Ġour Ġindustry Ġprediction Ġtask . </s> To Ġexamine Ġwhether Ġthe Ġmetadata Ġfields Ġcan Ġhelp Ġin Ġthe Ġprediction Ġof Ġa Ġuser 's Ġindustry , Ġwe Ġbuild Ġclass ifiers Ġusing Ġthe Ġdifferent Ġmetadata Ġelements . ĠFor Ġeach Ġmetadata Ġelement Ġthat Ġhas Ġa Ġtextual Ġvalue , Ġwe Ġuse Ġall Ġthe Ġwords Ġin Ġthe Ġtraining Ġset Ġfor Ġthat Ġfield Ġas Ġfeatures . ĠThe Ġonly Ġtwo Ġexceptions Ġare Ġthe Ġstate Ġfield , Ġwhich Ġis Ġencoded Ġas Ġone Ġfeature Ġthat Ġcan Ġtake Ġone Ġout Ġof Ġ50 Ġdifferent Ġvalues Ġrepresenting Ġthe Ġ50 ĠU . S . Ġstates ; Ġand Ġthe Ġgender Ġfield , Ġwhich Ġis Ġencoded Ġas Ġa Ġfeature Ġwith Ġa Ġdistinct Ġvalue Ġfor Ġeach Ġuser Ġgender Ġoption : Ġundefined , Ġmale , Ġor Ġfemale . </s> As Ġshown Ġin ĠTable ĠT AB REF 9 Ġ, Ġwe Ġbuild Ġfour Ġdifferent Ġclass ifiers Ġusing Ġthe Ġmult in omial ĠNB Ġalgorithm : ĠOcc u Ġ( which Ġuses Ġthe Ġwords Ġfound Ġin Ġthe Ġoccupation Ġprofile Ġelement ), ĠIntro Ġ( introdu ction ), ĠInter Ġ( interest s ), Ġand ĠGl oc Ġ( comb ined Ġgender , Ġcity , Ġstate ). </s> In Ġgeneral , Ġall Ġthe Ġmetadata Ġclass ifiers Ġperform Ġbetter Ġthan Ġour Ġmajority Ġbaseline Ġ( ĠIN LINE FORM 0 Ġof Ġ18 . 88 %). ĠFor Ġthe ĠGl oc Ġclass ifier , Ġthis Ġresult Ġis Ġin Ġalignment Ġwith Ġprevious Ġstudies ĠB IB REF 24 Ġ. ĠHowever , Ġthe Ġonly Ġmetadata Ġclass ifier Ġthat Ġoutper forms Ġthe Ġcontent Ġclass ifier Ġis Ġthe ĠOcc u Ġclass ifier , Ġwhich Ġdespite Ġmissing Ġand Ġnoisy Ġoccupation Ġvalues Ġexceeds Ġthe Ġcontent Ġclass ifier 's Ġperformance Ġby Ġan Ġabsolute Ġ3 . 2 %. </s> To Ġinvestigate Ġthe Ġpromise Ġof Ġcombining Ġthe Ġfive Ġdifferent Ġclass ifiers Ġwe Ġhave Ġbuilt Ġso Ġfar , Ġwe Ġcalculate Ġtheir Ġinter - pred iction Ġagreement Ġusing ĠFle iss 's ĠKappa ĠB IB REF 25 Ġ, Ġas Ġwell Ġas Ġthe Ġlower Ġprediction Ġbounds Ġusing Ġthe Ġdouble Ġfault Ġmeasure ĠB IB REF 26 Ġ. ĠThe ĠKappa Ġvalues , Ġpresented Ġin Ġthe Ġlower Ġleft Ġside Ġof ĠTable ĠT AB REF 10 Ġ, Ġexpress Ġthe Ġclassification Ġagreement Ġfor Ġcateg orical Ġitems , Ġin Ġthis Ġcase Ġthe Ġusers ' Ġindustry . ĠLower Ġvalues , Ġespecially Ġvalues Ġbelow Ġ30 %, Ġmean Ġsmaller Ġagreement . ĠSince Ġall Ġfive Ġclass ifiers Ġhave Ġbetter - than - bas eline Ġaccuracy , Ġthis Ġlow Ġagreement Ġsuggests Ġthat Ġtheir Ġpredictions Ġcould Ġpotentially Ġbe Ġcombined Ġto Ġachieve Ġa Ġbetter Ġaccumulated Ġresult . </s> Moreover , Ġthe Ġdouble Ġfault Ġmeasure Ġvalues , Ġwhich Ġare Ġpresented Ġin Ġthe Ġtop - right Ġhand Ġside Ġof ĠTable ĠT AB REF 10 Ġ, Ġexpress Ġthe Ġproportion Ġof Ġtest Ġcases Ġfor Ġwhich Ġboth Ġof Ġthe Ġtwo Ġrespective Ġclass ifiers Ġmake Ġfalse Ġpredictions , Ġessentially Ġproviding Ġthe Ġlowest Ġerror Ġbound Ġfor Ġthe Ġpair wise Ġensemble Ġclass ifier Ġperformance . ĠThe Ġlower Ġthose Ġnumbers Ġare , Ġthe Ġgreater Ġthe Ġaccuracy Ġpotential Ġof Ġany Ġmeta - class ification Ġscheme Ġthat Ġcombines Ġthose Ġclass ifiers . ĠOnce Ġagain , Ġthe Ġlow Ġdouble Ġfault Ġmeasure Ġvalues Ġsuggest Ġpotential Ġgain Ġfrom Ġa Ġcombination Ġof Ġthe Ġbase Ġclass ifiers Ġinto Ġan Ġensemble Ġof Ġmodels . </s> After Ġestablishing Ġthe Ġpromise Ġof Ġcreating Ġan Ġensemble Ġof Ġclass ifiers , Ġwe Ġimplement Ġtwo Ġmeta - class ification Ġapproaches . ĠFirst , Ġwe Ġcombine Ġour Ġclass ifiers Ġusing Ġfeatures Ġconc aten ation Ġ( or Ġearly Ġfusion ). ĠStarting Ġwith Ġour Ġcontent - based Ġclass ifier Ġ( Text ), Ġwe Ġsuccess ively Ġadd Ġthe Ġfeatures Ġderived Ġfrom Ġeach Ġmetadata Ġelement . ĠThe Ġresults , Ġboth Ġmicro - Ġand Ġmacro - acc uracy , Ġare Ġpresented Ġin ĠTable ĠT AB REF 12 Ġ. ĠEven Ġthough Ġall Ġthese Ġfour Ġfeature Ġconc aten ation Ġen semb les Ġoutper form Ġthe Ġcontent - based Ġclass ifier Ġin Ġthe Ġdevelopment Ġset , Ġthey Ġfail Ġto Ġoutper form Ġthe ĠOcc u Ġclass ifier . </s> Second , Ġwe Ġexplore Ġthe Ġpotential Ġof Ġusing Ġstacked Ġgeneral ization Ġ( or Ġlate Ġfusion ) ĠB IB REF 27 Ġ. ĠThe Ġbase Ġclass ifiers , Ġreferred Ġto Ġas ĠL 0 Ġclass ifiers , Ġare Ġtrained Ġon Ġdifferent Ġfolds Ġof Ġthe Ġtraining Ġset Ġand Ġused Ġto Ġpredict Ġthe Ġclass Ġof Ġthe Ġremaining Ġinstances . ĠThose Ġpredictions Ġare Ġthen Ġused Ġtogether Ġwith Ġthe Ġtrue Ġlabel Ġof Ġthe Ġtraining Ġinstances Ġto Ġtrain Ġa Ġsecond Ġclass ifier , Ġreferred Ġto Ġas Ġthe ĠL 1 Ġclass ifier : Ġthis ĠL 1 Ġis Ġused Ġto Ġproduce Ġthe Ġfinal Ġprediction Ġon Ġboth Ġthe Ġdevelopment Ġdata Ġand Ġthe Ġtest Ġdata . ĠTrad itionally , Ġstacking Ġuses Ġdifferent Ġmachine Ġlearning Ġalgorithms Ġon Ġthe Ġsame Ġtraining Ġdata . ĠHowever Ġin Ġour Ġcase , Ġwe Ġuse Ġthe Ġsame Ġalgorithm Ġ( mult in omial ĠNB ) Ġon Ġheter ogeneous Ġdata Ġ( i . e ., Ġdifferent Ġtypes Ġof Ġdata Ġsuch Ġas Ġcontent , Ġoccupation , Ġintroduction , Ġinterests , Ġgender , Ġcity Ġand Ġstate ) Ġin Ġorder Ġto Ġexploit Ġall Ġavailable Ġsources Ġof Ġinformation . </s> The Ġensemble Ġlearning Ġresults Ġon Ġthe Ġdevelopment Ġset Ġare Ġshown Ġin ĠTable ĠT AB REF 12 Ġ. ĠWe Ġnotice Ġa Ġconstant Ġimprovement Ġfor Ġboth Ġmetrics Ġwhen Ġadding Ġmore Ġclass ifiers Ġto Ġour Ġensemble Ġexcept Ġfor Ġthe ĠGl oc Ġclass ifier , Ġwhich Ġslightly Ġreduces Ġthe Ġperformance . ĠThe Ġbest Ġresult Ġis Ġachieved Ġusing Ġan Ġensemble Ġof Ġthe ĠText , ĠOcc u , ĠIntro , Ġand ĠInter ĠL 0 Ġclass ifiers ; Ġthe Ġrespective Ġperformance Ġon Ġthe Ġtest Ġset Ġis Ġan ĠIN LINE FORM 0 Ġof Ġ0 . 643 Ġand Ġan ĠIN LINE FORM 1 Ġof Ġ0 . 5 64 . ĠFinally , Ġwe Ġpresent Ġin ĠFigure ĠFIG REF 11 Ġthe Ġprediction Ġaccuracy Ġfor Ġthe Ġfinal Ġclass ifier Ġfor Ġeach Ġof Ġthe Ġdifferent Ġindustries Ġin Ġour Ġtest Ġdataset . ĠEv idently , Ġsome Ġindustries Ġare Ġeasier Ġto Ġpredict Ġthan Ġothers . ĠFor Ġexample , Ġwhile Ġthe ĠReal ĠEstate Ġand ĠReligion Ġindustries Ġachieve Ġaccuracy Ġfigures Ġabove Ġ80 %, Ġother Ġindustries , Ġsuch Ġas Ġthe ĠBanking Ġindustry , Ġare Ġpredicted Ġcorrectly Ġin Ġless Ġthan Ġ17 % Ġof Ġthe Ġtime . ĠA ne cd otal Ġevidence Ġdrawn Ġfrom Ġthe Ġexamination Ġof Ġthe Ġconfusion Ġmatrix Ġdoes Ġnot Ġencourage Ġany Ġstrong Ġassociation Ġof Ġthe ĠBanking Ġclass Ġwith Ġany Ġother . ĠThe Ġmis class ifications Ġare Ġroughly Ġuniform Ġacross Ġall Ġother Ġclasses , Ġsuggesting Ġthat Ġthe Ġusers Ġin Ġthe ĠBanking Ġindustry Ġuse Ġlanguage Ġin Ġa Ġnon - dist ingu ishing Ġway . </s> Qual itative ĠAnalysis </s> In Ġthis Ġsection , Ġwe Ġprovide Ġa Ġqualitative Ġanalysis Ġof Ġthe Ġlanguage Ġof Ġthe Ġdifferent Ġindustries . </s> Top - Ranked ĠWords </s> To Ġconduct Ġa Ġqualitative Ġexploration Ġof Ġwhich Ġwords Ġindicate Ġthe Ġindustry Ġof Ġa Ġuser , ĠTable ĠT AB REF 14 Ġshows Ġthe Ġthree Ġtop - ranking Ġcontent Ġwords Ġfor Ġthe Ġdifferent Ġindustries Ġusing Ġthe ĠA FR Ġmethod . </s> Not Ġsurprisingly , Ġthe Ġtop Ġranked Ġwords Ġalign Ġwell Ġwith Ġwhat Ġwe Ġwould Ġintuitive ly Ġexpect Ġfor Ġeach Ġindustry . ĠEven Ġthough Ġmost Ġof Ġthese Ġwords Ġare Ġpotentially Ġused Ġby Ġmany Ġusers Ġregardless Ġof Ġtheir Ġindustry Ġin Ġour Ġdataset , Ġthey Ġare Ġstill Ġdistinguished Ġby Ġthe ĠA FR Ġmethod Ġbecause Ġof Ġthe Ġdifferent Ġfrequencies Ġof Ġthese Ġwords Ġin Ġthe Ġtext Ġof Ġeach Ġindustry . </s> Indust ry - specific ĠWord ĠSimilar ities </s> Next , Ġwe Ġexamine Ġhow Ġthe Ġmeaning Ġof Ġa Ġword Ġis Ġshaped Ġby Ġthe Ġcontext Ġin Ġwhich Ġit Ġis Ġuttered . ĠIn Ġparticular , Ġwe Ġqual itatively Ġinvestigate Ġhow Ġthe Ġspeakers ' Ġindustry Ġaffects Ġmeaning Ġby Ġlearning Ġvector - space Ġrepresentations Ġof Ġwords Ġthat Ġtake Ġinto Ġaccount Ġsuch Ġcontextual Ġinformation . ĠTo Ġachieve Ġthis , Ġwe Ġapply Ġthe Ġcontextual ized Ġword Ġembed d ings Ġproposed Ġby ĠBam man Ġet Ġal . ĠBam man 14 , Ġwhich Ġare Ġbased Ġon Ġan Ġextension Ġof Ġthe ĠâĢ ľ skip - gram " Ġlanguage Ġmodel ĠB IB REF 28 Ġ. </s> In Ġaddition Ġto Ġlearning Ġa Ġglobal Ġrepresentation Ġfor Ġeach Ġword , Ġthese Ġcontextual ized Ġembed d ings Ġcompute Ġone Ġdeviation Ġfrom Ġthe Ġcommon Ġword Ġembed ding Ġrepresentation Ġfor Ġeach Ġcontextual Ġvariable , Ġin Ġthis Ġcase , Ġan Ġindustry Ġoption . ĠThese Ġdeviations Ġcapture Ġthe Ġterms ' Ġmeaning Ġvariations Ġ( sh ifts Ġin Ġthe ĠIN LINE FORM 0 Ġ- dimensional Ġspace Ġof Ġthe Ġrepresentations , Ġwhere ĠIN LINE FORM 1 Ġin Ġour Ġexperiments ) Ġin Ġthe Ġtext Ġof Ġthe Ġdifferent Ġindustries , Ġhowever Ġall Ġthe Ġembed d ings Ġare Ġin Ġthe Ġsame Ġvector Ġspace Ġto Ġallow Ġfor Ġcomparisons Ġto Ġone Ġanother . </s> Using Ġthe Ġword Ġrepresentations Ġlearned Ġfor Ġeach Ġindustry , Ġwe Ġpresent Ġin ĠTable ĠT AB REF 16 Ġthe Ġterms Ġin Ġthe ĠTechnology Ġand Ġthe ĠTourism Ġindustries Ġthat Ġhave Ġthe Ġhighest Ġcos ine Ġsimilarity Ġwith Ġa Ġjob - related Ġword , Ġcustomers . ĠSimilarly , ĠTable ĠT AB REF 17 Ġshows Ġthe Ġwords Ġin Ġthe ĠEnvironment Ġand Ġthe ĠTourism Ġindustries Ġthat Ġare Ġclosest Ġin Ġmeaning Ġto Ġa Ġgeneral Ġinterest Ġword , Ġfood . ĠMore Ġexamples Ġare Ġgiven Ġin Ġthe ĠAppendix ĠSEC REF 8 Ġ. </s> The Ġterms Ġthat Ġrank Ġhighest Ġin Ġeach Ġindustry Ġare Ġnoticeably Ġdifferent . ĠFor Ġexample , Ġas Ġseen Ġin ĠTable ĠT AB REF 17 Ġ, Ġwhile Ġfood Ġin Ġthe ĠEnvironment Ġindustry Ġis Ġsimilar Ġto Ġnut r itionally Ġand Ġlocally , Ġin Ġthe ĠTourism Ġindustry Ġthe Ġsame Ġword Ġrelates Ġmore Ġto Ġterms Ġsuch Ġas Ġdelicious Ġand Ġpast ries . ĠThese Ġresults Ġnot Ġonly Ġemphasize Ġthe Ġexisting Ġdifferences Ġin Ġhow Ġpeople Ġin Ġdifferent Ġindustries Ġperceive Ġcertain Ġterms , Ġbut Ġthey Ġalso Ġdemonstrate Ġthat Ġthose Ġdifferences Ġcan Ġeffectively Ġbe Ġcaptured Ġin Ġthe Ġresulting Ġword Ġembed d ings . </s> Em otional ĠOrient ation Ġper ĠIndustry Ġand ĠGender </s> As Ġa Ġfinal Ġanalysis , Ġwe Ġexplore Ġhow Ġwords Ġthat Ġare Ġemotionally Ġcharged Ġrelate Ġto Ġdifferent Ġindustries . ĠTo Ġquantify Ġthe Ġemotional Ġorientation Ġof Ġa Ġtext , Ġwe Ġuse Ġthe ĠPositive ĠEm otion Ġand ĠNegative ĠEm otion Ġcategories Ġin Ġthe ĠL ingu istic ĠInquiry Ġand ĠWord ĠCount Ġ( LI WC ) Ġdictionary ĠB IB REF 29 Ġ. ĠThe ĠLI WC Ġdictionary Ġcontains Ġlists Ġof Ġwords Ġthat Ġhave Ġbeen Ġshown Ġto Ġcorrelate Ġwith Ġthe Ġpsychological Ġstates Ġof Ġpeople Ġthat Ġuse Ġthem ; Ġfor Ġexample , Ġthe ĠPositive ĠEm otion Ġcategory Ġcontains Ġwords Ġsuch Ġas ĠâĢ ľ happy , âĢ Ŀ ĠâĢ ľ pretty , âĢ Ŀ Ġand ĠâĢ ľ good . âĢ Ŀ </s> For Ġthe Ġtext Ġof Ġall Ġthe Ġusers Ġin Ġeach Ġindustry Ġwe Ġmeasure Ġthe Ġfrequencies Ġof ĠPositive ĠEm otion Ġand ĠNegative ĠEm otion Ġwords Ġnormalized Ġby Ġthe Ġtext 's Ġlength . ĠTable ĠT AB REF 20 Ġpresents Ġthe Ġindustries ' Ġranking Ġfor Ġboth Ġcategories Ġof Ġwords Ġbased Ġon Ġtheir Ġrelative Ġfrequencies Ġin Ġthe Ġtext Ġof Ġeach Ġindustry . </s> We Ġfurther Ġperform Ġa Ġbreakdown Ġper - gender , Ġwhere Ġwe Ġonce Ġagain Ġcalculate Ġthe Ġproportion Ġof Ġemotionally Ġcharged Ġwords Ġin Ġeach Ġindustry , Ġbut Ġseparately Ġfor Ġeach Ġgender . ĠWe Ġfind Ġthat Ġthe Ġindustry Ġrankings Ġof Ġthe Ġrelative Ġfrequencies ĠIN LINE FORM 0 Ġof Ġemotionally Ġcharged Ġwords Ġfor Ġthe Ġtwo Ġgenders Ġare Ġstatistically Ġsignificantly Ġcorrelated , Ġwhich Ġsuggests Ġthat Ġregardless Ġof Ġtheir Ġgender , Ġusers Ġuse Ġpositive Ġ( or Ġnegative ) Ġwords Ġwith Ġa Ġrelative Ġfrequency Ġthat Ġcorrelates Ġwith Ġtheir Ġindustry . Ġ( In Ġother Ġwords , Ġeven Ġif Ġe . g ., ĠFashion Ġhas Ġa Ġlarger Ġnumber Ġof Ġwomen Ġusers , Ġboth Ġmen Ġand Ġwomen Ġworking Ġin ĠFashion Ġwill Ġtend Ġto Ġuse Ġmore Ġpositive Ġwords Ġthan Ġthe Ġcorresponding Ġgender Ġin Ġanother Ġindustry Ġwith Ġa Ġlarger Ġnumber Ġof Ġmen Ġusers Ġsuch Ġas ĠAutom otive .) </s> Finally , Ġmotivated Ġby Ġprevious Ġfindings Ġof Ġcorrelations Ġbetween Ġjob Ġsatisfaction Ġand Ġgender Ġdominance Ġin Ġthe Ġworkplace ĠB IB REF 30 Ġ, Ġwe Ġexplore Ġthe Ġrelationship Ġbetween Ġthe Ġusage Ġof ĠPositive ĠEm otion Ġand ĠNegative ĠEm otion Ġwords Ġand Ġthe Ġgender Ġdominance Ġin Ġan Ġindustry . ĠAlthough Ġwe Ġfind Ġthat Ġthere Ġare Ġsubstantial Ġgender Ġim bal ances Ġin Ġeach Ġindustry Ġ( App endix ĠSEC REF 9 Ġ), Ġwe Ġdid Ġnot Ġfind Ġany Ġstatistically Ġsignificant Ġcorrelation Ġbetween Ġthe Ġgender Ġdominance Ġratio Ġin Ġthe Ġdifferent Ġindustries Ġand Ġthe Ġusage Ġof Ġpositive Ġ( or Ġnegative ) Ġemotional Ġwords Ġin Ġeither Ġgender Ġin Ġour Ġdataset . </s> Conclusion </s> In Ġthis Ġpaper , Ġwe Ġexamined Ġthe Ġtask Ġof Ġpredicting Ġa Ġsocial Ġmedia Ġuser 's Ġindustry . ĠWe Ġintroduced Ġan Ġannot ated Ġdataset Ġof Ġover Ġ20 , 000 Ġblog Ġusers Ġand Ġapplied Ġa Ġcontent - based Ġclass ifier Ġin Ġconjunction Ġwith Ġtwo Ġfeature Ġselection Ġmethods Ġfor Ġan Ġoverall Ġaccuracy Ġof Ġup Ġto Ġ0 . 5 34 , Ġwhich Ġrepresents Ġa Ġlarge Ġimprovement Ġover Ġthe Ġmajority Ġclass Ġbaseline Ġof Ġ0 . 188 . </s> We Ġalso Ġdemonstrated Ġhow Ġthe Ġuser Ġmetadata Ġcan Ġbe Ġincorporated Ġin Ġour Ġclass ifiers . ĠAlthough Ġconc aten ation Ġof Ġfeatures Ġdrawn Ġboth Ġfrom Ġblog Ġcontent Ġand Ġprofile Ġelements Ġdid Ġnot Ġyield Ġany Ġclear Ġimprovements Ġover Ġthe Ġbest Ġindividual Ġclass ifiers , Ġwe Ġfound Ġthat Ġstacking Ġimproves Ġthe Ġprediction Ġaccuracy Ġto Ġan Ġoverall Ġaccuracy Ġof Ġ0 . 643 , Ġas Ġmeasured Ġon Ġour Ġtest Ġdataset . ĠA Ġmore Ġin - depth Ġanalysis Ġshowed Ġthat Ġnot Ġall Ġindustries Ġare Ġequally Ġeasy Ġto Ġpredict : Ġwhile Ġindustries Ġsuch Ġas ĠReal ĠEstate Ġand ĠReligion Ġare Ġclearly Ġdistinguish able Ġwith Ġaccuracy Ġfigures Ġover Ġ0 . 80 , Ġothers Ġsuch Ġas ĠBanking Ġare Ġmuch Ġharder Ġto Ġpredict . </s> Finally , Ġwe Ġpresented Ġa Ġqualitative Ġanalysis Ġto Ġprovide Ġsome Ġinsights Ġinto Ġthe Ġlanguage Ġof Ġdifferent Ġindustries , Ġwhich Ġhighlighted Ġdifferences Ġin Ġthe Ġtop - ranked Ġwords Ġin Ġeach Ġindustry , Ġword Ġsemantic Ġsimilarities , Ġand Ġthe Ġrelative Ġfrequency Ġof Ġemotionally Ġcharged Ġwords . </s> Acknowled gments </s> This Ġmaterial Ġis Ġbased Ġin Ġpart Ġupon Ġwork Ġsupported Ġby Ġthe ĠNational ĠScience ĠFoundation Ġ(# 13 44 257 ) Ġand Ġby Ġthe ĠJohn ĠTemple ton ĠFoundation Ġ(# 48 503 ). ĠAny Ġopinions , Ġfindings , Ġand Ġconclusions Ġor Ġrecommendations Ġexpressed Ġin Ġthis Ġmaterial Ġare Ġthose Ġof Ġthe Ġauthors Ġand Ġdo Ġnot Ġnecessarily Ġreflect Ġthe Ġviews Ġof Ġthe ĠNational ĠScience ĠFoundation Ġor Ġthe ĠJohn ĠTemple ton ĠFoundation . </s> Additional ĠExamples Ġof ĠWord ĠSimilar ities
Introduction Over the past two decades, the emergence of social media has enabled the proliferation of traceable human behavior. The content posted by users can reflect who their friends are, what topics they are interested in, or which company they are working for. At the same time, users are listing a number of profile fields to define themselves to others. The utilization of such metadata has proven important in facilitating further developments of applications in advertising BIBREF0 , personalization BIBREF1 , and recommender systems BIBREF2 . However, profile information can be limited, depending on the platform, or it is often deliberately omitted BIBREF3 . To uncloak this information, a number of studies have utilized social media users' footprints to approximate their profiles. This paper explores the potential of predicting a user's industry –the aggregate of enterprises in a particular field– by identifying industry indicative text in social media. The accurate prediction of users' industry can have a big impact on targeted advertising by minimizing wasted advertising BIBREF4 and improved personalized user experience. A number of studies in the social sciences have associated language use with social factors such as occupation, social class, education, and income BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 . An additional goal of this paper is to examine such findings, and in particular the link between language and occupational class, through a data-driven approach. In addition, we explore how meaning changes depending on the occupational context. By leveraging word embeddings, we seek to quantify how, for example, cloud might mean a separate concept (e.g., condensed water vapor) in the text written by users that work in environmental jobs while it might be used differently by users in technology occupations (e.g., Internet-based computing). Specifically, this paper makes four main contributions. First, we build a large, industry-annotated dataset that contains over 20,000 blog users. In addition to their posted text, we also link a number of user metadata including their gender, location, occupation, introduction and interests. Second, we build content-based classifiers for the industry prediction task and study the effect of incorporating textual features from the users' profile metadata using various meta-classification techniques, significantly improving both the overall accuracy and the average per industry accuracy. Next, after examining which words are indicative for each industry, we build vector-space representations of word meanings and calculate one deviation for each industry, illustrating how meaning is differentiated based on the users' industries. We qualitatively examine the resulting industry-informed semantic representations of words by listing the words per industry that are most similar to job related and general interest terms. Finally, we rank the different industries based on the normalized relative frequencies of emotionally charged words (positive and negative) and, in addition, discover that, for both genders, these frequencies do not statistically significantly correlate with an industry's gender dominance ratio. After discussing related work in Section SECREF2 , we present the dataset used in this study in Section SECREF3 . In Section SECREF4 we evaluate two feature selection methods and examine the industry inference problem using the text of the users' postings. We then augment our content-based classifier by building an ensemble that incorporates several metadata classifiers. We list the most industry indicative words and expose how each industrial semantic field varies with respect to a variety of terms in Section SECREF5 . We explore how the frequencies of emotionally charged words in each gender correlate with the industries and their respective gender dominance ratio and, finally, conclude in Section SECREF6 . Related Work Alongside the wide adoption of social media by the public, researchers have been leveraging the newly available data to create and refine models of users' behavior and profiling. There exists a myriad research that analyzes language in order to profile social media users. Some studies sought to characterize users' personality BIBREF9 , BIBREF10 , while others sequenced the expressed emotions BIBREF11 , studied mental disorders BIBREF12 , and the progression of health conditions BIBREF13 . At the same time, a number of researchers sought to predict the social media users' age and/or gender BIBREF14 , BIBREF15 , BIBREF16 , while others targeted and analyzed the ethnicity, nationality, and race of the users BIBREF17 , BIBREF18 , BIBREF19 . One of the profile fields that has drawn a great deal of attention is the location of a user. Among others, Hecht et al. Hecht11 predicted Twitter users' locations using machine learning on nationwide and state levels. Later, Han et al. Han14 identified location indicative words to predict the location of Twitter users down to the city level. As a separate line of research, a number of studies have focused on discovering the political orientation of users BIBREF15 , BIBREF20 , BIBREF21 . Finally, Li et al. Li14a proposed a way to model major life events such as getting married, moving to a new place, or graduating. In a subsequent study, BIBREF22 described a weakly supervised information extraction method that was used in conjunction with social network information to identify the name of a user's spouse, the college they attended, and the company where they are employed. The line of work that is most closely related to our research is the one concerned with understanding the relation between people's language and their industry. Previous research from the fields of psychology and economics have explored the potential for predicting one's occupation from their ability to use math and verbal symbols BIBREF23 and the relationship between job-types and demographics BIBREF24 . More recently, Huang et al. Huang15 used machine learning to classify Sina Weibo users to twelve different platform-defined occupational classes highlighting the effect of homophily in user interactions. This work examined only users that have been verified by the Sina Weibo platform, introducing a potential bias in the resulting dataset. Finally, Preotiuc-Pietro et al. Preoctiuc15 predicted the occupational class of Twitter users using the Standard Occupational Classification (SOC) system, which groups the different jobs based on skill requirements. In that work, the data collection process was limited to only users that specifically mentioned their occupation in their self-description in a way that could be directly mapped to a SOC occupational class. The mapping between a substring of their self-description and a SOC occupational class was done manually. Because of the manual annotation step, their method was not scalable; moreover, because they identified the occupation class inside a user self-description, only a very small fraction of the Twitter users could be included (in their case, 5,191 users). Both of these recent studies are based on micro-blogging platforms, which inherently restrict the number of characters that a post can have, and consequently the way that users can express themselves. Moreover, both studies used off-the-shelf occupational taxonomies (rather than self-declared occupation categories), resulting in classes that are either too generic (e.g., media, welfare and electronic are three of the twelve Sina Weibo categories), or too intermixed (e.g., an assistant accountant is in a different class from an accountant in SOC). To address these limitations, we investigate the industry prediction task in a large blog corpus consisting of over 20K American users, 40K web-blogs, and 560K blog posts. Dataset We compile our industry-annotated dataset by identifying blogger profiles located in the U.S. on the profile finder on http://www.blogger.com, and scraping only those users that had the industry profile element completed. For each of these bloggers, we retrieve all their blogs, and for each of these blogs we download the 21 most recent blog postings. We then clean these blog posts of HTML tags and tokenize them, and drop those bloggers whose cumulative textual content in their posts is less than 600 characters. Following these guidelines, we identified all the U.S. bloggers with completed industry information. Traditionally, standardized industry taxonomies organize economic activities into groups based on similar production processes, products or services, delivery systems or behavior in financial markets. Following such assumptions and regardless of their many similarities, a tomato farmer would be categorized into a distinct industry from a tobacco farmer. As demonstrated in Preotiuc-Pietro et al. Preoctiuc15 such groupings can cause unwarranted misclassifications. The Blogger platform provides a total of 39 different industry options. Even though a completed industry value is an implicit text annotation, we acknowledge the same problem noted in previous studies: some categories are too broad, while others are very similar. To remedy this and following Guibert et al. Guibert71, who argued that the denominations used in a classification must reflect the purpose of the study, we group the different Blogger industries based on similar educational background and similar technical terminology. To do that, we exclude very general categories and merge conceptually similar ones. Examples of broad categories are the Education and the Student options: a teacher could be teaching in any concentration, while a student could be enrolled in any discipline. Examples of conceptually similar categories are the Investment Banking and the Banking options. The final set of categories is shown in Table TABREF1 , along with the number of users in each category. The resulting dataset consists of 22,880 users, 41,094 blogs, and 561,003 posts. Table TABREF2 presents additional statistics of our dataset. Text-based Industry Modeling After collecting our dataset, we split it into three sets: a train set, a development set, and a test set. The sizes of these sets are 17,880, 2,500, and 2,500 users, respectively, with users randomly assigned to these sets. In all the experiments that follow, we evaluate our classifiers by training them on the train set, configure the parameters and measure performance on the development set, and finally report the prediction accuracy and results on the test set. Note that all the experiments are performed at user level, i.e., all the data for one user is compiled into one instance in our data sets. To measure the performance of our classifiers, we use the prediction accuracy. However, as shown in Table TABREF1 , the available data is skewed across categories, which could lead to somewhat distorted accuracy numbers depending on how well a model learns to predict the most populous classes. Moreover, accuracy alone does not provide a great deal of insight into the individual performance per industry, which is one of the main objectives in this study. Therefore, in our results below, we report: (1) micro-accuracy ( INLINEFORM0 ), calculated as the percentage of correctly classified instances out of all the instances in the development (test) data; and (2) macro-accuracy ( INLINEFORM1 ), calculated as the average of the per-category accuracies, where the per-category accuracy is the percentage of correctly classified instances out of the instances belonging to one category in the development (test) data. Leveraging Blog Content In this section, we seek the effectiveness of using solely textual features obtained from the users' postings to predict their industry. The industry prediction baseline Majority is set by discovering the most frequently featured class in our training set and picking that class in all predictions in the respective development or testing set. After excluding all the words that are not used by at least three separate users in our training set, we build our AllWords model by counting the frequencies of all the remaining words and training a multinomial Naive Bayes classifier. As seen in Figure FIGREF3 , we can far exceed the Majority baseline performance by incorporating basic language signals into machine learning algorithms (173% INLINEFORM0 improvement). We additionally explore the potential of improving our text classification task by applying a number of feature ranking methods and selecting varying proportions of top ranked features in an attempt to exclude noisy features. We start by ranking the different features, w, according to their Information Gain Ratio score (IGR) with respect to every industry, i, and training our classifier using different proportions of the top features. INLINEFORM0 INLINEFORM1 Even though we find that using the top 95% of all the features already exceeds the performance of the All Words model on the development data, we further experiment with ranking our features with a more aggressive formula that heavily promotes the features that are tightly associated with any industry category. Therefore, for every word in our training set, we define our newly introduced ranking method, the Aggressive Feature Ranking (AFR), as: INLINEFORM0 In Figure FIGREF3 we illustrate the performance of all four methods in our industry prediction task on the development data. Note that for each method, we provide both the accuracy ( INLINEFORM0 ) and the average per-class accuracy ( INLINEFORM1 ). The Majority and All Words methods apply to all the features; therefore, they are represented as a straight line in the figure. The IGR and AFR methods are applied to varying subsets of the features using a 5% step. Our experiments demonstrate that the word choice that the users make in their posts correlates with their industry. The first observation in Figure FIGREF3 is that the INLINEFORM0 is proportional to INLINEFORM1 ; as INLINEFORM2 increases, so does INLINEFORM3 . Secondly, the best result on the development set is achieved by using the top 90% of the features using the AFR method. Lastly, the improvements of the IGR and AFR feature selections are not substantially better in comparison to All Words (at most 5% improvement between All Words and AFR), which suggest that only a few noisy features exist and most of the words play some role in shaping the “language" of an industry. As a final evaluation, we apply on the test data the classifier found to work best on the development data (AFR feature selection, top 90% features), for an INLINEFORM0 of 0.534 and INLINEFORM1 of 0.477. Leveraging User Metadata Together with the industry information and the most recent postings of each blogger, we also download a number of accompanying profile elements. Using these additional elements, we explore the potential of incorporating users' metadata in our classifiers. Table TABREF7 shows the different user metadata we consider together with their coverage percentage (not all users provide a value for all of the profile elements). With the exception of the gender field, the remaining metadata elements shown in Table TABREF7 are completed by the users as a freely editable text field. This introduces a considerable amount of noise in the set of possible metadata values. Examples of noise in the occupation field include values such as “Retired”, “I work.”, or “momma” which are not necessarily informative for our industry prediction task. To examine whether the metadata fields can help in the prediction of a user's industry, we build classifiers using the different metadata elements. For each metadata element that has a textual value, we use all the words in the training set for that field as features. The only two exceptions are the state field, which is encoded as one feature that can take one out of 50 different values representing the 50 U.S. states; and the gender field, which is encoded as a feature with a distinct value for each user gender option: undefined, male, or female. As shown in Table TABREF9 , we build four different classifiers using the multinomial NB algorithm: Occu (which uses the words found in the occupation profile element), Intro (introduction), Inter (interests), and Gloc (combined gender, city, state). In general, all the metadata classifiers perform better than our majority baseline ( INLINEFORM0 of 18.88%). For the Gloc classifier, this result is in alignment with previous studies BIBREF24 . However, the only metadata classifier that outperforms the content classifier is the Occu classifier, which despite missing and noisy occupation values exceeds the content classifier's performance by an absolute 3.2%. To investigate the promise of combining the five different classifiers we have built so far, we calculate their inter-prediction agreement using Fleiss's Kappa BIBREF25 , as well as the lower prediction bounds using the double fault measure BIBREF26 . The Kappa values, presented in the lower left side of Table TABREF10 , express the classification agreement for categorical items, in this case the users' industry. Lower values, especially values below 30%, mean smaller agreement. Since all five classifiers have better-than-baseline accuracy, this low agreement suggests that their predictions could potentially be combined to achieve a better accumulated result. Moreover, the double fault measure values, which are presented in the top-right hand side of Table TABREF10 , express the proportion of test cases for which both of the two respective classifiers make false predictions, essentially providing the lowest error bound for the pairwise ensemble classifier performance. The lower those numbers are, the greater the accuracy potential of any meta-classification scheme that combines those classifiers. Once again, the low double fault measure values suggest potential gain from a combination of the base classifiers into an ensemble of models. After establishing the promise of creating an ensemble of classifiers, we implement two meta-classification approaches. First, we combine our classifiers using features concatenation (or early fusion). Starting with our content-based classifier (Text), we successively add the features derived from each metadata element. The results, both micro- and macro-accuracy, are presented in Table TABREF12 . Even though all these four feature concatenation ensembles outperform the content-based classifier in the development set, they fail to outperform the Occu classifier. Second, we explore the potential of using stacked generalization (or late fusion) BIBREF27 . The base classifiers, referred to as L0 classifiers, are trained on different folds of the training set and used to predict the class of the remaining instances. Those predictions are then used together with the true label of the training instances to train a second classifier, referred to as the L1 classifier: this L1 is used to produce the final prediction on both the development data and the test data. Traditionally, stacking uses different machine learning algorithms on the same training data. However in our case, we use the same algorithm (multinomial NB) on heterogeneous data (i.e., different types of data such as content, occupation, introduction, interests, gender, city and state) in order to exploit all available sources of information. The ensemble learning results on the development set are shown in Table TABREF12 . We notice a constant improvement for both metrics when adding more classifiers to our ensemble except for the Gloc classifier, which slightly reduces the performance. The best result is achieved using an ensemble of the Text, Occu, Intro, and Inter L0 classifiers; the respective performance on the test set is an INLINEFORM0 of 0.643 and an INLINEFORM1 of 0.564. Finally, we present in Figure FIGREF11 the prediction accuracy for the final classifier for each of the different industries in our test dataset. Evidently, some industries are easier to predict than others. For example, while the Real Estate and Religion industries achieve accuracy figures above 80%, other industries, such as the Banking industry, are predicted correctly in less than 17% of the time. Anecdotal evidence drawn from the examination of the confusion matrix does not encourage any strong association of the Banking class with any other. The misclassifications are roughly uniform across all other classes, suggesting that the users in the Banking industry use language in a non-distinguishing way. Qualitative Analysis In this section, we provide a qualitative analysis of the language of the different industries. Top-Ranked Words To conduct a qualitative exploration of which words indicate the industry of a user, Table TABREF14 shows the three top-ranking content words for the different industries using the AFR method. Not surprisingly, the top ranked words align well with what we would intuitively expect for each industry. Even though most of these words are potentially used by many users regardless of their industry in our dataset, they are still distinguished by the AFR method because of the different frequencies of these words in the text of each industry. Industry-specific Word Similarities Next, we examine how the meaning of a word is shaped by the context in which it is uttered. In particular, we qualitatively investigate how the speakers' industry affects meaning by learning vector-space representations of words that take into account such contextual information. To achieve this, we apply the contextualized word embeddings proposed by Bamman et al. Bamman14, which are based on an extension of the “skip-gram" language model BIBREF28 . In addition to learning a global representation for each word, these contextualized embeddings compute one deviation from the common word embedding representation for each contextual variable, in this case, an industry option. These deviations capture the terms' meaning variations (shifts in the INLINEFORM0 -dimensional space of the representations, where INLINEFORM1 in our experiments) in the text of the different industries, however all the embeddings are in the same vector space to allow for comparisons to one another. Using the word representations learned for each industry, we present in Table TABREF16 the terms in the Technology and the Tourism industries that have the highest cosine similarity with a job-related word, customers. Similarly, Table TABREF17 shows the words in the Environment and the Tourism industries that are closest in meaning to a general interest word, food. More examples are given in the Appendix SECREF8 . The terms that rank highest in each industry are noticeably different. For example, as seen in Table TABREF17 , while food in the Environment industry is similar to nutritionally and locally, in the Tourism industry the same word relates more to terms such as delicious and pastries. These results not only emphasize the existing differences in how people in different industries perceive certain terms, but they also demonstrate that those differences can effectively be captured in the resulting word embeddings. Emotional Orientation per Industry and Gender As a final analysis, we explore how words that are emotionally charged relate to different industries. To quantify the emotional orientation of a text, we use the Positive Emotion and Negative Emotion categories in the Linguistic Inquiry and Word Count (LIWC) dictionary BIBREF29 . The LIWC dictionary contains lists of words that have been shown to correlate with the psychological states of people that use them; for example, the Positive Emotion category contains words such as “happy,” “pretty,” and “good.” For the text of all the users in each industry we measure the frequencies of Positive Emotion and Negative Emotion words normalized by the text's length. Table TABREF20 presents the industries' ranking for both categories of words based on their relative frequencies in the text of each industry. We further perform a breakdown per-gender, where we once again calculate the proportion of emotionally charged words in each industry, but separately for each gender. We find that the industry rankings of the relative frequencies INLINEFORM0 of emotionally charged words for the two genders are statistically significantly correlated, which suggests that regardless of their gender, users use positive (or negative) words with a relative frequency that correlates with their industry. (In other words, even if e.g., Fashion has a larger number of women users, both men and women working in Fashion will tend to use more positive words than the corresponding gender in another industry with a larger number of men users such as Automotive.) Finally, motivated by previous findings of correlations between job satisfaction and gender dominance in the workplace BIBREF30 , we explore the relationship between the usage of Positive Emotion and Negative Emotion words and the gender dominance in an industry. Although we find that there are substantial gender imbalances in each industry (Appendix SECREF9 ), we did not find any statistically significant correlation between the gender dominance ratio in the different industries and the usage of positive (or negative) emotional words in either gender in our dataset. Conclusion In this paper, we examined the task of predicting a social media user's industry. We introduced an annotated dataset of over 20,000 blog users and applied a content-based classifier in conjunction with two feature selection methods for an overall accuracy of up to 0.534, which represents a large improvement over the majority class baseline of 0.188. We also demonstrated how the user metadata can be incorporated in our classifiers. Although concatenation of features drawn both from blog content and profile elements did not yield any clear improvements over the best individual classifiers, we found that stacking improves the prediction accuracy to an overall accuracy of 0.643, as measured on our test dataset. A more in-depth analysis showed that not all industries are equally easy to predict: while industries such as Real Estate and Religion are clearly distinguishable with accuracy figures over 0.80, others such as Banking are much harder to predict. Finally, we presented a qualitative analysis to provide some insights into the language of different industries, which highlighted differences in the top-ranked words in each industry, word semantic similarities, and the relative frequency of emotionally charged words. Acknowledgments This material is based in part upon work supported by the National Science Foundation (#1344257) and by the John Templeton Foundation (#48503). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or the John Templeton Foundation. Additional Examples of Word Similarities
How many users do they look at?
692c9c5d9ff9cd3e0ce8b5e4fa68dda9bd23dec1
How Ġmany Ġusers Ġdo Ġthey Ġlook Ġat ?
22 , 880 Ġusers
<s> How Ġmany Ġusers Ġdo Ġthey Ġlook Ġat ? </s> Introduction </s> Over Ġthe Ġpast Ġtwo Ġdecades , Ġthe Ġemergence Ġof Ġsocial Ġmedia Ġhas Ġenabled Ġthe Ġproliferation Ġof Ġtrace able Ġhuman Ġbehavior . ĠThe Ġcontent Ġposted Ġby Ġusers Ġcan Ġreflect Ġwho Ġtheir Ġfriends Ġare , Ġwhat Ġtopics Ġthey Ġare Ġinterested Ġin , Ġor Ġwhich Ġcompany Ġthey Ġare Ġworking Ġfor . ĠAt Ġthe Ġsame Ġtime , Ġusers Ġare Ġlisting Ġa Ġnumber Ġof Ġprofile Ġfields Ġto Ġdefine Ġthemselves Ġto Ġothers . ĠThe Ġutilization Ġof Ġsuch Ġmetadata Ġhas Ġproven Ġimportant Ġin Ġfacilitating Ġfurther Ġdevelopments Ġof Ġapplications Ġin Ġadvertising ĠB IB REF 0 Ġ, Ġpersonal ization ĠB IB REF 1 Ġ, Ġand Ġrecomm ender Ġsystems ĠB IB REF 2 Ġ. ĠHowever , Ġprofile Ġinformation Ġcan Ġbe Ġlimited , Ġdepending Ġon Ġthe Ġplatform , Ġor Ġit Ġis Ġoften Ġdeliberately Ġomitted ĠB IB REF 3 Ġ. ĠTo Ġun cl oak Ġthis Ġinformation , Ġa Ġnumber Ġof Ġstudies Ġhave Ġutilized Ġsocial Ġmedia Ġusers ' Ġfootprints Ġto Ġapproximate Ġtheir Ġprofiles . </s> This Ġpaper Ġexplores Ġthe Ġpotential Ġof Ġpredicting Ġa Ġuser 's Ġindustry ĠâĢĵ the Ġaggregate Ġof Ġenterprises Ġin Ġa Ġparticular Ġfield âĢĵ Ġby Ġidentifying Ġindustry Ġindicative Ġtext Ġin Ġsocial Ġmedia . ĠThe Ġaccurate Ġprediction Ġof Ġusers ' Ġindustry Ġcan Ġhave Ġa Ġbig Ġimpact Ġon Ġtargeted Ġadvertising Ġby Ġminimizing Ġwasted Ġadvertising ĠB IB REF 4 Ġand Ġimproved Ġpersonalized Ġuser Ġexperience . ĠA Ġnumber Ġof Ġstudies Ġin Ġthe Ġsocial Ġsciences Ġhave Ġassociated Ġlanguage Ġuse Ġwith Ġsocial Ġfactors Ġsuch Ġas Ġoccupation , Ġsocial Ġclass , Ġeducation , Ġand Ġincome ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ. ĠAn Ġadditional Ġgoal Ġof Ġthis Ġpaper Ġis Ġto Ġexamine Ġsuch Ġfindings , Ġand Ġin Ġparticular Ġthe Ġlink Ġbetween Ġlanguage Ġand Ġoccupational Ġclass , Ġthrough Ġa Ġdata - driven Ġapproach . </s> In Ġaddition , Ġwe Ġexplore Ġhow Ġmeaning Ġchanges Ġdepending Ġon Ġthe Ġoccupational Ġcontext . ĠBy Ġleveraging Ġword Ġembed d ings , Ġwe Ġseek Ġto Ġquantify Ġhow , Ġfor Ġexample , Ġcloud Ġmight Ġmean Ġa Ġseparate Ġconcept Ġ( e . g ., Ġcondensed Ġwater Ġvapor ) Ġin Ġthe Ġtext Ġwritten Ġby Ġusers Ġthat Ġwork Ġin Ġenvironmental Ġjobs Ġwhile Ġit Ġmight Ġbe Ġused Ġdifferently Ġby Ġusers Ġin Ġtechnology Ġoccupations Ġ( e . g ., ĠInternet - based Ġcomputing ). </s> Specifically , Ġthis Ġpaper Ġmakes Ġfour Ġmain Ġcontributions . ĠFirst , Ġwe Ġbuild Ġa Ġlarge , Ġindustry - annot ated Ġdataset Ġthat Ġcontains Ġover Ġ20 , 000 Ġblog Ġusers . ĠIn Ġaddition Ġto Ġtheir Ġposted Ġtext , Ġwe Ġalso Ġlink Ġa Ġnumber Ġof Ġuser Ġmetadata Ġincluding Ġtheir Ġgender , Ġlocation , Ġoccupation , Ġintroduction Ġand Ġinterests . </s> Second , Ġwe Ġbuild Ġcontent - based Ġclass ifiers Ġfor Ġthe Ġindustry Ġprediction Ġtask Ġand Ġstudy Ġthe Ġeffect Ġof Ġincorporating Ġtextual Ġfeatures Ġfrom Ġthe Ġusers ' Ġprofile Ġmetadata Ġusing Ġvarious Ġmeta - class ification Ġtechniques , Ġsignificantly Ġimproving Ġboth Ġthe Ġoverall Ġaccuracy Ġand Ġthe Ġaverage Ġper Ġindustry Ġaccuracy . </s> Next , Ġafter Ġexamining Ġwhich Ġwords Ġare Ġindicative Ġfor Ġeach Ġindustry , Ġwe Ġbuild Ġvector - space Ġrepresentations Ġof Ġword Ġmeanings Ġand Ġcalculate Ġone Ġdeviation Ġfor Ġeach Ġindustry , Ġillustrating Ġhow Ġmeaning Ġis Ġdifferentiated Ġbased Ġon Ġthe Ġusers ' Ġindustries . ĠWe Ġqual itatively Ġexamine Ġthe Ġresulting Ġindustry - informed Ġsemantic Ġrepresentations Ġof Ġwords Ġby Ġlisting Ġthe Ġwords Ġper Ġindustry Ġthat Ġare Ġmost Ġsimilar Ġto Ġjob Ġrelated Ġand Ġgeneral Ġinterest Ġterms . </s> Finally , Ġwe Ġrank Ġthe Ġdifferent Ġindustries Ġbased Ġon Ġthe Ġnormalized Ġrelative Ġfrequencies Ġof Ġemotionally Ġcharged Ġwords Ġ( positive Ġand Ġnegative ) Ġand , Ġin Ġaddition , Ġdiscover Ġthat , Ġfor Ġboth Ġgenders , Ġthese Ġfrequencies Ġdo Ġnot Ġstatistically Ġsignificantly Ġcorrelate Ġwith Ġan Ġindustry 's Ġgender Ġdominance Ġratio . </s> After Ġdiscussing Ġrelated Ġwork Ġin ĠSection ĠSEC REF 2 Ġ, Ġwe Ġpresent Ġthe Ġdataset Ġused Ġin Ġthis Ġstudy Ġin ĠSection ĠSEC REF 3 Ġ. ĠIn ĠSection ĠSEC REF 4 Ġwe Ġevaluate Ġtwo Ġfeature Ġselection Ġmethods Ġand Ġexamine Ġthe Ġindustry Ġinference Ġproblem Ġusing Ġthe Ġtext Ġof Ġthe Ġusers ' Ġpostings . ĠWe Ġthen Ġaugment Ġour Ġcontent - based Ġclass ifier Ġby Ġbuilding Ġan Ġensemble Ġthat Ġincorporates Ġseveral Ġmetadata Ġclass ifiers . ĠWe Ġlist Ġthe Ġmost Ġindustry Ġindicative Ġwords Ġand Ġexpose Ġhow Ġeach Ġindustrial Ġsemantic Ġfield Ġvaries Ġwith Ġrespect Ġto Ġa Ġvariety Ġof Ġterms Ġin ĠSection ĠSEC REF 5 Ġ. ĠWe Ġexplore Ġhow Ġthe Ġfrequencies Ġof Ġemotionally Ġcharged Ġwords Ġin Ġeach Ġgender Ġcorrelate Ġwith Ġthe Ġindustries Ġand Ġtheir Ġrespective Ġgender Ġdominance Ġratio Ġand , Ġfinally , Ġconclude Ġin ĠSection ĠSEC REF 6 Ġ. </s> Related ĠWork </s> Along side Ġthe Ġwide Ġadoption Ġof Ġsocial Ġmedia Ġby Ġthe Ġpublic , Ġresearchers Ġhave Ġbeen Ġleveraging Ġthe Ġnewly Ġavailable Ġdata Ġto Ġcreate Ġand Ġrefine Ġmodels Ġof Ġusers ' Ġbehavior Ġand Ġprofiling . ĠThere Ġexists Ġa Ġmyriad Ġresearch Ġthat Ġanaly zes Ġlanguage Ġin Ġorder Ġto Ġprofile Ġsocial Ġmedia Ġusers . ĠSome Ġstudies Ġsought Ġto Ġcharacterize Ġusers ' Ġpersonality ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ, Ġwhile Ġothers Ġsequ enced Ġthe Ġexpressed Ġemotions ĠB IB REF 11 Ġ, Ġstudied Ġmental Ġdisorders ĠB IB REF 12 Ġ, Ġand Ġthe Ġprogression Ġof Ġhealth Ġconditions ĠB IB REF 13 Ġ. ĠAt Ġthe Ġsame Ġtime , Ġa Ġnumber Ġof Ġresearchers Ġsought Ġto Ġpredict Ġthe Ġsocial Ġmedia Ġusers ' Ġage Ġand / or Ġgender ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 16 Ġ, Ġwhile Ġothers Ġtargeted Ġand Ġanalyzed Ġthe Ġethnicity , Ġnationality , Ġand Ġrace Ġof Ġthe Ġusers ĠB IB REF 17 Ġ, ĠB IB REF 18 Ġ, ĠB IB REF 19 Ġ. ĠOne Ġof Ġthe Ġprofile Ġfields Ġthat Ġhas Ġdrawn Ġa Ġgreat Ġdeal Ġof Ġattention Ġis Ġthe Ġlocation Ġof Ġa Ġuser . ĠAmong Ġothers , ĠHe cht Ġet Ġal . ĠHe cht 11 Ġpredicted ĠTwitter Ġusers ' Ġlocations Ġusing Ġmachine Ġlearning Ġon Ġnationwide Ġand Ġstate Ġlevels . ĠLater , ĠHan Ġet Ġal . ĠHan 14 Ġidentified Ġlocation Ġindicative Ġwords Ġto Ġpredict Ġthe Ġlocation Ġof ĠTwitter Ġusers Ġdown Ġto Ġthe Ġcity Ġlevel . </s> As Ġa Ġseparate Ġline Ġof Ġresearch , Ġa Ġnumber Ġof Ġstudies Ġhave Ġfocused Ġon Ġdiscovering Ġthe Ġpolitical Ġorientation Ġof Ġusers ĠB IB REF 15 Ġ, ĠB IB REF 20 Ġ, ĠB IB REF 21 Ġ. ĠFinally , ĠLi Ġet Ġal . ĠLi 14 a Ġproposed Ġa Ġway Ġto Ġmodel Ġmajor Ġlife Ġevents Ġsuch Ġas Ġgetting Ġmarried , Ġmoving Ġto Ġa Ġnew Ġplace , Ġor Ġgraduating . ĠIn Ġa Ġsubsequent Ġstudy , ĠB IB REF 22 Ġdescribed Ġa Ġweak ly Ġsupervised Ġinformation Ġextraction Ġmethod Ġthat Ġwas Ġused Ġin Ġconjunction Ġwith Ġsocial Ġnetwork Ġinformation Ġto Ġidentify Ġthe Ġname Ġof Ġa Ġuser 's Ġspouse , Ġthe Ġcollege Ġthey Ġattended , Ġand Ġthe Ġcompany Ġwhere Ġthey Ġare Ġemployed . </s> The Ġline Ġof Ġwork Ġthat Ġis Ġmost Ġclosely Ġrelated Ġto Ġour Ġresearch Ġis Ġthe Ġone Ġconcerned Ġwith Ġunderstanding Ġthe Ġrelation Ġbetween Ġpeople 's Ġlanguage Ġand Ġtheir Ġindustry . ĠPrevious Ġresearch Ġfrom Ġthe Ġfields Ġof Ġpsychology Ġand Ġeconomics Ġhave Ġexplored Ġthe Ġpotential Ġfor Ġpredicting Ġone 's Ġoccupation Ġfrom Ġtheir Ġability Ġto Ġuse Ġmath Ġand Ġverbal Ġsymbols ĠB IB REF 23 Ġand Ġthe Ġrelationship Ġbetween Ġjob - types Ġand Ġdemographics ĠB IB REF 24 Ġ. ĠMore Ġrecently , ĠHuang Ġet Ġal . ĠHuang 15 Ġused Ġmachine Ġlearning Ġto Ġclassify ĠSina ĠWe ibo Ġusers Ġto Ġtwelve Ġdifferent Ġplatform - defined Ġoccupational Ġclasses Ġhighlighting Ġthe Ġeffect Ġof Ġhom oph ily Ġin Ġuser Ġinteractions . ĠThis Ġwork Ġexamined Ġonly Ġusers Ġthat Ġhave Ġbeen Ġverified Ġby Ġthe ĠSina ĠWe ibo Ġplatform , Ġintroducing Ġa Ġpotential Ġbias Ġin Ġthe Ġresulting Ġdataset . ĠFinally , ĠPre oti uc - P iet ro Ġet Ġal . ĠPre oct i uc 15 Ġpredicted Ġthe Ġoccupational Ġclass Ġof ĠTwitter Ġusers Ġusing Ġthe ĠStandard ĠOccup ational ĠClassification Ġ( S OC ) Ġsystem , Ġwhich Ġgroups Ġthe Ġdifferent Ġjobs Ġbased Ġon Ġskill Ġrequirements . ĠIn Ġthat Ġwork , Ġthe Ġdata Ġcollection Ġprocess Ġwas Ġlimited Ġto Ġonly Ġusers Ġthat Ġspecifically Ġmentioned Ġtheir Ġoccupation Ġin Ġtheir Ġself - description Ġin Ġa Ġway Ġthat Ġcould Ġbe Ġdirectly Ġmapped Ġto Ġa ĠSOC Ġoccupational Ġclass . ĠThe Ġmapping Ġbetween Ġa Ġsubst ring Ġof Ġtheir Ġself - description Ġand Ġa ĠSOC Ġoccupational Ġclass Ġwas Ġdone Ġmanually . ĠBecause Ġof Ġthe Ġmanual Ġannotation Ġstep , Ġtheir Ġmethod Ġwas Ġnot Ġscalable ; Ġmoreover , Ġbecause Ġthey Ġidentified Ġthe Ġoccupation Ġclass Ġinside Ġa Ġuser Ġself - description , Ġonly Ġa Ġvery Ġsmall Ġfraction Ġof Ġthe ĠTwitter Ġusers Ġcould Ġbe Ġincluded Ġ( in Ġtheir Ġcase , Ġ5 , 191 Ġusers ). </s> Both Ġof Ġthese Ġrecent Ġstudies Ġare Ġbased Ġon Ġmicro - blog ging Ġplatforms , Ġwhich Ġinherently Ġrestrict Ġthe Ġnumber Ġof Ġcharacters Ġthat Ġa Ġpost Ġcan Ġhave , Ġand Ġconsequently Ġthe Ġway Ġthat Ġusers Ġcan Ġexpress Ġthemselves . </s> Moreover , Ġboth Ġstudies Ġused Ġoff - the - she lf Ġoccupational Ġtax onom ies Ġ( rather Ġthan Ġself - decl ared Ġoccupation Ġcategories ), Ġresulting Ġin Ġclasses Ġthat Ġare Ġeither Ġtoo Ġgeneric Ġ( e . g ., Ġmedia , Ġwelfare Ġand Ġelectronic Ġare Ġthree Ġof Ġthe Ġtwelve ĠSina ĠWe ibo Ġcategories ), Ġor Ġtoo Ġinter m ixed Ġ( e . g ., Ġan Ġassistant Ġaccountant Ġis Ġin Ġa Ġdifferent Ġclass Ġfrom Ġan Ġaccountant Ġin ĠSOC ). ĠTo Ġaddress Ġthese Ġlimitations , Ġwe Ġinvestigate Ġthe Ġindustry Ġprediction Ġtask Ġin Ġa Ġlarge Ġblog Ġcorpus Ġconsisting Ġof Ġover Ġ20 K ĠAmerican Ġusers , Ġ40 K Ġweb - blogs , Ġand Ġ560 K Ġblog Ġposts . </s> Dat as et </s> We Ġcompile Ġour Ġindustry - annot ated Ġdataset Ġby Ġidentifying Ġblogger Ġprofiles Ġlocated Ġin Ġthe ĠU . S . Ġon Ġthe Ġprofile Ġfind er Ġon Ġhttp :// www . blog ger . com , Ġand Ġscraping Ġonly Ġthose Ġusers Ġthat Ġhad Ġthe Ġindustry Ġprofile Ġelement Ġcompleted . </s> For Ġeach Ġof Ġthese Ġbloggers , Ġwe Ġretrieve Ġall Ġtheir Ġblogs , Ġand Ġfor Ġeach Ġof Ġthese Ġblogs Ġwe Ġdownload Ġthe Ġ21 Ġmost Ġrecent Ġblog Ġpostings . ĠWe Ġthen Ġclean Ġthese Ġblog Ġposts Ġof ĠHTML Ġtags Ġand Ġtoken ize Ġthem , Ġand Ġdrop Ġthose Ġbloggers Ġwhose Ġcumulative Ġtextual Ġcontent Ġin Ġtheir Ġposts Ġis Ġless Ġthan Ġ600 Ġcharacters . ĠFollowing Ġthese Ġguidelines , Ġwe Ġidentified Ġall Ġthe ĠU . S . Ġbloggers Ġwith Ġcompleted Ġindustry Ġinformation . </s> Tr ad itionally , Ġstandardized Ġindustry Ġtax onom ies Ġorganize Ġeconomic Ġactivities Ġinto Ġgroups Ġbased Ġon Ġsimilar Ġproduction Ġprocesses , Ġproducts Ġor Ġservices , Ġdelivery Ġsystems Ġor Ġbehavior Ġin Ġfinancial Ġmarkets . ĠFollowing Ġsuch Ġassumptions Ġand Ġregardless Ġof Ġtheir Ġmany Ġsimilarities , Ġa Ġtomato Ġfarmer Ġwould Ġbe Ġcategorized Ġinto Ġa Ġdistinct Ġindustry Ġfrom Ġa Ġtobacco Ġfarmer . ĠAs Ġdemonstrated Ġin ĠPre oti uc - P iet ro Ġet Ġal . ĠPre oct i uc 15 Ġsuch Ġgroup ings Ġcan Ġcause Ġunw arranted Ġmis class ifications . </s> The ĠBlog ger Ġplatform Ġprovides Ġa Ġtotal Ġof Ġ39 Ġdifferent Ġindustry Ġoptions . ĠEven Ġthough Ġa Ġcompleted Ġindustry Ġvalue Ġis Ġan Ġimplicit Ġtext Ġannotation , Ġwe Ġacknowledge Ġthe Ġsame Ġproblem Ġnoted Ġin Ġprevious Ġstudies : Ġsome Ġcategories Ġare Ġtoo Ġbroad , Ġwhile Ġothers Ġare Ġvery Ġsimilar . ĠTo Ġremedy Ġthis Ġand Ġfollowing ĠGu iber t Ġet Ġal . ĠGu iber t 71 , Ġwho Ġargued Ġthat Ġthe Ġdenominations Ġused Ġin Ġa Ġclassification Ġmust Ġreflect Ġthe Ġpurpose Ġof Ġthe Ġstudy , Ġwe Ġgroup Ġthe Ġdifferent ĠBlog ger Ġindustries Ġbased Ġon Ġsimilar Ġeducational Ġbackground Ġand Ġsimilar Ġtechnical Ġterminology . ĠTo Ġdo Ġthat , Ġwe Ġexclude Ġvery Ġgeneral Ġcategories Ġand Ġmerge Ġconcept ually Ġsimilar Ġones . ĠExamples Ġof Ġbroad Ġcategories Ġare Ġthe ĠEducation Ġand Ġthe ĠStudent Ġoptions : Ġa Ġteacher Ġcould Ġbe Ġteaching Ġin Ġany Ġconcentration , Ġwhile Ġa Ġstudent Ġcould Ġbe Ġenrolled Ġin Ġany Ġdiscipline . ĠExamples Ġof Ġconcept ually Ġsimilar Ġcategories Ġare Ġthe ĠInvestment ĠBanking Ġand Ġthe ĠBanking Ġoptions . </s> The Ġfinal Ġset Ġof Ġcategories Ġis Ġshown Ġin ĠTable ĠT AB REF 1 Ġ, Ġalong Ġwith Ġthe Ġnumber Ġof Ġusers Ġin Ġeach Ġcategory . ĠThe Ġresulting Ġdataset Ġconsists Ġof Ġ22 , 880 Ġusers , Ġ41 , 09 4 Ġblogs , Ġand Ġ5 61 , 003 Ġposts . ĠTable ĠT AB REF 2 Ġpresents Ġadditional Ġstatistics Ġof Ġour Ġdataset . </s> Text - based ĠIndustry ĠModel ing </s> After Ġcollecting Ġour Ġdataset , Ġwe Ġsplit Ġit Ġinto Ġthree Ġsets : Ġa Ġtrain Ġset , Ġa Ġdevelopment Ġset , Ġand Ġa Ġtest Ġset . ĠThe Ġsizes Ġof Ġthese Ġsets Ġare Ġ17 , 880 , Ġ2 , 500 , Ġand Ġ2 , 500 Ġusers , Ġrespectively , Ġwith Ġusers Ġrandomly Ġassigned Ġto Ġthese Ġsets . ĠIn Ġall Ġthe Ġexperiments Ġthat Ġfollow , Ġwe Ġevaluate Ġour Ġclass ifiers Ġby Ġtraining Ġthem Ġon Ġthe Ġtrain Ġset , Ġconfigure Ġthe Ġparameters Ġand Ġmeasure Ġperformance Ġon Ġthe Ġdevelopment Ġset , Ġand Ġfinally Ġreport Ġthe Ġprediction Ġaccuracy Ġand Ġresults Ġon Ġthe Ġtest Ġset . ĠNote Ġthat Ġall Ġthe Ġexperiments Ġare Ġperformed Ġat Ġuser Ġlevel , Ġi . e ., Ġall Ġthe Ġdata Ġfor Ġone Ġuser Ġis Ġcompiled Ġinto Ġone Ġinstance Ġin Ġour Ġdata Ġsets . </s> To Ġmeasure Ġthe Ġperformance Ġof Ġour Ġclass ifiers , Ġwe Ġuse Ġthe Ġprediction Ġaccuracy . ĠHowever , Ġas Ġshown Ġin ĠTable ĠT AB REF 1 Ġ, Ġthe Ġavailable Ġdata Ġis Ġskewed Ġacross Ġcategories , Ġwhich Ġcould Ġlead Ġto Ġsomewhat Ġdistorted Ġaccuracy Ġnumbers Ġdepending Ġon Ġhow Ġwell Ġa Ġmodel Ġlearns Ġto Ġpredict Ġthe Ġmost Ġpopulous Ġclasses . ĠMoreover , Ġaccuracy Ġalone Ġdoes Ġnot Ġprovide Ġa Ġgreat Ġdeal Ġof Ġinsight Ġinto Ġthe Ġindividual Ġperformance Ġper Ġindustry , Ġwhich Ġis Ġone Ġof Ġthe Ġmain Ġobjectives Ġin Ġthis Ġstudy . ĠTherefore , Ġin Ġour Ġresults Ġbelow , Ġwe Ġreport : Ġ( 1 ) Ġmicro - acc uracy Ġ( ĠIN LINE FORM 0 Ġ), Ġcalculated Ġas Ġthe Ġpercentage Ġof Ġcorrectly Ġclassified Ġinstances Ġout Ġof Ġall Ġthe Ġinstances Ġin Ġthe Ġdevelopment Ġ( test ) Ġdata ; Ġand Ġ( 2 ) Ġmacro - acc uracy Ġ( ĠIN LINE FORM 1 Ġ), Ġcalculated Ġas Ġthe Ġaverage Ġof Ġthe Ġper - category Ġaccur acies , Ġwhere Ġthe Ġper - category Ġaccuracy Ġis Ġthe Ġpercentage Ġof Ġcorrectly Ġclassified Ġinstances Ġout Ġof Ġthe Ġinstances Ġbelonging Ġto Ġone Ġcategory Ġin Ġthe Ġdevelopment Ġ( test ) Ġdata . </s> L ever aging ĠBlog ĠContent </s> In Ġthis Ġsection , Ġwe Ġseek Ġthe Ġeffectiveness Ġof Ġusing Ġsolely Ġtextual Ġfeatures Ġobtained Ġfrom Ġthe Ġusers ' Ġpostings Ġto Ġpredict Ġtheir Ġindustry . </s> The Ġindustry Ġprediction Ġbaseline ĠMajority Ġis Ġset Ġby Ġdiscovering Ġthe Ġmost Ġfrequently Ġfeatured Ġclass Ġin Ġour Ġtraining Ġset Ġand Ġpicking Ġthat Ġclass Ġin Ġall Ġpredictions Ġin Ġthe Ġrespective Ġdevelopment Ġor Ġtesting Ġset . </s> After Ġexcluding Ġall Ġthe Ġwords Ġthat Ġare Ġnot Ġused Ġby Ġat Ġleast Ġthree Ġseparate Ġusers Ġin Ġour Ġtraining Ġset , Ġwe Ġbuild Ġour ĠAll Words Ġmodel Ġby Ġcounting Ġthe Ġfrequencies Ġof Ġall Ġthe Ġremaining Ġwords Ġand Ġtraining Ġa Ġmult in omial ĠNa ive ĠBay es Ġclass ifier . ĠAs Ġseen Ġin ĠFigure ĠFIG REF 3 Ġ, Ġwe Ġcan Ġfar Ġexceed Ġthe ĠMajority Ġbaseline Ġperformance Ġby Ġincorporating Ġbasic Ġlanguage Ġsignals Ġinto Ġmachine Ġlearning Ġalgorithms Ġ( 173 % ĠIN LINE FORM 0 Ġimprovement ). </s> We Ġadditionally Ġexplore Ġthe Ġpotential Ġof Ġimproving Ġour Ġtext Ġclassification Ġtask Ġby Ġapplying Ġa Ġnumber Ġof Ġfeature Ġranking Ġmethods Ġand Ġselecting Ġvarying Ġproportions Ġof Ġtop Ġranked Ġfeatures Ġin Ġan Ġattempt Ġto Ġexclude Ġnoisy Ġfeatures . ĠWe Ġstart Ġby Ġranking Ġthe Ġdifferent Ġfeatures , Ġw , Ġaccording Ġto Ġtheir ĠInformation ĠGain ĠRatio Ġscore Ġ( IG R ) Ġwith Ġrespect Ġto Ġevery Ġindustry , Ġi , Ġand Ġtraining Ġour Ġclass ifier Ġusing Ġdifferent Ġproportions Ġof Ġthe Ġtop Ġfeatures . ĠIN LINE FORM 0 ĠIN LINE FORM 1 </s> Even Ġthough Ġwe Ġfind Ġthat Ġusing Ġthe Ġtop Ġ95 % Ġof Ġall Ġthe Ġfeatures Ġalready Ġexceeds Ġthe Ġperformance Ġof Ġthe ĠAll ĠWords Ġmodel Ġon Ġthe Ġdevelopment Ġdata , Ġwe Ġfurther Ġexperiment Ġwith Ġranking Ġour Ġfeatures Ġwith Ġa Ġmore Ġaggressive Ġformula Ġthat Ġheavily Ġpromotes Ġthe Ġfeatures Ġthat Ġare Ġtightly Ġassociated Ġwith Ġany Ġindustry Ġcategory . ĠTherefore , Ġfor Ġevery Ġword Ġin Ġour Ġtraining Ġset , Ġwe Ġdefine Ġour Ġnewly Ġintroduced Ġranking Ġmethod , Ġthe ĠAgg ressive ĠFeature ĠRanking Ġ( AF R ), Ġas : ĠIN LINE FORM 0 </s> In ĠFigure ĠFIG REF 3 Ġwe Ġillustrate Ġthe Ġperformance Ġof Ġall Ġfour Ġmethods Ġin Ġour Ġindustry Ġprediction Ġtask Ġon Ġthe Ġdevelopment Ġdata . ĠNote Ġthat Ġfor Ġeach Ġmethod , Ġwe Ġprovide Ġboth Ġthe Ġaccuracy Ġ( ĠIN LINE FORM 0 Ġ) Ġand Ġthe Ġaverage Ġper - class Ġaccuracy Ġ( ĠIN LINE FORM 1 Ġ). ĠThe ĠMajority Ġand ĠAll ĠWords Ġmethods Ġapply Ġto Ġall Ġthe Ġfeatures ; Ġtherefore , Ġthey Ġare Ġrepresented Ġas Ġa Ġstraight Ġline Ġin Ġthe Ġfigure . ĠThe ĠI GR Ġand ĠA FR Ġmethods Ġare Ġapplied Ġto Ġvarying Ġsubs ets Ġof Ġthe Ġfeatures Ġusing Ġa Ġ5 % Ġstep . </s> Our Ġexperiments Ġdemonstrate Ġthat Ġthe Ġword Ġchoice Ġthat Ġthe Ġusers Ġmake Ġin Ġtheir Ġposts Ġcorrelates Ġwith Ġtheir Ġindustry . ĠThe Ġfirst Ġobservation Ġin ĠFigure ĠFIG REF 3 Ġis Ġthat Ġthe ĠIN LINE FORM 0 Ġis Ġproportional Ġto ĠIN LINE FORM 1 Ġ; Ġas ĠIN LINE FORM 2 Ġincreases , Ġso Ġdoes ĠIN LINE FORM 3 Ġ. ĠSecondly , Ġthe Ġbest Ġresult Ġon Ġthe Ġdevelopment Ġset Ġis Ġachieved Ġby Ġusing Ġthe Ġtop Ġ90 % Ġof Ġthe Ġfeatures Ġusing Ġthe ĠA FR Ġmethod . ĠLastly , Ġthe Ġimprovements Ġof Ġthe ĠI GR Ġand ĠA FR Ġfeature Ġselections Ġare Ġnot Ġsubstantially Ġbetter Ġin Ġcomparison Ġto ĠAll ĠWords Ġ( at Ġmost Ġ5 % Ġimprovement Ġbetween ĠAll ĠWords Ġand ĠA FR ), Ġwhich Ġsuggest Ġthat Ġonly Ġa Ġfew Ġnoisy Ġfeatures Ġexist Ġand Ġmost Ġof Ġthe Ġwords Ġplay Ġsome Ġrole Ġin Ġshaping Ġthe ĠâĢ ľ language " Ġof Ġan Ġindustry . </s> As Ġa Ġfinal Ġevaluation , Ġwe Ġapply Ġon Ġthe Ġtest Ġdata Ġthe Ġclass ifier Ġfound Ġto Ġwork Ġbest Ġon Ġthe Ġdevelopment Ġdata Ġ( AF R Ġfeature Ġselection , Ġtop Ġ90 % Ġfeatures ), Ġfor Ġan ĠIN LINE FORM 0 Ġof Ġ0 . 5 34 Ġand ĠIN LINE FORM 1 Ġof Ġ0 . 477 . </s> L ever aging ĠUser ĠMet adata </s> Together Ġwith Ġthe Ġindustry Ġinformation Ġand Ġthe Ġmost Ġrecent Ġpostings Ġof Ġeach Ġblogger , Ġwe Ġalso Ġdownload Ġa Ġnumber Ġof Ġaccompanying Ġprofile Ġelements . ĠUsing Ġthese Ġadditional Ġelements , Ġwe Ġexplore Ġthe Ġpotential Ġof Ġincorporating Ġusers ' Ġmetadata Ġin Ġour Ġclass ifiers . </s> Table ĠT AB REF 7 Ġshows Ġthe Ġdifferent Ġuser Ġmetadata Ġwe Ġconsider Ġtogether Ġwith Ġtheir Ġcoverage Ġpercentage Ġ( not Ġall Ġusers Ġprovide Ġa Ġvalue Ġfor Ġall Ġof Ġthe Ġprofile Ġelements ). ĠWith Ġthe Ġexception Ġof Ġthe Ġgender Ġfield , Ġthe Ġremaining Ġmetadata Ġelements Ġshown Ġin ĠTable ĠT AB REF 7 Ġare Ġcompleted Ġby Ġthe Ġusers Ġas Ġa Ġfreely Ġedit able Ġtext Ġfield . ĠThis Ġintroduces Ġa Ġconsiderable Ġamount Ġof Ġnoise Ġin Ġthe Ġset Ġof Ġpossible Ġmetadata Ġvalues . ĠExamples Ġof Ġnoise Ġin Ġthe Ġoccupation Ġfield Ġinclude Ġvalues Ġsuch Ġas ĠâĢ ľ Ret ired âĢ Ŀ , ĠâĢ ľ I Ġwork . âĢ Ŀ , Ġor ĠâĢ ľ m omm a âĢ Ŀ Ġwhich Ġare Ġnot Ġnecessarily Ġinformative Ġfor Ġour Ġindustry Ġprediction Ġtask . </s> To Ġexamine Ġwhether Ġthe Ġmetadata Ġfields Ġcan Ġhelp Ġin Ġthe Ġprediction Ġof Ġa Ġuser 's Ġindustry , Ġwe Ġbuild Ġclass ifiers Ġusing Ġthe Ġdifferent Ġmetadata Ġelements . ĠFor Ġeach Ġmetadata Ġelement Ġthat Ġhas Ġa Ġtextual Ġvalue , Ġwe Ġuse Ġall Ġthe Ġwords Ġin Ġthe Ġtraining Ġset Ġfor Ġthat Ġfield Ġas Ġfeatures . ĠThe Ġonly Ġtwo Ġexceptions Ġare Ġthe Ġstate Ġfield , Ġwhich Ġis Ġencoded Ġas Ġone Ġfeature Ġthat Ġcan Ġtake Ġone Ġout Ġof Ġ50 Ġdifferent Ġvalues Ġrepresenting Ġthe Ġ50 ĠU . S . Ġstates ; Ġand Ġthe Ġgender Ġfield , Ġwhich Ġis Ġencoded Ġas Ġa Ġfeature Ġwith Ġa Ġdistinct Ġvalue Ġfor Ġeach Ġuser Ġgender Ġoption : Ġundefined , Ġmale , Ġor Ġfemale . </s> As Ġshown Ġin ĠTable ĠT AB REF 9 Ġ, Ġwe Ġbuild Ġfour Ġdifferent Ġclass ifiers Ġusing Ġthe Ġmult in omial ĠNB Ġalgorithm : ĠOcc u Ġ( which Ġuses Ġthe Ġwords Ġfound Ġin Ġthe Ġoccupation Ġprofile Ġelement ), ĠIntro Ġ( introdu ction ), ĠInter Ġ( interest s ), Ġand ĠGl oc Ġ( comb ined Ġgender , Ġcity , Ġstate ). </s> In Ġgeneral , Ġall Ġthe Ġmetadata Ġclass ifiers Ġperform Ġbetter Ġthan Ġour Ġmajority Ġbaseline Ġ( ĠIN LINE FORM 0 Ġof Ġ18 . 88 %). ĠFor Ġthe ĠGl oc Ġclass ifier , Ġthis Ġresult Ġis Ġin Ġalignment Ġwith Ġprevious Ġstudies ĠB IB REF 24 Ġ. ĠHowever , Ġthe Ġonly Ġmetadata Ġclass ifier Ġthat Ġoutper forms Ġthe Ġcontent Ġclass ifier Ġis Ġthe ĠOcc u Ġclass ifier , Ġwhich Ġdespite Ġmissing Ġand Ġnoisy Ġoccupation Ġvalues Ġexceeds Ġthe Ġcontent Ġclass ifier 's Ġperformance Ġby Ġan Ġabsolute Ġ3 . 2 %. </s> To Ġinvestigate Ġthe Ġpromise Ġof Ġcombining Ġthe Ġfive Ġdifferent Ġclass ifiers Ġwe Ġhave Ġbuilt Ġso Ġfar , Ġwe Ġcalculate Ġtheir Ġinter - pred iction Ġagreement Ġusing ĠFle iss 's ĠKappa ĠB IB REF 25 Ġ, Ġas Ġwell Ġas Ġthe Ġlower Ġprediction Ġbounds Ġusing Ġthe Ġdouble Ġfault Ġmeasure ĠB IB REF 26 Ġ. ĠThe ĠKappa Ġvalues , Ġpresented Ġin Ġthe Ġlower Ġleft Ġside Ġof ĠTable ĠT AB REF 10 Ġ, Ġexpress Ġthe Ġclassification Ġagreement Ġfor Ġcateg orical Ġitems , Ġin Ġthis Ġcase Ġthe Ġusers ' Ġindustry . ĠLower Ġvalues , Ġespecially Ġvalues Ġbelow Ġ30 %, Ġmean Ġsmaller Ġagreement . ĠSince Ġall Ġfive Ġclass ifiers Ġhave Ġbetter - than - bas eline Ġaccuracy , Ġthis Ġlow Ġagreement Ġsuggests Ġthat Ġtheir Ġpredictions Ġcould Ġpotentially Ġbe Ġcombined Ġto Ġachieve Ġa Ġbetter Ġaccumulated Ġresult . </s> Moreover , Ġthe Ġdouble Ġfault Ġmeasure Ġvalues , Ġwhich Ġare Ġpresented Ġin Ġthe Ġtop - right Ġhand Ġside Ġof ĠTable ĠT AB REF 10 Ġ, Ġexpress Ġthe Ġproportion Ġof Ġtest Ġcases Ġfor Ġwhich Ġboth Ġof Ġthe Ġtwo Ġrespective Ġclass ifiers Ġmake Ġfalse Ġpredictions , Ġessentially Ġproviding Ġthe Ġlowest Ġerror Ġbound Ġfor Ġthe Ġpair wise Ġensemble Ġclass ifier Ġperformance . ĠThe Ġlower Ġthose Ġnumbers Ġare , Ġthe Ġgreater Ġthe Ġaccuracy Ġpotential Ġof Ġany Ġmeta - class ification Ġscheme Ġthat Ġcombines Ġthose Ġclass ifiers . ĠOnce Ġagain , Ġthe Ġlow Ġdouble Ġfault Ġmeasure Ġvalues Ġsuggest Ġpotential Ġgain Ġfrom Ġa Ġcombination Ġof Ġthe Ġbase Ġclass ifiers Ġinto Ġan Ġensemble Ġof Ġmodels . </s> After Ġestablishing Ġthe Ġpromise Ġof Ġcreating Ġan Ġensemble Ġof Ġclass ifiers , Ġwe Ġimplement Ġtwo Ġmeta - class ification Ġapproaches . ĠFirst , Ġwe Ġcombine Ġour Ġclass ifiers Ġusing Ġfeatures Ġconc aten ation Ġ( or Ġearly Ġfusion ). ĠStarting Ġwith Ġour Ġcontent - based Ġclass ifier Ġ( Text ), Ġwe Ġsuccess ively Ġadd Ġthe Ġfeatures Ġderived Ġfrom Ġeach Ġmetadata Ġelement . ĠThe Ġresults , Ġboth Ġmicro - Ġand Ġmacro - acc uracy , Ġare Ġpresented Ġin ĠTable ĠT AB REF 12 Ġ. ĠEven Ġthough Ġall Ġthese Ġfour Ġfeature Ġconc aten ation Ġen semb les Ġoutper form Ġthe Ġcontent - based Ġclass ifier Ġin Ġthe Ġdevelopment Ġset , Ġthey Ġfail Ġto Ġoutper form Ġthe ĠOcc u Ġclass ifier . </s> Second , Ġwe Ġexplore Ġthe Ġpotential Ġof Ġusing Ġstacked Ġgeneral ization Ġ( or Ġlate Ġfusion ) ĠB IB REF 27 Ġ. ĠThe Ġbase Ġclass ifiers , Ġreferred Ġto Ġas ĠL 0 Ġclass ifiers , Ġare Ġtrained Ġon Ġdifferent Ġfolds Ġof Ġthe Ġtraining Ġset Ġand Ġused Ġto Ġpredict Ġthe Ġclass Ġof Ġthe Ġremaining Ġinstances . ĠThose Ġpredictions Ġare Ġthen Ġused Ġtogether Ġwith Ġthe Ġtrue Ġlabel Ġof Ġthe Ġtraining Ġinstances Ġto Ġtrain Ġa Ġsecond Ġclass ifier , Ġreferred Ġto Ġas Ġthe ĠL 1 Ġclass ifier : Ġthis ĠL 1 Ġis Ġused Ġto Ġproduce Ġthe Ġfinal Ġprediction Ġon Ġboth Ġthe Ġdevelopment Ġdata Ġand Ġthe Ġtest Ġdata . ĠTrad itionally , Ġstacking Ġuses Ġdifferent Ġmachine Ġlearning Ġalgorithms Ġon Ġthe Ġsame Ġtraining Ġdata . ĠHowever Ġin Ġour Ġcase , Ġwe Ġuse Ġthe Ġsame Ġalgorithm Ġ( mult in omial ĠNB ) Ġon Ġheter ogeneous Ġdata Ġ( i . e ., Ġdifferent Ġtypes Ġof Ġdata Ġsuch Ġas Ġcontent , Ġoccupation , Ġintroduction , Ġinterests , Ġgender , Ġcity Ġand Ġstate ) Ġin Ġorder Ġto Ġexploit Ġall Ġavailable Ġsources Ġof Ġinformation . </s> The Ġensemble Ġlearning Ġresults Ġon Ġthe Ġdevelopment Ġset Ġare Ġshown Ġin ĠTable ĠT AB REF 12 Ġ. ĠWe Ġnotice Ġa Ġconstant Ġimprovement Ġfor Ġboth Ġmetrics Ġwhen Ġadding Ġmore Ġclass ifiers Ġto Ġour Ġensemble Ġexcept Ġfor Ġthe ĠGl oc Ġclass ifier , Ġwhich Ġslightly Ġreduces Ġthe Ġperformance . ĠThe Ġbest Ġresult Ġis Ġachieved Ġusing Ġan Ġensemble Ġof Ġthe ĠText , ĠOcc u , ĠIntro , Ġand ĠInter ĠL 0 Ġclass ifiers ; Ġthe Ġrespective Ġperformance Ġon Ġthe Ġtest Ġset Ġis Ġan ĠIN LINE FORM 0 Ġof Ġ0 . 643 Ġand Ġan ĠIN LINE FORM 1 Ġof Ġ0 . 5 64 . ĠFinally , Ġwe Ġpresent Ġin ĠFigure ĠFIG REF 11 Ġthe Ġprediction Ġaccuracy Ġfor Ġthe Ġfinal Ġclass ifier Ġfor Ġeach Ġof Ġthe Ġdifferent Ġindustries Ġin Ġour Ġtest Ġdataset . ĠEv idently , Ġsome Ġindustries Ġare Ġeasier Ġto Ġpredict Ġthan Ġothers . ĠFor Ġexample , Ġwhile Ġthe ĠReal ĠEstate Ġand ĠReligion Ġindustries Ġachieve Ġaccuracy Ġfigures Ġabove Ġ80 %, Ġother Ġindustries , Ġsuch Ġas Ġthe ĠBanking Ġindustry , Ġare Ġpredicted Ġcorrectly Ġin Ġless Ġthan Ġ17 % Ġof Ġthe Ġtime . ĠA ne cd otal Ġevidence Ġdrawn Ġfrom Ġthe Ġexamination Ġof Ġthe Ġconfusion Ġmatrix Ġdoes Ġnot Ġencourage Ġany Ġstrong Ġassociation Ġof Ġthe ĠBanking Ġclass Ġwith Ġany Ġother . ĠThe Ġmis class ifications Ġare Ġroughly Ġuniform Ġacross Ġall Ġother Ġclasses , Ġsuggesting Ġthat Ġthe Ġusers Ġin Ġthe ĠBanking Ġindustry Ġuse Ġlanguage Ġin Ġa Ġnon - dist ingu ishing Ġway . </s> Qual itative ĠAnalysis </s> In Ġthis Ġsection , Ġwe Ġprovide Ġa Ġqualitative Ġanalysis Ġof Ġthe Ġlanguage Ġof Ġthe Ġdifferent Ġindustries . </s> Top - Ranked ĠWords </s> To Ġconduct Ġa Ġqualitative Ġexploration Ġof Ġwhich Ġwords Ġindicate Ġthe Ġindustry Ġof Ġa Ġuser , ĠTable ĠT AB REF 14 Ġshows Ġthe Ġthree Ġtop - ranking Ġcontent Ġwords Ġfor Ġthe Ġdifferent Ġindustries Ġusing Ġthe ĠA FR Ġmethod . </s> Not Ġsurprisingly , Ġthe Ġtop Ġranked Ġwords Ġalign Ġwell Ġwith Ġwhat Ġwe Ġwould Ġintuitive ly Ġexpect Ġfor Ġeach Ġindustry . ĠEven Ġthough Ġmost Ġof Ġthese Ġwords Ġare Ġpotentially Ġused Ġby Ġmany Ġusers Ġregardless Ġof Ġtheir Ġindustry Ġin Ġour Ġdataset , Ġthey Ġare Ġstill Ġdistinguished Ġby Ġthe ĠA FR Ġmethod Ġbecause Ġof Ġthe Ġdifferent Ġfrequencies Ġof Ġthese Ġwords Ġin Ġthe Ġtext Ġof Ġeach Ġindustry . </s> Indust ry - specific ĠWord ĠSimilar ities </s> Next , Ġwe Ġexamine Ġhow Ġthe Ġmeaning Ġof Ġa Ġword Ġis Ġshaped Ġby Ġthe Ġcontext Ġin Ġwhich Ġit Ġis Ġuttered . ĠIn Ġparticular , Ġwe Ġqual itatively Ġinvestigate Ġhow Ġthe Ġspeakers ' Ġindustry Ġaffects Ġmeaning Ġby Ġlearning Ġvector - space Ġrepresentations Ġof Ġwords Ġthat Ġtake Ġinto Ġaccount Ġsuch Ġcontextual Ġinformation . ĠTo Ġachieve Ġthis , Ġwe Ġapply Ġthe Ġcontextual ized Ġword Ġembed d ings Ġproposed Ġby ĠBam man Ġet Ġal . ĠBam man 14 , Ġwhich Ġare Ġbased Ġon Ġan Ġextension Ġof Ġthe ĠâĢ ľ skip - gram " Ġlanguage Ġmodel ĠB IB REF 28 Ġ. </s> In Ġaddition Ġto Ġlearning Ġa Ġglobal Ġrepresentation Ġfor Ġeach Ġword , Ġthese Ġcontextual ized Ġembed d ings Ġcompute Ġone Ġdeviation Ġfrom Ġthe Ġcommon Ġword Ġembed ding Ġrepresentation Ġfor Ġeach Ġcontextual Ġvariable , Ġin Ġthis Ġcase , Ġan Ġindustry Ġoption . ĠThese Ġdeviations Ġcapture Ġthe Ġterms ' Ġmeaning Ġvariations Ġ( sh ifts Ġin Ġthe ĠIN LINE FORM 0 Ġ- dimensional Ġspace Ġof Ġthe Ġrepresentations , Ġwhere ĠIN LINE FORM 1 Ġin Ġour Ġexperiments ) Ġin Ġthe Ġtext Ġof Ġthe Ġdifferent Ġindustries , Ġhowever Ġall Ġthe Ġembed d ings Ġare Ġin Ġthe Ġsame Ġvector Ġspace Ġto Ġallow Ġfor Ġcomparisons Ġto Ġone Ġanother . </s> Using Ġthe Ġword Ġrepresentations Ġlearned Ġfor Ġeach Ġindustry , Ġwe Ġpresent Ġin ĠTable ĠT AB REF 16 Ġthe Ġterms Ġin Ġthe ĠTechnology Ġand Ġthe ĠTourism Ġindustries Ġthat Ġhave Ġthe Ġhighest Ġcos ine Ġsimilarity Ġwith Ġa Ġjob - related Ġword , Ġcustomers . ĠSimilarly , ĠTable ĠT AB REF 17 Ġshows Ġthe Ġwords Ġin Ġthe ĠEnvironment Ġand Ġthe ĠTourism Ġindustries Ġthat Ġare Ġclosest Ġin Ġmeaning Ġto Ġa Ġgeneral Ġinterest Ġword , Ġfood . ĠMore Ġexamples Ġare Ġgiven Ġin Ġthe ĠAppendix ĠSEC REF 8 Ġ. </s> The Ġterms Ġthat Ġrank Ġhighest Ġin Ġeach Ġindustry Ġare Ġnoticeably Ġdifferent . ĠFor Ġexample , Ġas Ġseen Ġin ĠTable ĠT AB REF 17 Ġ, Ġwhile Ġfood Ġin Ġthe ĠEnvironment Ġindustry Ġis Ġsimilar Ġto Ġnut r itionally Ġand Ġlocally , Ġin Ġthe ĠTourism Ġindustry Ġthe Ġsame Ġword Ġrelates Ġmore Ġto Ġterms Ġsuch Ġas Ġdelicious Ġand Ġpast ries . ĠThese Ġresults Ġnot Ġonly Ġemphasize Ġthe Ġexisting Ġdifferences Ġin Ġhow Ġpeople Ġin Ġdifferent Ġindustries Ġperceive Ġcertain Ġterms , Ġbut Ġthey Ġalso Ġdemonstrate Ġthat Ġthose Ġdifferences Ġcan Ġeffectively Ġbe Ġcaptured Ġin Ġthe Ġresulting Ġword Ġembed d ings . </s> Em otional ĠOrient ation Ġper ĠIndustry Ġand ĠGender </s> As Ġa Ġfinal Ġanalysis , Ġwe Ġexplore Ġhow Ġwords Ġthat Ġare Ġemotionally Ġcharged Ġrelate Ġto Ġdifferent Ġindustries . ĠTo Ġquantify Ġthe Ġemotional Ġorientation Ġof Ġa Ġtext , Ġwe Ġuse Ġthe ĠPositive ĠEm otion Ġand ĠNegative ĠEm otion Ġcategories Ġin Ġthe ĠL ingu istic ĠInquiry Ġand ĠWord ĠCount Ġ( LI WC ) Ġdictionary ĠB IB REF 29 Ġ. ĠThe ĠLI WC Ġdictionary Ġcontains Ġlists Ġof Ġwords Ġthat Ġhave Ġbeen Ġshown Ġto Ġcorrelate Ġwith Ġthe Ġpsychological Ġstates Ġof Ġpeople Ġthat Ġuse Ġthem ; Ġfor Ġexample , Ġthe ĠPositive ĠEm otion Ġcategory Ġcontains Ġwords Ġsuch Ġas ĠâĢ ľ happy , âĢ Ŀ ĠâĢ ľ pretty , âĢ Ŀ Ġand ĠâĢ ľ good . âĢ Ŀ </s> For Ġthe Ġtext Ġof Ġall Ġthe Ġusers Ġin Ġeach Ġindustry Ġwe Ġmeasure Ġthe Ġfrequencies Ġof ĠPositive ĠEm otion Ġand ĠNegative ĠEm otion Ġwords Ġnormalized Ġby Ġthe Ġtext 's Ġlength . ĠTable ĠT AB REF 20 Ġpresents Ġthe Ġindustries ' Ġranking Ġfor Ġboth Ġcategories Ġof Ġwords Ġbased Ġon Ġtheir Ġrelative Ġfrequencies Ġin Ġthe Ġtext Ġof Ġeach Ġindustry . </s> We Ġfurther Ġperform Ġa Ġbreakdown Ġper - gender , Ġwhere Ġwe Ġonce Ġagain Ġcalculate Ġthe Ġproportion Ġof Ġemotionally Ġcharged Ġwords Ġin Ġeach Ġindustry , Ġbut Ġseparately Ġfor Ġeach Ġgender . ĠWe Ġfind Ġthat Ġthe Ġindustry Ġrankings Ġof Ġthe Ġrelative Ġfrequencies ĠIN LINE FORM 0 Ġof Ġemotionally Ġcharged Ġwords Ġfor Ġthe Ġtwo Ġgenders Ġare Ġstatistically Ġsignificantly Ġcorrelated , Ġwhich Ġsuggests Ġthat Ġregardless Ġof Ġtheir Ġgender , Ġusers Ġuse Ġpositive Ġ( or Ġnegative ) Ġwords Ġwith Ġa Ġrelative Ġfrequency Ġthat Ġcorrelates Ġwith Ġtheir Ġindustry . Ġ( In Ġother Ġwords , Ġeven Ġif Ġe . g ., ĠFashion Ġhas Ġa Ġlarger Ġnumber Ġof Ġwomen Ġusers , Ġboth Ġmen Ġand Ġwomen Ġworking Ġin ĠFashion Ġwill Ġtend Ġto Ġuse Ġmore Ġpositive Ġwords Ġthan Ġthe Ġcorresponding Ġgender Ġin Ġanother Ġindustry Ġwith Ġa Ġlarger Ġnumber Ġof Ġmen Ġusers Ġsuch Ġas ĠAutom otive .) </s> Finally , Ġmotivated Ġby Ġprevious Ġfindings Ġof Ġcorrelations Ġbetween Ġjob Ġsatisfaction Ġand Ġgender Ġdominance Ġin Ġthe Ġworkplace ĠB IB REF 30 Ġ, Ġwe Ġexplore Ġthe Ġrelationship Ġbetween Ġthe Ġusage Ġof ĠPositive ĠEm otion Ġand ĠNegative ĠEm otion Ġwords Ġand Ġthe Ġgender Ġdominance Ġin Ġan Ġindustry . ĠAlthough Ġwe Ġfind Ġthat Ġthere Ġare Ġsubstantial Ġgender Ġim bal ances Ġin Ġeach Ġindustry Ġ( App endix ĠSEC REF 9 Ġ), Ġwe Ġdid Ġnot Ġfind Ġany Ġstatistically Ġsignificant Ġcorrelation Ġbetween Ġthe Ġgender Ġdominance Ġratio Ġin Ġthe Ġdifferent Ġindustries Ġand Ġthe Ġusage Ġof Ġpositive Ġ( or Ġnegative ) Ġemotional Ġwords Ġin Ġeither Ġgender Ġin Ġour Ġdataset . </s> Conclusion </s> In Ġthis Ġpaper , Ġwe Ġexamined Ġthe Ġtask Ġof Ġpredicting Ġa Ġsocial Ġmedia Ġuser 's Ġindustry . ĠWe Ġintroduced Ġan Ġannot ated Ġdataset Ġof Ġover Ġ20 , 000 Ġblog Ġusers Ġand Ġapplied Ġa Ġcontent - based Ġclass ifier Ġin Ġconjunction Ġwith Ġtwo Ġfeature Ġselection Ġmethods Ġfor Ġan Ġoverall Ġaccuracy Ġof Ġup Ġto Ġ0 . 5 34 , Ġwhich Ġrepresents Ġa Ġlarge Ġimprovement Ġover Ġthe Ġmajority Ġclass Ġbaseline Ġof Ġ0 . 188 . </s> We Ġalso Ġdemonstrated Ġhow Ġthe Ġuser Ġmetadata Ġcan Ġbe Ġincorporated Ġin Ġour Ġclass ifiers . ĠAlthough Ġconc aten ation Ġof Ġfeatures Ġdrawn Ġboth Ġfrom Ġblog Ġcontent Ġand Ġprofile Ġelements Ġdid Ġnot Ġyield Ġany Ġclear Ġimprovements Ġover Ġthe Ġbest Ġindividual Ġclass ifiers , Ġwe Ġfound Ġthat Ġstacking Ġimproves Ġthe Ġprediction Ġaccuracy Ġto Ġan Ġoverall Ġaccuracy Ġof Ġ0 . 643 , Ġas Ġmeasured Ġon Ġour Ġtest Ġdataset . ĠA Ġmore Ġin - depth Ġanalysis Ġshowed Ġthat Ġnot Ġall Ġindustries Ġare Ġequally Ġeasy Ġto Ġpredict : Ġwhile Ġindustries Ġsuch Ġas ĠReal ĠEstate Ġand ĠReligion Ġare Ġclearly Ġdistinguish able Ġwith Ġaccuracy Ġfigures Ġover Ġ0 . 80 , Ġothers Ġsuch Ġas ĠBanking Ġare Ġmuch Ġharder Ġto Ġpredict . </s> Finally , Ġwe Ġpresented Ġa Ġqualitative Ġanalysis Ġto Ġprovide Ġsome Ġinsights Ġinto Ġthe Ġlanguage Ġof Ġdifferent Ġindustries , Ġwhich Ġhighlighted Ġdifferences Ġin Ġthe Ġtop - ranked Ġwords Ġin Ġeach Ġindustry , Ġword Ġsemantic Ġsimilarities , Ġand Ġthe Ġrelative Ġfrequency Ġof Ġemotionally Ġcharged Ġwords . </s> Acknowled gments </s> This Ġmaterial Ġis Ġbased Ġin Ġpart Ġupon Ġwork Ġsupported Ġby Ġthe ĠNational ĠScience ĠFoundation Ġ(# 13 44 257 ) Ġand Ġby Ġthe ĠJohn ĠTemple ton ĠFoundation Ġ(# 48 503 ). ĠAny Ġopinions , Ġfindings , Ġand Ġconclusions Ġor Ġrecommendations Ġexpressed Ġin Ġthis Ġmaterial Ġare Ġthose Ġof Ġthe Ġauthors Ġand Ġdo Ġnot Ġnecessarily Ġreflect Ġthe Ġviews Ġof Ġthe ĠNational ĠScience ĠFoundation Ġor Ġthe ĠJohn ĠTemple ton ĠFoundation . </s> Additional ĠExamples Ġof ĠWord ĠSimilar ities