No
BOOLEAN
No
BOOLEAN
We tested the proposed UTCNN on two different datasets: FBFans and CreateDebate. FBFans is a privately-owned, single-topic, Chinese, unbalanced, social media dataset, and CreateDebate is a public, multiple-topic, English, balanced, forum dataset. Results using these two datasets show the applicability and superiority for different topics, languages, data distributions, and platforms. The CreateDebate dataset was collected from an English online debate forum discussing four topics: abortion (ABO), gay rights (GAY), Obama (OBA), and marijuana (MAR). The posts are annotated as for (F) and against (A). Replies to posts in this dataset are also labeled with stance and hence use the same data format as posts. The labeling results are shown in the right-hand side of Table TABREF12 . We observe that the dataset is more balanced than the FBFans dataset. In addition, there are 977 unique users in the dataset. To compare with Hasan and Ng's work, we conducted five-fold cross-validation and present the annotation results as the average number of all folds BIBREF9 , BIBREF5 .
1611.03599
Introduction </s> This Ġwork Ġis Ġlic enced Ġunder Ġa ĠCreative ĠCommons ĠAttribution Ġ4 . 0 ĠInternational ĠLicense . ĠLicense Ġdetails : Ġhttp :// creat ive comm ons . org / lic enses / by / 4 . 0 / ĠDeep Ġneural Ġnetworks Ġhave Ġbeen Ġwidely Ġused Ġin Ġtext Ġclassification Ġand Ġhave Ġachieved Ġpromising Ġresults ĠB IB REF 0 Ġ, ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ. ĠMost Ġfocus Ġon Ġcontent Ġinformation Ġand Ġuse Ġmodels Ġsuch Ġas Ġconv olution al Ġneural Ġnetworks Ġ( CNN ) ĠB IB REF 3 Ġor Ġrecursive Ġneural Ġnetworks ĠB IB REF 4 Ġ. ĠHowever , Ġfor Ġuser - generated Ġposts Ġon Ġsocial Ġmedia Ġlike ĠFacebook Ġor ĠTwitter , Ġthere Ġis Ġmore Ġinformation Ġthat Ġshould Ġnot Ġbe Ġignored . ĠOn Ġsocial Ġmedia Ġplatforms , Ġa Ġuser Ġcan Ġact Ġeither Ġas Ġthe Ġauthor Ġof Ġa Ġpost Ġor Ġas Ġa Ġreader Ġwho Ġexpresses Ġhis Ġor Ġher Ġcomments Ġabout Ġthe Ġpost . </s> In Ġthis Ġpaper , Ġwe Ġclassify Ġposts Ġtaking Ġinto Ġaccount Ġpost Ġauthors hip , Ġlikes , Ġtopics , Ġand Ġcomments . ĠIn Ġparticular , Ġusers Ġand Ġtheir ĠâĢ ľ l ikes âĢ Ŀ Ġhold Ġstrong Ġpotential Ġfor Ġtext Ġmining . ĠFor Ġexample , Ġgiven Ġa Ġset Ġof Ġposts Ġthat Ġare Ġrelated Ġto Ġa Ġspecific Ġtopic , Ġa Ġuser 's Ġlikes Ġand Ġdisl ikes Ġprovide Ġclues Ġfor Ġstance Ġlabeling . ĠFrom Ġa Ġuser Ġpoint Ġof Ġview , Ġusers Ġwith Ġpositive Ġattitudes Ġtoward Ġthe Ġissue Ġleave Ġpositive Ġcomments Ġon Ġthe Ġposts Ġwith Ġpraise Ġor Ġeven Ġjust Ġthe Ġpost 's Ġcontent ; Ġfrom Ġa Ġpost Ġpoint Ġof Ġview , Ġpositive Ġposts Ġattract Ġusers Ġwho Ġhold Ġpositive Ġstances . ĠWe Ġalso Ġinvestigate Ġthe Ġinfluence Ġof Ġtopics : Ġdifferent Ġtopics Ġare Ġassociated Ġwith Ġdifferent Ġstance Ġlabeling Ġtendencies Ġand Ġword Ġusage . ĠFor Ġexample Ġwe Ġdiscuss Ġwomen 's Ġrights Ġand Ġunwanted Ġbabies Ġon Ġthe Ġtopic Ġof Ġabortion , Ġbut Ġwe Ġcriticize Ġmedicine Ġusage Ġor Ġcrime Ġwhen Ġon Ġthe Ġtopic Ġof Ġmarijuana ĠB IB REF 5 Ġ. ĠEven Ġfor Ġposts Ġon Ġa Ġspecific Ġtopic Ġlike Ġnuclear Ġpower , Ġa Ġvariety Ġof Ġarguments Ġare Ġraised : Ġgreen Ġenergy , Ġradiation , Ġair Ġpollution , Ġand Ġso Ġon . ĠAs Ġfor Ġcomments , Ġwe Ġtreat Ġthem Ġas Ġadditional Ġtext Ġinformation . ĠThe Ġarguments Ġin Ġthe Ġcomments Ġand Ġthe Ġcommenters Ġ( the Ġusers Ġwho Ġleave Ġthe Ġcomments ) Ġprovide Ġhints Ġon Ġthe Ġpost 's Ġcontent Ġand Ġfurther Ġfacilitate Ġstance Ġclassification . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġthe Ġuser - topic - comment Ġneural Ġnetwork Ġ( UTC NN ), Ġa Ġdeep Ġlearning Ġmodel Ġthat Ġutilizes Ġuser , Ġtopic , Ġand Ġcomment Ġinformation . ĠWe Ġattempt Ġto Ġlearn Ġuser Ġand Ġtopic Ġrepresentations Ġwhich Ġencode Ġuser Ġinteractions Ġand Ġtopic Ġinfluences Ġto Ġfurther Ġenhance Ġtext Ġclassification , Ġand Ġwe Ġalso Ġincorporate Ġcomment Ġinformation . ĠWe Ġevaluate Ġthis Ġmodel Ġon Ġa Ġpost Ġstance Ġclassification Ġtask Ġon Ġforum - style Ġsocial Ġmedia Ġplatforms . ĠThe Ġcontributions Ġof Ġthis Ġpaper Ġare Ġas Ġfollows : Ġ1 . ĠWe Ġpropose ĠUTC NN , Ġa Ġneural Ġnetwork Ġfor Ġtext Ġin Ġmodern Ġsocial Ġmedia Ġchannels Ġas Ġwell Ġas Ġlegacy Ġsocial Ġmedia , Ġforums , Ġand Ġmessage Ġboards ĠâĢĶ Ġanywhere Ġthat Ġreveals Ġusers , Ġtheir Ġtastes , Ġas Ġwell Ġas Ġtheir Ġreplies Ġto Ġposts . Ġ2 . ĠWhen Ġclass ifying Ġsocial Ġmedia Ġpost Ġstances , Ġwe Ġleverage Ġusers , Ġincluding Ġauthors Ġand Ġlik ers . ĠUser Ġembed d ings Ġcan Ġbe Ġgenerated Ġeven Ġfor Ġusers Ġwho Ġhave Ġnever Ġposted Ġanything . Ġ3 . ĠWe Ġincorporate Ġa Ġtopic Ġmodel Ġto Ġautomatically Ġassign Ġtopics Ġto Ġeach Ġpost Ġin Ġa Ġsingle Ġtopic Ġdataset . Ġ4 . ĠWe Ġshow Ġthat Ġoverall , Ġthe Ġproposed Ġmethod Ġachieves Ġthe Ġhighest Ġperformance Ġin Ġall Ġinstances , Ġand Ġthat Ġall Ġof Ġthe Ġinformation Ġextracted , Ġwhether Ġusers , Ġtopics , Ġor Ġcomments , Ġstill Ġhas Ġits Ġcontributions . </s> Extra - L ingu istic ĠFeatures Ġfor ĠSt ance ĠClassification </s> In Ġthis Ġpaper Ġwe Ġaim Ġto Ġuse Ġtext Ġas Ġwell Ġas Ġother Ġfeatures Ġto Ġsee Ġhow Ġthey Ġcomplement Ġeach Ġother Ġin Ġa Ġdeep Ġlearning Ġmodel . ĠIn Ġthe Ġstance Ġclassification Ġdomain , Ġprevious Ġwork Ġhas Ġshowed Ġthat Ġtext Ġfeatures Ġare Ġlimited , Ġsuggesting Ġthat Ġadding Ġextra - ling u istic Ġconstraints Ġcould Ġimprove Ġperformance ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ. ĠFor Ġexample , ĠHasan Ġand ĠNg Ġas Ġwell Ġas ĠThomas Ġet Ġal . Ġrequire Ġthat Ġposts Ġwritten Ġby Ġthe Ġsame Ġauthor Ġhave Ġthe Ġsame Ġstance ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ. ĠThe Ġaddition Ġof Ġthis Ġconstraint Ġyields Ġaccuracy Ġimprovements Ġof Ġ1 âĢĵ 7 % Ġfor Ġsome Ġmodels Ġand Ġdatasets . ĠHasan Ġand ĠNg Ġlater Ġadded Ġuser - inter action Ġconstraints Ġand Ġideology Ġconstraints ĠB IB REF 7 Ġ: Ġthe Ġformer Ġmodels Ġthe Ġrelationship Ġamong Ġposts Ġin Ġa Ġsequence Ġof Ġreplies Ġand Ġthe Ġlatter Ġmodels Ġinter - topic Ġrelationships , Ġe . g ., Ġusers Ġwho Ġoppose Ġabortion Ġcould Ġbe Ġconservative Ġand Ġthus Ġare Ġlikely Ġto Ġoppose Ġgay Ġrights . </s> For Ġwork Ġfocusing Ġon Ġonline Ġforum Ġtext , Ġsince Ġposts Ġare Ġlinked Ġthrough Ġuser Ġreplies , Ġsequential Ġlabeling Ġmethods Ġhave Ġbeen Ġused Ġto Ġmodel Ġrelationships Ġbetween Ġposts . ĠFor Ġexample , ĠHasan Ġand ĠNg Ġuse Ġhidden ĠMark ov Ġmodels Ġ( HM Ms ) Ġto Ġmodel Ġdependent Ġrelationships Ġto Ġthe Ġpreceding Ġpost ĠB IB REF 9 Ġ; ĠBur foot Ġet Ġal . Ġuse Ġiter ative Ġclassification Ġto Ġrepeatedly Ġgenerate Ġnew Ġestimates Ġbased Ġon Ġthe Ġcurrent Ġstate Ġof Ġknowledge ĠB IB REF 11 Ġ; ĠS rid har Ġet Ġal . Ġuse Ġprob abil istic Ġsoft Ġlogic Ġ( PS L ) Ġto Ġmodel Ġreply Ġlinks Ġvia Ġcollaborative Ġfiltering ĠB IB REF 12 Ġ. ĠIn Ġthe ĠFacebook Ġdataset Ġwe Ġstudy , Ġwe Ġuse Ġcomments Ġinstead Ġof Ġreply Ġlinks . ĠHowever , Ġas Ġthe Ġultimate Ġgoal Ġin Ġthis Ġpaper Ġis Ġpredicting Ġnot Ġcomment Ġstance Ġbut Ġpost Ġstance , Ġwe Ġtreat Ġcomments Ġas Ġextra Ġinformation Ġfor Ġuse Ġin Ġpredicting Ġpost Ġstance . </s> Deep ĠLearning Ġon ĠExtra - L ingu istic ĠFeatures </s> In Ġrecent Ġyears Ġneural Ġnetwork Ġmodels Ġhave Ġbeen Ġapplied Ġto Ġdocument Ġsentiment Ġclassification ĠB IB REF 13 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 2 Ġ. ĠText Ġfeatures Ġcan Ġbe Ġused Ġin Ġdeep Ġnetworks Ġto Ġcapture Ġtext Ġsemantics Ġor Ġsentiment . ĠFor Ġexample , ĠDong Ġet Ġal . Ġuse Ġan Ġadaptive Ġlayer Ġin Ġa Ġrecursive Ġneural Ġnetwork Ġfor Ġtarget - dependent ĠTwitter Ġsentiment Ġanalysis , Ġwhere Ġtargets Ġare Ġtopics Ġsuch Ġas Ġwindows Ġ7 Ġor Ġt aylor Ġswift ĠB IB REF 16 Ġ, ĠB IB REF 17 Ġ; Ġrecursive Ġneural Ġtens or Ġnetworks Ġ( R NT Ns ) Ġutilize Ġsentence Ġparse Ġtrees Ġto Ġcapture Ġsentence - level Ġsentiment Ġfor Ġmovie Ġreviews ĠB IB REF 4 Ġ; ĠLe Ġand ĠMik ol ov Ġpredict Ġsentiment Ġby Ġusing Ġparagraph Ġvectors Ġto Ġmodel Ġeach Ġparagraph Ġas Ġa Ġcontinuous Ġrepresentation ĠB IB REF 18 Ġ. ĠThey Ġshow Ġthat Ġperformance Ġcan Ġthus Ġbe Ġimproved Ġby Ġmore Ġdelicate Ġtext Ġmodels . </s> Others Ġhave Ġsuggested Ġusing Ġextra - ling u istic Ġfeatures Ġto Ġimprove Ġthe Ġdeep Ġlearning Ġmodel . ĠThe Ġuser - word Ġcomposition Ġvector Ġmodel Ġ( U WC VM ) ĠB IB REF 19 Ġis Ġinspired Ġby Ġthe Ġpossibility Ġthat Ġthe Ġstrength Ġof Ġsentiment Ġwords Ġis Ġuser - specific ; Ġto Ġcapture Ġthis Ġthey Ġadd Ġuser Ġembed d ings Ġin Ġtheir Ġmodel . ĠIn ĠUP NN , Ġa Ġlater Ġextension , Ġthey Ġfurther Ġadd Ġa Ġproduct - word Ġcomposition Ġas Ġproduct Ġembed d ings , Ġarguing Ġthat Ġproducts Ġcan Ġalso Ġshow Ġdifferent Ġtendencies Ġof Ġbeing Ġrated Ġor Ġreviewed ĠB IB REF 20 Ġ. ĠTheir Ġaddition Ġof Ġuser Ġinformation Ġyielded Ġ2 âĢĵ 10 % Ġimprovements Ġin Ġaccuracy Ġas Ġcompared Ġto Ġthe Ġabove - mentioned ĠR NT N Ġand Ġparagraph Ġvector Ġmethods . ĠWe Ġalso Ġseek Ġto Ġinject Ġuser Ġinformation Ġinto Ġthe Ġneural Ġnetwork Ġmodel . ĠIn Ġcomparison Ġto Ġthe Ġresearch Ġof ĠTang Ġet Ġal . Ġon Ġsentiment Ġclassification Ġfor Ġproduct Ġreviews , Ġthe Ġdifference Ġis Ġtwo - fold . ĠFirst , Ġwe Ġtake Ġinto Ġaccount Ġmultiple Ġusers Ġ( one Ġauthor Ġand Ġpotentially Ġmany Ġlik ers ) Ġfor Ġone Ġpost , Ġwhereas Ġonly Ġone Ġuser Ġ( the Ġreviewer ) Ġis Ġinvolved Ġin Ġa Ġreview . ĠSecond , Ġwe Ġadd Ġcomment Ġinformation Ġto Ġprovide Ġmore Ġfeatures Ġfor Ġpost Ġstance Ġclassification . ĠNone Ġof Ġthese Ġtwo Ġfactors Ġhave Ġbeen Ġconsidered Ġpreviously Ġin Ġa Ġdeep Ġlearning Ġmodel Ġfor Ġtext Ġstance Ġclassification . ĠTherefore , Ġwe Ġpropose ĠUTC NN , Ġwhich Ġgenerates Ġand Ġutilizes Ġuser Ġembed d ings Ġfor Ġall Ġusers ĠâĢĶ Ġeven Ġfor Ġthose Ġwho Ġhave Ġnot Ġauthored Ġany Ġposts ĠâĢĶ Ġand Ġincorporates Ġcomments Ġto Ġfurther Ġimprove Ġperformance . </s> Method </s> In Ġthis Ġsection , Ġwe Ġfirst Ġdescribe ĠCNN - based Ġdocument Ġcomposition , Ġwhich Ġcaptures Ġuser - Ġand Ġtopic - dependent Ġdocument - level Ġsemantic Ġrepresentation Ġfrom Ġword Ġrepresentations . ĠThen Ġwe Ġshow Ġhow Ġto Ġadd Ġcomment Ġinformation Ġto Ġconstruct Ġthe Ġuser - topic - comment Ġneural Ġnetwork Ġ( UTC NN ). </s> User - Ġand ĠTopic - dependent ĠDocument ĠCom position </s> As Ġshown Ġin ĠFigure ĠFIG REF 4 Ġ, Ġwe Ġuse Ġa Ġgeneral ĠCNN ĠB IB REF 3 Ġand Ġtwo Ġsemantic Ġtransformations Ġfor Ġdocument Ġcomposition Ġ. ĠWe Ġare Ġgiven Ġa Ġdocument Ġwith Ġan Ġengaged Ġuser ĠIN LINE FORM 0 Ġ, Ġa Ġtopic ĠIN LINE FORM 1 Ġ, Ġand Ġits Ġcomposite ĠIN LINE FORM 2 Ġwords , Ġeach Ġword ĠIN LINE FORM 3 Ġof Ġwhich Ġis Ġassociated Ġwith Ġa Ġword Ġembed ding ĠIN LINE FORM 4 Ġwhere ĠIN LINE FORM 5 Ġis Ġthe Ġvector Ġdimension . ĠFor Ġeach Ġword Ġembed ding ĠIN LINE FORM 6 Ġ, Ġwe Ġapply Ġtwo Ġdot Ġoperations Ġas Ġshown Ġin ĠEqu ation ĠEQ REF 6 Ġ: ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġmodels Ġthe Ġuser Ġreading Ġpreference Ġfor Ġcertain Ġsemantics , Ġand ĠIN LINE FORM 1 Ġmodels Ġthe Ġtopic Ġsemantics ; ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġare Ġthe Ġdimensions Ġof Ġtransformed Ġuser Ġand Ġtopic Ġembed d ings Ġrespectively . ĠWe Ġuse ĠIN LINE FORM 4 Ġto Ġmodel Ġsem antically Ġwhat Ġeach Ġuser Ġprefers Ġto Ġread Ġand / or Ġwrite , Ġand Ġuse ĠIN LINE FORM 5 Ġto Ġmodel Ġthe Ġsemantics Ġof Ġeach Ġtopic . ĠThe Ġdot Ġoperation Ġof ĠIN LINE FORM 6 Ġand ĠIN LINE FORM 7 Ġtransforms Ġthe Ġglobal Ġrepresentation ĠIN LINE FORM 8 Ġto Ġa Ġuser - dependent Ġrepresentation . ĠLikewise , Ġthe Ġdot Ġoperation Ġof ĠIN LINE FORM 9 Ġand ĠIN LINE FORM 10 Ġtransforms ĠIN LINE FORM 11 Ġto Ġa Ġtopic - dependent Ġrepresentation . </s> After Ġthe Ġtwo Ġdot Ġoperations Ġon ĠIN LINE FORM 0 Ġ, Ġwe Ġhave Ġuser - dependent Ġand Ġtopic - dependent Ġword Ġvectors ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ, Ġwhich Ġare Ġconc aten ated Ġto Ġform Ġa Ġuser - Ġand Ġtopic - dependent Ġword Ġvector ĠIN LINE FORM 3 Ġ. ĠThen Ġthe Ġtransformed Ġword Ġembed d ings ĠIN LINE FORM 4 Ġare Ġused Ġas Ġthe ĠCNN Ġinput . ĠHere Ġwe Ġapply Ġthree Ġconv olution al Ġlayers Ġon Ġthe Ġconc aten ated Ġtransformed Ġword Ġembed d ings ĠIN LINE FORM 5 Ġ: ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġindex Ġof Ġwords ; ĠIN LINE FORM 1 Ġis Ġa Ġnon - linear Ġactivation Ġfunction Ġ( we Ġuse ĠIN LINE FORM 2 Ġ); ĠIN LINE FORM 5 Ġis Ġthe Ġconv olution al Ġfilter Ġwith Ġinput Ġlength ĠIN LINE FORM 6 Ġand Ġoutput Ġlength ĠIN LINE FORM 7 Ġ, Ġwhere ĠIN LINE FORM 8 Ġis Ġthe Ġwindow Ġsize Ġof Ġthe Ġconv olution al Ġoperation ; Ġand ĠIN LINE FORM 9 Ġand ĠIN LINE FORM 10 Ġare Ġthe Ġoutput Ġand Ġbias Ġof Ġthe Ġconv olution Ġlayer ĠIN LINE FORM 11 Ġ, Ġrespectively . ĠIn Ġour Ġexperiments , Ġthe Ġthree Ġwindow Ġsizes ĠIN LINE FORM 12 Ġin Ġthe Ġthree Ġconv olution Ġlayers Ġare Ġone , Ġtwo , Ġand Ġthree , Ġencoding Ġun ig ram , Ġbig ram , Ġand Ġtrig ram Ġsemantics Ġaccordingly . </s> After Ġthe Ġconv olution al Ġlayer , Ġwe Ġadd Ġa Ġmaximum Ġpool ing Ġlayer Ġamong Ġconv olution al Ġoutputs Ġto Ġobtain Ġthe Ġun ig ram , Ġbig ram , Ġand Ġtrig ram Ġn - gram Ġrepresentations . ĠThis Ġis Ġsucceeded Ġby Ġan Ġaverage Ġpool ing Ġlayer Ġfor Ġan Ġelement - wise Ġaverage Ġof Ġthe Ġthree Ġmaxim ized Ġconv olution Ġoutputs . </s> UTC NN ĠModel ĠDescription </s> Figure ĠFIG REF 10 Ġillustrates Ġthe ĠUTC NN Ġmodel . ĠAs Ġmore Ġthan Ġone Ġuser Ġmay Ġinteract Ġwith Ġa Ġgiven Ġpost , Ġwe Ġfirst Ġadd Ġa Ġmaximum Ġpool ing Ġlayer Ġafter Ġthe Ġuser Ġmatrix Ġembed ding Ġlayer Ġand Ġuser Ġvector Ġembed ding Ġlayer Ġto Ġform Ġa Ġmoderator Ġmatrix Ġembed ding ĠIN LINE FORM 0 Ġand Ġa Ġmoderator Ġvector Ġembed ding ĠIN LINE FORM 1 Ġfor Ġmoderator ĠIN LINE FORM 2 Ġrespectively , Ġwhere ĠIN LINE FORM 3 Ġis Ġused Ġfor Ġthe Ġsemantic Ġtransformation Ġin Ġthe Ġdocument Ġcomposition Ġprocess , Ġas Ġmentioned Ġin Ġthe Ġprevious Ġsection . ĠThe Ġterm Ġmoderator Ġhere Ġis Ġto Ġdenote Ġthe Ġpseudo Ġuser Ġwho Ġprovides Ġthe Ġoverall Ġsemantic / sent iment Ġof Ġall Ġthe Ġengaged Ġusers Ġfor Ġone Ġdocument . ĠThe Ġembed ding ĠIN LINE FORM 4 Ġmodels Ġthe Ġmoderator Ġstance Ġpreference , Ġthat Ġis , Ġthe Ġpattern Ġof Ġthe Ġrevealed Ġuser Ġstance : Ġwhether Ġa Ġuser Ġis Ġwilling Ġto Ġshow Ġhis Ġpreference , Ġwhether Ġa Ġuser Ġlikes Ġto Ġshow Ġimpartial ity Ġwith Ġneutral Ġstatements Ġand Ġreasonable Ġarguments , Ġor Ġjust Ġwants Ġto Ġshow Ġstrong Ġsupport Ġfor Ġone Ġstance . ĠIdeally , Ġthe Ġlatent Ġuser Ġstance Ġis Ġmodeled Ġby ĠIN LINE FORM 5 Ġfor Ġeach Ġuser . ĠLikewise , Ġfor Ġtopic Ġinformation , Ġa Ġmaximum Ġpool ing Ġlayer Ġis Ġadded Ġafter Ġthe Ġtopic Ġmatrix Ġembed ding Ġlayer Ġand Ġtopic Ġvector Ġembed ding Ġlayer Ġto Ġform Ġa Ġjoint Ġtopic Ġmatrix Ġembed ding ĠIN LINE FORM 6 Ġand Ġa Ġjoint Ġtopic Ġvector Ġembed ding ĠIN LINE FORM 7 Ġfor Ġtopic ĠIN LINE FORM 8 Ġrespectively , Ġwhere ĠIN LINE FORM 9 Ġmodels Ġthe Ġsemantic Ġtransformation Ġof Ġtopic ĠIN LINE FORM 10 Ġas Ġin Ġusers Ġand ĠIN LINE FORM 11 Ġmodels Ġthe Ġtopic Ġstance Ġtendency . ĠThe Ġlatent Ġtopic Ġstance Ġis Ġalso Ġmodeled Ġby ĠIN LINE FORM 12 Ġfor Ġeach Ġtopic . </s> As Ġfor Ġcomments , Ġwe Ġview Ġthem Ġas Ġshort Ġdocuments Ġwith Ġauthors Ġonly Ġbut Ġwithout Ġlik ers Ġnor Ġtheir Ġown Ġcomments . ĠTherefore Ġwe Ġapply Ġdocument Ġcomposition Ġon Ġcomments Ġalthough Ġhere Ġusers Ġare Ġcommenters Ġ( users Ġwho Ġcomment ). ĠIt Ġis Ġnoticed Ġthat Ġthe Ġword Ġembed d ings ĠIN LINE FORM 0 Ġfor Ġthe Ġsame Ġword Ġin Ġthe Ġposts Ġand Ġcomments Ġare Ġthe Ġsame , Ġbut Ġafter Ġbeing Ġtransformed Ġto ĠIN LINE FORM 1 Ġin Ġthe Ġdocument Ġcomposition Ġprocess Ġshown Ġin ĠFigure ĠFIG REF 4 Ġ, Ġthey Ġmight Ġbecome Ġdifferent Ġbecause Ġof Ġtheir Ġdifferent Ġengaged Ġusers . ĠThe Ġoutput Ġcomment Ġrepresentation Ġtogether Ġwith Ġthe Ġcommenter Ġvector Ġembed ding ĠIN LINE FORM 2 Ġand Ġtopic Ġvector Ġembed ding ĠIN LINE FORM 3 Ġare Ġconc aten ated Ġand Ġa Ġmaximum Ġpool ing Ġlayer Ġis Ġadded Ġto Ġselect Ġthe Ġmost Ġimportant Ġfeature Ġfor Ġcomments . ĠInstead Ġof Ġrequiring Ġthat Ġthe Ġcomment Ġstance Ġagree Ġwith Ġthe Ġpost , ĠUTC NN Ġsimply Ġextracts Ġthe Ġmost Ġimportant Ġfeatures Ġof Ġthe Ġcomment Ġcontents ; Ġthey Ġcould Ġbe Ġhelpful , Ġwhether Ġthey Ġshow Ġobvious Ġagreement Ġor Ġdisagreement . ĠTherefore Ġwhen Ġcombining Ġcomment Ġinformation Ġhere , Ġthe Ġmaximum Ġpool ing Ġlayer Ġis Ġmore Ġappropriate Ġthan Ġother Ġpool ing Ġor Ġmerging Ġlayers . ĠIndeed , Ġwe Ġbelieve Ġthis Ġis Ġone Ġreason Ġfor ĠUTC NN 's Ġperformance Ġgains . </s> Finally , Ġthe Ġpooled Ġcomment Ġrepresentation , Ġtogether Ġwith Ġuser Ġvector Ġembed ding ĠIN LINE FORM 0 Ġ, Ġtopic Ġvector Ġembed ding ĠIN LINE FORM 1 Ġ, Ġand Ġdocument Ġrepresentation Ġare Ġfed Ġto Ġa Ġfully Ġconnected Ġnetwork , Ġand Ġsoft max Ġis Ġapplied Ġto Ġyield Ġthe Ġfinal Ġstance Ġlabel Ġprediction Ġfor Ġthe Ġpost . </s> Exper iment </s> We Ġstart Ġwith Ġthe Ġexperimental Ġdataset Ġand Ġthen Ġdescribe Ġthe Ġtraining Ġprocess Ġas Ġwell Ġas Ġthe Ġimplementation Ġof Ġthe Ġbas elines . ĠWe Ġalso Ġimplement Ġseveral Ġvariations Ġto Ġreveal Ġthe Ġeffects Ġof Ġfeatures : Ġauthors , Ġlik ers , Ġcomment , Ġand Ġcommenters . ĠIn Ġthe Ġresults Ġsection Ġwe Ġcompare Ġour Ġmodel Ġwith Ġrelated Ġwork . </s> Dat as et </s> We Ġtested Ġthe Ġproposed ĠUTC NN Ġon Ġtwo Ġdifferent Ġdatasets : ĠFB Fans Ġand ĠCreate Deb ate . ĠFB Fans Ġis Ġa Ġprivately - owned , Ġsingle - topic , ĠChinese , Ġun balanced , Ġsocial Ġmedia Ġdataset , Ġand ĠCreate Deb ate Ġis Ġa Ġpublic , Ġmultiple - topic , ĠEnglish , Ġbalanced , Ġforum Ġdataset . ĠResults Ġusing Ġthese Ġtwo Ġdatasets Ġshow Ġthe Ġapplic ability Ġand Ġsuperiority Ġfor Ġdifferent Ġtopics , Ġlanguages , Ġdata Ġdistributions , Ġand Ġplatforms . </s> The ĠFB Fans Ġdataset Ġcontains Ġdata Ġfrom Ġanti - nuclear - power ĠChinese ĠFacebook Ġfan Ġgroups Ġfrom ĠSeptember Ġ2013 Ġto ĠAugust Ġ2014 , Ġincluding Ġposts Ġand Ġtheir Ġauthor Ġand Ġlik er ĠIDs . ĠThere Ġare Ġa Ġtotal Ġof Ġ2 , 496 Ġauthors , Ġ505 , 137 Ġlik ers , Ġ33 , 686 Ġcommenters , Ġand Ġ505 , 412 Ġunique Ġusers . ĠTwo Ġannot ators Ġwere Ġasked Ġto Ġtake Ġinto Ġaccount Ġonly Ġthe Ġpost Ġcontent Ġto Ġlabel Ġthe Ġstance Ġof Ġthe Ġposts Ġin Ġthe Ġwhole Ġdataset Ġas Ġsupportive , Ġneutral , Ġor Ġun support ive Ġ( here after Ġden oted Ġas ĠSup , ĠNe u , Ġand ĠUn s ). ĠSup / Un s Ġposts Ġwere Ġthose Ġin Ġsupport Ġof Ġor Ġagainst Ġanti - re const ruction ; ĠNe u Ġposts Ġwere Ġthose Ġev inc ing Ġa Ġneutral Ġstandpoint Ġon Ġthe Ġtopic , Ġor Ġwere Ġirrelevant . ĠRaw Ġagreement Ġbetween Ġannot ators Ġis Ġ0 . 91 , Ġindicating Ġhigh Ġagreement . ĠSpecifically , ĠCohen âĢ Ļ s ĠKappa Ġfor ĠNe u Ġand Ġnot ĠNe u Ġlabeling Ġis Ġ0 . 58 Ġ( moderate ), Ġand Ġfor ĠSup Ġor ĠUn s Ġlabeling Ġis Ġ0 . 84 Ġ( almost Ġperfect ). ĠPosts Ġwith Ġinconsistent Ġlabels Ġwere Ġfiltered Ġout , Ġand Ġthe Ġdevelopment Ġand Ġtesting Ġsets Ġwere Ġrandomly Ġselected Ġfrom Ġwhat Ġwas Ġleft . ĠPosts Ġin Ġthe Ġdevelopment Ġand Ġtesting Ġsets Ġinvolved Ġat Ġleast Ġone Ġuser Ġwho Ġappeared Ġin Ġthe Ġtraining Ġset . ĠThe Ġnumber Ġof Ġposts Ġfor Ġeach Ġstance Ġis Ġshown Ġon Ġthe Ġleft - hand Ġside Ġof ĠTable ĠT AB REF 12 Ġ. ĠAbout Ġtwenty Ġpercent Ġof Ġthe Ġposts Ġwere Ġlabeled Ġwith Ġa Ġstance , Ġand Ġthe Ġnumber Ġof Ġsupportive Ġ( Sup ) Ġposts Ġwas Ġmuch Ġlarger Ġthan Ġthat Ġof Ġthe Ġun support ive Ġ( Un s ) Ġones : Ġthis Ġis Ġthus Ġhighly Ġskewed Ġdata , Ġwhich Ġcompl icates Ġstance Ġclassification . ĠOn Ġaverage , Ġ161 . 1 Ġusers Ġwere Ġinvolved Ġin Ġone Ġpost . ĠThe Ġmaximum Ġwas Ġ23 , 297 Ġand Ġthe Ġminimum Ġwas Ġone Ġ( the Ġauthor ). ĠFor Ġcomments , Ġon Ġaverage Ġthere Ġwere Ġ3 Ġcomments Ġper Ġpost . ĠThe Ġmaximum Ġwas Ġ1 , 09 2 Ġand Ġthe Ġminimum Ġwas Ġzero . </s> To Ġtest Ġwhether Ġthe Ġassumption Ġof Ġthis Ġpaper ĠâĢĵ Ġposts Ġattract Ġusers Ġwho Ġhold Ġthe Ġsame Ġstance Ġto Ġlike Ġthem ĠâĢĵ Ġis Ġreliable , Ġwe Ġexamine Ġthe Ġlikes Ġfrom Ġauthors Ġof Ġdifferent Ġstances . ĠPosts Ġin ĠFB Fans Ġdataset Ġare Ġused Ġfor Ġthis Ġanalysis . ĠWe Ġcalculate Ġthe Ġlike Ġstatistics Ġof Ġeach Ġdistinct Ġauthor Ġfrom Ġthese Ġ32 , 595 Ġposts . ĠAs Ġthe Ġnumbers Ġof Ġauthors Ġin Ġthe ĠSup , ĠNe u Ġand ĠUn s Ġstances Ġare Ġlargely Ġim balanced , Ġthese Ġnumbers Ġare Ġnormalized Ġby Ġthe Ġnumber Ġof Ġusers Ġof Ġeach Ġstance . ĠTable ĠT AB REF 13 Ġshows Ġthe Ġresults . ĠPosts Ġwith Ġstances Ġ( i . e ., Ġnot Ġneutral ) Ġattract Ġusers Ġof Ġthe Ġsame Ġstance . ĠNeutral Ġposts Ġalso Ġattract Ġboth Ġsupportive Ġand Ġneutral Ġusers , Ġlike Ġwhat Ġwe Ġobserve Ġin Ġsupportive Ġposts , Ġbut Ġjust Ġthe Ġneutral Ġposts Ġcan Ġattract Ġeven Ġmore Ġneutral Ġlik ers . ĠThese Ġresults Ġdo Ġsuggest Ġthat Ġusers Ġprefer Ġposts Ġof Ġthe Ġsame Ġstance , Ġor Ġat Ġleast Ġposts Ġof Ġno Ġobvious Ġstance Ġwhich Ġmight Ġcause Ġannoyance Ġwhen Ġreading , Ġand Ġhence Ġsupport Ġthe Ġuser Ġmodeling Ġin Ġour Ġapproach . </s> The ĠCreate Deb ate Ġdataset Ġwas Ġcollected Ġfrom Ġan ĠEnglish Ġonline Ġdebate Ġforum Ġdiscussing Ġfour Ġtopics : Ġabortion Ġ( AB O ), Ġgay Ġrights Ġ( G AY ), ĠObama Ġ( O BA ), Ġand Ġmarijuana Ġ( MAR ). ĠThe Ġposts Ġare Ġannot ated Ġas Ġfor Ġ( F ) Ġand Ġagainst Ġ( A ). ĠRepl ies Ġto Ġposts Ġin Ġthis Ġdataset Ġare Ġalso Ġlabeled Ġwith Ġstance Ġand Ġhence Ġuse Ġthe Ġsame Ġdata Ġformat Ġas Ġposts . ĠThe Ġlabeling Ġresults Ġare Ġshown Ġin Ġthe Ġright - hand Ġside Ġof ĠTable ĠT AB REF 12 Ġ. ĠWe Ġobserve Ġthat Ġthe Ġdataset Ġis Ġmore Ġbalanced Ġthan Ġthe ĠFB Fans Ġdataset . ĠIn Ġaddition , Ġthere Ġare Ġ9 77 Ġunique Ġusers Ġin Ġthe Ġdataset . ĠTo Ġcompare Ġwith ĠHasan Ġand ĠNg 's Ġwork , Ġwe Ġconducted Ġfive - fold Ġcross - valid ation Ġand Ġpresent Ġthe Ġannotation Ġresults Ġas Ġthe Ġaverage Ġnumber Ġof Ġall Ġfolds ĠB IB REF 9 Ġ, ĠB IB REF 5 Ġ. </s> The ĠFB Fans Ġdataset Ġhas Ġmore Ġintegrated Ġfunctions Ġthan Ġthe ĠCreate Deb ate Ġdataset ; Ġthus Ġour Ġmodel Ġcan Ġutilize Ġall Ġlinguistic Ġand Ġextra - ling u istic Ġfeatures . ĠFor Ġthe ĠCreate Deb ate Ġdataset , Ġon Ġthe Ġother Ġhand , Ġthe Ġlike Ġand Ġcomment Ġfeatures Ġare Ġnot Ġavailable Ġ( as Ġthere Ġis Ġa Ġstance Ġlabel Ġfor Ġeach Ġreply , Ġreplies Ġare Ġevaluated Ġas Ġposts Ġas Ġother Ġprevious Ġwork ) Ġbut Ġwe Ġstill Ġimplemented Ġour Ġmodel Ġusing Ġthe Ġcontent , Ġauthor , Ġand Ġtopic Ġinformation . </s> Settings </s> In Ġthe ĠUTC NN Ġtraining Ġprocess , Ġcross - ent ropy Ġwas Ġused Ġas Ġthe Ġloss Ġfunction Ġand ĠAda Grad Ġas Ġthe Ġoptim izer . ĠFor ĠFB Fans Ġdataset , Ġwe Ġlearned Ġthe Ġ50 - dimensional Ġword Ġembed d ings Ġon Ġthe Ġwhole Ġdataset Ġusing ĠGl o Ve ĠB IB REF 21 Ġto Ġcapture Ġthe Ġword Ġsemantics ; Ġfor ĠCreate Deb ate Ġdataset Ġwe Ġused Ġthe Ġpublicly Ġavailable ĠEnglish Ġ50 - dimensional Ġword Ġembed d ings , Ġpre - trained Ġalso Ġusing ĠGl o Ve . ĠThese Ġword Ġembed d ings Ġwere Ġfixed Ġin Ġthe Ġtraining Ġprocess . ĠThe Ġlearning Ġrate Ġwas Ġset Ġto Ġ0 . 03 . ĠAll Ġuser Ġand Ġtopic Ġembed d ings Ġwere Ġrandomly Ġinitialized Ġin Ġthe Ġrange Ġof Ġ[- 0 . 1 Ġ0 . 1 ]. ĠMatrix Ġembed d ings Ġfor Ġusers Ġand Ġtopics Ġwere Ġsized Ġat Ġ250 Ġ( ĠIN LINE FORM 0 Ġ); Ġvector Ġembed d ings Ġfor Ġusers Ġand Ġtopics Ġwere Ġset Ġto Ġlength Ġ10 . </s> We Ġapplied Ġthe ĠL DA Ġtopic Ġmodel ĠB IB REF 22 Ġon Ġthe ĠFB Fans Ġdataset Ġto Ġdetermine Ġthe Ġlatent Ġtopics Ġwith Ġwhich Ġto Ġbuild Ġtopic Ġembed d ings , Ġas Ġthere Ġis Ġonly Ġone Ġgeneral Ġknown Ġtopic : Ġnuclear Ġpower Ġplants . ĠWe Ġlearned Ġ100 Ġlatent Ġtopics Ġand Ġassigned Ġthe Ġtop Ġthree Ġtopics Ġfor Ġeach Ġpost . ĠFor Ġthe ĠCreate Deb ate Ġdataset , Ġwhich Ġitself Ġconstitutes Ġfour Ġtopics , Ġthe Ġtopic Ġlabels Ġfor Ġposts Ġwere Ġused Ġdirectly Ġwithout Ġadditionally Ġapplying ĠL DA . </s> For Ġthe ĠFB Fans Ġdata Ġwe Ġreport Ġclass - based Ġf - sc ores Ġas Ġwell Ġas Ġthe Ġmacro - average Ġf - score Ġ( ĠIN LINE FORM 0 Ġ) Ġshown Ġin Ġequation ĠEQ REF 19 Ġ. ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġare Ġthe Ġaverage Ġprecision Ġand Ġrecall Ġof Ġthe Ġthree Ġclass . ĠWe Ġadopted Ġthe Ġmacro - average Ġf - score Ġas Ġthe Ġevaluation Ġmetric Ġfor Ġthe Ġoverall Ġperformance Ġbecause Ġ( 1 ) Ġthe Ġexperimental Ġdataset Ġis Ġseverely Ġim balanced , Ġwhich Ġis Ġcommon Ġfor Ġcontentious Ġissues ; Ġand Ġ( 2 ) Ġfor Ġstance Ġclassification , Ġcontent Ġin Ġminor - class Ġposts Ġis Ġusually Ġmore Ġimportant Ġfor Ġfurther Ġapplications . ĠFor Ġthe ĠCreate Deb ate Ġdataset , Ġaccuracy Ġwas Ġadopted Ġas Ġthe Ġevaluation Ġmetric Ġto Ġcompare Ġthe Ġresults Ġwith Ġrelated Ġwork ĠB IB REF 7 Ġ, ĠB IB REF 9 Ġ, ĠB IB REF 12 Ġ. </s> Bas elines </s> We Ġpit Ġour Ġmodel Ġagainst Ġthe Ġfollowing Ġbas elines : Ġ1 ) ĠS VM Ġwith Ġun ig ram , Ġbig ram , Ġand Ġtrig ram Ġfeatures , Ġwhich Ġis Ġa Ġstandard Ġyet Ġrather Ġstrong Ġclass ifier Ġfor Ġtext Ġfeatures ; Ġ2 ) ĠS VM Ġwith Ġaverage Ġword Ġembed ding , Ġwhere Ġa Ġdocument Ġis Ġrepresented Ġas Ġa Ġcontinuous Ġrepresentation Ġby Ġaveraging Ġthe Ġembed d ings Ġof Ġthe Ġcomposite Ġwords ; Ġ3 ) ĠS VM Ġwith Ġaverage Ġtransformed Ġword Ġembed d ings Ġ( the ĠIN LINE FORM 0 Ġin Ġequation ĠEQ REF 6 Ġ), Ġwhere Ġa Ġdocument Ġis Ġrepresented Ġas Ġa Ġcontinuous Ġrepresentation Ġby Ġaveraging Ġthe Ġtransformed Ġembed d ings Ġof Ġthe Ġcomposite Ġwords ; Ġ4 ) Ġtwo Ġmature Ġdeep Ġlearning Ġmodels Ġon Ġtext Ġclassification , ĠCNN ĠB IB REF 3 Ġand ĠRec urrent ĠConv olution al ĠNeural ĠNetworks Ġ( RC NN ) ĠB IB REF 0 Ġ, Ġwhere Ġthe Ġhyper param eters Ġare Ġbased Ġon Ġtheir Ġwork ; Ġ5 ) Ġthe Ġabove ĠS VM Ġand Ġdeep Ġlearning Ġmodels Ġwith Ġcomment Ġinformation ; Ġ6 ) ĠUTC NN Ġwithout Ġuser Ġinformation , Ġrepresenting Ġa Ġpure - text ĠCNN Ġmodel Ġwhere Ġwe Ġuse Ġthe Ġsame Ġuser Ġmatrix Ġand Ġuser Ġembed d ings ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġfor Ġeach Ġuser ; Ġ7 ) ĠUTC NN Ġwithout Ġthe ĠL DA Ġmodel , Ġrepresenting Ġhow ĠUTC NN Ġworks Ġwith Ġa Ġsingle - topic Ġdataset ; Ġ8 ) ĠUTC NN Ġwithout Ġcomments , Ġin Ġwhich Ġthe Ġmodel Ġpredicts Ġthe Ġstance Ġlabel Ġgiven Ġonly Ġuser Ġand Ġtopic Ġinformation . ĠAll Ġthese Ġmodels Ġwere Ġtrained Ġon Ġthe Ġtraining Ġset , Ġand Ġparameters Ġas Ġwell Ġas Ġthe ĠS VM Ġkernel Ġselections Ġ( linear Ġor ĠRB F ) Ġwere Ġfine - tun ed Ġon Ġthe Ġdevelopment Ġset . ĠAlso , Ġwe Ġadopt Ġovers am pling Ġon ĠSV Ms , ĠCNN Ġand ĠRC NN Ġbecause Ġthe ĠFB Fans Ġdataset Ġis Ġhighly Ġim balanced . </s> Results Ġon ĠFB Fans ĠDat as et </s> In ĠTable ĠT AB REF 22 Ġwe Ġshow Ġthe Ġresults Ġof ĠUTC NN Ġand Ġthe Ġbas elines Ġon Ġthe ĠFB Fans Ġdataset . ĠHere ĠMajority Ġyields Ġgood Ġperformance Ġon ĠNe u Ġsince ĠFB Fans Ġis Ġhighly Ġbiased Ġto Ġthe Ġneutral Ġclass . ĠThe ĠS VM Ġmodels Ġperform Ġwell Ġon ĠSup Ġand ĠNe u Ġbut Ġperform Ġpoorly Ġfor ĠUn s , Ġshowing Ġthat Ġcontent Ġinformation Ġin Ġitself Ġis Ġinsufficient Ġto Ġpredict Ġstance Ġlabels , Ġespecially Ġfor Ġthe Ġminor Ġclass . ĠWith Ġthe Ġtransformed Ġword Ġembed ding Ġfeature , ĠS VM Ġcan Ġachieve Ġcomparable Ġperformance Ġas ĠS VM Ġwith Ġn - gram Ġfeature . ĠHowever , Ġthe Ġmuch Ġfewer Ġfeature Ġdimension Ġof Ġthe Ġtransformed Ġword Ġembed ding Ġmakes ĠS VM Ġwith Ġword Ġembed d ings Ġa Ġmore Ġefficient Ġchoice Ġfor Ġmodeling Ġthe Ġlarge Ġscale Ġsocial Ġmedia Ġdataset . ĠFor Ġthe ĠCNN Ġand ĠRC NN Ġmodels , Ġthey Ġperform Ġslightly Ġbetter Ġthan Ġmost Ġof Ġthe ĠS VM Ġmodels Ġbut Ġstill , Ġthe Ġcontent Ġinformation Ġis Ġinsufficient Ġto Ġachieve Ġa Ġgood Ġperformance Ġon Ġthe ĠUn s Ġposts . ĠAs Ġto Ġadding Ġcomment Ġinformation Ġto Ġthese Ġmodels , Ġsince Ġthe Ġcommenters Ġdo Ġnot Ġalways Ġhold Ġthe Ġsame Ġstance Ġas Ġthe Ġauthor , Ġsimply Ġadding Ġcomments Ġand Ġpost Ġcontents Ġtogether Ġmerely Ġadds Ġnoise Ġto Ġthe Ġmodel . </s> Among Ġall ĠUTC NN Ġvariations , Ġwe Ġfind Ġthat Ġuser Ġinformation Ġis Ġmost Ġimportant , Ġfollowed Ġby Ġtopic Ġand Ġcomment Ġinformation . ĠUTC NN Ġwithout Ġuser Ġinformation Ġshows Ġresults Ġsimilar Ġto ĠSV Ms ĠâĢĶ Ġit Ġdoes Ġwell Ġfor ĠSup Ġand ĠNe u Ġbut Ġdetects Ġno ĠUn s . ĠIts Ġbest Ġf - sc ores Ġon Ġboth ĠSup Ġand ĠNe u Ġamong Ġall Ġmethods Ġshow Ġthat Ġwith Ġenough Ġtraining Ġdata , Ġcontent - based Ġmodels Ġcan Ġperform Ġwell ; Ġat Ġthe Ġsame Ġtime , Ġthe Ġlack Ġof Ġuser Ġinformation Ġresults Ġin Ġtoo Ġfew Ġclues Ġfor Ġminor - class Ġposts Ġto Ġeither Ġpredict Ġtheir Ġstance Ġdirectly Ġor Ġlink Ġthem Ġto Ġother Ġusers Ġand Ġposts Ġfor Ġimproved Ġperformance . ĠThe Ġ17 . 5 % Ġimprovement Ġwhen Ġadding Ġuser Ġinformation Ġsuggests Ġthat Ġuser Ġinformation Ġis Ġespecially Ġuseful Ġwhen Ġthe Ġdataset Ġis Ġhighly Ġim balanced . ĠAll Ġmodels Ġthat Ġconsider Ġuser Ġinformation Ġpredict Ġthe Ġminority Ġclass Ġsuccessfully . ĠU CT NN Ġwithout Ġtopic Ġinformation Ġworks Ġwell Ġbut Ġachieves Ġlower Ġperformance Ġthan Ġthe Ġfull ĠUTC NN Ġmodel . ĠThe Ġ4 . 9 % Ġperformance Ġgain Ġbrought Ġby ĠL DA Ġshows Ġthat Ġalthough Ġit Ġis Ġsatisfactory Ġfor Ġsingle Ġtopic Ġdatasets , Ġadding Ġthat Ġlatent Ġtopics Ġstill Ġbenefits Ġperformance : Ġeven Ġwhen Ġwe Ġare Ġdiscussing Ġthe Ġsame Ġtopic , Ġwe Ġuse Ġdifferent Ġarguments Ġand Ġsupporting Ġevidence . ĠLastly , Ġwe Ġget Ġ4 . 8 % Ġimprovement Ġwhen Ġadding Ġcomment Ġinformation Ġand Ġit Ġachieves Ġcomparable Ġperformance Ġto ĠUTC NN Ġwithout Ġtopic Ġinformation , Ġwhich Ġshows Ġthat Ġcomments Ġalso Ġbenefit Ġperformance . ĠFor Ġplatforms Ġwhere Ġuser ĠIDs Ġare Ġpixel ated Ġor Ġotherwise Ġhidden , Ġadding Ġcomments Ġto Ġa Ġtext Ġmodel Ġstill Ġimproves Ġperformance . ĠIn Ġits Ġintegration Ġof Ġuser , Ġcontent , Ġand Ġcomment Ġinformation , Ġthe Ġfull ĠUTC NN Ġproduces Ġthe Ġhighest Ġf - sc ores Ġon Ġall ĠSup , ĠNe u , Ġand ĠUn s Ġstances Ġamong Ġmodels Ġthat Ġpredict Ġthe ĠUn s Ġclass , Ġand Ġthe Ġhighest Ġmacro - average Ġf - score Ġoverall . ĠThis Ġshows Ġits Ġability Ġto Ġbalance Ġa Ġbiased Ġdataset Ġand Ġsupports Ġour Ġclaim Ġthat ĠUTC NN Ġsuccessfully Ġbridges Ġcontent Ġand Ġuser , Ġtopic , Ġand Ġcomment Ġinformation Ġfor Ġstance Ġclassification Ġon Ġsocial Ġmedia Ġtext . ĠAnother Ġmerit Ġof ĠUTC NN Ġis Ġthat Ġit Ġdoes Ġnot Ġrequire Ġa Ġbalanced Ġtraining Ġdata . ĠThis Ġis Ġsupported Ġby Ġits Ġoutper forming Ġother Ġmodels Ġthough Ġno Ġovers am pling Ġtechnique Ġis Ġapplied Ġto Ġthe ĠUTC NN Ġrelated Ġexperiments Ġas Ġshown Ġin Ġthis Ġpaper . ĠThus Ġwe Ġcan Ġconclude Ġthat Ġthe Ġuser Ġinformation Ġprovides Ġstrong Ġclues Ġand Ġit Ġis Ġstill Ġrich Ġeven Ġin Ġthe Ġminority Ġclass . </s> We Ġalso Ġinvestigate Ġthe Ġsemantic Ġdifference Ġwhen Ġa Ġuser Ġacts Ġas Ġan Ġauthor / l iker Ġor Ġa Ġcommenter . ĠWe Ġevaluated Ġa Ġvariation Ġin Ġwhich Ġall Ġembed d ings Ġfrom Ġthe Ġsame Ġuser Ġwere Ġforced Ġto Ġbe Ġidentical Ġ( this Ġis Ġthe ĠUTC NN Ġshared Ġuser Ġembed ding Ġsetting Ġin ĠTable ĠT AB REF 22 Ġ). ĠThis Ġsetting Ġyielded Ġonly Ġa Ġ2 . 5 % Ġimprovement Ġover Ġthe Ġmodel Ġwithout Ġcomments , Ġwhich Ġis Ġnot Ġstatistically Ġsignificant . ĠHowever , Ġwhen Ġseparating Ġauthors / l ikers Ġand Ġcommenters Ġembed d ings Ġ( i . e ., Ġthe ĠUTC NN Ġfull Ġmodel ), Ġwe Ġachieved Ġmuch Ġgreater Ġimprovements Ġ( 4 . 8 %). ĠWe Ġattribute Ġthis Ġresult Ġto Ġthe Ġtendency Ġof Ġusers Ġto Ġuse Ġdifferent Ġwording Ġfor Ġdifferent Ġroles Ġ( for Ġinstance Ġauthor Ġvs Ġcommenter ). ĠThis Ġis Ġobserved Ġwhen Ġthe Ġuser , Ġacting Ġas Ġan Ġauthor , Ġattempts Ġto Ġsupport Ġher Ġargument Ġagainst Ġnuclear Ġpower Ġby Ġusing Ġimprovements Ġin Ġsolar Ġpower ; Ġwhen Ġacting Ġas Ġa Ġcommenter , Ġthough , Ġshe Ġinteracts Ġwith Ġpost Ġcontents Ġby Ġcriticizing Ġpast Ġpoliticians Ġwho Ġsupported Ġnuclear Ġpower Ġor Ġby Ġarguing Ġthat Ġthe Ġproposed Ġevacuation Ġplan Ġin Ġcase Ġof Ġa Ġnuclear Ġaccident Ġis Ġridiculous . ĠBased Ġon Ġthis Ġfinding , Ġin Ġthe Ġfinal ĠUTC NN Ġsetting Ġwe Ġtrain Ġtwo Ġuser Ġmatrix Ġembed d ings Ġfor Ġone Ġuser : Ġone Ġfor Ġthe Ġauthor / l iker Ġrole Ġand Ġthe Ġother Ġfor Ġthe Ġcommenter Ġrole . </s> Results Ġon ĠCreate Deb ate ĠDat as et </s> Table ĠT AB REF 24 Ġshows Ġthe Ġresults Ġof ĠUTC NN , Ġbas elines Ġas Ġwe Ġimplemented Ġon Ġthe ĠFB Fans Ġd ats et Ġand Ġrelated Ġwork Ġon Ġthe ĠCreate Deb ate Ġdataset . ĠWe Ġdo Ġnot Ġadopt Ġovers am pling Ġon Ġthese Ġmodels Ġbecause Ġthe ĠCreate Deb ate Ġdataset Ġis Ġalmost Ġbalanced . ĠIn Ġprevious Ġwork , Ġinteger Ġlinear Ġprogramming Ġ( IL P ) Ġor Ġlinear - chain Ġconditional Ġrandom Ġfields Ġ( CR Fs ) Ġwere Ġproposed Ġto Ġintegrate Ġtext Ġfeatures , Ġauthor , Ġideology , Ġand Ġuser - inter action Ġconstraints , Ġwhere Ġtext Ġfeatures Ġare Ġun ig ram , Ġbig ram , Ġand ĠPOS - depend encies ; Ġthe Ġauthor Ġconstraint Ġtends Ġto Ġrequire Ġthat Ġposts Ġfrom Ġthe Ġsame Ġauthor Ġfor Ġthe Ġsame Ġtopic Ġhold Ġthe Ġsame Ġstance ; Ġthe Ġideology Ġconstraint Ġaims Ġto Ġcapture Ġinf erences Ġbetween Ġtopics Ġfor Ġthe Ġsame Ġauthor ; Ġthe Ġuser - inter action Ġconstraint Ġmodels Ġrelationships Ġamong Ġposts Ġvia Ġuser Ġinteractions Ġsuch Ġas Ġreplies ĠB IB REF 7 Ġ, ĠB IB REF 9 Ġ. </s> The ĠS VM Ġwith Ġn - gram Ġor Ġaverage Ġword Ġembed ding Ġfeature Ġperforms Ġjust Ġsimilar Ġto Ġthe Ġmajority . ĠHowever , Ġwith Ġthe Ġtransformed Ġword Ġembed ding , Ġit Ġachieves Ġsuperior Ġresults . ĠIt Ġshows Ġthat Ġthe Ġlearned Ġuser Ġand Ġtopic Ġembed d ings Ġreally Ġcapture Ġthe Ġuser Ġand Ġtopic Ġsemantics . ĠThis Ġfinding Ġis Ġnot Ġso Ġobvious Ġin Ġthe ĠFB Fans Ġdataset Ġand Ġit Ġmight Ġbe Ġdue Ġto Ġthe Ġunfavorable Ġdata Ġske wn ess Ġfor ĠS VM . ĠAs Ġfor ĠCNN Ġand ĠRC NN , Ġthey Ġperform Ġslightly Ġbetter Ġthan Ġmost ĠSV Ms Ġas Ġwe Ġfound Ġin ĠTable ĠT AB REF 22 Ġfor ĠFB Fans . </s> Compared Ġto Ġthe ĠIL P ĠB IB REF 7 Ġand ĠCR F ĠB IB REF 9 Ġmethods , Ġthe ĠUTC NN Ġuser Ġembed d ings Ġencode Ġauthor Ġand Ġuser - inter action Ġconstraints , Ġwhere Ġthe Ġideology Ġconstraint Ġis Ġmodeled Ġby Ġthe Ġtopic Ġembed d ings Ġand Ġtext Ġfeatures Ġare Ġmodeled Ġby Ġthe ĠCNN . ĠThe Ġsignificant Ġimprovement Ġachieved Ġby ĠUTC NN Ġsuggests Ġthe Ġlatent Ġrepresentations Ġare Ġmore Ġeffective Ġthan Ġovert Ġmodel Ġconstraints . </s> The ĠPS L Ġmodel ĠB IB REF 12 Ġjointly Ġlabels Ġboth Ġauthor Ġand Ġpost Ġstance Ġusing Ġprob abil istic Ġsoft Ġlogic Ġ( PS L ) ĠB IB REF 23 Ġby Ġconsidering Ġtext Ġfeatures Ġand Ġreply Ġlinks Ġbetween Ġauthors Ġand Ġposts Ġas Ġin ĠHasan Ġand ĠNg 's Ġwork . ĠTable ĠT AB REF 24 Ġreports Ġthe Ġresult Ġof Ġtheir Ġbest ĠAD Ġsetting , Ġwhich Ġrepresents Ġthe Ġfull Ġjoint Ġstance / dis ag reement Ġcollective Ġmodel Ġon Ġposts Ġand Ġis Ġhence Ġmore Ġrelevant Ġto ĠUTC NN . ĠIn Ġcontrast Ġto Ġtheir Ġmodel , Ġthe ĠUTC NN Ġuser Ġembed d ings Ġrepresent Ġrelationships Ġbetween Ġauthors , Ġbut ĠUTC NN Ġmodels Ġdo Ġnot Ġutilize Ġlink Ġinformation Ġbetween Ġposts . ĠThough Ġthe ĠPS L Ġmodel Ġhas Ġthe Ġadvantage Ġof Ġbeing Ġable Ġto Ġjointly Ġlabel Ġthe Ġstances Ġof Ġauthors Ġand Ġposts , Ġits Ġperformance Ġon Ġposts Ġis Ġlower Ġthan Ġthe Ġthat Ġfor Ġthe ĠIL P Ġor ĠCR F Ġmodels . ĠUTC NN Ġsignificantly Ġoutper forms Ġthese Ġmodels Ġon Ġposts Ġand Ġhas Ġthe Ġpotential Ġto Ġpredict Ġuser Ġstances Ġthrough Ġthe Ġgenerated Ġuser Ġembed d ings . </s> For Ġthe ĠCreate Deb ate Ġdataset , Ġwe Ġalso Ġevaluated Ġperformance Ġwhen Ġnot Ġusing Ġtopic Ġembed d ings Ġor Ġuser Ġembed d ings ; Ġas Ġreplies Ġin Ġthis Ġdataset Ġare Ġviewed Ġas Ġposts , Ġthe Ġsetting Ġwithout Ġcomment Ġembed d ings Ġis Ġnot Ġavailable . ĠTable ĠT AB REF 24 Ġshows Ġthe Ġsame Ġfindings Ġas ĠTable ĠT AB REF 22 Ġ: Ġthe Ġ21 % Ġimprovement Ġin Ġaccuracy Ġdemonstrates Ġthat Ġuser Ġinformation Ġis Ġthe Ġmost Ġvital . ĠThis Ġfinding Ġalso Ġsupports Ġthe Ġresults Ġin Ġthe Ġrelated Ġwork : Ġuser Ġconstraints Ġare Ġuseful Ġand Ġcan Ġyield Ġ11 . 2 % Ġimprovement Ġin Ġaccuracy ĠB IB REF 7 Ġ. ĠFurther Ġconsidering Ġtopic Ġinformation Ġyields Ġ3 . 4 % Ġimprovement , Ġsuggesting Ġthat Ġknowing Ġthe Ġsubject Ġof Ġdebates Ġprovides Ġuseful Ġinformation . ĠIn Ġsum , ĠTable ĠT AB REF 22 Ġtogether Ġwith ĠTable ĠT AB REF 24 Ġshow Ġthat ĠUTC NN Ġachieves Ġpromising Ġperformance Ġregardless Ġof Ġtopic , Ġlanguage , Ġdata Ġdistribution , Ġand Ġplatform . </s> Conclusion </s> We Ġhave Ġproposed ĠUTC NN , Ġa Ġneural Ġnetwork Ġmodel Ġthat Ġincorporates Ġuser , Ġtopic , Ġcontent Ġand Ġcomment Ġinformation Ġfor Ġstance Ġclassification Ġon Ġsocial Ġmedia Ġtexts . ĠUTC NN Ġlearns Ġuser Ġembed d ings Ġfor Ġall Ġusers Ġwith Ġminimum Ġactive Ġdegree , Ġi . e ., Ġone Ġpost Ġor Ġone Ġlike . ĠTopic Ġinformation Ġobtained Ġfrom Ġthe Ġtopic Ġmodel Ġor Ġthe Ġpre - defined Ġlabels Ġfurther Ġimproves Ġthe ĠUTC NN Ġmodel . ĠIn Ġaddition , Ġcomment Ġinformation Ġprovides Ġadditional Ġclues Ġfor Ġstance Ġclassification . ĠWe Ġhave Ġshown Ġthat ĠUTC NN Ġachieves Ġpromising Ġand Ġbalanced Ġresults . ĠIn Ġthe Ġfuture Ġwe Ġplan Ġto Ġexplore Ġthe Ġeffectiveness Ġof Ġthe ĠUTC NN Ġuser Ġembed d ings Ġfor Ġauthor Ġstance Ġclassification . </s> Acknowled gements </s> Research Ġof Ġthis Ġpaper Ġwas Ġpartially Ġsupported Ġby ĠMinistry Ġof ĠScience Ġand ĠTechnology , ĠTaiwan , Ġunder Ġthe Ġcontract ĠM OST Ġ104 - 22 21 - E - 001 - 024 - MY 2 .
Introduction This work is licenced under a Creative Commons Attribution 4.0 International License. License details: http://creativecommons.org/licenses/by/4.0/ Deep neural networks have been widely used in text classification and have achieved promising results BIBREF0 , BIBREF1 , BIBREF2 . Most focus on content information and use models such as convolutional neural networks (CNN) BIBREF3 or recursive neural networks BIBREF4 . However, for user-generated posts on social media like Facebook or Twitter, there is more information that should not be ignored. On social media platforms, a user can act either as the author of a post or as a reader who expresses his or her comments about the post. In this paper, we classify posts taking into account post authorship, likes, topics, and comments. In particular, users and their “likes” hold strong potential for text mining. For example, given a set of posts that are related to a specific topic, a user's likes and dislikes provide clues for stance labeling. From a user point of view, users with positive attitudes toward the issue leave positive comments on the posts with praise or even just the post's content; from a post point of view, positive posts attract users who hold positive stances. We also investigate the influence of topics: different topics are associated with different stance labeling tendencies and word usage. For example we discuss women's rights and unwanted babies on the topic of abortion, but we criticize medicine usage or crime when on the topic of marijuana BIBREF5 . Even for posts on a specific topic like nuclear power, a variety of arguments are raised: green energy, radiation, air pollution, and so on. As for comments, we treat them as additional text information. The arguments in the comments and the commenters (the users who leave the comments) provide hints on the post's content and further facilitate stance classification. In this paper, we propose the user-topic-comment neural network (UTCNN), a deep learning model that utilizes user, topic, and comment information. We attempt to learn user and topic representations which encode user interactions and topic influences to further enhance text classification, and we also incorporate comment information. We evaluate this model on a post stance classification task on forum-style social media platforms. The contributions of this paper are as follows: 1. We propose UTCNN, a neural network for text in modern social media channels as well as legacy social media, forums, and message boards — anywhere that reveals users, their tastes, as well as their replies to posts. 2. When classifying social media post stances, we leverage users, including authors and likers. User embeddings can be generated even for users who have never posted anything. 3. We incorporate a topic model to automatically assign topics to each post in a single topic dataset. 4. We show that overall, the proposed method achieves the highest performance in all instances, and that all of the information extracted, whether users, topics, or comments, still has its contributions. Extra-Linguistic Features for Stance Classification In this paper we aim to use text as well as other features to see how they complement each other in a deep learning model. In the stance classification domain, previous work has showed that text features are limited, suggesting that adding extra-linguistic constraints could improve performance BIBREF6 , BIBREF7 , BIBREF8 . For example, Hasan and Ng as well as Thomas et al. require that posts written by the same author have the same stance BIBREF9 , BIBREF10 . The addition of this constraint yields accuracy improvements of 1–7% for some models and datasets. Hasan and Ng later added user-interaction constraints and ideology constraints BIBREF7 : the former models the relationship among posts in a sequence of replies and the latter models inter-topic relationships, e.g., users who oppose abortion could be conservative and thus are likely to oppose gay rights. For work focusing on online forum text, since posts are linked through user replies, sequential labeling methods have been used to model relationships between posts. For example, Hasan and Ng use hidden Markov models (HMMs) to model dependent relationships to the preceding post BIBREF9 ; Burfoot et al. use iterative classification to repeatedly generate new estimates based on the current state of knowledge BIBREF11 ; Sridhar et al. use probabilistic soft logic (PSL) to model reply links via collaborative filtering BIBREF12 . In the Facebook dataset we study, we use comments instead of reply links. However, as the ultimate goal in this paper is predicting not comment stance but post stance, we treat comments as extra information for use in predicting post stance. Deep Learning on Extra-Linguistic Features In recent years neural network models have been applied to document sentiment classification BIBREF13 , BIBREF4 , BIBREF14 , BIBREF15 , BIBREF2 . Text features can be used in deep networks to capture text semantics or sentiment. For example, Dong et al. use an adaptive layer in a recursive neural network for target-dependent Twitter sentiment analysis, where targets are topics such as windows 7 or taylor swift BIBREF16 , BIBREF17 ; recursive neural tensor networks (RNTNs) utilize sentence parse trees to capture sentence-level sentiment for movie reviews BIBREF4 ; Le and Mikolov predict sentiment by using paragraph vectors to model each paragraph as a continuous representation BIBREF18 . They show that performance can thus be improved by more delicate text models. Others have suggested using extra-linguistic features to improve the deep learning model. The user-word composition vector model (UWCVM) BIBREF19 is inspired by the possibility that the strength of sentiment words is user-specific; to capture this they add user embeddings in their model. In UPNN, a later extension, they further add a product-word composition as product embeddings, arguing that products can also show different tendencies of being rated or reviewed BIBREF20 . Their addition of user information yielded 2–10% improvements in accuracy as compared to the above-mentioned RNTN and paragraph vector methods. We also seek to inject user information into the neural network model. In comparison to the research of Tang et al. on sentiment classification for product reviews, the difference is two-fold. First, we take into account multiple users (one author and potentially many likers) for one post, whereas only one user (the reviewer) is involved in a review. Second, we add comment information to provide more features for post stance classification. None of these two factors have been considered previously in a deep learning model for text stance classification. Therefore, we propose UTCNN, which generates and utilizes user embeddings for all users — even for those who have not authored any posts — and incorporates comments to further improve performance. Method In this section, we first describe CNN-based document composition, which captures user- and topic-dependent document-level semantic representation from word representations. Then we show how to add comment information to construct the user-topic-comment neural network (UTCNN). User- and Topic-dependent Document Composition As shown in Figure FIGREF4 , we use a general CNN BIBREF3 and two semantic transformations for document composition . We are given a document with an engaged user INLINEFORM0 , a topic INLINEFORM1 , and its composite INLINEFORM2 words, each word INLINEFORM3 of which is associated with a word embedding INLINEFORM4 where INLINEFORM5 is the vector dimension. For each word embedding INLINEFORM6 , we apply two dot operations as shown in Equation EQREF6 : DISPLAYFORM0 where INLINEFORM0 models the user reading preference for certain semantics, and INLINEFORM1 models the topic semantics; INLINEFORM2 and INLINEFORM3 are the dimensions of transformed user and topic embeddings respectively. We use INLINEFORM4 to model semantically what each user prefers to read and/or write, and use INLINEFORM5 to model the semantics of each topic. The dot operation of INLINEFORM6 and INLINEFORM7 transforms the global representation INLINEFORM8 to a user-dependent representation. Likewise, the dot operation of INLINEFORM9 and INLINEFORM10 transforms INLINEFORM11 to a topic-dependent representation. After the two dot operations on INLINEFORM0 , we have user-dependent and topic-dependent word vectors INLINEFORM1 and INLINEFORM2 , which are concatenated to form a user- and topic-dependent word vector INLINEFORM3 . Then the transformed word embeddings INLINEFORM4 are used as the CNN input. Here we apply three convolutional layers on the concatenated transformed word embeddings INLINEFORM5 : DISPLAYFORM0 where INLINEFORM0 is the index of words; INLINEFORM1 is a non-linear activation function (we use INLINEFORM2 ); INLINEFORM5 is the convolutional filter with input length INLINEFORM6 and output length INLINEFORM7 , where INLINEFORM8 is the window size of the convolutional operation; and INLINEFORM9 and INLINEFORM10 are the output and bias of the convolution layer INLINEFORM11 , respectively. In our experiments, the three window sizes INLINEFORM12 in the three convolution layers are one, two, and three, encoding unigram, bigram, and trigram semantics accordingly. After the convolutional layer, we add a maximum pooling layer among convolutional outputs to obtain the unigram, bigram, and trigram n-gram representations. This is succeeded by an average pooling layer for an element-wise average of the three maximized convolution outputs. UTCNN Model Description Figure FIGREF10 illustrates the UTCNN model. As more than one user may interact with a given post, we first add a maximum pooling layer after the user matrix embedding layer and user vector embedding layer to form a moderator matrix embedding INLINEFORM0 and a moderator vector embedding INLINEFORM1 for moderator INLINEFORM2 respectively, where INLINEFORM3 is used for the semantic transformation in the document composition process, as mentioned in the previous section. The term moderator here is to denote the pseudo user who provides the overall semantic/sentiment of all the engaged users for one document. The embedding INLINEFORM4 models the moderator stance preference, that is, the pattern of the revealed user stance: whether a user is willing to show his preference, whether a user likes to show impartiality with neutral statements and reasonable arguments, or just wants to show strong support for one stance. Ideally, the latent user stance is modeled by INLINEFORM5 for each user. Likewise, for topic information, a maximum pooling layer is added after the topic matrix embedding layer and topic vector embedding layer to form a joint topic matrix embedding INLINEFORM6 and a joint topic vector embedding INLINEFORM7 for topic INLINEFORM8 respectively, where INLINEFORM9 models the semantic transformation of topic INLINEFORM10 as in users and INLINEFORM11 models the topic stance tendency. The latent topic stance is also modeled by INLINEFORM12 for each topic. As for comments, we view them as short documents with authors only but without likers nor their own comments. Therefore we apply document composition on comments although here users are commenters (users who comment). It is noticed that the word embeddings INLINEFORM0 for the same word in the posts and comments are the same, but after being transformed to INLINEFORM1 in the document composition process shown in Figure FIGREF4 , they might become different because of their different engaged users. The output comment representation together with the commenter vector embedding INLINEFORM2 and topic vector embedding INLINEFORM3 are concatenated and a maximum pooling layer is added to select the most important feature for comments. Instead of requiring that the comment stance agree with the post, UTCNN simply extracts the most important features of the comment contents; they could be helpful, whether they show obvious agreement or disagreement. Therefore when combining comment information here, the maximum pooling layer is more appropriate than other pooling or merging layers. Indeed, we believe this is one reason for UTCNN's performance gains. Finally, the pooled comment representation, together with user vector embedding INLINEFORM0 , topic vector embedding INLINEFORM1 , and document representation are fed to a fully connected network, and softmax is applied to yield the final stance label prediction for the post. Experiment We start with the experimental dataset and then describe the training process as well as the implementation of the baselines. We also implement several variations to reveal the effects of features: authors, likers, comment, and commenters. In the results section we compare our model with related work. Dataset We tested the proposed UTCNN on two different datasets: FBFans and CreateDebate. FBFans is a privately-owned, single-topic, Chinese, unbalanced, social media dataset, and CreateDebate is a public, multiple-topic, English, balanced, forum dataset. Results using these two datasets show the applicability and superiority for different topics, languages, data distributions, and platforms. The FBFans dataset contains data from anti-nuclear-power Chinese Facebook fan groups from September 2013 to August 2014, including posts and their author and liker IDs. There are a total of 2,496 authors, 505,137 likers, 33,686 commenters, and 505,412 unique users. Two annotators were asked to take into account only the post content to label the stance of the posts in the whole dataset as supportive, neutral, or unsupportive (hereafter denoted as Sup, Neu, and Uns). Sup/Uns posts were those in support of or against anti-reconstruction; Neu posts were those evincing a neutral standpoint on the topic, or were irrelevant. Raw agreement between annotators is 0.91, indicating high agreement. Specifically, Cohen’s Kappa for Neu and not Neu labeling is 0.58 (moderate), and for Sup or Uns labeling is 0.84 (almost perfect). Posts with inconsistent labels were filtered out, and the development and testing sets were randomly selected from what was left. Posts in the development and testing sets involved at least one user who appeared in the training set. The number of posts for each stance is shown on the left-hand side of Table TABREF12 . About twenty percent of the posts were labeled with a stance, and the number of supportive (Sup) posts was much larger than that of the unsupportive (Uns) ones: this is thus highly skewed data, which complicates stance classification. On average, 161.1 users were involved in one post. The maximum was 23,297 and the minimum was one (the author). For comments, on average there were 3 comments per post. The maximum was 1,092 and the minimum was zero. To test whether the assumption of this paper – posts attract users who hold the same stance to like them – is reliable, we examine the likes from authors of different stances. Posts in FBFans dataset are used for this analysis. We calculate the like statistics of each distinct author from these 32,595 posts. As the numbers of authors in the Sup, Neu and Uns stances are largely imbalanced, these numbers are normalized by the number of users of each stance. Table TABREF13 shows the results. Posts with stances (i.e., not neutral) attract users of the same stance. Neutral posts also attract both supportive and neutral users, like what we observe in supportive posts, but just the neutral posts can attract even more neutral likers. These results do suggest that users prefer posts of the same stance, or at least posts of no obvious stance which might cause annoyance when reading, and hence support the user modeling in our approach. The CreateDebate dataset was collected from an English online debate forum discussing four topics: abortion (ABO), gay rights (GAY), Obama (OBA), and marijuana (MAR). The posts are annotated as for (F) and against (A). Replies to posts in this dataset are also labeled with stance and hence use the same data format as posts. The labeling results are shown in the right-hand side of Table TABREF12 . We observe that the dataset is more balanced than the FBFans dataset. In addition, there are 977 unique users in the dataset. To compare with Hasan and Ng's work, we conducted five-fold cross-validation and present the annotation results as the average number of all folds BIBREF9 , BIBREF5 . The FBFans dataset has more integrated functions than the CreateDebate dataset; thus our model can utilize all linguistic and extra-linguistic features. For the CreateDebate dataset, on the other hand, the like and comment features are not available (as there is a stance label for each reply, replies are evaluated as posts as other previous work) but we still implemented our model using the content, author, and topic information. Settings In the UTCNN training process, cross-entropy was used as the loss function and AdaGrad as the optimizer. For FBFans dataset, we learned the 50-dimensional word embeddings on the whole dataset using GloVe BIBREF21 to capture the word semantics; for CreateDebate dataset we used the publicly available English 50-dimensional word embeddings, pre-trained also using GloVe. These word embeddings were fixed in the training process. The learning rate was set to 0.03. All user and topic embeddings were randomly initialized in the range of [-0.1 0.1]. Matrix embeddings for users and topics were sized at 250 ( INLINEFORM0 ); vector embeddings for users and topics were set to length 10. We applied the LDA topic model BIBREF22 on the FBFans dataset to determine the latent topics with which to build topic embeddings, as there is only one general known topic: nuclear power plants. We learned 100 latent topics and assigned the top three topics for each post. For the CreateDebate dataset, which itself constitutes four topics, the topic labels for posts were used directly without additionally applying LDA. For the FBFans data we report class-based f-scores as well as the macro-average f-score ( INLINEFORM0 ) shown in equation EQREF19 . DISPLAYFORM0 where INLINEFORM0 and INLINEFORM1 are the average precision and recall of the three class. We adopted the macro-average f-score as the evaluation metric for the overall performance because (1) the experimental dataset is severely imbalanced, which is common for contentious issues; and (2) for stance classification, content in minor-class posts is usually more important for further applications. For the CreateDebate dataset, accuracy was adopted as the evaluation metric to compare the results with related work BIBREF7 , BIBREF9 , BIBREF12 . Baselines We pit our model against the following baselines: 1) SVM with unigram, bigram, and trigram features, which is a standard yet rather strong classifier for text features; 2) SVM with average word embedding, where a document is represented as a continuous representation by averaging the embeddings of the composite words; 3) SVM with average transformed word embeddings (the INLINEFORM0 in equation EQREF6 ), where a document is represented as a continuous representation by averaging the transformed embeddings of the composite words; 4) two mature deep learning models on text classification, CNN BIBREF3 and Recurrent Convolutional Neural Networks (RCNN) BIBREF0 , where the hyperparameters are based on their work; 5) the above SVM and deep learning models with comment information; 6) UTCNN without user information, representing a pure-text CNN model where we use the same user matrix and user embeddings INLINEFORM1 and INLINEFORM2 for each user; 7) UTCNN without the LDA model, representing how UTCNN works with a single-topic dataset; 8) UTCNN without comments, in which the model predicts the stance label given only user and topic information. All these models were trained on the training set, and parameters as well as the SVM kernel selections (linear or RBF) were fine-tuned on the development set. Also, we adopt oversampling on SVMs, CNN and RCNN because the FBFans dataset is highly imbalanced. Results on FBFans Dataset In Table TABREF22 we show the results of UTCNN and the baselines on the FBFans dataset. Here Majority yields good performance on Neu since FBFans is highly biased to the neutral class. The SVM models perform well on Sup and Neu but perform poorly for Uns, showing that content information in itself is insufficient to predict stance labels, especially for the minor class. With the transformed word embedding feature, SVM can achieve comparable performance as SVM with n-gram feature. However, the much fewer feature dimension of the transformed word embedding makes SVM with word embeddings a more efficient choice for modeling the large scale social media dataset. For the CNN and RCNN models, they perform slightly better than most of the SVM models but still, the content information is insufficient to achieve a good performance on the Uns posts. As to adding comment information to these models, since the commenters do not always hold the same stance as the author, simply adding comments and post contents together merely adds noise to the model. Among all UTCNN variations, we find that user information is most important, followed by topic and comment information. UTCNN without user information shows results similar to SVMs — it does well for Sup and Neu but detects no Uns. Its best f-scores on both Sup and Neu among all methods show that with enough training data, content-based models can perform well; at the same time, the lack of user information results in too few clues for minor-class posts to either predict their stance directly or link them to other users and posts for improved performance. The 17.5% improvement when adding user information suggests that user information is especially useful when the dataset is highly imbalanced. All models that consider user information predict the minority class successfully. UCTNN without topic information works well but achieves lower performance than the full UTCNN model. The 4.9% performance gain brought by LDA shows that although it is satisfactory for single topic datasets, adding that latent topics still benefits performance: even when we are discussing the same topic, we use different arguments and supporting evidence. Lastly, we get 4.8% improvement when adding comment information and it achieves comparable performance to UTCNN without topic information, which shows that comments also benefit performance. For platforms where user IDs are pixelated or otherwise hidden, adding comments to a text model still improves performance. In its integration of user, content, and comment information, the full UTCNN produces the highest f-scores on all Sup, Neu, and Uns stances among models that predict the Uns class, and the highest macro-average f-score overall. This shows its ability to balance a biased dataset and supports our claim that UTCNN successfully bridges content and user, topic, and comment information for stance classification on social media text. Another merit of UTCNN is that it does not require a balanced training data. This is supported by its outperforming other models though no oversampling technique is applied to the UTCNN related experiments as shown in this paper. Thus we can conclude that the user information provides strong clues and it is still rich even in the minority class. We also investigate the semantic difference when a user acts as an author/liker or a commenter. We evaluated a variation in which all embeddings from the same user were forced to be identical (this is the UTCNN shared user embedding setting in Table TABREF22 ). This setting yielded only a 2.5% improvement over the model without comments, which is not statistically significant. However, when separating authors/likers and commenters embeddings (i.e., the UTCNN full model), we achieved much greater improvements (4.8%). We attribute this result to the tendency of users to use different wording for different roles (for instance author vs commenter). This is observed when the user, acting as an author, attempts to support her argument against nuclear power by using improvements in solar power; when acting as a commenter, though, she interacts with post contents by criticizing past politicians who supported nuclear power or by arguing that the proposed evacuation plan in case of a nuclear accident is ridiculous. Based on this finding, in the final UTCNN setting we train two user matrix embeddings for one user: one for the author/liker role and the other for the commenter role. Results on CreateDebate Dataset Table TABREF24 shows the results of UTCNN, baselines as we implemented on the FBFans datset and related work on the CreateDebate dataset. We do not adopt oversampling on these models because the CreateDebate dataset is almost balanced. In previous work, integer linear programming (ILP) or linear-chain conditional random fields (CRFs) were proposed to integrate text features, author, ideology, and user-interaction constraints, where text features are unigram, bigram, and POS-dependencies; the author constraint tends to require that posts from the same author for the same topic hold the same stance; the ideology constraint aims to capture inferences between topics for the same author; the user-interaction constraint models relationships among posts via user interactions such as replies BIBREF7 , BIBREF9 . The SVM with n-gram or average word embedding feature performs just similar to the majority. However, with the transformed word embedding, it achieves superior results. It shows that the learned user and topic embeddings really capture the user and topic semantics. This finding is not so obvious in the FBFans dataset and it might be due to the unfavorable data skewness for SVM. As for CNN and RCNN, they perform slightly better than most SVMs as we found in Table TABREF22 for FBFans. Compared to the ILP BIBREF7 and CRF BIBREF9 methods, the UTCNN user embeddings encode author and user-interaction constraints, where the ideology constraint is modeled by the topic embeddings and text features are modeled by the CNN. The significant improvement achieved by UTCNN suggests the latent representations are more effective than overt model constraints. The PSL model BIBREF12 jointly labels both author and post stance using probabilistic soft logic (PSL) BIBREF23 by considering text features and reply links between authors and posts as in Hasan and Ng's work. Table TABREF24 reports the result of their best AD setting, which represents the full joint stance/disagreement collective model on posts and is hence more relevant to UTCNN. In contrast to their model, the UTCNN user embeddings represent relationships between authors, but UTCNN models do not utilize link information between posts. Though the PSL model has the advantage of being able to jointly label the stances of authors and posts, its performance on posts is lower than the that for the ILP or CRF models. UTCNN significantly outperforms these models on posts and has the potential to predict user stances through the generated user embeddings. For the CreateDebate dataset, we also evaluated performance when not using topic embeddings or user embeddings; as replies in this dataset are viewed as posts, the setting without comment embeddings is not available. Table TABREF24 shows the same findings as Table TABREF22 : the 21% improvement in accuracy demonstrates that user information is the most vital. This finding also supports the results in the related work: user constraints are useful and can yield 11.2% improvement in accuracy BIBREF7 . Further considering topic information yields 3.4% improvement, suggesting that knowing the subject of debates provides useful information. In sum, Table TABREF22 together with Table TABREF24 show that UTCNN achieves promising performance regardless of topic, language, data distribution, and platform. Conclusion We have proposed UTCNN, a neural network model that incorporates user, topic, content and comment information for stance classification on social media texts. UTCNN learns user embeddings for all users with minimum active degree, i.e., one post or one like. Topic information obtained from the topic model or the pre-defined labels further improves the UTCNN model. In addition, comment information provides additional clues for stance classification. We have shown that UTCNN achieves promising and balanced results. In the future we plan to explore the effectiveness of the UTCNN user embeddings for author stance classification. Acknowledgements Research of this paper was partially supported by Ministry of Science and Technology, Taiwan, under the contract MOST 104-2221-E-001-024-MY2.
Did they collected the two datasets?
e330e162ec29722f5ec9f83853d129c9e0693d65
Did Ġthey Ġcollected Ġthe Ġtwo Ġdatasets ?
No
<s> Did Ġthey Ġcollected Ġthe Ġtwo Ġdatasets ? </s> Introduction </s> This Ġwork Ġis Ġlic enced Ġunder Ġa ĠCreative ĠCommons ĠAttribution Ġ4 . 0 ĠInternational ĠLicense . ĠLicense Ġdetails : Ġhttp :// creat ive comm ons . org / lic enses / by / 4 . 0 / ĠDeep Ġneural Ġnetworks Ġhave Ġbeen Ġwidely Ġused Ġin Ġtext Ġclassification Ġand Ġhave Ġachieved Ġpromising Ġresults ĠB IB REF 0 Ġ, ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ. ĠMost Ġfocus Ġon Ġcontent Ġinformation Ġand Ġuse Ġmodels Ġsuch Ġas Ġconv olution al Ġneural Ġnetworks Ġ( CNN ) ĠB IB REF 3 Ġor Ġrecursive Ġneural Ġnetworks ĠB IB REF 4 Ġ. ĠHowever , Ġfor Ġuser - generated Ġposts Ġon Ġsocial Ġmedia Ġlike ĠFacebook Ġor ĠTwitter , Ġthere Ġis Ġmore Ġinformation Ġthat Ġshould Ġnot Ġbe Ġignored . ĠOn Ġsocial Ġmedia Ġplatforms , Ġa Ġuser Ġcan Ġact Ġeither Ġas Ġthe Ġauthor Ġof Ġa Ġpost Ġor Ġas Ġa Ġreader Ġwho Ġexpresses Ġhis Ġor Ġher Ġcomments Ġabout Ġthe Ġpost . </s> In Ġthis Ġpaper , Ġwe Ġclassify Ġposts Ġtaking Ġinto Ġaccount Ġpost Ġauthors hip , Ġlikes , Ġtopics , Ġand Ġcomments . ĠIn Ġparticular , Ġusers Ġand Ġtheir ĠâĢ ľ l ikes âĢ Ŀ Ġhold Ġstrong Ġpotential Ġfor Ġtext Ġmining . ĠFor Ġexample , Ġgiven Ġa Ġset Ġof Ġposts Ġthat Ġare Ġrelated Ġto Ġa Ġspecific Ġtopic , Ġa Ġuser 's Ġlikes Ġand Ġdisl ikes Ġprovide Ġclues Ġfor Ġstance Ġlabeling . ĠFrom Ġa Ġuser Ġpoint Ġof Ġview , Ġusers Ġwith Ġpositive Ġattitudes Ġtoward Ġthe Ġissue Ġleave Ġpositive Ġcomments Ġon Ġthe Ġposts Ġwith Ġpraise Ġor Ġeven Ġjust Ġthe Ġpost 's Ġcontent ; Ġfrom Ġa Ġpost Ġpoint Ġof Ġview , Ġpositive Ġposts Ġattract Ġusers Ġwho Ġhold Ġpositive Ġstances . ĠWe Ġalso Ġinvestigate Ġthe Ġinfluence Ġof Ġtopics : Ġdifferent Ġtopics Ġare Ġassociated Ġwith Ġdifferent Ġstance Ġlabeling Ġtendencies Ġand Ġword Ġusage . ĠFor Ġexample Ġwe Ġdiscuss Ġwomen 's Ġrights Ġand Ġunwanted Ġbabies Ġon Ġthe Ġtopic Ġof Ġabortion , Ġbut Ġwe Ġcriticize Ġmedicine Ġusage Ġor Ġcrime Ġwhen Ġon Ġthe Ġtopic Ġof Ġmarijuana ĠB IB REF 5 Ġ. ĠEven Ġfor Ġposts Ġon Ġa Ġspecific Ġtopic Ġlike Ġnuclear Ġpower , Ġa Ġvariety Ġof Ġarguments Ġare Ġraised : Ġgreen Ġenergy , Ġradiation , Ġair Ġpollution , Ġand Ġso Ġon . ĠAs Ġfor Ġcomments , Ġwe Ġtreat Ġthem Ġas Ġadditional Ġtext Ġinformation . ĠThe Ġarguments Ġin Ġthe Ġcomments Ġand Ġthe Ġcommenters Ġ( the Ġusers Ġwho Ġleave Ġthe Ġcomments ) Ġprovide Ġhints Ġon Ġthe Ġpost 's Ġcontent Ġand Ġfurther Ġfacilitate Ġstance Ġclassification . </s> In Ġthis Ġpaper , Ġwe Ġpropose Ġthe Ġuser - topic - comment Ġneural Ġnetwork Ġ( UTC NN ), Ġa Ġdeep Ġlearning Ġmodel Ġthat Ġutilizes Ġuser , Ġtopic , Ġand Ġcomment Ġinformation . ĠWe Ġattempt Ġto Ġlearn Ġuser Ġand Ġtopic Ġrepresentations Ġwhich Ġencode Ġuser Ġinteractions Ġand Ġtopic Ġinfluences Ġto Ġfurther Ġenhance Ġtext Ġclassification , Ġand Ġwe Ġalso Ġincorporate Ġcomment Ġinformation . ĠWe Ġevaluate Ġthis Ġmodel Ġon Ġa Ġpost Ġstance Ġclassification Ġtask Ġon Ġforum - style Ġsocial Ġmedia Ġplatforms . ĠThe Ġcontributions Ġof Ġthis Ġpaper Ġare Ġas Ġfollows : Ġ1 . ĠWe Ġpropose ĠUTC NN , Ġa Ġneural Ġnetwork Ġfor Ġtext Ġin Ġmodern Ġsocial Ġmedia Ġchannels Ġas Ġwell Ġas Ġlegacy Ġsocial Ġmedia , Ġforums , Ġand Ġmessage Ġboards ĠâĢĶ Ġanywhere Ġthat Ġreveals Ġusers , Ġtheir Ġtastes , Ġas Ġwell Ġas Ġtheir Ġreplies Ġto Ġposts . Ġ2 . ĠWhen Ġclass ifying Ġsocial Ġmedia Ġpost Ġstances , Ġwe Ġleverage Ġusers , Ġincluding Ġauthors Ġand Ġlik ers . ĠUser Ġembed d ings Ġcan Ġbe Ġgenerated Ġeven Ġfor Ġusers Ġwho Ġhave Ġnever Ġposted Ġanything . Ġ3 . ĠWe Ġincorporate Ġa Ġtopic Ġmodel Ġto Ġautomatically Ġassign Ġtopics Ġto Ġeach Ġpost Ġin Ġa Ġsingle Ġtopic Ġdataset . Ġ4 . ĠWe Ġshow Ġthat Ġoverall , Ġthe Ġproposed Ġmethod Ġachieves Ġthe Ġhighest Ġperformance Ġin Ġall Ġinstances , Ġand Ġthat Ġall Ġof Ġthe Ġinformation Ġextracted , Ġwhether Ġusers , Ġtopics , Ġor Ġcomments , Ġstill Ġhas Ġits Ġcontributions . </s> Extra - L ingu istic ĠFeatures Ġfor ĠSt ance ĠClassification </s> In Ġthis Ġpaper Ġwe Ġaim Ġto Ġuse Ġtext Ġas Ġwell Ġas Ġother Ġfeatures Ġto Ġsee Ġhow Ġthey Ġcomplement Ġeach Ġother Ġin Ġa Ġdeep Ġlearning Ġmodel . ĠIn Ġthe Ġstance Ġclassification Ġdomain , Ġprevious Ġwork Ġhas Ġshowed Ġthat Ġtext Ġfeatures Ġare Ġlimited , Ġsuggesting Ġthat Ġadding Ġextra - ling u istic Ġconstraints Ġcould Ġimprove Ġperformance ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ, ĠB IB REF 8 Ġ. ĠFor Ġexample , ĠHasan Ġand ĠNg Ġas Ġwell Ġas ĠThomas Ġet Ġal . Ġrequire Ġthat Ġposts Ġwritten Ġby Ġthe Ġsame Ġauthor Ġhave Ġthe Ġsame Ġstance ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ. ĠThe Ġaddition Ġof Ġthis Ġconstraint Ġyields Ġaccuracy Ġimprovements Ġof Ġ1 âĢĵ 7 % Ġfor Ġsome Ġmodels Ġand Ġdatasets . ĠHasan Ġand ĠNg Ġlater Ġadded Ġuser - inter action Ġconstraints Ġand Ġideology Ġconstraints ĠB IB REF 7 Ġ: Ġthe Ġformer Ġmodels Ġthe Ġrelationship Ġamong Ġposts Ġin Ġa Ġsequence Ġof Ġreplies Ġand Ġthe Ġlatter Ġmodels Ġinter - topic Ġrelationships , Ġe . g ., Ġusers Ġwho Ġoppose Ġabortion Ġcould Ġbe Ġconservative Ġand Ġthus Ġare Ġlikely Ġto Ġoppose Ġgay Ġrights . </s> For Ġwork Ġfocusing Ġon Ġonline Ġforum Ġtext , Ġsince Ġposts Ġare Ġlinked Ġthrough Ġuser Ġreplies , Ġsequential Ġlabeling Ġmethods Ġhave Ġbeen Ġused Ġto Ġmodel Ġrelationships Ġbetween Ġposts . ĠFor Ġexample , ĠHasan Ġand ĠNg Ġuse Ġhidden ĠMark ov Ġmodels Ġ( HM Ms ) Ġto Ġmodel Ġdependent Ġrelationships Ġto Ġthe Ġpreceding Ġpost ĠB IB REF 9 Ġ; ĠBur foot Ġet Ġal . Ġuse Ġiter ative Ġclassification Ġto Ġrepeatedly Ġgenerate Ġnew Ġestimates Ġbased Ġon Ġthe Ġcurrent Ġstate Ġof Ġknowledge ĠB IB REF 11 Ġ; ĠS rid har Ġet Ġal . Ġuse Ġprob abil istic Ġsoft Ġlogic Ġ( PS L ) Ġto Ġmodel Ġreply Ġlinks Ġvia Ġcollaborative Ġfiltering ĠB IB REF 12 Ġ. ĠIn Ġthe ĠFacebook Ġdataset Ġwe Ġstudy , Ġwe Ġuse Ġcomments Ġinstead Ġof Ġreply Ġlinks . ĠHowever , Ġas Ġthe Ġultimate Ġgoal Ġin Ġthis Ġpaper Ġis Ġpredicting Ġnot Ġcomment Ġstance Ġbut Ġpost Ġstance , Ġwe Ġtreat Ġcomments Ġas Ġextra Ġinformation Ġfor Ġuse Ġin Ġpredicting Ġpost Ġstance . </s> Deep ĠLearning Ġon ĠExtra - L ingu istic ĠFeatures </s> In Ġrecent Ġyears Ġneural Ġnetwork Ġmodels Ġhave Ġbeen Ġapplied Ġto Ġdocument Ġsentiment Ġclassification ĠB IB REF 13 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 2 Ġ. ĠText Ġfeatures Ġcan Ġbe Ġused Ġin Ġdeep Ġnetworks Ġto Ġcapture Ġtext Ġsemantics Ġor Ġsentiment . ĠFor Ġexample , ĠDong Ġet Ġal . Ġuse Ġan Ġadaptive Ġlayer Ġin Ġa Ġrecursive Ġneural Ġnetwork Ġfor Ġtarget - dependent ĠTwitter Ġsentiment Ġanalysis , Ġwhere Ġtargets Ġare Ġtopics Ġsuch Ġas Ġwindows Ġ7 Ġor Ġt aylor Ġswift ĠB IB REF 16 Ġ, ĠB IB REF 17 Ġ; Ġrecursive Ġneural Ġtens or Ġnetworks Ġ( R NT Ns ) Ġutilize Ġsentence Ġparse Ġtrees Ġto Ġcapture Ġsentence - level Ġsentiment Ġfor Ġmovie Ġreviews ĠB IB REF 4 Ġ; ĠLe Ġand ĠMik ol ov Ġpredict Ġsentiment Ġby Ġusing Ġparagraph Ġvectors Ġto Ġmodel Ġeach Ġparagraph Ġas Ġa Ġcontinuous Ġrepresentation ĠB IB REF 18 Ġ. ĠThey Ġshow Ġthat Ġperformance Ġcan Ġthus Ġbe Ġimproved Ġby Ġmore Ġdelicate Ġtext Ġmodels . </s> Others Ġhave Ġsuggested Ġusing Ġextra - ling u istic Ġfeatures Ġto Ġimprove Ġthe Ġdeep Ġlearning Ġmodel . ĠThe Ġuser - word Ġcomposition Ġvector Ġmodel Ġ( U WC VM ) ĠB IB REF 19 Ġis Ġinspired Ġby Ġthe Ġpossibility Ġthat Ġthe Ġstrength Ġof Ġsentiment Ġwords Ġis Ġuser - specific ; Ġto Ġcapture Ġthis Ġthey Ġadd Ġuser Ġembed d ings Ġin Ġtheir Ġmodel . ĠIn ĠUP NN , Ġa Ġlater Ġextension , Ġthey Ġfurther Ġadd Ġa Ġproduct - word Ġcomposition Ġas Ġproduct Ġembed d ings , Ġarguing Ġthat Ġproducts Ġcan Ġalso Ġshow Ġdifferent Ġtendencies Ġof Ġbeing Ġrated Ġor Ġreviewed ĠB IB REF 20 Ġ. ĠTheir Ġaddition Ġof Ġuser Ġinformation Ġyielded Ġ2 âĢĵ 10 % Ġimprovements Ġin Ġaccuracy Ġas Ġcompared Ġto Ġthe Ġabove - mentioned ĠR NT N Ġand Ġparagraph Ġvector Ġmethods . ĠWe Ġalso Ġseek Ġto Ġinject Ġuser Ġinformation Ġinto Ġthe Ġneural Ġnetwork Ġmodel . ĠIn Ġcomparison Ġto Ġthe Ġresearch Ġof ĠTang Ġet Ġal . Ġon Ġsentiment Ġclassification Ġfor Ġproduct Ġreviews , Ġthe Ġdifference Ġis Ġtwo - fold . ĠFirst , Ġwe Ġtake Ġinto Ġaccount Ġmultiple Ġusers Ġ( one Ġauthor Ġand Ġpotentially Ġmany Ġlik ers ) Ġfor Ġone Ġpost , Ġwhereas Ġonly Ġone Ġuser Ġ( the Ġreviewer ) Ġis Ġinvolved Ġin Ġa Ġreview . ĠSecond , Ġwe Ġadd Ġcomment Ġinformation Ġto Ġprovide Ġmore Ġfeatures Ġfor Ġpost Ġstance Ġclassification . ĠNone Ġof Ġthese Ġtwo Ġfactors Ġhave Ġbeen Ġconsidered Ġpreviously Ġin Ġa Ġdeep Ġlearning Ġmodel Ġfor Ġtext Ġstance Ġclassification . ĠTherefore , Ġwe Ġpropose ĠUTC NN , Ġwhich Ġgenerates Ġand Ġutilizes Ġuser Ġembed d ings Ġfor Ġall Ġusers ĠâĢĶ Ġeven Ġfor Ġthose Ġwho Ġhave Ġnot Ġauthored Ġany Ġposts ĠâĢĶ Ġand Ġincorporates Ġcomments Ġto Ġfurther Ġimprove Ġperformance . </s> Method </s> In Ġthis Ġsection , Ġwe Ġfirst Ġdescribe ĠCNN - based Ġdocument Ġcomposition , Ġwhich Ġcaptures Ġuser - Ġand Ġtopic - dependent Ġdocument - level Ġsemantic Ġrepresentation Ġfrom Ġword Ġrepresentations . ĠThen Ġwe Ġshow Ġhow Ġto Ġadd Ġcomment Ġinformation Ġto Ġconstruct Ġthe Ġuser - topic - comment Ġneural Ġnetwork Ġ( UTC NN ). </s> User - Ġand ĠTopic - dependent ĠDocument ĠCom position </s> As Ġshown Ġin ĠFigure ĠFIG REF 4 Ġ, Ġwe Ġuse Ġa Ġgeneral ĠCNN ĠB IB REF 3 Ġand Ġtwo Ġsemantic Ġtransformations Ġfor Ġdocument Ġcomposition Ġ. ĠWe Ġare Ġgiven Ġa Ġdocument Ġwith Ġan Ġengaged Ġuser ĠIN LINE FORM 0 Ġ, Ġa Ġtopic ĠIN LINE FORM 1 Ġ, Ġand Ġits Ġcomposite ĠIN LINE FORM 2 Ġwords , Ġeach Ġword ĠIN LINE FORM 3 Ġof Ġwhich Ġis Ġassociated Ġwith Ġa Ġword Ġembed ding ĠIN LINE FORM 4 Ġwhere ĠIN LINE FORM 5 Ġis Ġthe Ġvector Ġdimension . ĠFor Ġeach Ġword Ġembed ding ĠIN LINE FORM 6 Ġ, Ġwe Ġapply Ġtwo Ġdot Ġoperations Ġas Ġshown Ġin ĠEqu ation ĠEQ REF 6 Ġ: ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġmodels Ġthe Ġuser Ġreading Ġpreference Ġfor Ġcertain Ġsemantics , Ġand ĠIN LINE FORM 1 Ġmodels Ġthe Ġtopic Ġsemantics ; ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġare Ġthe Ġdimensions Ġof Ġtransformed Ġuser Ġand Ġtopic Ġembed d ings Ġrespectively . ĠWe Ġuse ĠIN LINE FORM 4 Ġto Ġmodel Ġsem antically Ġwhat Ġeach Ġuser Ġprefers Ġto Ġread Ġand / or Ġwrite , Ġand Ġuse ĠIN LINE FORM 5 Ġto Ġmodel Ġthe Ġsemantics Ġof Ġeach Ġtopic . ĠThe Ġdot Ġoperation Ġof ĠIN LINE FORM 6 Ġand ĠIN LINE FORM 7 Ġtransforms Ġthe Ġglobal Ġrepresentation ĠIN LINE FORM 8 Ġto Ġa Ġuser - dependent Ġrepresentation . ĠLikewise , Ġthe Ġdot Ġoperation Ġof ĠIN LINE FORM 9 Ġand ĠIN LINE FORM 10 Ġtransforms ĠIN LINE FORM 11 Ġto Ġa Ġtopic - dependent Ġrepresentation . </s> After Ġthe Ġtwo Ġdot Ġoperations Ġon ĠIN LINE FORM 0 Ġ, Ġwe Ġhave Ġuser - dependent Ġand Ġtopic - dependent Ġword Ġvectors ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġ, Ġwhich Ġare Ġconc aten ated Ġto Ġform Ġa Ġuser - Ġand Ġtopic - dependent Ġword Ġvector ĠIN LINE FORM 3 Ġ. ĠThen Ġthe Ġtransformed Ġword Ġembed d ings ĠIN LINE FORM 4 Ġare Ġused Ġas Ġthe ĠCNN Ġinput . ĠHere Ġwe Ġapply Ġthree Ġconv olution al Ġlayers Ġon Ġthe Ġconc aten ated Ġtransformed Ġword Ġembed d ings ĠIN LINE FORM 5 Ġ: ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġis Ġthe Ġindex Ġof Ġwords ; ĠIN LINE FORM 1 Ġis Ġa Ġnon - linear Ġactivation Ġfunction Ġ( we Ġuse ĠIN LINE FORM 2 Ġ); ĠIN LINE FORM 5 Ġis Ġthe Ġconv olution al Ġfilter Ġwith Ġinput Ġlength ĠIN LINE FORM 6 Ġand Ġoutput Ġlength ĠIN LINE FORM 7 Ġ, Ġwhere ĠIN LINE FORM 8 Ġis Ġthe Ġwindow Ġsize Ġof Ġthe Ġconv olution al Ġoperation ; Ġand ĠIN LINE FORM 9 Ġand ĠIN LINE FORM 10 Ġare Ġthe Ġoutput Ġand Ġbias Ġof Ġthe Ġconv olution Ġlayer ĠIN LINE FORM 11 Ġ, Ġrespectively . ĠIn Ġour Ġexperiments , Ġthe Ġthree Ġwindow Ġsizes ĠIN LINE FORM 12 Ġin Ġthe Ġthree Ġconv olution Ġlayers Ġare Ġone , Ġtwo , Ġand Ġthree , Ġencoding Ġun ig ram , Ġbig ram , Ġand Ġtrig ram Ġsemantics Ġaccordingly . </s> After Ġthe Ġconv olution al Ġlayer , Ġwe Ġadd Ġa Ġmaximum Ġpool ing Ġlayer Ġamong Ġconv olution al Ġoutputs Ġto Ġobtain Ġthe Ġun ig ram , Ġbig ram , Ġand Ġtrig ram Ġn - gram Ġrepresentations . ĠThis Ġis Ġsucceeded Ġby Ġan Ġaverage Ġpool ing Ġlayer Ġfor Ġan Ġelement - wise Ġaverage Ġof Ġthe Ġthree Ġmaxim ized Ġconv olution Ġoutputs . </s> UTC NN ĠModel ĠDescription </s> Figure ĠFIG REF 10 Ġillustrates Ġthe ĠUTC NN Ġmodel . ĠAs Ġmore Ġthan Ġone Ġuser Ġmay Ġinteract Ġwith Ġa Ġgiven Ġpost , Ġwe Ġfirst Ġadd Ġa Ġmaximum Ġpool ing Ġlayer Ġafter Ġthe Ġuser Ġmatrix Ġembed ding Ġlayer Ġand Ġuser Ġvector Ġembed ding Ġlayer Ġto Ġform Ġa Ġmoderator Ġmatrix Ġembed ding ĠIN LINE FORM 0 Ġand Ġa Ġmoderator Ġvector Ġembed ding ĠIN LINE FORM 1 Ġfor Ġmoderator ĠIN LINE FORM 2 Ġrespectively , Ġwhere ĠIN LINE FORM 3 Ġis Ġused Ġfor Ġthe Ġsemantic Ġtransformation Ġin Ġthe Ġdocument Ġcomposition Ġprocess , Ġas Ġmentioned Ġin Ġthe Ġprevious Ġsection . ĠThe Ġterm Ġmoderator Ġhere Ġis Ġto Ġdenote Ġthe Ġpseudo Ġuser Ġwho Ġprovides Ġthe Ġoverall Ġsemantic / sent iment Ġof Ġall Ġthe Ġengaged Ġusers Ġfor Ġone Ġdocument . ĠThe Ġembed ding ĠIN LINE FORM 4 Ġmodels Ġthe Ġmoderator Ġstance Ġpreference , Ġthat Ġis , Ġthe Ġpattern Ġof Ġthe Ġrevealed Ġuser Ġstance : Ġwhether Ġa Ġuser Ġis Ġwilling Ġto Ġshow Ġhis Ġpreference , Ġwhether Ġa Ġuser Ġlikes Ġto Ġshow Ġimpartial ity Ġwith Ġneutral Ġstatements Ġand Ġreasonable Ġarguments , Ġor Ġjust Ġwants Ġto Ġshow Ġstrong Ġsupport Ġfor Ġone Ġstance . ĠIdeally , Ġthe Ġlatent Ġuser Ġstance Ġis Ġmodeled Ġby ĠIN LINE FORM 5 Ġfor Ġeach Ġuser . ĠLikewise , Ġfor Ġtopic Ġinformation , Ġa Ġmaximum Ġpool ing Ġlayer Ġis Ġadded Ġafter Ġthe Ġtopic Ġmatrix Ġembed ding Ġlayer Ġand Ġtopic Ġvector Ġembed ding Ġlayer Ġto Ġform Ġa Ġjoint Ġtopic Ġmatrix Ġembed ding ĠIN LINE FORM 6 Ġand Ġa Ġjoint Ġtopic Ġvector Ġembed ding ĠIN LINE FORM 7 Ġfor Ġtopic ĠIN LINE FORM 8 Ġrespectively , Ġwhere ĠIN LINE FORM 9 Ġmodels Ġthe Ġsemantic Ġtransformation Ġof Ġtopic ĠIN LINE FORM 10 Ġas Ġin Ġusers Ġand ĠIN LINE FORM 11 Ġmodels Ġthe Ġtopic Ġstance Ġtendency . ĠThe Ġlatent Ġtopic Ġstance Ġis Ġalso Ġmodeled Ġby ĠIN LINE FORM 12 Ġfor Ġeach Ġtopic . </s> As Ġfor Ġcomments , Ġwe Ġview Ġthem Ġas Ġshort Ġdocuments Ġwith Ġauthors Ġonly Ġbut Ġwithout Ġlik ers Ġnor Ġtheir Ġown Ġcomments . ĠTherefore Ġwe Ġapply Ġdocument Ġcomposition Ġon Ġcomments Ġalthough Ġhere Ġusers Ġare Ġcommenters Ġ( users Ġwho Ġcomment ). ĠIt Ġis Ġnoticed Ġthat Ġthe Ġword Ġembed d ings ĠIN LINE FORM 0 Ġfor Ġthe Ġsame Ġword Ġin Ġthe Ġposts Ġand Ġcomments Ġare Ġthe Ġsame , Ġbut Ġafter Ġbeing Ġtransformed Ġto ĠIN LINE FORM 1 Ġin Ġthe Ġdocument Ġcomposition Ġprocess Ġshown Ġin ĠFigure ĠFIG REF 4 Ġ, Ġthey Ġmight Ġbecome Ġdifferent Ġbecause Ġof Ġtheir Ġdifferent Ġengaged Ġusers . ĠThe Ġoutput Ġcomment Ġrepresentation Ġtogether Ġwith Ġthe Ġcommenter Ġvector Ġembed ding ĠIN LINE FORM 2 Ġand Ġtopic Ġvector Ġembed ding ĠIN LINE FORM 3 Ġare Ġconc aten ated Ġand Ġa Ġmaximum Ġpool ing Ġlayer Ġis Ġadded Ġto Ġselect Ġthe Ġmost Ġimportant Ġfeature Ġfor Ġcomments . ĠInstead Ġof Ġrequiring Ġthat Ġthe Ġcomment Ġstance Ġagree Ġwith Ġthe Ġpost , ĠUTC NN Ġsimply Ġextracts Ġthe Ġmost Ġimportant Ġfeatures Ġof Ġthe Ġcomment Ġcontents ; Ġthey Ġcould Ġbe Ġhelpful , Ġwhether Ġthey Ġshow Ġobvious Ġagreement Ġor Ġdisagreement . ĠTherefore Ġwhen Ġcombining Ġcomment Ġinformation Ġhere , Ġthe Ġmaximum Ġpool ing Ġlayer Ġis Ġmore Ġappropriate Ġthan Ġother Ġpool ing Ġor Ġmerging Ġlayers . ĠIndeed , Ġwe Ġbelieve Ġthis Ġis Ġone Ġreason Ġfor ĠUTC NN 's Ġperformance Ġgains . </s> Finally , Ġthe Ġpooled Ġcomment Ġrepresentation , Ġtogether Ġwith Ġuser Ġvector Ġembed ding ĠIN LINE FORM 0 Ġ, Ġtopic Ġvector Ġembed ding ĠIN LINE FORM 1 Ġ, Ġand Ġdocument Ġrepresentation Ġare Ġfed Ġto Ġa Ġfully Ġconnected Ġnetwork , Ġand Ġsoft max Ġis Ġapplied Ġto Ġyield Ġthe Ġfinal Ġstance Ġlabel Ġprediction Ġfor Ġthe Ġpost . </s> Exper iment </s> We Ġstart Ġwith Ġthe Ġexperimental Ġdataset Ġand Ġthen Ġdescribe Ġthe Ġtraining Ġprocess Ġas Ġwell Ġas Ġthe Ġimplementation Ġof Ġthe Ġbas elines . ĠWe Ġalso Ġimplement Ġseveral Ġvariations Ġto Ġreveal Ġthe Ġeffects Ġof Ġfeatures : Ġauthors , Ġlik ers , Ġcomment , Ġand Ġcommenters . ĠIn Ġthe Ġresults Ġsection Ġwe Ġcompare Ġour Ġmodel Ġwith Ġrelated Ġwork . </s> Dat as et </s> We Ġtested Ġthe Ġproposed ĠUTC NN Ġon Ġtwo Ġdifferent Ġdatasets : ĠFB Fans Ġand ĠCreate Deb ate . ĠFB Fans Ġis Ġa Ġprivately - owned , Ġsingle - topic , ĠChinese , Ġun balanced , Ġsocial Ġmedia Ġdataset , Ġand ĠCreate Deb ate Ġis Ġa Ġpublic , Ġmultiple - topic , ĠEnglish , Ġbalanced , Ġforum Ġdataset . ĠResults Ġusing Ġthese Ġtwo Ġdatasets Ġshow Ġthe Ġapplic ability Ġand Ġsuperiority Ġfor Ġdifferent Ġtopics , Ġlanguages , Ġdata Ġdistributions , Ġand Ġplatforms . </s> The ĠFB Fans Ġdataset Ġcontains Ġdata Ġfrom Ġanti - nuclear - power ĠChinese ĠFacebook Ġfan Ġgroups Ġfrom ĠSeptember Ġ2013 Ġto ĠAugust Ġ2014 , Ġincluding Ġposts Ġand Ġtheir Ġauthor Ġand Ġlik er ĠIDs . ĠThere Ġare Ġa Ġtotal Ġof Ġ2 , 496 Ġauthors , Ġ505 , 137 Ġlik ers , Ġ33 , 686 Ġcommenters , Ġand Ġ505 , 412 Ġunique Ġusers . ĠTwo Ġannot ators Ġwere Ġasked Ġto Ġtake Ġinto Ġaccount Ġonly Ġthe Ġpost Ġcontent Ġto Ġlabel Ġthe Ġstance Ġof Ġthe Ġposts Ġin Ġthe Ġwhole Ġdataset Ġas Ġsupportive , Ġneutral , Ġor Ġun support ive Ġ( here after Ġden oted Ġas ĠSup , ĠNe u , Ġand ĠUn s ). ĠSup / Un s Ġposts Ġwere Ġthose Ġin Ġsupport Ġof Ġor Ġagainst Ġanti - re const ruction ; ĠNe u Ġposts Ġwere Ġthose Ġev inc ing Ġa Ġneutral Ġstandpoint Ġon Ġthe Ġtopic , Ġor Ġwere Ġirrelevant . ĠRaw Ġagreement Ġbetween Ġannot ators Ġis Ġ0 . 91 , Ġindicating Ġhigh Ġagreement . ĠSpecifically , ĠCohen âĢ Ļ s ĠKappa Ġfor ĠNe u Ġand Ġnot ĠNe u Ġlabeling Ġis Ġ0 . 58 Ġ( moderate ), Ġand Ġfor ĠSup Ġor ĠUn s Ġlabeling Ġis Ġ0 . 84 Ġ( almost Ġperfect ). ĠPosts Ġwith Ġinconsistent Ġlabels Ġwere Ġfiltered Ġout , Ġand Ġthe Ġdevelopment Ġand Ġtesting Ġsets Ġwere Ġrandomly Ġselected Ġfrom Ġwhat Ġwas Ġleft . ĠPosts Ġin Ġthe Ġdevelopment Ġand Ġtesting Ġsets Ġinvolved Ġat Ġleast Ġone Ġuser Ġwho Ġappeared Ġin Ġthe Ġtraining Ġset . ĠThe Ġnumber Ġof Ġposts Ġfor Ġeach Ġstance Ġis Ġshown Ġon Ġthe Ġleft - hand Ġside Ġof ĠTable ĠT AB REF 12 Ġ. ĠAbout Ġtwenty Ġpercent Ġof Ġthe Ġposts Ġwere Ġlabeled Ġwith Ġa Ġstance , Ġand Ġthe Ġnumber Ġof Ġsupportive Ġ( Sup ) Ġposts Ġwas Ġmuch Ġlarger Ġthan Ġthat Ġof Ġthe Ġun support ive Ġ( Un s ) Ġones : Ġthis Ġis Ġthus Ġhighly Ġskewed Ġdata , Ġwhich Ġcompl icates Ġstance Ġclassification . ĠOn Ġaverage , Ġ161 . 1 Ġusers Ġwere Ġinvolved Ġin Ġone Ġpost . ĠThe Ġmaximum Ġwas Ġ23 , 297 Ġand Ġthe Ġminimum Ġwas Ġone Ġ( the Ġauthor ). ĠFor Ġcomments , Ġon Ġaverage Ġthere Ġwere Ġ3 Ġcomments Ġper Ġpost . ĠThe Ġmaximum Ġwas Ġ1 , 09 2 Ġand Ġthe Ġminimum Ġwas Ġzero . </s> To Ġtest Ġwhether Ġthe Ġassumption Ġof Ġthis Ġpaper ĠâĢĵ Ġposts Ġattract Ġusers Ġwho Ġhold Ġthe Ġsame Ġstance Ġto Ġlike Ġthem ĠâĢĵ Ġis Ġreliable , Ġwe Ġexamine Ġthe Ġlikes Ġfrom Ġauthors Ġof Ġdifferent Ġstances . ĠPosts Ġin ĠFB Fans Ġdataset Ġare Ġused Ġfor Ġthis Ġanalysis . ĠWe Ġcalculate Ġthe Ġlike Ġstatistics Ġof Ġeach Ġdistinct Ġauthor Ġfrom Ġthese Ġ32 , 595 Ġposts . ĠAs Ġthe Ġnumbers Ġof Ġauthors Ġin Ġthe ĠSup , ĠNe u Ġand ĠUn s Ġstances Ġare Ġlargely Ġim balanced , Ġthese Ġnumbers Ġare Ġnormalized Ġby Ġthe Ġnumber Ġof Ġusers Ġof Ġeach Ġstance . ĠTable ĠT AB REF 13 Ġshows Ġthe Ġresults . ĠPosts Ġwith Ġstances Ġ( i . e ., Ġnot Ġneutral ) Ġattract Ġusers Ġof Ġthe Ġsame Ġstance . ĠNeutral Ġposts Ġalso Ġattract Ġboth Ġsupportive Ġand Ġneutral Ġusers , Ġlike Ġwhat Ġwe Ġobserve Ġin Ġsupportive Ġposts , Ġbut Ġjust Ġthe Ġneutral Ġposts Ġcan Ġattract Ġeven Ġmore Ġneutral Ġlik ers . ĠThese Ġresults Ġdo Ġsuggest Ġthat Ġusers Ġprefer Ġposts Ġof Ġthe Ġsame Ġstance , Ġor Ġat Ġleast Ġposts Ġof Ġno Ġobvious Ġstance Ġwhich Ġmight Ġcause Ġannoyance Ġwhen Ġreading , Ġand Ġhence Ġsupport Ġthe Ġuser Ġmodeling Ġin Ġour Ġapproach . </s> The ĠCreate Deb ate Ġdataset Ġwas Ġcollected Ġfrom Ġan ĠEnglish Ġonline Ġdebate Ġforum Ġdiscussing Ġfour Ġtopics : Ġabortion Ġ( AB O ), Ġgay Ġrights Ġ( G AY ), ĠObama Ġ( O BA ), Ġand Ġmarijuana Ġ( MAR ). ĠThe Ġposts Ġare Ġannot ated Ġas Ġfor Ġ( F ) Ġand Ġagainst Ġ( A ). ĠRepl ies Ġto Ġposts Ġin Ġthis Ġdataset Ġare Ġalso Ġlabeled Ġwith Ġstance Ġand Ġhence Ġuse Ġthe Ġsame Ġdata Ġformat Ġas Ġposts . ĠThe Ġlabeling Ġresults Ġare Ġshown Ġin Ġthe Ġright - hand Ġside Ġof ĠTable ĠT AB REF 12 Ġ. ĠWe Ġobserve Ġthat Ġthe Ġdataset Ġis Ġmore Ġbalanced Ġthan Ġthe ĠFB Fans Ġdataset . ĠIn Ġaddition , Ġthere Ġare Ġ9 77 Ġunique Ġusers Ġin Ġthe Ġdataset . ĠTo Ġcompare Ġwith ĠHasan Ġand ĠNg 's Ġwork , Ġwe Ġconducted Ġfive - fold Ġcross - valid ation Ġand Ġpresent Ġthe Ġannotation Ġresults Ġas Ġthe Ġaverage Ġnumber Ġof Ġall Ġfolds ĠB IB REF 9 Ġ, ĠB IB REF 5 Ġ. </s> The ĠFB Fans Ġdataset Ġhas Ġmore Ġintegrated Ġfunctions Ġthan Ġthe ĠCreate Deb ate Ġdataset ; Ġthus Ġour Ġmodel Ġcan Ġutilize Ġall Ġlinguistic Ġand Ġextra - ling u istic Ġfeatures . ĠFor Ġthe ĠCreate Deb ate Ġdataset , Ġon Ġthe Ġother Ġhand , Ġthe Ġlike Ġand Ġcomment Ġfeatures Ġare Ġnot Ġavailable Ġ( as Ġthere Ġis Ġa Ġstance Ġlabel Ġfor Ġeach Ġreply , Ġreplies Ġare Ġevaluated Ġas Ġposts Ġas Ġother Ġprevious Ġwork ) Ġbut Ġwe Ġstill Ġimplemented Ġour Ġmodel Ġusing Ġthe Ġcontent , Ġauthor , Ġand Ġtopic Ġinformation . </s> Settings </s> In Ġthe ĠUTC NN Ġtraining Ġprocess , Ġcross - ent ropy Ġwas Ġused Ġas Ġthe Ġloss Ġfunction Ġand ĠAda Grad Ġas Ġthe Ġoptim izer . ĠFor ĠFB Fans Ġdataset , Ġwe Ġlearned Ġthe Ġ50 - dimensional Ġword Ġembed d ings Ġon Ġthe Ġwhole Ġdataset Ġusing ĠGl o Ve ĠB IB REF 21 Ġto Ġcapture Ġthe Ġword Ġsemantics ; Ġfor ĠCreate Deb ate Ġdataset Ġwe Ġused Ġthe Ġpublicly Ġavailable ĠEnglish Ġ50 - dimensional Ġword Ġembed d ings , Ġpre - trained Ġalso Ġusing ĠGl o Ve . ĠThese Ġword Ġembed d ings Ġwere Ġfixed Ġin Ġthe Ġtraining Ġprocess . ĠThe Ġlearning Ġrate Ġwas Ġset Ġto Ġ0 . 03 . ĠAll Ġuser Ġand Ġtopic Ġembed d ings Ġwere Ġrandomly Ġinitialized Ġin Ġthe Ġrange Ġof Ġ[- 0 . 1 Ġ0 . 1 ]. ĠMatrix Ġembed d ings Ġfor Ġusers Ġand Ġtopics Ġwere Ġsized Ġat Ġ250 Ġ( ĠIN LINE FORM 0 Ġ); Ġvector Ġembed d ings Ġfor Ġusers Ġand Ġtopics Ġwere Ġset Ġto Ġlength Ġ10 . </s> We Ġapplied Ġthe ĠL DA Ġtopic Ġmodel ĠB IB REF 22 Ġon Ġthe ĠFB Fans Ġdataset Ġto Ġdetermine Ġthe Ġlatent Ġtopics Ġwith Ġwhich Ġto Ġbuild Ġtopic Ġembed d ings , Ġas Ġthere Ġis Ġonly Ġone Ġgeneral Ġknown Ġtopic : Ġnuclear Ġpower Ġplants . ĠWe Ġlearned Ġ100 Ġlatent Ġtopics Ġand Ġassigned Ġthe Ġtop Ġthree Ġtopics Ġfor Ġeach Ġpost . ĠFor Ġthe ĠCreate Deb ate Ġdataset , Ġwhich Ġitself Ġconstitutes Ġfour Ġtopics , Ġthe Ġtopic Ġlabels Ġfor Ġposts Ġwere Ġused Ġdirectly Ġwithout Ġadditionally Ġapplying ĠL DA . </s> For Ġthe ĠFB Fans Ġdata Ġwe Ġreport Ġclass - based Ġf - sc ores Ġas Ġwell Ġas Ġthe Ġmacro - average Ġf - score Ġ( ĠIN LINE FORM 0 Ġ) Ġshown Ġin Ġequation ĠEQ REF 19 Ġ. ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġand ĠIN LINE FORM 1 Ġare Ġthe Ġaverage Ġprecision Ġand Ġrecall Ġof Ġthe Ġthree Ġclass . ĠWe Ġadopted Ġthe Ġmacro - average Ġf - score Ġas Ġthe Ġevaluation Ġmetric Ġfor Ġthe Ġoverall Ġperformance Ġbecause Ġ( 1 ) Ġthe Ġexperimental Ġdataset Ġis Ġseverely Ġim balanced , Ġwhich Ġis Ġcommon Ġfor Ġcontentious Ġissues ; Ġand Ġ( 2 ) Ġfor Ġstance Ġclassification , Ġcontent Ġin Ġminor - class Ġposts Ġis Ġusually Ġmore Ġimportant Ġfor Ġfurther Ġapplications . ĠFor Ġthe ĠCreate Deb ate Ġdataset , Ġaccuracy Ġwas Ġadopted Ġas Ġthe Ġevaluation Ġmetric Ġto Ġcompare Ġthe Ġresults Ġwith Ġrelated Ġwork ĠB IB REF 7 Ġ, ĠB IB REF 9 Ġ, ĠB IB REF 12 Ġ. </s> Bas elines </s> We Ġpit Ġour Ġmodel Ġagainst Ġthe Ġfollowing Ġbas elines : Ġ1 ) ĠS VM Ġwith Ġun ig ram , Ġbig ram , Ġand Ġtrig ram Ġfeatures , Ġwhich Ġis Ġa Ġstandard Ġyet Ġrather Ġstrong Ġclass ifier Ġfor Ġtext Ġfeatures ; Ġ2 ) ĠS VM Ġwith Ġaverage Ġword Ġembed ding , Ġwhere Ġa Ġdocument Ġis Ġrepresented Ġas Ġa Ġcontinuous Ġrepresentation Ġby Ġaveraging Ġthe Ġembed d ings Ġof Ġthe Ġcomposite Ġwords ; Ġ3 ) ĠS VM Ġwith Ġaverage Ġtransformed Ġword Ġembed d ings Ġ( the ĠIN LINE FORM 0 Ġin Ġequation ĠEQ REF 6 Ġ), Ġwhere Ġa Ġdocument Ġis Ġrepresented Ġas Ġa Ġcontinuous Ġrepresentation Ġby Ġaveraging Ġthe Ġtransformed Ġembed d ings Ġof Ġthe Ġcomposite Ġwords ; Ġ4 ) Ġtwo Ġmature Ġdeep Ġlearning Ġmodels Ġon Ġtext Ġclassification , ĠCNN ĠB IB REF 3 Ġand ĠRec urrent ĠConv olution al ĠNeural ĠNetworks Ġ( RC NN ) ĠB IB REF 0 Ġ, Ġwhere Ġthe Ġhyper param eters Ġare Ġbased Ġon Ġtheir Ġwork ; Ġ5 ) Ġthe Ġabove ĠS VM Ġand Ġdeep Ġlearning Ġmodels Ġwith Ġcomment Ġinformation ; Ġ6 ) ĠUTC NN Ġwithout Ġuser Ġinformation , Ġrepresenting Ġa Ġpure - text ĠCNN Ġmodel Ġwhere Ġwe Ġuse Ġthe Ġsame Ġuser Ġmatrix Ġand Ġuser Ġembed d ings ĠIN LINE FORM 1 Ġand ĠIN LINE FORM 2 Ġfor Ġeach Ġuser ; Ġ7 ) ĠUTC NN Ġwithout Ġthe ĠL DA Ġmodel , Ġrepresenting Ġhow ĠUTC NN Ġworks Ġwith Ġa Ġsingle - topic Ġdataset ; Ġ8 ) ĠUTC NN Ġwithout Ġcomments , Ġin Ġwhich Ġthe Ġmodel Ġpredicts Ġthe Ġstance Ġlabel Ġgiven Ġonly Ġuser Ġand Ġtopic Ġinformation . ĠAll Ġthese Ġmodels Ġwere Ġtrained Ġon Ġthe Ġtraining Ġset , Ġand Ġparameters Ġas Ġwell Ġas Ġthe ĠS VM Ġkernel Ġselections Ġ( linear Ġor ĠRB F ) Ġwere Ġfine - tun ed Ġon Ġthe Ġdevelopment Ġset . ĠAlso , Ġwe Ġadopt Ġovers am pling Ġon ĠSV Ms , ĠCNN Ġand ĠRC NN Ġbecause Ġthe ĠFB Fans Ġdataset Ġis Ġhighly Ġim balanced . </s> Results Ġon ĠFB Fans ĠDat as et </s> In ĠTable ĠT AB REF 22 Ġwe Ġshow Ġthe Ġresults Ġof ĠUTC NN Ġand Ġthe Ġbas elines Ġon Ġthe ĠFB Fans Ġdataset . ĠHere ĠMajority Ġyields Ġgood Ġperformance Ġon ĠNe u Ġsince ĠFB Fans Ġis Ġhighly Ġbiased Ġto Ġthe Ġneutral Ġclass . ĠThe ĠS VM Ġmodels Ġperform Ġwell Ġon ĠSup Ġand ĠNe u Ġbut Ġperform Ġpoorly Ġfor ĠUn s , Ġshowing Ġthat Ġcontent Ġinformation Ġin Ġitself Ġis Ġinsufficient Ġto Ġpredict Ġstance Ġlabels , Ġespecially Ġfor Ġthe Ġminor Ġclass . ĠWith Ġthe Ġtransformed Ġword Ġembed ding Ġfeature , ĠS VM Ġcan Ġachieve Ġcomparable Ġperformance Ġas ĠS VM Ġwith Ġn - gram Ġfeature . ĠHowever , Ġthe Ġmuch Ġfewer Ġfeature Ġdimension Ġof Ġthe Ġtransformed Ġword Ġembed ding Ġmakes ĠS VM Ġwith Ġword Ġembed d ings Ġa Ġmore Ġefficient Ġchoice Ġfor Ġmodeling Ġthe Ġlarge Ġscale Ġsocial Ġmedia Ġdataset . ĠFor Ġthe ĠCNN Ġand ĠRC NN Ġmodels , Ġthey Ġperform Ġslightly Ġbetter Ġthan Ġmost Ġof Ġthe ĠS VM Ġmodels Ġbut Ġstill , Ġthe Ġcontent Ġinformation Ġis Ġinsufficient Ġto Ġachieve Ġa Ġgood Ġperformance Ġon Ġthe ĠUn s Ġposts . ĠAs Ġto Ġadding Ġcomment Ġinformation Ġto Ġthese Ġmodels , Ġsince Ġthe Ġcommenters Ġdo Ġnot Ġalways Ġhold Ġthe Ġsame Ġstance Ġas Ġthe Ġauthor , Ġsimply Ġadding Ġcomments Ġand Ġpost Ġcontents Ġtogether Ġmerely Ġadds Ġnoise Ġto Ġthe Ġmodel . </s> Among Ġall ĠUTC NN Ġvariations , Ġwe Ġfind Ġthat Ġuser Ġinformation Ġis Ġmost Ġimportant , Ġfollowed Ġby Ġtopic Ġand Ġcomment Ġinformation . ĠUTC NN Ġwithout Ġuser Ġinformation Ġshows Ġresults Ġsimilar Ġto ĠSV Ms ĠâĢĶ Ġit Ġdoes Ġwell Ġfor ĠSup Ġand ĠNe u Ġbut Ġdetects Ġno ĠUn s . ĠIts Ġbest Ġf - sc ores Ġon Ġboth ĠSup Ġand ĠNe u Ġamong Ġall Ġmethods Ġshow Ġthat Ġwith Ġenough Ġtraining Ġdata , Ġcontent - based Ġmodels Ġcan Ġperform Ġwell ; Ġat Ġthe Ġsame Ġtime , Ġthe Ġlack Ġof Ġuser Ġinformation Ġresults Ġin Ġtoo Ġfew Ġclues Ġfor Ġminor - class Ġposts Ġto Ġeither Ġpredict Ġtheir Ġstance Ġdirectly Ġor Ġlink Ġthem Ġto Ġother Ġusers Ġand Ġposts Ġfor Ġimproved Ġperformance . ĠThe Ġ17 . 5 % Ġimprovement Ġwhen Ġadding Ġuser Ġinformation Ġsuggests Ġthat Ġuser Ġinformation Ġis Ġespecially Ġuseful Ġwhen Ġthe Ġdataset Ġis Ġhighly Ġim balanced . ĠAll Ġmodels Ġthat Ġconsider Ġuser Ġinformation Ġpredict Ġthe Ġminority Ġclass Ġsuccessfully . ĠU CT NN Ġwithout Ġtopic Ġinformation Ġworks Ġwell Ġbut Ġachieves Ġlower Ġperformance Ġthan Ġthe Ġfull ĠUTC NN Ġmodel . ĠThe Ġ4 . 9 % Ġperformance Ġgain Ġbrought Ġby ĠL DA Ġshows Ġthat Ġalthough Ġit Ġis Ġsatisfactory Ġfor Ġsingle Ġtopic Ġdatasets , Ġadding Ġthat Ġlatent Ġtopics Ġstill Ġbenefits Ġperformance : Ġeven Ġwhen Ġwe Ġare Ġdiscussing Ġthe Ġsame Ġtopic , Ġwe Ġuse Ġdifferent Ġarguments Ġand Ġsupporting Ġevidence . ĠLastly , Ġwe Ġget Ġ4 . 8 % Ġimprovement Ġwhen Ġadding Ġcomment Ġinformation Ġand Ġit Ġachieves Ġcomparable Ġperformance Ġto ĠUTC NN Ġwithout Ġtopic Ġinformation , Ġwhich Ġshows Ġthat Ġcomments Ġalso Ġbenefit Ġperformance . ĠFor Ġplatforms Ġwhere Ġuser ĠIDs Ġare Ġpixel ated Ġor Ġotherwise Ġhidden , Ġadding Ġcomments Ġto Ġa Ġtext Ġmodel Ġstill Ġimproves Ġperformance . ĠIn Ġits Ġintegration Ġof Ġuser , Ġcontent , Ġand Ġcomment Ġinformation , Ġthe Ġfull ĠUTC NN Ġproduces Ġthe Ġhighest Ġf - sc ores Ġon Ġall ĠSup , ĠNe u , Ġand ĠUn s Ġstances Ġamong Ġmodels Ġthat Ġpredict Ġthe ĠUn s Ġclass , Ġand Ġthe Ġhighest Ġmacro - average Ġf - score Ġoverall . ĠThis Ġshows Ġits Ġability Ġto Ġbalance Ġa Ġbiased Ġdataset Ġand Ġsupports Ġour Ġclaim Ġthat ĠUTC NN Ġsuccessfully Ġbridges Ġcontent Ġand Ġuser , Ġtopic , Ġand Ġcomment Ġinformation Ġfor Ġstance Ġclassification Ġon Ġsocial Ġmedia Ġtext . ĠAnother Ġmerit Ġof ĠUTC NN Ġis Ġthat Ġit Ġdoes Ġnot Ġrequire Ġa Ġbalanced Ġtraining Ġdata . ĠThis Ġis Ġsupported Ġby Ġits Ġoutper forming Ġother Ġmodels Ġthough Ġno Ġovers am pling Ġtechnique Ġis Ġapplied Ġto Ġthe ĠUTC NN Ġrelated Ġexperiments Ġas Ġshown Ġin Ġthis Ġpaper . ĠThus Ġwe Ġcan Ġconclude Ġthat Ġthe Ġuser Ġinformation Ġprovides Ġstrong Ġclues Ġand Ġit Ġis Ġstill Ġrich Ġeven Ġin Ġthe Ġminority Ġclass . </s> We Ġalso Ġinvestigate Ġthe Ġsemantic Ġdifference Ġwhen Ġa Ġuser Ġacts Ġas Ġan Ġauthor / l iker Ġor Ġa Ġcommenter . ĠWe Ġevaluated Ġa Ġvariation Ġin Ġwhich Ġall Ġembed d ings Ġfrom Ġthe Ġsame Ġuser Ġwere Ġforced Ġto Ġbe Ġidentical Ġ( this Ġis Ġthe ĠUTC NN Ġshared Ġuser Ġembed ding Ġsetting Ġin ĠTable ĠT AB REF 22 Ġ). ĠThis Ġsetting Ġyielded Ġonly Ġa Ġ2 . 5 % Ġimprovement Ġover Ġthe Ġmodel Ġwithout Ġcomments , Ġwhich Ġis Ġnot Ġstatistically Ġsignificant . ĠHowever , Ġwhen Ġseparating Ġauthors / l ikers Ġand Ġcommenters Ġembed d ings Ġ( i . e ., Ġthe ĠUTC NN Ġfull Ġmodel ), Ġwe Ġachieved Ġmuch Ġgreater Ġimprovements Ġ( 4 . 8 %). ĠWe Ġattribute Ġthis Ġresult Ġto Ġthe Ġtendency Ġof Ġusers Ġto Ġuse Ġdifferent Ġwording Ġfor Ġdifferent Ġroles Ġ( for Ġinstance Ġauthor Ġvs Ġcommenter ). ĠThis Ġis Ġobserved Ġwhen Ġthe Ġuser , Ġacting Ġas Ġan Ġauthor , Ġattempts Ġto Ġsupport Ġher Ġargument Ġagainst Ġnuclear Ġpower Ġby Ġusing Ġimprovements Ġin Ġsolar Ġpower ; Ġwhen Ġacting Ġas Ġa Ġcommenter , Ġthough , Ġshe Ġinteracts Ġwith Ġpost Ġcontents Ġby Ġcriticizing Ġpast Ġpoliticians Ġwho Ġsupported Ġnuclear Ġpower Ġor Ġby Ġarguing Ġthat Ġthe Ġproposed Ġevacuation Ġplan Ġin Ġcase Ġof Ġa Ġnuclear Ġaccident Ġis Ġridiculous . ĠBased Ġon Ġthis Ġfinding , Ġin Ġthe Ġfinal ĠUTC NN Ġsetting Ġwe Ġtrain Ġtwo Ġuser Ġmatrix Ġembed d ings Ġfor Ġone Ġuser : Ġone Ġfor Ġthe Ġauthor / l iker Ġrole Ġand Ġthe Ġother Ġfor Ġthe Ġcommenter Ġrole . </s> Results Ġon ĠCreate Deb ate ĠDat as et </s> Table ĠT AB REF 24 Ġshows Ġthe Ġresults Ġof ĠUTC NN , Ġbas elines Ġas Ġwe Ġimplemented Ġon Ġthe ĠFB Fans Ġd ats et Ġand Ġrelated Ġwork Ġon Ġthe ĠCreate Deb ate Ġdataset . ĠWe Ġdo Ġnot Ġadopt Ġovers am pling Ġon Ġthese Ġmodels Ġbecause Ġthe ĠCreate Deb ate Ġdataset Ġis Ġalmost Ġbalanced . ĠIn Ġprevious Ġwork , Ġinteger Ġlinear Ġprogramming Ġ( IL P ) Ġor Ġlinear - chain Ġconditional Ġrandom Ġfields Ġ( CR Fs ) Ġwere Ġproposed Ġto Ġintegrate Ġtext Ġfeatures , Ġauthor , Ġideology , Ġand Ġuser - inter action Ġconstraints , Ġwhere Ġtext Ġfeatures Ġare Ġun ig ram , Ġbig ram , Ġand ĠPOS - depend encies ; Ġthe Ġauthor Ġconstraint Ġtends Ġto Ġrequire Ġthat Ġposts Ġfrom Ġthe Ġsame Ġauthor Ġfor Ġthe Ġsame Ġtopic Ġhold Ġthe Ġsame Ġstance ; Ġthe Ġideology Ġconstraint Ġaims Ġto Ġcapture Ġinf erences Ġbetween Ġtopics Ġfor Ġthe Ġsame Ġauthor ; Ġthe Ġuser - inter action Ġconstraint Ġmodels Ġrelationships Ġamong Ġposts Ġvia Ġuser Ġinteractions Ġsuch Ġas Ġreplies ĠB IB REF 7 Ġ, ĠB IB REF 9 Ġ. </s> The ĠS VM Ġwith Ġn - gram Ġor Ġaverage Ġword Ġembed ding Ġfeature Ġperforms Ġjust Ġsimilar Ġto Ġthe Ġmajority . ĠHowever , Ġwith Ġthe Ġtransformed Ġword Ġembed ding , Ġit Ġachieves Ġsuperior Ġresults . ĠIt Ġshows Ġthat Ġthe Ġlearned Ġuser Ġand Ġtopic Ġembed d ings Ġreally Ġcapture Ġthe Ġuser Ġand Ġtopic Ġsemantics . ĠThis Ġfinding Ġis Ġnot Ġso Ġobvious Ġin Ġthe ĠFB Fans Ġdataset Ġand Ġit Ġmight Ġbe Ġdue Ġto Ġthe Ġunfavorable Ġdata Ġske wn ess Ġfor ĠS VM . ĠAs Ġfor ĠCNN Ġand ĠRC NN , Ġthey Ġperform Ġslightly Ġbetter Ġthan Ġmost ĠSV Ms Ġas Ġwe Ġfound Ġin ĠTable ĠT AB REF 22 Ġfor ĠFB Fans . </s> Compared Ġto Ġthe ĠIL P ĠB IB REF 7 Ġand ĠCR F ĠB IB REF 9 Ġmethods , Ġthe ĠUTC NN Ġuser Ġembed d ings Ġencode Ġauthor Ġand Ġuser - inter action Ġconstraints , Ġwhere Ġthe Ġideology Ġconstraint Ġis Ġmodeled Ġby Ġthe Ġtopic Ġembed d ings Ġand Ġtext Ġfeatures Ġare Ġmodeled Ġby Ġthe ĠCNN . ĠThe Ġsignificant Ġimprovement Ġachieved Ġby ĠUTC NN Ġsuggests Ġthe Ġlatent Ġrepresentations Ġare Ġmore Ġeffective Ġthan Ġovert Ġmodel Ġconstraints . </s> The ĠPS L Ġmodel ĠB IB REF 12 Ġjointly Ġlabels Ġboth Ġauthor Ġand Ġpost Ġstance Ġusing Ġprob abil istic Ġsoft Ġlogic Ġ( PS L ) ĠB IB REF 23 Ġby Ġconsidering Ġtext Ġfeatures Ġand Ġreply Ġlinks Ġbetween Ġauthors Ġand Ġposts Ġas Ġin ĠHasan Ġand ĠNg 's Ġwork . ĠTable ĠT AB REF 24 Ġreports Ġthe Ġresult Ġof Ġtheir Ġbest ĠAD Ġsetting , Ġwhich Ġrepresents Ġthe Ġfull Ġjoint Ġstance / dis ag reement Ġcollective Ġmodel Ġon Ġposts Ġand Ġis Ġhence Ġmore Ġrelevant Ġto ĠUTC NN . ĠIn Ġcontrast Ġto Ġtheir Ġmodel , Ġthe ĠUTC NN Ġuser Ġembed d ings Ġrepresent Ġrelationships Ġbetween Ġauthors , Ġbut ĠUTC NN Ġmodels Ġdo Ġnot Ġutilize Ġlink Ġinformation Ġbetween Ġposts . ĠThough Ġthe ĠPS L Ġmodel Ġhas Ġthe Ġadvantage Ġof Ġbeing Ġable Ġto Ġjointly Ġlabel Ġthe Ġstances Ġof Ġauthors Ġand Ġposts , Ġits Ġperformance Ġon Ġposts Ġis Ġlower Ġthan Ġthe Ġthat Ġfor Ġthe ĠIL P Ġor ĠCR F Ġmodels . ĠUTC NN Ġsignificantly Ġoutper forms Ġthese Ġmodels Ġon Ġposts Ġand Ġhas Ġthe Ġpotential Ġto Ġpredict Ġuser Ġstances Ġthrough Ġthe Ġgenerated Ġuser Ġembed d ings . </s> For Ġthe ĠCreate Deb ate Ġdataset , Ġwe Ġalso Ġevaluated Ġperformance Ġwhen Ġnot Ġusing Ġtopic Ġembed d ings Ġor Ġuser Ġembed d ings ; Ġas Ġreplies Ġin Ġthis Ġdataset Ġare Ġviewed Ġas Ġposts , Ġthe Ġsetting Ġwithout Ġcomment Ġembed d ings Ġis Ġnot Ġavailable . ĠTable ĠT AB REF 24 Ġshows Ġthe Ġsame Ġfindings Ġas ĠTable ĠT AB REF 22 Ġ: Ġthe Ġ21 % Ġimprovement Ġin Ġaccuracy Ġdemonstrates Ġthat Ġuser Ġinformation Ġis Ġthe Ġmost Ġvital . ĠThis Ġfinding Ġalso Ġsupports Ġthe Ġresults Ġin Ġthe Ġrelated Ġwork : Ġuser Ġconstraints Ġare Ġuseful Ġand Ġcan Ġyield Ġ11 . 2 % Ġimprovement Ġin Ġaccuracy ĠB IB REF 7 Ġ. ĠFurther Ġconsidering Ġtopic Ġinformation Ġyields Ġ3 . 4 % Ġimprovement , Ġsuggesting Ġthat Ġknowing Ġthe Ġsubject Ġof Ġdebates Ġprovides Ġuseful Ġinformation . ĠIn Ġsum , ĠTable ĠT AB REF 22 Ġtogether Ġwith ĠTable ĠT AB REF 24 Ġshow Ġthat ĠUTC NN Ġachieves Ġpromising Ġperformance Ġregardless Ġof Ġtopic , Ġlanguage , Ġdata Ġdistribution , Ġand Ġplatform . </s> Conclusion </s> We Ġhave Ġproposed ĠUTC NN , Ġa Ġneural Ġnetwork Ġmodel Ġthat Ġincorporates Ġuser , Ġtopic , Ġcontent Ġand Ġcomment Ġinformation Ġfor Ġstance Ġclassification Ġon Ġsocial Ġmedia Ġtexts . ĠUTC NN Ġlearns Ġuser Ġembed d ings Ġfor Ġall Ġusers Ġwith Ġminimum Ġactive Ġdegree , Ġi . e ., Ġone Ġpost Ġor Ġone Ġlike . ĠTopic Ġinformation Ġobtained Ġfrom Ġthe Ġtopic Ġmodel Ġor Ġthe Ġpre - defined Ġlabels Ġfurther Ġimproves Ġthe ĠUTC NN Ġmodel . ĠIn Ġaddition , Ġcomment Ġinformation Ġprovides Ġadditional Ġclues Ġfor Ġstance Ġclassification . ĠWe Ġhave Ġshown Ġthat ĠUTC NN Ġachieves Ġpromising Ġand Ġbalanced Ġresults . ĠIn Ġthe Ġfuture Ġwe Ġplan Ġto Ġexplore Ġthe Ġeffectiveness Ġof Ġthe ĠUTC NN Ġuser Ġembed d ings Ġfor Ġauthor Ġstance Ġclassification . </s> Acknowled gements </s> Research Ġof Ġthis Ġpaper Ġwas Ġpartially Ġsupported Ġby ĠMinistry Ġof ĠScience Ġand ĠTechnology , ĠTaiwan , Ġunder Ġthe Ġcontract ĠM OST Ġ104 - 22 21 - E - 001 - 024 - MY 2 .