Facebook status update messages
EXTRACTIVE
Facebook status update messages
EXTRACTIVE
We created our own causal explanation dataset by collecting 3,268 random Facebook status update messages. Three well-trained annotators manually labeled whether or not each message contains the causal explanation and obtained 1,598 causality messages with substantial agreement ( $\kappa =0.61$ ). We used the majority vote for our gold standard. Then, on each causality message, annotators identified which text spans are causal explanations.
We created our own causal explanation dataset by collecting 3,268 random Facebook status update messages. Three well-trained annotators manually labeled whether or not each message contains the causal explanation and obtained 1,598 causality messages with substantial agreement ( $\kappa =0.61$ ). We used the majority vote for our gold standard. Then, on each causality message, annotators identified which text spans are causal explanations.
1809.01202
Introduction </s> Ex plan ations Ġof Ġhappen ings Ġin Ġone 's Ġlife , Ġcausal Ġexplanations , Ġare Ġan Ġimportant Ġtopic Ġof Ġstudy Ġin Ġsocial , Ġpsychological , Ġeconomic , Ġand Ġbehavioral Ġsciences . ĠFor Ġexample , Ġpsychologists Ġhave Ġanalyzed Ġpeople 's Ġcausal Ġexplanatory Ġstyle ĠB IB REF 0 Ġand Ġfound Ġstrong Ġnegative Ġrelationships Ġwith Ġdepression , Ġpass ivity , Ġand Ġhostility , Ġas Ġwell Ġas Ġpositive Ġrelationships Ġwith Ġlife Ġsatisfaction , Ġquality Ġof Ġlife , Ġand Ġlength Ġof Ġlife ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 0 Ġ. </s> To Ġhelp Ġunderstand Ġthe Ġsignificance Ġof Ġcausal Ġexplanations , Ġconsider Ġhow Ġthey Ġare Ġapplied Ġto Ġmeasuring Ġoptimism Ġ( and Ġits Ġcon verse , Ġpessim ism ) ĠB IB REF 0 Ġ. ĠFor Ġexample , Ġin ĠâĢ ľ My Ġparser Ġfailed Ġbecause ĠI Ġalways Ġhave Ġbugs . âĢ Ŀ , Ġthe Ġemphasized Ġtext Ġspan Ġis Ġconsidered Ġa Ġcausal Ġexplanation Ġwhich Ġindicates Ġpessimistic Ġpersonality ĠâĢĵ Ġa Ġnegative Ġevent Ġwhere Ġthe Ġauthor Ġbelieves Ġthe Ġcause Ġis Ġpervasive . ĠHowever , Ġin ĠâĢ ľ My Ġparser Ġfailed Ġbecause ĠI Ġbarely Ġworked Ġon Ġthe Ġcode . âĢ Ŀ , Ġthe Ġexplanation Ġwould Ġbe Ġconsidered Ġa Ġsignal Ġof Ġoptimistic Ġpersonality ĠâĢĵ Ġa Ġnegative Ġevent Ġfor Ġwhich Ġthe Ġcause Ġis Ġbelieved Ġto Ġbe Ġshort - lived . </s> Language - based Ġmodels Ġwhich Ġcan Ġdetect Ġcausal Ġexplanations Ġfrom Ġeveryday Ġsocial Ġmedia Ġlanguage Ġcan Ġbe Ġused Ġfor Ġmore Ġthan Ġautom ating Ġoptimism Ġdetection . ĠLanguage - based Ġassessments Ġwould Ġenable Ġother Ġlarge - scale Ġdownstream Ġtasks : Ġtracking Ġprevailing Ġcausal Ġbeliefs Ġ( e . g ., Ġabout Ġclimate Ġchange Ġor Ġautism ), Ġbetter Ġextracting Ġprocess Ġknowledge Ġfrom Ġnon - fiction Ġ( e . g ., Ġgravity Ġcauses Ġobjects Ġto Ġmove Ġtoward Ġone Ġanother ), Ġor Ġdetecting Ġattribution Ġof Ġblame Ġor Ġpraise Ġin Ġproduct Ġor Ġservice Ġreviews Ġ( âĢ ľ I Ġloved Ġthis Ġrestaurant Ġbecause Ġthe Ġfish Ġwas Ġcooked Ġto Ġperfection âĢ Ŀ ). </s> In Ġthis Ġpaper , Ġwe Ġintroduce Ġcausal Ġexplanation Ġanalysis Ġand Ġits Ġsubt asks Ġof Ġdetecting Ġthe Ġpresence Ġof Ġcaus ality Ġ( ca us ality Ġprediction ) Ġand Ġidentifying Ġexplanatory Ġphrases Ġ( ca usal Ġexplanation Ġidentification ). ĠThere Ġare Ġmany Ġchallenges Ġto Ġachieving Ġthese Ġtask . ĠFirst , Ġthe Ġun gram matical Ġtexts Ġin Ġsocial Ġmedia Ġincur Ġpoor Ġsynt actic Ġparsing Ġresults Ġwhich Ġdrastically Ġaffect Ġthe Ġperformance Ġof Ġdiscourse Ġrelation Ġparsing Ġpipelines Ġ. ĠMany Ġcausal Ġrelations Ġare Ġimplicit Ġand Ġdo Ġnot Ġcontain Ġany Ġdiscourse Ġmarkers Ġ( e . g ., Ġ` because '). ĠFurther , ĠExplicit Ġcausal Ġrelations Ġare Ġalso Ġmore Ġdifficult Ġin Ġsocial Ġmedia Ġdue Ġto Ġthe Ġabundance Ġof Ġabbrevi ations Ġand Ġvariations Ġof Ġdiscourse Ġconnect ives Ġ( e . g ., Ġ` c uz ' Ġand Ġ` bc uz '). </s> Prev ailing Ġapproaches Ġfor Ġsocial Ġmedia Ġanalyses , Ġutilizing Ġtraditional Ġlinear Ġmodels Ġor Ġbag Ġof Ġwords Ġmodels Ġ( e . g ., ĠS VM Ġtrained Ġwith Ġn - gram , Ġpart - of - speech Ġ( POS ) Ġtags , Ġor Ġlex icon - based Ġfeatures ) Ġalone Ġdo Ġnot Ġseem Ġappropriate Ġfor Ġthis Ġtask Ġsince Ġthey Ġsimply Ġcannot Ġsegment Ġthe Ġtext Ġinto Ġmeaningful Ġdiscourse Ġunits Ġor Ġdiscourse Ġarguments Ġsuch Ġas Ġclauses Ġor Ġsentences Ġrather Ġthan Ġrandom Ġconsecutive Ġtoken Ġsequences Ġor Ġspecific Ġword Ġtokens . ĠEven Ġwhen Ġthe Ġdiscourse Ġunits Ġare Ġclear , Ġpars ers Ġmay Ġstill Ġfail Ġto Ġaccurately Ġidentify Ġdiscourse Ġrelations Ġsince Ġthe Ġcontent Ġof Ġsocial Ġmedia Ġis Ġquite Ġdifferent Ġthan Ġthat Ġof Ġnews wire Ġwhich Ġis Ġtypically Ġused Ġfor Ġdiscourse Ġparsing . </s> In Ġorder Ġto Ġovercome Ġthese Ġdifficulties Ġof Ġdiscourse Ġrelation Ġparsing Ġin Ġsocial Ġmedia , Ġwe Ġsimplify Ġand Ġminimize Ġthe Ġuse Ġof Ġsynt actic Ġparsing Ġresults Ġand Ġcapture Ġrelations Ġbetween Ġdiscourse Ġarguments , Ġand Ġinvestigate Ġthe Ġuse Ġof Ġa Ġrecursive Ġneural Ġnetwork Ġmodel Ġ( R NN ). ĠRecent Ġwork Ġhas Ġshown Ġthat ĠR NN s Ġare Ġeffective Ġfor Ġutilizing Ġdiscourse Ġstructures Ġfor Ġtheir Ġdownstream Ġtasks ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, Ġbut Ġthey Ġhave Ġyet Ġto Ġbe Ġdirectly Ġused Ġfor Ġdiscourse Ġrelation Ġprediction Ġin Ġsocial Ġmedia . ĠWe Ġevaluated Ġour Ġmodel Ġby Ġcomparing Ġit Ġto Ġoff - the - she lf Ġend - to - end Ġdiscourse Ġrelation Ġpars ers Ġand Ġtraditional Ġmodels . ĠWe Ġfound Ġthat Ġthe ĠS VM Ġand Ġrandom Ġforest Ġclass ifiers Ġwork Ġbetter Ġthan Ġthe ĠL ST M Ġclass ifier Ġfor Ġthe Ġcaus ality Ġdetection , Ġwhile Ġthe ĠL ST M Ġclass ifier Ġoutper forms Ġother Ġmodels Ġfor Ġidentifying Ġcausal Ġexplanation . </s> The Ġcontributions Ġof Ġthis Ġwork Ġinclude : Ġ( 1 ) Ġthe Ġproposal Ġof Ġmodels Ġfor Ġboth Ġ( a ) Ġcaus ality Ġprediction Ġand Ġ( b ) Ġcausal Ġexplanation Ġidentification , Ġ( 2 ) Ġthe Ġextensive Ġevaluation Ġof Ġa Ġvariety Ġof Ġmodels Ġfrom Ġsocial Ġmedia Ġclassification Ġmodels Ġand Ġdiscourse Ġrelation Ġpars ers Ġto ĠR NN - based Ġapplication Ġmodels , Ġdemonstrating Ġthat Ġfeature - based Ġmodels Ġwork Ġbest Ġfor Ġcaus ality Ġprediction Ġwhile ĠR NN s Ġare Ġsuperior Ġfor Ġthe Ġmore Ġdifficult Ġtask Ġof Ġcausal Ġexplanation Ġidentification , Ġ( 3 ) Ġperformance Ġanalysis Ġon Ġarchitectural Ġdifferences Ġof Ġthe Ġpipeline Ġand Ġthe Ġclass ifier Ġstructures , Ġ( 4 ) Ġexploration Ġof Ġthe Ġapplications Ġof Ġcausal Ġexplanation Ġto Ġdownstream Ġtasks , Ġand Ġ( 5 ) Ġrelease Ġof Ġa Ġnovel , Ġanonym ized Ġcaus ality ĠFacebook Ġdataset Ġalong Ġwith Ġour Ġcaus ality Ġprediction Ġand Ġcausal Ġexplanation Ġidentification Ġmodels . </s> Related ĠWork </s> Ident ifying Ġcausal Ġexplanations Ġin Ġdocuments Ġcan Ġbe Ġviewed Ġas Ġdiscourse Ġrelation Ġparsing . ĠThe ĠPenn ĠDisc ourse ĠTree bank Ġ( PD TB ) ĠB IB REF 7 Ġhas Ġa Ġ` Cause ' Ġand Ġ` Pr ag matic ĠCause ' Ġdiscourse Ġtype Ġunder Ġa Ġgeneral Ġ` Cont ing ency ' Ġclass Ġand ĠR het orical ĠStructure ĠTheory Ġ( R ST ) ĠB IB REF 8 Ġhas Ġa Ġ` Relations Ġof ĠCause '. ĠIn Ġmost Ġcases , Ġthe Ġdevelopment Ġof Ġdiscourse Ġpars ers Ġhas Ġtaken Ġplace Ġin - domain , Ġwhere Ġresearchers Ġhave Ġused Ġthe Ġexisting Ġannotations Ġof Ġdiscourse Ġarguments Ġin Ġnews wire Ġtext Ġ( e . g . ĠWall ĠStreet ĠJournal ) Ġfrom Ġthe Ġdiscourse Ġtree bank Ġand Ġfocused Ġon Ġexploring Ġdifferent Ġfeatures Ġand Ġoptimizing Ġvarious Ġtypes Ġof Ġmodels Ġfor Ġpredicting Ġrelations ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġ. ĠIn Ġorder Ġto Ġfurther Ġdevelop Ġautomated Ġsystems , Ġresearchers Ġhave Ġproposed Ġend - to - end Ġdiscourse Ġrelation Ġpars ers , Ġbuilding Ġmodels Ġwhich Ġare Ġtrained Ġand Ġevaluated Ġon Ġthe Ġannot ated ĠPD TB Ġand ĠR ST ĠDisc ourse ĠTree bank Ġ( R ST ĠDT ). ĠThese Ġcorpor a Ġconsist Ġof Ġdocuments Ġfrom ĠWall ĠStreet ĠJournal Ġ( WS J ) Ġwhich Ġare Ġmuch Ġmore Ġwell - organized Ġand Ġgram matical Ġthan Ġsocial Ġmedia Ġtexts ĠB IB REF 12 Ġ, ĠB IB REF 13 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ. </s> Only Ġa Ġfew Ġworks Ġhave Ġattempted Ġto Ġparse Ġdiscourse Ġrelations Ġfor Ġout - of - domain Ġproblems Ġsuch Ġas Ġtext Ġcategor izations Ġon Ġsocial Ġmedia Ġtexts ; ĠJi Ġand ĠB hat ia Ġused Ġmodels Ġwhich Ġare Ġpret rained Ġwith ĠR ST ĠDT Ġfor Ġbuilding Ġdiscourse Ġstructures Ġfrom Ġmovie Ġreviews , Ġand ĠSon Ġadapted Ġthe ĠPD TB Ġdiscourse Ġrelation Ġparsing Ġapproach Ġfor Ġcapturing Ġcounter fact ual Ġcondition als Ġfrom Ġtweets ĠB IB REF 4 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 16 Ġ. ĠThese Ġworks Ġhad Ġsubstantial Ġdifferences Ġto Ġwhat Ġpropose Ġin Ġthis Ġpaper . ĠFirst , ĠJi Ġand ĠB hat ia Ġused Ġa Ġpret rained Ġmodel Ġ( not Ġfully Ġoptimal Ġfor Ġsome Ġparts Ġof Ġthe Ġgiven Ġtask ) Ġin Ġtheir Ġpipeline ; ĠJi 's Ġmodel Ġperformed Ġworse Ġthan Ġthe Ġbaseline Ġon Ġthe Ġcategor ization Ġof Ġlegislative Ġbills , Ġwhich Ġis Ġthought Ġto Ġbe Ġdue Ġto Ġlegislative Ġdiscourse Ġstructures Ġdiffering Ġfrom Ġthose Ġof Ġthe Ġtraining Ġset Ġ( WS J Ġcorpus ). ĠB hat ia Ġalso Ġused Ġa Ġpret rained Ġmodel Ġfinding Ġthat Ġutilizing Ġdiscourse Ġrelation Ġfeatures Ġdid Ġnot Ġboost Ġaccuracy ĠB IB REF 4 Ġ, ĠB IB REF 3 Ġ. ĠBoth ĠB hat ia Ġand ĠSon Ġused Ġmanual Ġschemes Ġwhich Ġmay Ġlimit Ġthe Ġcoverage Ġof Ġcertain Ġtypes Ġof Ġpositive Ġsamples âĢĵ ĠB hat ia Ġused Ġa Ġhand - crafted Ġschema Ġfor Ġweight ing Ġdiscourse Ġstructures Ġfor Ġthe Ġneural Ġnetwork Ġmodel Ġand ĠSon Ġmanually Ġdeveloped Ġseven Ġsurface Ġforms Ġof Ġcounter fact ual Ġthinking Ġfor Ġthe Ġrule - based Ġsystem ĠB IB REF 4 Ġ, ĠB IB REF 16 Ġ. ĠWe Ġuse Ġsocial - media - specific Ġfeatures Ġfrom Ġpret rained Ġmodels Ġwhich Ġare Ġdirectly Ġtrained Ġon Ġtweets Ġand Ġwe Ġavoid Ġany Ġhand - crafted Ġrules Ġexcept Ġfor Ġthose Ġincluded Ġin Ġthe Ġexisting Ġdiscourse Ġargument Ġextraction Ġtechniques . </s> The Ġautomated Ġsystems Ġfor Ġdiscourse Ġrelation Ġparsing Ġinvolve Ġmultiple Ġsubt asks Ġfrom Ġsegment ing Ġthe Ġwhole Ġtext Ġinto Ġdiscourse Ġarguments Ġto Ġclass ifying Ġdiscourse Ġrelations Ġbetween Ġthe Ġarguments . ĠPast Ġresearch Ġhas Ġfound Ġthat Ġdifferent Ġtypes Ġof Ġmodels Ġand Ġfeatures Ġyield Ġvarying Ġperformance Ġfor Ġeach Ġsubt ask . ĠSome Ġhave Ġoptimized Ġmodels Ġfor Ġdiscourse Ġrelation Ġclassification Ġ( i . e . Ġgiven Ġa Ġdocument Ġindicating Ġif Ġthe Ġrelation Ġexisting ) Ġwithout Ġdiscourse Ġargument Ġparsing Ġusing Ġmodels Ġsuch Ġas ĠNa ive - Bay es Ġor ĠSV Ms , Ġachieve Ġrelatively Ġstronger Ġaccur acies Ġbut Ġa Ġsimpler Ġtask Ġthan Ġthat Ġassociated Ġwith Ġdiscourse Ġarguments ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġ, ĠB IB REF 9 Ġ. ĠResearchers Ġwho , Ġinstead , Ġtried Ġto Ġbuild Ġthe Ġend - to - end Ġparsing Ġpipelines Ġconsidered Ġa Ġwider Ġrange Ġof Ġapproaches Ġincluding Ġsequence Ġmodels Ġand ĠR NN s ĠB IB REF 12 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 17 Ġ. ĠParticularly , Ġwhen Ġthey Ġtried Ġto Ġutilize Ġthe Ġdiscourse Ġstructures Ġfor Ġout - domain Ġapplications , Ġthey Ġused ĠR NN - based Ġmodels Ġand Ġfound Ġthat Ġthose Ġmodels Ġare Ġadvantageous Ġfor Ġtheir Ġdownstream Ġtasks ĠB IB REF 4 Ġ, ĠB IB REF 3 Ġ. </s> In Ġour Ġcase , Ġfor Ġidentifying Ġcausal Ġexplanations Ġfrom Ġsocial Ġmedia Ġusing Ġdiscourse Ġstructure , Ġwe Ġbuild Ġan ĠR NN - based Ġmodel Ġfor Ġits Ġstructural Ġeffectiveness Ġin Ġthis Ġtask Ġ( see Ġdetails Ġin Ġsection ĠUID 13 Ġ). ĠHowever , Ġwe Ġalso Ġnote Ġthat Ġsimpler Ġmodels Ġsuch Ġas ĠSV Ms Ġand Ġlog istic Ġregression Ġobtained Ġthe Ġstate - of - the - art Ġperformances Ġfor Ġtext Ġcategor ization Ġtasks Ġin Ġsocial Ġmedia ĠB IB REF 18 Ġ, ĠB IB REF 19 Ġ, Ġso Ġwe Ġbuild Ġrelatively Ġsimple Ġmodels Ġwith Ġdifferent Ġproperties Ġfor Ġeach Ġstage Ġof Ġthe Ġfull Ġpipeline Ġof Ġour Ġparser . </s> Methods </s> We Ġbuild Ġour Ġmodel Ġbased Ġon ĠPD TB - style Ġdiscourse Ġrelation Ġparsing Ġsince ĠPD TB Ġhas Ġa Ġrelatively Ġsimpler Ġtext Ġsegment ation Ġmethod ; Ġfor Ġexplicit Ġdiscourse Ġrelations , Ġit Ġfinds Ġthe Ġpresence Ġof Ġdiscourse Ġconnect ives Ġwithin Ġa Ġdocument Ġand Ġextracts Ġdiscourse Ġarguments Ġwhich Ġparam et ri ze Ġthe Ġconnect ive Ġwhile Ġfor Ġimplicit Ġrelations , Ġit Ġconsiders Ġall Ġadjacent Ġsentences Ġas Ġcandidate Ġdiscourse Ġarguments . </s> Dat as et </s> We Ġcreated Ġour Ġown Ġcausal Ġexplanation Ġdataset Ġby Ġcollecting Ġ3 , 268 Ġrandom ĠFacebook Ġstatus Ġupdate Ġmessages . ĠThree Ġwell - trained Ġannot ators Ġmanually Ġlabeled Ġwhether Ġor Ġnot Ġeach Ġmessage Ġcontains Ġthe Ġcausal Ġexplanation Ġand Ġobtained Ġ1 , 598 Ġcaus ality Ġmessages Ġwith Ġsubstantial Ġagreement Ġ( Ġ$\ k appa Ġ= 0 . 61 $ Ġ). ĠWe Ġused Ġthe Ġmajority Ġvote Ġfor Ġour Ġgold Ġstandard . ĠThen , Ġon Ġeach Ġcaus ality Ġmessage , Ġannot ators Ġidentified Ġwhich Ġtext Ġspans Ġare Ġcausal Ġexplanations . </s> For Ġeach Ġtask , Ġwe Ġused Ġ80 % Ġof Ġthe Ġdataset Ġfor Ġtraining Ġour Ġmodel Ġand Ġ10 % Ġfor Ġtuning Ġthe Ġhyper param eters Ġof Ġour Ġmodels . ĠFinally , Ġwe Ġevaluated Ġall Ġof Ġour Ġmodels Ġon Ġthe Ġremaining Ġ10 % Ġ( Table Ġ1 Ġand ĠTable Ġ2 Ġ). ĠFor Ġcausal Ġexplanation Ġdetection Ġtask , Ġwe Ġextracted Ġdiscourse Ġarguments Ġusing Ġour Ġparser Ġand Ġselected Ġdiscourse Ġarguments Ġwhich Ġmost Ġcover Ġthe Ġannot ated Ġcausal Ġexplanation Ġtext Ġspan Ġas Ġour Ġgold Ġstandard . </s> Model </s> We Ġbuild Ġtwo Ġtypes Ġof Ġmodels . ĠFirst , Ġwe Ġdevelop Ġfeature - based Ġmodels Ġwhich Ġutilize Ġfeatures Ġof Ġthe Ġsuccessful Ġmodels Ġin Ġsocial Ġmedia Ġanalysis Ġand Ġcausal Ġrelation Ġdiscourse Ġparsing . ĠThen , Ġwe Ġbuild Ġa Ġrecursive Ġneural Ġnetwork Ġmodel Ġwhich Ġuses Ġdistributed Ġrepresentation Ġof Ġdiscourse Ġarguments Ġas Ġthis Ġapproach Ġcan Ġeven Ġcapture Ġlatent Ġproperties Ġof Ġcausal Ġrelations Ġwhich Ġmay Ġexist Ġbetween Ġdistant Ġdiscourse Ġarguments . ĠWe Ġspecifically Ġselected Ġbid irection al ĠL ST M Ġsince Ġthe Ġmodel Ġwith Ġthe Ġdiscourse Ġdistribution al Ġstructure Ġbuilt Ġin Ġthis Ġform Ġoutper formed Ġthe Ġtraditional Ġmodels Ġin Ġsimilar ĠN LP Ġdownstream Ġtasks ĠB IB REF 3 Ġ. </s> As Ġthe Ġfirst Ġstep Ġof Ġour Ġpipeline , Ġwe Ġuse ĠTwe eb o Ġparser ĠB IB REF 20 Ġto Ġextract Ġsynt actic Ġfeatures Ġfrom Ġmessages . ĠThen , Ġwe Ġdem arc ate Ġsentences Ġusing Ġpunct uation Ġ( ` ,' ) Ġtag Ġand Ġperiods . ĠAmong Ġthose Ġsentences , Ġwe Ġfind Ġdiscourse Ġconnect ives Ġdefined Ġin ĠPD TB Ġannotation Ġalong Ġwith Ġa ĠTweet ĠPOS Ġtag Ġfor Ġconjunction Ġwords Ġwhich Ġcan Ġalso Ġbe Ġa Ġdiscourse Ġmarker . ĠIn Ġorder Ġto Ġdecide Ġwhether Ġthese Ġconnect ives Ġare Ġreally Ġdiscourse Ġconnect ives Ġ( e . g ., ĠI Ġwent Ġhome , Ġbut Ġhe Ġstayed ) Ġas Ġopposed Ġto Ġsimple Ġconnections Ġof Ġtwo Ġwords Ġ( I Ġlike Ġapple Ġand Ġbanana ) Ġwe Ġsee Ġif Ġverb Ġphrases Ġexist Ġbefore Ġand Ġafter Ġthe Ġconnect ive Ġby Ġusing Ġdependency Ġparsing Ġresults . ĠAlthough Ġdiscourse Ġconnect ive Ġdis amb ig uation Ġis Ġa Ġcomplicated Ġtask Ġwhich Ġcan Ġbe Ġmuch Ġimproved Ġby Ġsynt actic Ġfeatures ĠB IB REF 21 Ġ, Ġwe Ġtry Ġto Ġminimize Ġeffects Ġof Ġsynt actic Ġparsing Ġand Ġsimplify Ġit Ġsince Ġit Ġis Ġhighly Ġerror - prone Ġin Ġsocial Ġmedia . ĠFinally , Ġaccording Ġto Ġvisual Ġinspection , Ġem oj is Ġ( ` E ' Ġtag ) Ġare Ġcrucial Ġfor Ġdiscourse Ġrelation Ġin Ġsocial Ġmedia Ġso Ġwe Ġtake Ġthem Ġas Ġseparate Ġdiscourse Ġarguments Ġ( e . g ., in ĠâĢ ľ My Ġtest Ġresult ... Ġ:( âĢ Ŀ Ġthe Ġsad Ġfeeling Ġis Ġcaused Ġby Ġthe Ġtest Ġresult , Ġbut Ġit Ġcannot Ġbe Ġcaptured Ġby Ġplain Ġword Ġtokens ). </s> We Ġtrained Ġa Ġlinear ĠS VM , Ġan Ġr bf ĠS VM , Ġand Ġa Ġrandom Ġforest Ġwith ĠN - gram , Ġchar ater ĠN - gram , Ġand Ġtweet ĠPOS Ġtags , Ġsentiment Ġtags , Ġaverage Ġword Ġlengths Ġand Ġword Ġcounts Ġfrom Ġeach Ġmessage Ġas Ġthey Ġhave Ġa Ġpivotal Ġrole Ġin Ġthe Ġmodels Ġfor Ġmany ĠN LP Ġdownstream Ġtasks Ġin Ġsocial Ġmedia ĠB IB REF 19 Ġ, ĠB IB REF 18 Ġ. ĠIn Ġaddition Ġto Ġthese Ġfeatures , Ġwe Ġalso Ġextracted ĠFirst - Last , ĠFirst 3 Ġfeatures Ġand ĠWord ĠP airs Ġfrom Ġevery Ġadjacent Ġpair Ġof Ġdiscourse Ġarguments Ġsince Ġthese Ġfeatures Ġwere Ġmost Ġhelpful Ġfor Ġcausal Ġrelation Ġprediction ĠB IB REF 9 Ġ. ĠFirst - Last , ĠFirst 3 Ġfeatures Ġare Ġfirst Ġand Ġlast Ġword Ġand Ġfirst Ġthree Ġwords Ġof Ġtwo Ġdiscourse Ġarguments Ġof Ġthe Ġrelation , Ġand ĠWord ĠP airs Ġare Ġthe Ġcross Ġproduct Ġof Ġwords Ġof Ġthose Ġdiscourse Ġarguments . ĠThese Ġtwo Ġfeatures Ġenable Ġour Ġmodel Ġto Ġcapture Ġinteraction Ġbetween Ġtwo Ġdiscourse Ġarguments . ĠB IB REF 9 Ġreported Ġthat Ġthese Ġtwo Ġfeatures Ġalong Ġwith Ġverbs , Ġmod ality , Ġcontext , Ġand Ġpol arity Ġ( which Ġcan Ġbe Ġcaptured Ġby ĠN - gram s , Ġsentiment Ġtags Ġand ĠPOS Ġtags Ġin Ġour Ġprevious Ġfeatures ) Ġobtained Ġthe Ġbest Ġperformance Ġfor Ġpredicting ĠCont ing ency Ġclass Ġto Ġwhich Ġcaus ality Ġbelongs . </s> We Ġload Ġthe ĠGL O VE Ġword Ġembed ding ĠB IB REF 22 Ġtrained Ġin ĠTwitter Ġfor Ġeach Ġtoken Ġof Ġextracted Ġdiscourse Ġarguments Ġfrom Ġmessages . ĠFor Ġthe Ġdistribution al Ġrepresentation Ġof Ġdiscourse Ġarguments , Ġwe Ġrun Ġa ĠWord - level ĠL ST M Ġon Ġthe Ġwords ' Ġembed d ings Ġwithin Ġeach Ġdiscourse Ġargument Ġand Ġconc aten ate Ġlast Ġhidden Ġstate Ġvectors Ġof Ġforward ĠL ST M Ġ( Ġ$\ over right arrow { h } $ Ġ) Ġand Ġbackward ĠL ST M Ġ( Ġ$\ over left arrow { h } $ Ġ) Ġwhich Ġis Ġsuggested Ġby ĠB IB REF 3 Ġ( Ġ$ DA Ġ= Ġ[ \ over right arrow { h }; \ over left arrow { h } ] $ Ġ). ĠThen , Ġwe Ġfeed Ġthe Ġsequence Ġof Ġthe Ġvector Ġrepresentation Ġof Ġdiscourse Ġarguments Ġto Ġthe ĠDisc ourse - argument - level ĠL ST M Ġ( DA - level ĠL ST M ) Ġto Ġmake Ġa Ġfinal Ġprediction Ġwith Ġlog Ġsoft max Ġfunction . ĠWith Ġthis Ġstructure , Ġthe Ġmodel Ġcan Ġlearn Ġthe Ġrepresentation Ġof Ġinteraction Ġof Ġtokens Ġinside Ġeach Ġdiscourse Ġargument , Ġthen Ġcapture Ġdiscourse Ġrelations Ġacross Ġall Ġof Ġthe Ġdiscourse Ġarguments Ġin Ġeach Ġmessage Ġ( Figure Ġ2 Ġ). ĠIn Ġorder Ġto Ġprevent Ġthe Ġover fitting , Ġwe Ġadded Ġa Ġdrop out Ġlayer Ġbetween Ġthe ĠWord - level ĠL ST M Ġand Ġthe ĠDA - level ĠL ST M Ġlayer . </s> We Ġalso Ġexplore Ġsubs ets Ġof Ġthe Ġfull ĠR NN Ġarchitecture , Ġspecifically Ġwith Ġone Ġof Ġthe Ġtwo ĠL ST M Ġlayers Ġremoved . ĠIn Ġthe Ġfirst Ġmodel Ġvariant , Ġwe Ġdirectly Ġinput Ġall Ġword Ġembed d ings Ġof Ġa Ġwhole Ġmessage Ġto Ġa ĠBi L ST M Ġlayer Ġand Ġmake Ġprediction Ġ( Word ĠL ST M ) Ġwithout Ġthe Ġhelp Ġof Ġthe Ġdistribution al Ġvector Ġrepresentations Ġof Ġdiscourse Ġarguments . ĠIn Ġthe Ġsecond Ġmodel Ġvariant , Ġwe Ġtake Ġthe Ġaverage Ġof Ġall Ġword Ġembed d ings Ġof Ġeach Ġdiscourse Ġargument Ġ( Ġ$ DA _ k = \ frac { 1 }{ N _ k } Ġ\ sum Ġ_ { i = 1 } ^{ N _ k } W _{ i } $ Ġ), Ġand Ġuse Ġthem Ġas Ġinputs Ġto Ġa ĠBi L ST M Ġlayer Ġ( DA ĠAVG ĠL ST M ) Ġas Ġthe Ġaverage Ġvector Ġof Ġembed d ings Ġwere Ġquite Ġeffective Ġfor Ġrepresenting Ġthe Ġwhole Ġsequence ĠB IB REF 3 Ġ, ĠB IB REF 5 Ġ. ĠAs Ġwith Ġthe Ġfull Ġarchitectures , Ġfor ĠCP Ġboth Ġof Ġthese Ġvariants Ġends Ġwith Ġa Ġmany - to - one Ġclassification Ġper Ġmessage , Ġwhile Ġthe ĠCE I Ġmodel Ġends Ġwith Ġa Ġsequence Ġof Ġclass ifications . </s> Exper iment </s> We Ġexplored Ġthree Ġtypes Ġof Ġmodels Ġ( RB F ĠS VM , ĠLinear ĠS VM , Ġand ĠRandom ĠForest ĠClass ifier ) Ġwhich Ġhave Ġpreviously Ġbeen Ġshown Ġempir ically Ġuseful Ġfor Ġthe Ġlanguage Ġanalysis Ġin Ġsocial Ġmedia . ĠWe Ġfiltered Ġout Ġlow Ġfrequency ĠWord ĠP airs Ġfeatures Ġas Ġthey Ġtend Ġto Ġbe Ġnoisy Ġand Ġsparse ĠB IB REF 9 Ġ. ĠThen , Ġwe Ġconducted Ġun ivariate Ġfeature Ġselection Ġto Ġrestrict Ġall Ġremaining Ġfeatures Ġto Ġthose Ġshowing Ġat Ġleast Ġa Ġsmall Ġrelationship Ġwith Ġthe Ġoutcome . ĠSpecifically , Ġwe Ġkeep Ġall Ġfeatures Ġpassing Ġa Ġfamily - wise Ġerror Ġrate Ġof Ġ$\ alpha Ġ= Ġ60 $ Ġwith Ġthe Ġgiven Ġoutcome . ĠAfter Ġcomparing Ġthe Ġperformance Ġof Ġthe Ġoptimized Ġversion Ġof Ġeach Ġmodel , Ġwe Ġalso Ġconducted Ġa Ġfeature Ġab lation Ġtest Ġon Ġthe Ġbest Ġmodel Ġin Ġorder Ġto Ġsee Ġhow Ġmuch Ġeach Ġfeature Ġcontributes Ġto Ġthe Ġcaus ality Ġprediction . </s> We Ġused Ġbid irection al ĠL ST Ms Ġfor Ġcaus ality Ġclassification Ġand Ġcausal Ġexplanation Ġidentification Ġsince Ġthe Ġdiscourse Ġarguments Ġfor Ġcausal Ġexplanation Ġcan Ġshow Ġup Ġeither Ġbefore Ġand Ġafter Ġthe Ġeffected Ġevents Ġor Ġresults Ġand Ġwe Ġwant Ġour Ġmodel Ġto Ġbe Ġoptimized Ġfor Ġboth Ġcases . ĠHowever , Ġthere Ġis Ġa Ġrisk Ġof Ġover fitting Ġdue Ġto Ġthe Ġdataset Ġwhich Ġis Ġrelatively Ġsmall Ġfor Ġthe Ġhigh Ġcomplexity Ġof Ġthe Ġmodel , Ġso Ġwe Ġadded Ġa Ġdrop out Ġlayer Ġ( p = 0 . 3 ) Ġbetween Ġthe ĠWord - level ĠL ST M Ġand Ġthe ĠDA - level ĠL ST M . </s> For Ġtuning Ġour Ġmodel , Ġwe Ġexplore Ġthe Ġdimension ality Ġof Ġword Ġvector Ġand ĠL ST M Ġhidden Ġstate Ġvectors Ġof Ġdiscourse Ġarguments Ġof Ġ25 , Ġ50 , Ġ100 , Ġand Ġ200 Ġas Ġpret rained ĠGL O VE Ġvectors Ġwere Ġtrained Ġin Ġthis Ġsetting . ĠFor Ġoptimization , Ġwe Ġused ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ) Ġand ĠAdam ĠB IB REF 23 Ġwith Ġlearning Ġrates Ġ0 . 01 Ġand Ġ0 . 001 . </s> We Ġignore Ġmissing Ġword Ġembed d ings Ġbecause Ġour Ġdataset Ġis Ġquite Ġsmall Ġfor Ġret raining Ġnew Ġword Ġembed d ings . ĠHowever , Ġif Ġembed d ings Ġare Ġextracted Ġas Ġseparate Ġdiscourse Ġarguments , Ġwe Ġused Ġthe Ġaverage Ġof Ġall Ġvectors Ġof Ġall Ġdiscourse Ġarguments Ġin Ġthat Ġmessage . ĠAverage Ġembed d ings Ġhave Ġperformed Ġwell Ġfor Ġrepresenting Ġtext Ġsequences Ġin Ġother Ġtasks ĠB IB REF 5 Ġ. </s> We Ġfirst Ġuse Ġstate - of - the - art ĠPD TB Ġtag gers Ġfor Ġour Ġbaseline ĠB IB REF 13 Ġ, ĠB IB REF 12 Ġfor Ġthe Ġevaluation Ġof Ġthe Ġcaus ality Ġprediction Ġof Ġour Ġmodels Ġ( ĠB IB REF 12 Ġrequires Ġsentences Ġextracted Ġfrom Ġthe Ġtext Ġas Ġits Ġinput , Ġso Ġwe Ġused Ġour Ġparser Ġto Ġextract Ġsentences Ġfrom Ġthe Ġmessage ). ĠThen , Ġwe Ġcompare Ġhow Ġmodels Ġwork Ġfor Ġeach Ġtask Ġand Ġdis assembled Ġthem Ġto Ġinspect Ġhow Ġeach Ġpart Ġof Ġthe Ġmodels Ġcan Ġaffect Ġtheir Ġfinal Ġprediction Ġperformances . ĠWe Ġconducted ĠMcN em ar 's Ġtest Ġto Ġdetermine Ġwhether Ġthe Ġperformance Ġdifferences Ġare Ġstatistically Ġsignificant Ġat Ġ$ p Ġ< Ġ. 05 $ Ġ. </s> Results </s> We Ġinvestigated Ġvarious Ġmodels Ġfor Ġboth Ġcaus ality Ġdetection Ġand Ġexplanation Ġidentification . ĠBased Ġon Ġtheir Ġperformances Ġon Ġthe Ġtask , Ġwe Ġanalyzed Ġthe Ġrelationships Ġbetween Ġthe Ġtypes Ġof Ġmodels Ġand Ġthe Ġtasks , Ġand Ġscrutin ized Ġfurther Ġfor Ġthe Ġbest Ġperforming Ġmodels . ĠFor Ġperformance Ġanalysis , Ġwe Ġreported Ġweighted ĠF 1 Ġof Ġclasses . </s> C aus ality ĠPrediction </s> In Ġorder Ġto Ġclassify Ġwhether Ġa Ġmessage Ġcontains Ġcausal Ġrelation , Ġwe Ġcompared Ġoff - the - she lf ĠPD TB Ġpars ers , Ġlinear ĠS VM , ĠRB F ĠS VM , ĠRandom Ġforest Ġand ĠL ST M Ġclass ifiers . ĠThe Ġoff - the - she lf Ġpars ers Ġachieved Ġthe Ġlowest Ġaccur acies Ġ( ĠB IB REF 12 Ġand ĠB IB REF 13 Ġin ĠTable Ġ3 Ġ). ĠThis Ġresult Ġcan Ġbe Ġexpected Ġsince Ġ1 ) Ġthese Ġmodels Ġwere Ġtrained Ġwith Ġnews Ġarticles Ġand Ġ2 ) Ġthey Ġare Ġtrained Ġfor Ġall Ġpossible Ġdiscourse Ġrelations Ġin Ġaddition Ġto Ġcausal Ġrelations Ġ( e . g ., Ġcontrast , Ġcondition , Ġetc ). ĠAmong Ġour Ġsuggested Ġmodels , ĠS VM Ġand Ġrandom Ġforest Ġclass ifier Ġperformed Ġbetter Ġthan ĠL ST M Ġand , Ġin Ġthe Ġgeneral Ġtrend , Ġthe Ġmore Ġcomplex Ġthe Ġmodels Ġwere , Ġthe Ġworse Ġthey Ġperformed . ĠThis Ġsuggests Ġthat Ġthe Ġmodels Ġwith Ġmore Ġdirect Ġand Ġsimpler Ġlearning Ġmethods Ġwith Ġfeatures Ġmight Ġclassify Ġthe Ġcaus ality Ġmessages Ġbetter Ġthan Ġthe Ġones Ġmore Ġoptimized Ġfor Ġcapturing Ġdistribution al Ġinformation Ġor Ġnon - linear Ġrelationships Ġof Ġfeatures . </s> Table Ġ4 Ġshows Ġthe Ġresults Ġof Ġa Ġfeature Ġab lation Ġtest Ġto Ġsee Ġhow Ġeach Ġfeature Ġcontributes Ġto Ġcaus ality Ġclassification Ġperformance Ġof Ġthe Ġlinear ĠS VM Ġclass ifier . ĠPOS Ġtags Ġcaused Ġthe Ġlargest Ġdrop Ġin ĠF 1 . ĠWe Ġsuspect ĠPOS Ġtags Ġplayed Ġa Ġunique Ġrole Ġbecause Ġdiscourse Ġconnect ives Ġcan Ġhave Ġvarious Ġsurface Ġforms Ġ( e . g ., Ġbecause , Ġc uz , Ġbc uz , Ġetc ) Ġbut Ġstill Ġthe Ġsame ĠPOS Ġtag Ġ` P '. ĠAlso ĠPOS Ġtags Ġcan Ġcapture Ġthe Ġoccurrences Ġof Ġmod al Ġverbs , Ġa Ġfeature Ġpreviously Ġfound Ġto Ġbe Ġvery Ġuseful Ġfor Ġdetecting Ġsimilar Ġdiscourse Ġrelations ĠB IB REF 9 Ġ. ĠN - gram Ġfeatures Ġcaused Ġ0 . 022 ĠF 1 Ġdrop Ġwhile Ġsentiment Ġtags Ġdid Ġnot Ġaffect Ġthe Ġmodel Ġwhen Ġremoved . ĠUnlike Ġthe Ġprevious Ġwork Ġwhere ĠFirst - Last , ĠFirst 3 Ġand ĠWord Ġpairs Ġtended Ġto Ġgain Ġa Ġlarge ĠF 1 Ġincrease Ġfor Ġmultic lass Ġdiscourse Ġrelation Ġprediction , Ġin Ġour Ġcase , Ġthey Ġdid Ġnot Ġaffect Ġthe Ġprediction Ġperformance Ġcompared Ġto Ġother Ġfeature Ġtypes Ġsuch Ġas ĠPOS Ġtags Ġor ĠN - gram s . </s> Ca usal ĠExplan ation ĠIdentification </s> In Ġthis Ġtask , Ġthe Ġmodel Ġidentifies Ġcausal Ġexplanations Ġgiven Ġthe Ġdiscourse Ġarguments Ġof Ġthe Ġcaus ality Ġmessage . ĠWe Ġexplored Ġover Ġthe Ġsame Ġmodels Ġas Ġthose Ġwe Ġused Ġfor Ġcaus ality Ġ( s ans Ġthe Ġoutput Ġlayer ), Ġand Ġfound Ġthe Ġalmost Ġopposite Ġtrend Ġof Ġperformances Ġ( see ĠTable Ġ5 Ġ). ĠThe ĠLinear ĠS VM Ġobtained Ġlowest ĠF 1 Ġwhile Ġthe ĠL ST M Ġmodel Ġmade Ġthe Ġbest Ġidentification Ġperformance . ĠAs Ġopposed Ġto Ġthe Ġsimple Ġbinary Ġclassification Ġof Ġthe Ġcaus ality Ġmessages , Ġin Ġorder Ġto Ġdetect Ġcausal Ġexplanation , Ġit Ġis Ġmore Ġbeneficial Ġto Ġconsider Ġthe Ġrelation Ġacross Ġdiscourse Ġarguments Ġof Ġthe Ġwhole Ġmessage Ġand Ġimplicit Ġdistribution al Ġrepresentation Ġdue Ġto Ġthe Ġimplicit Ġcausal Ġrelations Ġbetween Ġtwo Ġdistant Ġarguments . </s> Arch itect ural ĠVari ants </s> For Ġcaus ality Ġprediction , Ġwe Ġexperimented Ġwith Ġonly Ġword Ġtokens Ġin Ġthe Ġwhole Ġmessage Ġwithout Ġhelp Ġof ĠWord - level ĠL ST M Ġlayer Ġ( Word ĠL ST M ), Ġand ĠF 1 Ġdropped Ġby Ġ0 . 0 64 Ġ( CP Ġin ĠTable Ġ6 Ġ). ĠAlso , Ġwhen Ġwe Ġused Ġthe Ġaverage Ġof Ġthe Ġsequence Ġof Ġword Ġembed d ings Ġof Ġeach Ġdiscourse Ġargument Ġas Ġan Ġinput Ġto Ġthe ĠDA - level ĠL ST M Ġand Ġit Ġcaused ĠF 1 Ġdrop Ġof Ġ0 . 07 3 . ĠThis Ġsuggests Ġthat Ġthe Ġinformation Ġgained Ġfrom Ġboth Ġthe Ġinteraction Ġof Ġwords Ġin Ġand Ġin Ġbetween Ġdiscourse Ġarguments Ġhelp Ġwhen Ġthe Ġmodel Ġutilizes Ġthe Ġdistribution al Ġrepresentation Ġof Ġthe Ġtexts . </s> For Ġcausal Ġexplanation Ġidentification , Ġin Ġorder Ġto Ġtest Ġhow Ġthe ĠL ST M Ġclass ifier Ġworks Ġwithout Ġits Ġcapability Ġof Ġcapturing Ġthe Ġrelations Ġbetween Ġdiscourse Ġarguments , Ġwe Ġremoved ĠDA - level ĠL ST M Ġlayer Ġand Ġran Ġthe ĠL ST M Ġdirectly Ġon Ġthe Ġword Ġembed ding Ġsequence Ġfor Ġeach Ġdiscourse Ġargument Ġfor Ġclass ifying Ġwhether Ġthe Ġargument Ġis Ġcausal Ġexplanation , Ġand Ġthe Ġmodel Ġhad Ġ0 . 06 1 ĠF 1 Ġdrop Ġ( Word ĠL ST M Ġin ĠCE I Ġin ĠTable Ġ6 Ġ). ĠAlso , Ġwhen Ġwe Ġran ĠDA - level ĠL ST M Ġon Ġthe Ġaverage Ġvectors Ġof Ġthe Ġword Ġsequences Ġof Ġeach Ġdiscourse Ġargument Ġof Ġmessages , ĠF 1 Ġdecreased Ġto Ġ0 . 8 18 . ĠThis Ġfollows Ġthe Ġsimilar Ġpattern Ġobserved Ġfrom Ġother Ġtypes Ġof Ġmodels Ġperformances Ġ( i . e ., ĠSV Ms Ġand ĠRandom ĠForest Ġclass ifiers ) Ġthat Ġthe Ġmodels Ġwith Ġhigher Ġcomplexity Ġfor Ġcapturing Ġthe Ġinteraction Ġof Ġdiscourse Ġarguments Ġtend Ġto Ġidentify Ġcausal Ġexplanation Ġwith Ġthe Ġhigher Ġaccur acies . </s> For ĠCE I Ġtask , Ġwe Ġfound Ġthat Ġwhen Ġthe Ġmodel Ġran Ġon Ġthe Ġsequence Ġrepresentation Ġof Ġdiscourse Ġargument Ġ( DA ĠAVG ĠL ST M ), Ġits Ġperformance Ġwas Ġhigher Ġthan Ġthe Ġplain Ġsequence Ġof Ġword Ġembed d ings Ġ( Word ĠL ST M ). ĠFinally , Ġin Ġboth Ġsubt asks , Ġwhen Ġthe Ġmodels Ġran Ġon Ġboth ĠWord - level Ġand ĠDA - Level Ġ( Full ĠL ST M ), Ġthey Ġobtained Ġthe Ġhighest Ġperformance . </s> Complete ĠPipeline </s> E val uations Ġthus Ġfar Ġzero ed - in Ġon Ġeach Ġsubt ask Ġof Ġcausal Ġexplanation Ġanalysis Ġ( i . e . ĠCE I Ġonly Ġfocused Ġon Ġdata Ġalready Ġidentified Ġto Ġcontain Ġcausal Ġexplanations ). ĠHere , Ġwe Ġseek Ġto Ġevaluate Ġthe Ġcomplete Ġpipeline Ġof ĠCP Ġand ĠCE I , Ġstarting Ġfrom Ġall Ġof Ġtest Ġdata Ġ( those Ġor Ġwithout Ġcaus ality ) Ġand Ġevaluating Ġthe Ġfinal Ġaccuracy Ġof ĠCE I Ġpredictions . ĠThis Ġis Ġintended Ġto Ġevaluate ĠCE I Ġperformance Ġunder Ġan Ġapplied Ġsetting Ġwhere Ġone Ġdoes Ġnot Ġalready Ġknow Ġwhether Ġa Ġdocument Ġhas Ġa Ġcausal Ġexplanation . </s> There Ġare Ġseveral Ġapproaches Ġwe Ġcould Ġtake Ġto Ġperform ĠCE I Ġstarting Ġfrom Ġun annot ated Ġdata . ĠWe Ġcould Ġsimply Ġrun ĠCE I Ġprediction Ġby Ġitself Ġ( CE I ĠOnly ) Ġor Ġthe Ġpipeline Ġof ĠCP Ġfirst Ġand Ġthen Ġonly Ġrun ĠCE I Ġon Ġdocuments Ġpredicted Ġas Ġcausal Ġ( CP Ġ+ ĠCE I ). ĠFurther , Ġthe ĠCE I Ġmodel Ġcould Ġbe Ġtrained Ġonly Ġon Ġthose Ġdocuments Ġannot ated Ġcausal Ġ( as Ġwas Ġdone Ġin Ġthe Ġprevious Ġexperiments ) Ġor Ġon Ġall Ġtraining Ġdocuments Ġincluding Ġmany Ġthat Ġare Ġnot Ġcausal . </s> Table Ġ7 Ġshow Ġresults Ġvarying Ġthe Ġpipeline Ġand Ġhow ĠCE I Ġwas Ġtrained . ĠThough Ġall Ġsetups Ġperformed Ġdecent Ġ( Ġ$ F 1 Ġ> Ġ0 . 81 $ Ġ) Ġwe Ġsee Ġthat Ġthe Ġpip el ined Ġapproach , Ġfirst Ġpredicting Ġcaus ality Ġ( with Ġthe Ġlinear ĠS VM ) Ġand Ġthen Ġpredicting Ġcausal Ġexplanations Ġonly Ġfor Ġthose Ġwith Ġmarked Ġcausal Ġ( CP Ġ+ ĠCE I Ġ$ _{ ca usal } $ Ġ) Ġyielded Ġthe Ġstrongest Ġresults . ĠThis Ġalso Ġutilized Ġthe ĠCE I Ġmodel Ġonly Ġtrained Ġon Ġthose Ġannot ated Ġcausal . ĠBesides Ġperformance , Ġan Ġadded Ġbenefit Ġfrom Ġthis Ġtwo Ġstep Ġapproach Ġis Ġthat Ġthe ĠCP Ġstep Ġis Ġless Ġcomputational Ġintensive Ġof Ġthe ĠCE I Ġstep Ġand Ġapproximately Ġ2 / 3 Ġof Ġdocuments Ġwill Ġnever Ġneed Ġthe ĠCE I Ġstep Ġapplied . </s> We Ġhad Ġan Ġinevitable Ġlimitation Ġon Ġthe Ġsize Ġof Ġour Ġdataset , Ġsince Ġthere Ġis Ġno Ġother Ġcaus ality Ġdataset Ġover Ġsocial Ġmedia Ġand Ġthe Ġannotation Ġrequired Ġan Ġintensive Ġiter ative Ġprocess . ĠThis Ġmight Ġhave Ġlimited Ġperformances Ġof Ġmore Ġcomplex Ġmodels , Ġbut Ġconsidering Ġthe Ġprocessing Ġtime Ġand Ġthe Ġcomputation Ġload , Ġthe Ġcombination Ġof Ġthe Ġlinear Ġmodel Ġand Ġthe ĠR NN - based Ġmodel Ġof Ġour Ġpipeline Ġobtained Ġboth Ġthe Ġhigh Ġperformance Ġand Ġefficiency Ġfor Ġthe Ġpractical Ġapplications Ġto Ġdownstream Ġtasks . ĠIn Ġother Ġwords , Ġit 's Ġpossible Ġthe Ġlinear Ġmodel Ġwill Ġnot Ġperform Ġas Ġwell Ġif Ġthe Ġtraining Ġsize Ġis Ġincreased Ġsubstantially . ĠHowever , Ġa Ġlinear Ġmodel Ġcould Ġstill Ġbe Ġused Ġto Ġdo Ġa Ġfirst - pass , Ġcomput ationally Ġefficient Ġlabeling , Ġin Ġorder Ġto Ġshort list Ġsocial Ġmedia Ġposts Ġfor Ġfurther Ġlabeling Ġfrom Ġan ĠL ST M Ġor Ġmore Ġcomplex Ġmodel . </s> Expl oration </s> Here , Ġwe Ġexplore Ġthe Ġuse Ġof Ġcausal Ġexplanation Ġanalysis Ġfor Ġdownstream Ġtasks . ĠFirst Ġwe Ġlook Ġat Ġthe Ġrelationship Ġbetween Ġuse Ġof Ġcausal Ġexplanation Ġand Ġone 's Ġdemographics : Ġage Ġand Ġgender . ĠThen , Ġwe Ġconsider Ġtheir Ġuse Ġin Ġsentiment Ġanalysis Ġfor Ġextracting Ġthe Ġcauses Ġof Ġpol arity Ġratings . ĠResearch Ġinvolving Ġhuman Ġsubjects Ġwas Ġapproved Ġby Ġthe ĠUniversity Ġof ĠPennsylvania ĠInst itutional ĠReview ĠBoard . </s> Conclusion </s> We Ġdeveloped Ġa Ġpipeline Ġfor Ġcausal Ġexplanation Ġanalysis Ġover Ġsocial Ġmedia Ġtext , Ġincluding Ġboth Ġcaus ality Ġprediction Ġand Ġcausal Ġexplanation Ġidentification . ĠWe Ġexamined Ġa Ġvariety Ġof Ġmodel Ġtypes Ġand ĠR NN Ġarchitectures Ġfor Ġeach Ġpart Ġof Ġthe Ġpipeline , Ġfinding Ġan ĠS VM Ġbest Ġfor Ġcaus ality Ġprediction Ġand Ġa Ġhierarchy Ġof ĠBi L ST Ms Ġfor Ġcausal Ġexplanation Ġidentification , Ġsuggesting Ġthe Ġlater Ġtask Ġrelies Ġmore Ġheavily Ġon Ġsequential Ġinformation . ĠIn Ġfact , Ġwe Ġfound Ġreplacing Ġeither Ġlayer Ġof Ġthe Ġhierarchical ĠL ST M Ġarchitecture Ġ( the Ġword - level Ġor Ġthe ĠDA - level ) Ġwith Ġa Ġan Ġequivalent ĠâĢ ľ bag Ġof Ġfeatures âĢ Ŀ Ġapproach Ġresulted Ġin Ġreduced Ġaccuracy . ĠResults Ġof Ġour Ġwhole Ġpipeline Ġof Ġcausal Ġexplanation Ġanalysis Ġwere Ġfound Ġquite Ġstrong , Ġachieving Ġan Ġ$ F 1 = 0 . 8 68 $ Ġat Ġidentifying Ġdiscourse Ġarguments Ġthat Ġare Ġcausal Ġexplanations . </s> Finally , Ġwe Ġdemonstrated Ġuse Ġof Ġour Ġmodels Ġin Ġapplications , Ġfinding Ġassociations Ġbetween Ġdemographics Ġand Ġrate Ġof Ġmentioning Ġcausal Ġexplanations , Ġas Ġwell Ġas Ġshowing Ġdifferences Ġin Ġthe Ġtop Ġwords Ġpredictive Ġof Ġnegative Ġratings Ġin ĠYelp Ġreviews . ĠUt il ization Ġof Ġdiscourse Ġstructure Ġin Ġsocial Ġmedia Ġanalysis Ġhas Ġbeen Ġa Ġlargely Ġunt apped Ġarea Ġof Ġexploration , Ġperhaps Ġdue Ġto Ġits Ġperceived Ġdifficulty . ĠWe Ġhope Ġthe Ġstrong Ġresults Ġof Ġcausal Ġexplanation Ġidentification Ġhere Ġleads Ġto Ġthe Ġintegration Ġof Ġmore Ġsyntax Ġand Ġdeeper Ġsemantics Ġinto Ġsocial Ġmedia Ġanalyses Ġand Ġultimately Ġenables Ġnew Ġapplications Ġbeyond Ġthe Ġcurrent Ġstate Ġof Ġthe Ġart . </s> Acknowled gments </s> This Ġwork Ġwas Ġsupported , Ġin Ġpart , Ġby Ġa Ġgrant Ġfrom Ġthe ĠTemple ton ĠReligion ĠTrust Ġ( ID Ġ# TR T 00 48 ). ĠThe Ġfund ers Ġhad Ġno Ġrole Ġin Ġstudy Ġdesign , Ġdata Ġcollection Ġand Ġanalysis , Ġdecision Ġto Ġpublish , Ġor Ġpreparation Ġof Ġthe Ġmanuscript . ĠWe Ġalso Ġthank ĠLaura ĠSmith , ĠY iy i ĠChen , ĠGret a ĠJaw el Ġand ĠVanessa ĠHernandez Ġfor Ġtheir Ġwork Ġin Ġidentifying Ġcausal Ġexplanations .
Introduction Explanations of happenings in one's life, causal explanations, are an important topic of study in social, psychological, economic, and behavioral sciences. For example, psychologists have analyzed people's causal explanatory style BIBREF0 and found strong negative relationships with depression, passivity, and hostility, as well as positive relationships with life satisfaction, quality of life, and length of life BIBREF1 , BIBREF2 , BIBREF0 . To help understand the significance of causal explanations, consider how they are applied to measuring optimism (and its converse, pessimism) BIBREF0 . For example, in “My parser failed because I always have bugs.”, the emphasized text span is considered a causal explanation which indicates pessimistic personality – a negative event where the author believes the cause is pervasive. However, in “My parser failed because I barely worked on the code.”, the explanation would be considered a signal of optimistic personality – a negative event for which the cause is believed to be short-lived. Language-based models which can detect causal explanations from everyday social media language can be used for more than automating optimism detection. Language-based assessments would enable other large-scale downstream tasks: tracking prevailing causal beliefs (e.g., about climate change or autism), better extracting process knowledge from non-fiction (e.g., gravity causes objects to move toward one another), or detecting attribution of blame or praise in product or service reviews (“I loved this restaurant because the fish was cooked to perfection”). In this paper, we introduce causal explanation analysis and its subtasks of detecting the presence of causality (causality prediction) and identifying explanatory phrases (causal explanation identification). There are many challenges to achieving these task. First, the ungrammatical texts in social media incur poor syntactic parsing results which drastically affect the performance of discourse relation parsing pipelines . Many causal relations are implicit and do not contain any discourse markers (e.g., `because'). Further, Explicit causal relations are also more difficult in social media due to the abundance of abbreviations and variations of discourse connectives (e.g., `cuz' and `bcuz'). Prevailing approaches for social media analyses, utilizing traditional linear models or bag of words models (e.g., SVM trained with n-gram, part-of-speech (POS) tags, or lexicon-based features) alone do not seem appropriate for this task since they simply cannot segment the text into meaningful discourse units or discourse arguments such as clauses or sentences rather than random consecutive token sequences or specific word tokens. Even when the discourse units are clear, parsers may still fail to accurately identify discourse relations since the content of social media is quite different than that of newswire which is typically used for discourse parsing. In order to overcome these difficulties of discourse relation parsing in social media, we simplify and minimize the use of syntactic parsing results and capture relations between discourse arguments, and investigate the use of a recursive neural network model (RNN). Recent work has shown that RNNs are effective for utilizing discourse structures for their downstream tasks BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 , but they have yet to be directly used for discourse relation prediction in social media. We evaluated our model by comparing it to off-the-shelf end-to-end discourse relation parsers and traditional models. We found that the SVM and random forest classifiers work better than the LSTM classifier for the causality detection, while the LSTM classifier outperforms other models for identifying causal explanation. The contributions of this work include: (1) the proposal of models for both (a) causality prediction and (b) causal explanation identification, (2) the extensive evaluation of a variety of models from social media classification models and discourse relation parsers to RNN-based application models, demonstrating that feature-based models work best for causality prediction while RNNs are superior for the more difficult task of causal explanation identification, (3) performance analysis on architectural differences of the pipeline and the classifier structures, (4) exploration of the applications of causal explanation to downstream tasks, and (5) release of a novel, anonymized causality Facebook dataset along with our causality prediction and causal explanation identification models. Related Work Identifying causal explanations in documents can be viewed as discourse relation parsing. The Penn Discourse Treebank (PDTB) BIBREF7 has a `Cause' and `Pragmatic Cause' discourse type under a general `Contingency' class and Rhetorical Structure Theory (RST) BIBREF8 has a `Relations of Cause'. In most cases, the development of discourse parsers has taken place in-domain, where researchers have used the existing annotations of discourse arguments in newswire text (e.g. Wall Street Journal) from the discourse treebank and focused on exploring different features and optimizing various types of models for predicting relations BIBREF9 , BIBREF10 , BIBREF11 . In order to further develop automated systems, researchers have proposed end-to-end discourse relation parsers, building models which are trained and evaluated on the annotated PDTB and RST Discourse Treebank (RST DT). These corpora consist of documents from Wall Street Journal (WSJ) which are much more well-organized and grammatical than social media texts BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 . Only a few works have attempted to parse discourse relations for out-of-domain problems such as text categorizations on social media texts; Ji and Bhatia used models which are pretrained with RST DT for building discourse structures from movie reviews, and Son adapted the PDTB discourse relation parsing approach for capturing counterfactual conditionals from tweets BIBREF4 , BIBREF3 , BIBREF16 . These works had substantial differences to what propose in this paper. First, Ji and Bhatia used a pretrained model (not fully optimal for some parts of the given task) in their pipeline; Ji's model performed worse than the baseline on the categorization of legislative bills, which is thought to be due to legislative discourse structures differing from those of the training set (WSJ corpus). Bhatia also used a pretrained model finding that utilizing discourse relation features did not boost accuracy BIBREF4 , BIBREF3 . Both Bhatia and Son used manual schemes which may limit the coverage of certain types of positive samples– Bhatia used a hand-crafted schema for weighting discourse structures for the neural network model and Son manually developed seven surface forms of counterfactual thinking for the rule-based system BIBREF4 , BIBREF16 . We use social-media-specific features from pretrained models which are directly trained on tweets and we avoid any hand-crafted rules except for those included in the existing discourse argument extraction techniques. The automated systems for discourse relation parsing involve multiple subtasks from segmenting the whole text into discourse arguments to classifying discourse relations between the arguments. Past research has found that different types of models and features yield varying performance for each subtask. Some have optimized models for discourse relation classification (i.e. given a document indicating if the relation existing) without discourse argument parsing using models such as Naive-Bayes or SVMs, achieve relatively stronger accuracies but a simpler task than that associated with discourse arguments BIBREF10 , BIBREF11 , BIBREF9 . Researchers who, instead, tried to build the end-to-end parsing pipelines considered a wider range of approaches including sequence models and RNNs BIBREF12 , BIBREF15 , BIBREF14 , BIBREF17 . Particularly, when they tried to utilize the discourse structures for out-domain applications, they used RNN-based models and found that those models are advantageous for their downstream tasks BIBREF4 , BIBREF3 . In our case, for identifying causal explanations from social media using discourse structure, we build an RNN-based model for its structural effectiveness in this task (see details in section UID13 ). However, we also note that simpler models such as SVMs and logistic regression obtained the state-of-the-art performances for text categorization tasks in social media BIBREF18 , BIBREF19 , so we build relatively simple models with different properties for each stage of the full pipeline of our parser. Methods We build our model based on PDTB-style discourse relation parsing since PDTB has a relatively simpler text segmentation method; for explicit discourse relations, it finds the presence of discourse connectives within a document and extracts discourse arguments which parametrize the connective while for implicit relations, it considers all adjacent sentences as candidate discourse arguments. Dataset We created our own causal explanation dataset by collecting 3,268 random Facebook status update messages. Three well-trained annotators manually labeled whether or not each message contains the causal explanation and obtained 1,598 causality messages with substantial agreement ( $\kappa =0.61$ ). We used the majority vote for our gold standard. Then, on each causality message, annotators identified which text spans are causal explanations. For each task, we used 80% of the dataset for training our model and 10% for tuning the hyperparameters of our models. Finally, we evaluated all of our models on the remaining 10% (Table 1 and Table 2 ). For causal explanation detection task, we extracted discourse arguments using our parser and selected discourse arguments which most cover the annotated causal explanation text span as our gold standard. Model We build two types of models. First, we develop feature-based models which utilize features of the successful models in social media analysis and causal relation discourse parsing. Then, we build a recursive neural network model which uses distributed representation of discourse arguments as this approach can even capture latent properties of causal relations which may exist between distant discourse arguments. We specifically selected bidirectional LSTM since the model with the discourse distributional structure built in this form outperformed the traditional models in similar NLP downstream tasks BIBREF3 . As the first step of our pipeline, we use Tweebo parser BIBREF20 to extract syntactic features from messages. Then, we demarcate sentences using punctuation (`,') tag and periods. Among those sentences, we find discourse connectives defined in PDTB annotation along with a Tweet POS tag for conjunction words which can also be a discourse marker. In order to decide whether these connectives are really discourse connectives (e.g., I went home, but he stayed) as opposed to simple connections of two words (I like apple and banana) we see if verb phrases exist before and after the connective by using dependency parsing results. Although discourse connective disambiguation is a complicated task which can be much improved by syntactic features BIBREF21 , we try to minimize effects of syntactic parsing and simplify it since it is highly error-prone in social media. Finally, according to visual inspection, emojis (`E' tag) are crucial for discourse relation in social media so we take them as separate discourse arguments (e.g.,in “My test result... :(” the sad feeling is caused by the test result, but it cannot be captured by plain word tokens). We trained a linear SVM, an rbf SVM, and a random forest with N-gram, charater N-gram, and tweet POS tags, sentiment tags, average word lengths and word counts from each message as they have a pivotal role in the models for many NLP downstream tasks in social media BIBREF19 , BIBREF18 . In addition to these features, we also extracted First-Last, First3 features and Word Pairs from every adjacent pair of discourse arguments since these features were most helpful for causal relation prediction BIBREF9 . First-Last, First3 features are first and last word and first three words of two discourse arguments of the relation, and Word Pairs are the cross product of words of those discourse arguments. These two features enable our model to capture interaction between two discourse arguments. BIBREF9 reported that these two features along with verbs, modality, context, and polarity (which can be captured by N-grams, sentiment tags and POS tags in our previous features) obtained the best performance for predicting Contingency class to which causality belongs. We load the GLOVE word embedding BIBREF22 trained in Twitter for each token of extracted discourse arguments from messages. For the distributional representation of discourse arguments, we run a Word-level LSTM on the words' embeddings within each discourse argument and concatenate last hidden state vectors of forward LSTM ( $\overrightarrow{h}$ ) and backward LSTM ( $\overleftarrow{h}$ ) which is suggested by BIBREF3 ( $DA = [\overrightarrow{h};\overleftarrow{h}]$ ). Then, we feed the sequence of the vector representation of discourse arguments to the Discourse-argument-level LSTM (DA-level LSTM) to make a final prediction with log softmax function. With this structure, the model can learn the representation of interaction of tokens inside each discourse argument, then capture discourse relations across all of the discourse arguments in each message (Figure 2 ). In order to prevent the overfitting, we added a dropout layer between the Word-level LSTM and the DA-level LSTM layer. We also explore subsets of the full RNN architecture, specifically with one of the two LSTM layers removed. In the first model variant, we directly input all word embeddings of a whole message to a BiLSTM layer and make prediction (Word LSTM) without the help of the distributional vector representations of discourse arguments. In the second model variant, we take the average of all word embeddings of each discourse argument ( $DA_k=\frac{1}{N_k} \sum _{i=1}^{N_k}W_{i}$ ), and use them as inputs to a BiLSTM layer (DA AVG LSTM) as the average vector of embeddings were quite effective for representing the whole sequence BIBREF3 , BIBREF5 . As with the full architectures, for CP both of these variants ends with a many-to-one classification per message, while the CEI model ends with a sequence of classifications. Experiment We explored three types of models (RBF SVM, Linear SVM, and Random Forest Classifier) which have previously been shown empirically useful for the language analysis in social media. We filtered out low frequency Word Pairs features as they tend to be noisy and sparse BIBREF9 . Then, we conducted univariate feature selection to restrict all remaining features to those showing at least a small relationship with the outcome. Specifically, we keep all features passing a family-wise error rate of $\alpha = 60$ with the given outcome. After comparing the performance of the optimized version of each model, we also conducted a feature ablation test on the best model in order to see how much each feature contributes to the causality prediction. We used bidirectional LSTMs for causality classification and causal explanation identification since the discourse arguments for causal explanation can show up either before and after the effected events or results and we want our model to be optimized for both cases. However, there is a risk of overfitting due to the dataset which is relatively small for the high complexity of the model, so we added a dropout layer (p=0.3) between the Word-level LSTM and the DA-level LSTM. For tuning our model, we explore the dimensionality of word vector and LSTM hidden state vectors of discourse arguments of 25, 50, 100, and 200 as pretrained GLOVE vectors were trained in this setting. For optimization, we used Stochastic Gradient Descent (SGD) and Adam BIBREF23 with learning rates 0.01 and 0.001. We ignore missing word embeddings because our dataset is quite small for retraining new word embeddings. However, if embeddings are extracted as separate discourse arguments, we used the average of all vectors of all discourse arguments in that message. Average embeddings have performed well for representing text sequences in other tasks BIBREF5 . We first use state-of-the-art PDTB taggers for our baseline BIBREF13 , BIBREF12 for the evaluation of the causality prediction of our models ( BIBREF12 requires sentences extracted from the text as its input, so we used our parser to extract sentences from the message). Then, we compare how models work for each task and disassembled them to inspect how each part of the models can affect their final prediction performances. We conducted McNemar's test to determine whether the performance differences are statistically significant at $p < .05$ . Results We investigated various models for both causality detection and explanation identification. Based on their performances on the task, we analyzed the relationships between the types of models and the tasks, and scrutinized further for the best performing models. For performance analysis, we reported weighted F1 of classes. Causality Prediction In order to classify whether a message contains causal relation, we compared off-the-shelf PDTB parsers, linear SVM, RBF SVM, Random forest and LSTM classifiers. The off-the-shelf parsers achieved the lowest accuracies ( BIBREF12 and BIBREF13 in Table 3 ). This result can be expected since 1) these models were trained with news articles and 2) they are trained for all possible discourse relations in addition to causal relations (e.g., contrast, condition, etc). Among our suggested models, SVM and random forest classifier performed better than LSTM and, in the general trend, the more complex the models were, the worse they performed. This suggests that the models with more direct and simpler learning methods with features might classify the causality messages better than the ones more optimized for capturing distributional information or non-linear relationships of features. Table 4 shows the results of a feature ablation test to see how each feature contributes to causality classification performance of the linear SVM classifier. POS tags caused the largest drop in F1. We suspect POS tags played a unique role because discourse connectives can have various surface forms (e.g., because, cuz, bcuz, etc) but still the same POS tag `P'. Also POS tags can capture the occurrences of modal verbs, a feature previously found to be very useful for detecting similar discourse relations BIBREF9 . N-gram features caused 0.022 F1 drop while sentiment tags did not affect the model when removed. Unlike the previous work where First-Last, First3 and Word pairs tended to gain a large F1 increase for multiclass discourse relation prediction, in our case, they did not affect the prediction performance compared to other feature types such as POS tags or N-grams. Causal Explanation Identification In this task, the model identifies causal explanations given the discourse arguments of the causality message. We explored over the same models as those we used for causality (sans the output layer), and found the almost opposite trend of performances (see Table 5 ). The Linear SVM obtained lowest F1 while the LSTM model made the best identification performance. As opposed to the simple binary classification of the causality messages, in order to detect causal explanation, it is more beneficial to consider the relation across discourse arguments of the whole message and implicit distributional representation due to the implicit causal relations between two distant arguments. Architectural Variants For causality prediction, we experimented with only word tokens in the whole message without help of Word-level LSTM layer (Word LSTM), and F1 dropped by 0.064 (CP in Table 6 ). Also, when we used the average of the sequence of word embeddings of each discourse argument as an input to the DA-level LSTM and it caused F1 drop of 0.073. This suggests that the information gained from both the interaction of words in and in between discourse arguments help when the model utilizes the distributional representation of the texts. For causal explanation identification, in order to test how the LSTM classifier works without its capability of capturing the relations between discourse arguments, we removed DA-level LSTM layer and ran the LSTM directly on the word embedding sequence for each discourse argument for classifying whether the argument is causal explanation, and the model had 0.061 F1 drop (Word LSTM in CEI in Table 6 ). Also, when we ran DA-level LSTM on the average vectors of the word sequences of each discourse argument of messages, F1 decreased to 0.818. This follows the similar pattern observed from other types of models performances (i.e., SVMs and Random Forest classifiers) that the models with higher complexity for capturing the interaction of discourse arguments tend to identify causal explanation with the higher accuracies. For CEI task, we found that when the model ran on the sequence representation of discourse argument (DA AVG LSTM), its performance was higher than the plain sequence of word embeddings (Word LSTM). Finally, in both subtasks, when the models ran on both Word-level and DA-Level (Full LSTM), they obtained the highest performance. Complete Pipeline Evaluations thus far zeroed-in on each subtask of causal explanation analysis (i.e. CEI only focused on data already identified to contain causal explanations). Here, we seek to evaluate the complete pipeline of CP and CEI, starting from all of test data (those or without causality) and evaluating the final accuracy of CEI predictions. This is intended to evaluate CEI performance under an applied setting where one does not already know whether a document has a causal explanation. There are several approaches we could take to perform CEI starting from unannotated data. We could simply run CEI prediction by itself (CEI Only) or the pipeline of CP first and then only run CEI on documents predicted as causal (CP + CEI). Further, the CEI model could be trained only on those documents annotated causal (as was done in the previous experiments) or on all training documents including many that are not causal. Table 7 show results varying the pipeline and how CEI was trained. Though all setups performed decent ( $F1 > 0.81$ ) we see that the pipelined approach, first predicting causality (with the linear SVM) and then predicting causal explanations only for those with marked causal (CP + CEI $_{causal}$ ) yielded the strongest results. This also utilized the CEI model only trained on those annotated causal. Besides performance, an added benefit from this two step approach is that the CP step is less computational intensive of the CEI step and approximately 2/3 of documents will never need the CEI step applied. We had an inevitable limitation on the size of our dataset, since there is no other causality dataset over social media and the annotation required an intensive iterative process. This might have limited performances of more complex models, but considering the processing time and the computation load, the combination of the linear model and the RNN-based model of our pipeline obtained both the high performance and efficiency for the practical applications to downstream tasks. In other words, it's possible the linear model will not perform as well if the training size is increased substantially. However, a linear model could still be used to do a first-pass, computationally efficient labeling, in order to shortlist social media posts for further labeling from an LSTM or more complex model. Exploration Here, we explore the use of causal explanation analysis for downstream tasks. First we look at the relationship between use of causal explanation and one's demographics: age and gender. Then, we consider their use in sentiment analysis for extracting the causes of polarity ratings. Research involving human subjects was approved by the University of Pennsylvania Institutional Review Board. Conclusion We developed a pipeline for causal explanation analysis over social media text, including both causality prediction and causal explanation identification. We examined a variety of model types and RNN architectures for each part of the pipeline, finding an SVM best for causality prediction and a hierarchy of BiLSTMs for causal explanation identification, suggesting the later task relies more heavily on sequential information. In fact, we found replacing either layer of the hierarchical LSTM architecture (the word-level or the DA-level) with a an equivalent “bag of features” approach resulted in reduced accuracy. Results of our whole pipeline of causal explanation analysis were found quite strong, achieving an $F1=0.868$ at identifying discourse arguments that are causal explanations. Finally, we demonstrated use of our models in applications, finding associations between demographics and rate of mentioning causal explanations, as well as showing differences in the top words predictive of negative ratings in Yelp reviews. Utilization of discourse structure in social media analysis has been a largely untapped area of exploration, perhaps due to its perceived difficulty. We hope the strong results of causal explanation identification here leads to the integration of more syntax and deeper semantics into social media analyses and ultimately enables new applications beyond the current state of the art. Acknowledgments This work was supported, in part, by a grant from the Templeton Religion Trust (ID #TRT0048). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We also thank Laura Smith, Yiyi Chen, Greta Jawel and Vanessa Hernandez for their work in identifying causal explanations.
What types of social media did they consider?
a4d115220438c0ded06a91ad62337061389a6747
What Ġtypes Ġof Ġsocial Ġmedia Ġdid Ġthey Ġconsider ?
Facebook Ġstatus Ġupdate Ġmessages
<s> What Ġtypes Ġof Ġsocial Ġmedia Ġdid Ġthey Ġconsider ? </s> Introduction </s> Ex plan ations Ġof Ġhappen ings Ġin Ġone 's Ġlife , Ġcausal Ġexplanations , Ġare Ġan Ġimportant Ġtopic Ġof Ġstudy Ġin Ġsocial , Ġpsychological , Ġeconomic , Ġand Ġbehavioral Ġsciences . ĠFor Ġexample , Ġpsychologists Ġhave Ġanalyzed Ġpeople 's Ġcausal Ġexplanatory Ġstyle ĠB IB REF 0 Ġand Ġfound Ġstrong Ġnegative Ġrelationships Ġwith Ġdepression , Ġpass ivity , Ġand Ġhostility , Ġas Ġwell Ġas Ġpositive Ġrelationships Ġwith Ġlife Ġsatisfaction , Ġquality Ġof Ġlife , Ġand Ġlength Ġof Ġlife ĠB IB REF 1 Ġ, ĠB IB REF 2 Ġ, ĠB IB REF 0 Ġ. </s> To Ġhelp Ġunderstand Ġthe Ġsignificance Ġof Ġcausal Ġexplanations , Ġconsider Ġhow Ġthey Ġare Ġapplied Ġto Ġmeasuring Ġoptimism Ġ( and Ġits Ġcon verse , Ġpessim ism ) ĠB IB REF 0 Ġ. ĠFor Ġexample , Ġin ĠâĢ ľ My Ġparser Ġfailed Ġbecause ĠI Ġalways Ġhave Ġbugs . âĢ Ŀ , Ġthe Ġemphasized Ġtext Ġspan Ġis Ġconsidered Ġa Ġcausal Ġexplanation Ġwhich Ġindicates Ġpessimistic Ġpersonality ĠâĢĵ Ġa Ġnegative Ġevent Ġwhere Ġthe Ġauthor Ġbelieves Ġthe Ġcause Ġis Ġpervasive . ĠHowever , Ġin ĠâĢ ľ My Ġparser Ġfailed Ġbecause ĠI Ġbarely Ġworked Ġon Ġthe Ġcode . âĢ Ŀ , Ġthe Ġexplanation Ġwould Ġbe Ġconsidered Ġa Ġsignal Ġof Ġoptimistic Ġpersonality ĠâĢĵ Ġa Ġnegative Ġevent Ġfor Ġwhich Ġthe Ġcause Ġis Ġbelieved Ġto Ġbe Ġshort - lived . </s> Language - based Ġmodels Ġwhich Ġcan Ġdetect Ġcausal Ġexplanations Ġfrom Ġeveryday Ġsocial Ġmedia Ġlanguage Ġcan Ġbe Ġused Ġfor Ġmore Ġthan Ġautom ating Ġoptimism Ġdetection . ĠLanguage - based Ġassessments Ġwould Ġenable Ġother Ġlarge - scale Ġdownstream Ġtasks : Ġtracking Ġprevailing Ġcausal Ġbeliefs Ġ( e . g ., Ġabout Ġclimate Ġchange Ġor Ġautism ), Ġbetter Ġextracting Ġprocess Ġknowledge Ġfrom Ġnon - fiction Ġ( e . g ., Ġgravity Ġcauses Ġobjects Ġto Ġmove Ġtoward Ġone Ġanother ), Ġor Ġdetecting Ġattribution Ġof Ġblame Ġor Ġpraise Ġin Ġproduct Ġor Ġservice Ġreviews Ġ( âĢ ľ I Ġloved Ġthis Ġrestaurant Ġbecause Ġthe Ġfish Ġwas Ġcooked Ġto Ġperfection âĢ Ŀ ). </s> In Ġthis Ġpaper , Ġwe Ġintroduce Ġcausal Ġexplanation Ġanalysis Ġand Ġits Ġsubt asks Ġof Ġdetecting Ġthe Ġpresence Ġof Ġcaus ality Ġ( ca us ality Ġprediction ) Ġand Ġidentifying Ġexplanatory Ġphrases Ġ( ca usal Ġexplanation Ġidentification ). ĠThere Ġare Ġmany Ġchallenges Ġto Ġachieving Ġthese Ġtask . ĠFirst , Ġthe Ġun gram matical Ġtexts Ġin Ġsocial Ġmedia Ġincur Ġpoor Ġsynt actic Ġparsing Ġresults Ġwhich Ġdrastically Ġaffect Ġthe Ġperformance Ġof Ġdiscourse Ġrelation Ġparsing Ġpipelines Ġ. ĠMany Ġcausal Ġrelations Ġare Ġimplicit Ġand Ġdo Ġnot Ġcontain Ġany Ġdiscourse Ġmarkers Ġ( e . g ., Ġ` because '). ĠFurther , ĠExplicit Ġcausal Ġrelations Ġare Ġalso Ġmore Ġdifficult Ġin Ġsocial Ġmedia Ġdue Ġto Ġthe Ġabundance Ġof Ġabbrevi ations Ġand Ġvariations Ġof Ġdiscourse Ġconnect ives Ġ( e . g ., Ġ` c uz ' Ġand Ġ` bc uz '). </s> Prev ailing Ġapproaches Ġfor Ġsocial Ġmedia Ġanalyses , Ġutilizing Ġtraditional Ġlinear Ġmodels Ġor Ġbag Ġof Ġwords Ġmodels Ġ( e . g ., ĠS VM Ġtrained Ġwith Ġn - gram , Ġpart - of - speech Ġ( POS ) Ġtags , Ġor Ġlex icon - based Ġfeatures ) Ġalone Ġdo Ġnot Ġseem Ġappropriate Ġfor Ġthis Ġtask Ġsince Ġthey Ġsimply Ġcannot Ġsegment Ġthe Ġtext Ġinto Ġmeaningful Ġdiscourse Ġunits Ġor Ġdiscourse Ġarguments Ġsuch Ġas Ġclauses Ġor Ġsentences Ġrather Ġthan Ġrandom Ġconsecutive Ġtoken Ġsequences Ġor Ġspecific Ġword Ġtokens . ĠEven Ġwhen Ġthe Ġdiscourse Ġunits Ġare Ġclear , Ġpars ers Ġmay Ġstill Ġfail Ġto Ġaccurately Ġidentify Ġdiscourse Ġrelations Ġsince Ġthe Ġcontent Ġof Ġsocial Ġmedia Ġis Ġquite Ġdifferent Ġthan Ġthat Ġof Ġnews wire Ġwhich Ġis Ġtypically Ġused Ġfor Ġdiscourse Ġparsing . </s> In Ġorder Ġto Ġovercome Ġthese Ġdifficulties Ġof Ġdiscourse Ġrelation Ġparsing Ġin Ġsocial Ġmedia , Ġwe Ġsimplify Ġand Ġminimize Ġthe Ġuse Ġof Ġsynt actic Ġparsing Ġresults Ġand Ġcapture Ġrelations Ġbetween Ġdiscourse Ġarguments , Ġand Ġinvestigate Ġthe Ġuse Ġof Ġa Ġrecursive Ġneural Ġnetwork Ġmodel Ġ( R NN ). ĠRecent Ġwork Ġhas Ġshown Ġthat ĠR NN s Ġare Ġeffective Ġfor Ġutilizing Ġdiscourse Ġstructures Ġfor Ġtheir Ġdownstream Ġtasks ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ, ĠB IB REF 5 Ġ, ĠB IB REF 6 Ġ, Ġbut Ġthey Ġhave Ġyet Ġto Ġbe Ġdirectly Ġused Ġfor Ġdiscourse Ġrelation Ġprediction Ġin Ġsocial Ġmedia . ĠWe Ġevaluated Ġour Ġmodel Ġby Ġcomparing Ġit Ġto Ġoff - the - she lf Ġend - to - end Ġdiscourse Ġrelation Ġpars ers Ġand Ġtraditional Ġmodels . ĠWe Ġfound Ġthat Ġthe ĠS VM Ġand Ġrandom Ġforest Ġclass ifiers Ġwork Ġbetter Ġthan Ġthe ĠL ST M Ġclass ifier Ġfor Ġthe Ġcaus ality Ġdetection , Ġwhile Ġthe ĠL ST M Ġclass ifier Ġoutper forms Ġother Ġmodels Ġfor Ġidentifying Ġcausal Ġexplanation . </s> The Ġcontributions Ġof Ġthis Ġwork Ġinclude : Ġ( 1 ) Ġthe Ġproposal Ġof Ġmodels Ġfor Ġboth Ġ( a ) Ġcaus ality Ġprediction Ġand Ġ( b ) Ġcausal Ġexplanation Ġidentification , Ġ( 2 ) Ġthe Ġextensive Ġevaluation Ġof Ġa Ġvariety Ġof Ġmodels Ġfrom Ġsocial Ġmedia Ġclassification Ġmodels Ġand Ġdiscourse Ġrelation Ġpars ers Ġto ĠR NN - based Ġapplication Ġmodels , Ġdemonstrating Ġthat Ġfeature - based Ġmodels Ġwork Ġbest Ġfor Ġcaus ality Ġprediction Ġwhile ĠR NN s Ġare Ġsuperior Ġfor Ġthe Ġmore Ġdifficult Ġtask Ġof Ġcausal Ġexplanation Ġidentification , Ġ( 3 ) Ġperformance Ġanalysis Ġon Ġarchitectural Ġdifferences Ġof Ġthe Ġpipeline Ġand Ġthe Ġclass ifier Ġstructures , Ġ( 4 ) Ġexploration Ġof Ġthe Ġapplications Ġof Ġcausal Ġexplanation Ġto Ġdownstream Ġtasks , Ġand Ġ( 5 ) Ġrelease Ġof Ġa Ġnovel , Ġanonym ized Ġcaus ality ĠFacebook Ġdataset Ġalong Ġwith Ġour Ġcaus ality Ġprediction Ġand Ġcausal Ġexplanation Ġidentification Ġmodels . </s> Related ĠWork </s> Ident ifying Ġcausal Ġexplanations Ġin Ġdocuments Ġcan Ġbe Ġviewed Ġas Ġdiscourse Ġrelation Ġparsing . ĠThe ĠPenn ĠDisc ourse ĠTree bank Ġ( PD TB ) ĠB IB REF 7 Ġhas Ġa Ġ` Cause ' Ġand Ġ` Pr ag matic ĠCause ' Ġdiscourse Ġtype Ġunder Ġa Ġgeneral Ġ` Cont ing ency ' Ġclass Ġand ĠR het orical ĠStructure ĠTheory Ġ( R ST ) ĠB IB REF 8 Ġhas Ġa Ġ` Relations Ġof ĠCause '. ĠIn Ġmost Ġcases , Ġthe Ġdevelopment Ġof Ġdiscourse Ġpars ers Ġhas Ġtaken Ġplace Ġin - domain , Ġwhere Ġresearchers Ġhave Ġused Ġthe Ġexisting Ġannotations Ġof Ġdiscourse Ġarguments Ġin Ġnews wire Ġtext Ġ( e . g . ĠWall ĠStreet ĠJournal ) Ġfrom Ġthe Ġdiscourse Ġtree bank Ġand Ġfocused Ġon Ġexploring Ġdifferent Ġfeatures Ġand Ġoptimizing Ġvarious Ġtypes Ġof Ġmodels Ġfor Ġpredicting Ġrelations ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġ. ĠIn Ġorder Ġto Ġfurther Ġdevelop Ġautomated Ġsystems , Ġresearchers Ġhave Ġproposed Ġend - to - end Ġdiscourse Ġrelation Ġpars ers , Ġbuilding Ġmodels Ġwhich Ġare Ġtrained Ġand Ġevaluated Ġon Ġthe Ġannot ated ĠPD TB Ġand ĠR ST ĠDisc ourse ĠTree bank Ġ( R ST ĠDT ). ĠThese Ġcorpor a Ġconsist Ġof Ġdocuments Ġfrom ĠWall ĠStreet ĠJournal Ġ( WS J ) Ġwhich Ġare Ġmuch Ġmore Ġwell - organized Ġand Ġgram matical Ġthan Ġsocial Ġmedia Ġtexts ĠB IB REF 12 Ġ, ĠB IB REF 13 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 15 Ġ. </s> Only Ġa Ġfew Ġworks Ġhave Ġattempted Ġto Ġparse Ġdiscourse Ġrelations Ġfor Ġout - of - domain Ġproblems Ġsuch Ġas Ġtext Ġcategor izations Ġon Ġsocial Ġmedia Ġtexts ; ĠJi Ġand ĠB hat ia Ġused Ġmodels Ġwhich Ġare Ġpret rained Ġwith ĠR ST ĠDT Ġfor Ġbuilding Ġdiscourse Ġstructures Ġfrom Ġmovie Ġreviews , Ġand ĠSon Ġadapted Ġthe ĠPD TB Ġdiscourse Ġrelation Ġparsing Ġapproach Ġfor Ġcapturing Ġcounter fact ual Ġcondition als Ġfrom Ġtweets ĠB IB REF 4 Ġ, ĠB IB REF 3 Ġ, ĠB IB REF 16 Ġ. ĠThese Ġworks Ġhad Ġsubstantial Ġdifferences Ġto Ġwhat Ġpropose Ġin Ġthis Ġpaper . ĠFirst , ĠJi Ġand ĠB hat ia Ġused Ġa Ġpret rained Ġmodel Ġ( not Ġfully Ġoptimal Ġfor Ġsome Ġparts Ġof Ġthe Ġgiven Ġtask ) Ġin Ġtheir Ġpipeline ; ĠJi 's Ġmodel Ġperformed Ġworse Ġthan Ġthe Ġbaseline Ġon Ġthe Ġcategor ization Ġof Ġlegislative Ġbills , Ġwhich Ġis Ġthought Ġto Ġbe Ġdue Ġto Ġlegislative Ġdiscourse Ġstructures Ġdiffering Ġfrom Ġthose Ġof Ġthe Ġtraining Ġset Ġ( WS J Ġcorpus ). ĠB hat ia Ġalso Ġused Ġa Ġpret rained Ġmodel Ġfinding Ġthat Ġutilizing Ġdiscourse Ġrelation Ġfeatures Ġdid Ġnot Ġboost Ġaccuracy ĠB IB REF 4 Ġ, ĠB IB REF 3 Ġ. ĠBoth ĠB hat ia Ġand ĠSon Ġused Ġmanual Ġschemes Ġwhich Ġmay Ġlimit Ġthe Ġcoverage Ġof Ġcertain Ġtypes Ġof Ġpositive Ġsamples âĢĵ ĠB hat ia Ġused Ġa Ġhand - crafted Ġschema Ġfor Ġweight ing Ġdiscourse Ġstructures Ġfor Ġthe Ġneural Ġnetwork Ġmodel Ġand ĠSon Ġmanually Ġdeveloped Ġseven Ġsurface Ġforms Ġof Ġcounter fact ual Ġthinking Ġfor Ġthe Ġrule - based Ġsystem ĠB IB REF 4 Ġ, ĠB IB REF 16 Ġ. ĠWe Ġuse Ġsocial - media - specific Ġfeatures Ġfrom Ġpret rained Ġmodels Ġwhich Ġare Ġdirectly Ġtrained Ġon Ġtweets Ġand Ġwe Ġavoid Ġany Ġhand - crafted Ġrules Ġexcept Ġfor Ġthose Ġincluded Ġin Ġthe Ġexisting Ġdiscourse Ġargument Ġextraction Ġtechniques . </s> The Ġautomated Ġsystems Ġfor Ġdiscourse Ġrelation Ġparsing Ġinvolve Ġmultiple Ġsubt asks Ġfrom Ġsegment ing Ġthe Ġwhole Ġtext Ġinto Ġdiscourse Ġarguments Ġto Ġclass ifying Ġdiscourse Ġrelations Ġbetween Ġthe Ġarguments . ĠPast Ġresearch Ġhas Ġfound Ġthat Ġdifferent Ġtypes Ġof Ġmodels Ġand Ġfeatures Ġyield Ġvarying Ġperformance Ġfor Ġeach Ġsubt ask . ĠSome Ġhave Ġoptimized Ġmodels Ġfor Ġdiscourse Ġrelation Ġclassification Ġ( i . e . Ġgiven Ġa Ġdocument Ġindicating Ġif Ġthe Ġrelation Ġexisting ) Ġwithout Ġdiscourse Ġargument Ġparsing Ġusing Ġmodels Ġsuch Ġas ĠNa ive - Bay es Ġor ĠSV Ms , Ġachieve Ġrelatively Ġstronger Ġaccur acies Ġbut Ġa Ġsimpler Ġtask Ġthan Ġthat Ġassociated Ġwith Ġdiscourse Ġarguments ĠB IB REF 10 Ġ, ĠB IB REF 11 Ġ, ĠB IB REF 9 Ġ. ĠResearchers Ġwho , Ġinstead , Ġtried Ġto Ġbuild Ġthe Ġend - to - end Ġparsing Ġpipelines Ġconsidered Ġa Ġwider Ġrange Ġof Ġapproaches Ġincluding Ġsequence Ġmodels Ġand ĠR NN s ĠB IB REF 12 Ġ, ĠB IB REF 15 Ġ, ĠB IB REF 14 Ġ, ĠB IB REF 17 Ġ. ĠParticularly , Ġwhen Ġthey Ġtried Ġto Ġutilize Ġthe Ġdiscourse Ġstructures Ġfor Ġout - domain Ġapplications , Ġthey Ġused ĠR NN - based Ġmodels Ġand Ġfound Ġthat Ġthose Ġmodels Ġare Ġadvantageous Ġfor Ġtheir Ġdownstream Ġtasks ĠB IB REF 4 Ġ, ĠB IB REF 3 Ġ. </s> In Ġour Ġcase , Ġfor Ġidentifying Ġcausal Ġexplanations Ġfrom Ġsocial Ġmedia Ġusing Ġdiscourse Ġstructure , Ġwe Ġbuild Ġan ĠR NN - based Ġmodel Ġfor Ġits Ġstructural Ġeffectiveness Ġin Ġthis Ġtask Ġ( see Ġdetails Ġin Ġsection ĠUID 13 Ġ). ĠHowever , Ġwe Ġalso Ġnote Ġthat Ġsimpler Ġmodels Ġsuch Ġas ĠSV Ms Ġand Ġlog istic Ġregression Ġobtained Ġthe Ġstate - of - the - art Ġperformances Ġfor Ġtext Ġcategor ization Ġtasks Ġin Ġsocial Ġmedia ĠB IB REF 18 Ġ, ĠB IB REF 19 Ġ, Ġso Ġwe Ġbuild Ġrelatively Ġsimple Ġmodels Ġwith Ġdifferent Ġproperties Ġfor Ġeach Ġstage Ġof Ġthe Ġfull Ġpipeline Ġof Ġour Ġparser . </s> Methods </s> We Ġbuild Ġour Ġmodel Ġbased Ġon ĠPD TB - style Ġdiscourse Ġrelation Ġparsing Ġsince ĠPD TB Ġhas Ġa Ġrelatively Ġsimpler Ġtext Ġsegment ation Ġmethod ; Ġfor Ġexplicit Ġdiscourse Ġrelations , Ġit Ġfinds Ġthe Ġpresence Ġof Ġdiscourse Ġconnect ives Ġwithin Ġa Ġdocument Ġand Ġextracts Ġdiscourse Ġarguments Ġwhich Ġparam et ri ze Ġthe Ġconnect ive Ġwhile Ġfor Ġimplicit Ġrelations , Ġit Ġconsiders Ġall Ġadjacent Ġsentences Ġas Ġcandidate Ġdiscourse Ġarguments . </s> Dat as et </s> We Ġcreated Ġour Ġown Ġcausal Ġexplanation Ġdataset Ġby Ġcollecting Ġ3 , 268 Ġrandom ĠFacebook Ġstatus Ġupdate Ġmessages . ĠThree Ġwell - trained Ġannot ators Ġmanually Ġlabeled Ġwhether Ġor Ġnot Ġeach Ġmessage Ġcontains Ġthe Ġcausal Ġexplanation Ġand Ġobtained Ġ1 , 598 Ġcaus ality Ġmessages Ġwith Ġsubstantial Ġagreement Ġ( Ġ$\ k appa Ġ= 0 . 61 $ Ġ). ĠWe Ġused Ġthe Ġmajority Ġvote Ġfor Ġour Ġgold Ġstandard . ĠThen , Ġon Ġeach Ġcaus ality Ġmessage , Ġannot ators Ġidentified Ġwhich Ġtext Ġspans Ġare Ġcausal Ġexplanations . </s> For Ġeach Ġtask , Ġwe Ġused Ġ80 % Ġof Ġthe Ġdataset Ġfor Ġtraining Ġour Ġmodel Ġand Ġ10 % Ġfor Ġtuning Ġthe Ġhyper param eters Ġof Ġour Ġmodels . ĠFinally , Ġwe Ġevaluated Ġall Ġof Ġour Ġmodels Ġon Ġthe Ġremaining Ġ10 % Ġ( Table Ġ1 Ġand ĠTable Ġ2 Ġ). ĠFor Ġcausal Ġexplanation Ġdetection Ġtask , Ġwe Ġextracted Ġdiscourse Ġarguments Ġusing Ġour Ġparser Ġand Ġselected Ġdiscourse Ġarguments Ġwhich Ġmost Ġcover Ġthe Ġannot ated Ġcausal Ġexplanation Ġtext Ġspan Ġas Ġour Ġgold Ġstandard . </s> Model </s> We Ġbuild Ġtwo Ġtypes Ġof Ġmodels . ĠFirst , Ġwe Ġdevelop Ġfeature - based Ġmodels Ġwhich Ġutilize Ġfeatures Ġof Ġthe Ġsuccessful Ġmodels Ġin Ġsocial Ġmedia Ġanalysis Ġand Ġcausal Ġrelation Ġdiscourse Ġparsing . ĠThen , Ġwe Ġbuild Ġa Ġrecursive Ġneural Ġnetwork Ġmodel Ġwhich Ġuses Ġdistributed Ġrepresentation Ġof Ġdiscourse Ġarguments Ġas Ġthis Ġapproach Ġcan Ġeven Ġcapture Ġlatent Ġproperties Ġof Ġcausal Ġrelations Ġwhich Ġmay Ġexist Ġbetween Ġdistant Ġdiscourse Ġarguments . ĠWe Ġspecifically Ġselected Ġbid irection al ĠL ST M Ġsince Ġthe Ġmodel Ġwith Ġthe Ġdiscourse Ġdistribution al Ġstructure Ġbuilt Ġin Ġthis Ġform Ġoutper formed Ġthe Ġtraditional Ġmodels Ġin Ġsimilar ĠN LP Ġdownstream Ġtasks ĠB IB REF 3 Ġ. </s> As Ġthe Ġfirst Ġstep Ġof Ġour Ġpipeline , Ġwe Ġuse ĠTwe eb o Ġparser ĠB IB REF 20 Ġto Ġextract Ġsynt actic Ġfeatures Ġfrom Ġmessages . ĠThen , Ġwe Ġdem arc ate Ġsentences Ġusing Ġpunct uation Ġ( ` ,' ) Ġtag Ġand Ġperiods . ĠAmong Ġthose Ġsentences , Ġwe Ġfind Ġdiscourse Ġconnect ives Ġdefined Ġin ĠPD TB Ġannotation Ġalong Ġwith Ġa ĠTweet ĠPOS Ġtag Ġfor Ġconjunction Ġwords Ġwhich Ġcan Ġalso Ġbe Ġa Ġdiscourse Ġmarker . ĠIn Ġorder Ġto Ġdecide Ġwhether Ġthese Ġconnect ives Ġare Ġreally Ġdiscourse Ġconnect ives Ġ( e . g ., ĠI Ġwent Ġhome , Ġbut Ġhe Ġstayed ) Ġas Ġopposed Ġto Ġsimple Ġconnections Ġof Ġtwo Ġwords Ġ( I Ġlike Ġapple Ġand Ġbanana ) Ġwe Ġsee Ġif Ġverb Ġphrases Ġexist Ġbefore Ġand Ġafter Ġthe Ġconnect ive Ġby Ġusing Ġdependency Ġparsing Ġresults . ĠAlthough Ġdiscourse Ġconnect ive Ġdis amb ig uation Ġis Ġa Ġcomplicated Ġtask Ġwhich Ġcan Ġbe Ġmuch Ġimproved Ġby Ġsynt actic Ġfeatures ĠB IB REF 21 Ġ, Ġwe Ġtry Ġto Ġminimize Ġeffects Ġof Ġsynt actic Ġparsing Ġand Ġsimplify Ġit Ġsince Ġit Ġis Ġhighly Ġerror - prone Ġin Ġsocial Ġmedia . ĠFinally , Ġaccording Ġto Ġvisual Ġinspection , Ġem oj is Ġ( ` E ' Ġtag ) Ġare Ġcrucial Ġfor Ġdiscourse Ġrelation Ġin Ġsocial Ġmedia Ġso Ġwe Ġtake Ġthem Ġas Ġseparate Ġdiscourse Ġarguments Ġ( e . g ., in ĠâĢ ľ My Ġtest Ġresult ... Ġ:( âĢ Ŀ Ġthe Ġsad Ġfeeling Ġis Ġcaused Ġby Ġthe Ġtest Ġresult , Ġbut Ġit Ġcannot Ġbe Ġcaptured Ġby Ġplain Ġword Ġtokens ). </s> We Ġtrained Ġa Ġlinear ĠS VM , Ġan Ġr bf ĠS VM , Ġand Ġa Ġrandom Ġforest Ġwith ĠN - gram , Ġchar ater ĠN - gram , Ġand Ġtweet ĠPOS Ġtags , Ġsentiment Ġtags , Ġaverage Ġword Ġlengths Ġand Ġword Ġcounts Ġfrom Ġeach Ġmessage Ġas Ġthey Ġhave Ġa Ġpivotal Ġrole Ġin Ġthe Ġmodels Ġfor Ġmany ĠN LP Ġdownstream Ġtasks Ġin Ġsocial Ġmedia ĠB IB REF 19 Ġ, ĠB IB REF 18 Ġ. ĠIn Ġaddition Ġto Ġthese Ġfeatures , Ġwe Ġalso Ġextracted ĠFirst - Last , ĠFirst 3 Ġfeatures Ġand ĠWord ĠP airs Ġfrom Ġevery Ġadjacent Ġpair Ġof Ġdiscourse Ġarguments Ġsince Ġthese Ġfeatures Ġwere Ġmost Ġhelpful Ġfor Ġcausal Ġrelation Ġprediction ĠB IB REF 9 Ġ. ĠFirst - Last , ĠFirst 3 Ġfeatures Ġare Ġfirst Ġand Ġlast Ġword Ġand Ġfirst Ġthree Ġwords Ġof Ġtwo Ġdiscourse Ġarguments Ġof Ġthe Ġrelation , Ġand ĠWord ĠP airs Ġare Ġthe Ġcross Ġproduct Ġof Ġwords Ġof Ġthose Ġdiscourse Ġarguments . ĠThese Ġtwo Ġfeatures Ġenable Ġour Ġmodel Ġto Ġcapture Ġinteraction Ġbetween Ġtwo Ġdiscourse Ġarguments . ĠB IB REF 9 Ġreported Ġthat Ġthese Ġtwo Ġfeatures Ġalong Ġwith Ġverbs , Ġmod ality , Ġcontext , Ġand Ġpol arity Ġ( which Ġcan Ġbe Ġcaptured Ġby ĠN - gram s , Ġsentiment Ġtags Ġand ĠPOS Ġtags Ġin Ġour Ġprevious Ġfeatures ) Ġobtained Ġthe Ġbest Ġperformance Ġfor Ġpredicting ĠCont ing ency Ġclass Ġto Ġwhich Ġcaus ality Ġbelongs . </s> We Ġload Ġthe ĠGL O VE Ġword Ġembed ding ĠB IB REF 22 Ġtrained Ġin ĠTwitter Ġfor Ġeach Ġtoken Ġof Ġextracted Ġdiscourse Ġarguments Ġfrom Ġmessages . ĠFor Ġthe Ġdistribution al Ġrepresentation Ġof Ġdiscourse Ġarguments , Ġwe Ġrun Ġa ĠWord - level ĠL ST M Ġon Ġthe Ġwords ' Ġembed d ings Ġwithin Ġeach Ġdiscourse Ġargument Ġand Ġconc aten ate Ġlast Ġhidden Ġstate Ġvectors Ġof Ġforward ĠL ST M Ġ( Ġ$\ over right arrow { h } $ Ġ) Ġand Ġbackward ĠL ST M Ġ( Ġ$\ over left arrow { h } $ Ġ) Ġwhich Ġis Ġsuggested Ġby ĠB IB REF 3 Ġ( Ġ$ DA Ġ= Ġ[ \ over right arrow { h }; \ over left arrow { h } ] $ Ġ). ĠThen , Ġwe Ġfeed Ġthe Ġsequence Ġof Ġthe Ġvector Ġrepresentation Ġof Ġdiscourse Ġarguments Ġto Ġthe ĠDisc ourse - argument - level ĠL ST M Ġ( DA - level ĠL ST M ) Ġto Ġmake Ġa Ġfinal Ġprediction Ġwith Ġlog Ġsoft max Ġfunction . ĠWith Ġthis Ġstructure , Ġthe Ġmodel Ġcan Ġlearn Ġthe Ġrepresentation Ġof Ġinteraction Ġof Ġtokens Ġinside Ġeach Ġdiscourse Ġargument , Ġthen Ġcapture Ġdiscourse Ġrelations Ġacross Ġall Ġof Ġthe Ġdiscourse Ġarguments Ġin Ġeach Ġmessage Ġ( Figure Ġ2 Ġ). ĠIn Ġorder Ġto Ġprevent Ġthe Ġover fitting , Ġwe Ġadded Ġa Ġdrop out Ġlayer Ġbetween Ġthe ĠWord - level ĠL ST M Ġand Ġthe ĠDA - level ĠL ST M Ġlayer . </s> We Ġalso Ġexplore Ġsubs ets Ġof Ġthe Ġfull ĠR NN Ġarchitecture , Ġspecifically Ġwith Ġone Ġof Ġthe Ġtwo ĠL ST M Ġlayers Ġremoved . ĠIn Ġthe Ġfirst Ġmodel Ġvariant , Ġwe Ġdirectly Ġinput Ġall Ġword Ġembed d ings Ġof Ġa Ġwhole Ġmessage Ġto Ġa ĠBi L ST M Ġlayer Ġand Ġmake Ġprediction Ġ( Word ĠL ST M ) Ġwithout Ġthe Ġhelp Ġof Ġthe Ġdistribution al Ġvector Ġrepresentations Ġof Ġdiscourse Ġarguments . ĠIn Ġthe Ġsecond Ġmodel Ġvariant , Ġwe Ġtake Ġthe Ġaverage Ġof Ġall Ġword Ġembed d ings Ġof Ġeach Ġdiscourse Ġargument Ġ( Ġ$ DA _ k = \ frac { 1 }{ N _ k } Ġ\ sum Ġ_ { i = 1 } ^{ N _ k } W _{ i } $ Ġ), Ġand Ġuse Ġthem Ġas Ġinputs Ġto Ġa ĠBi L ST M Ġlayer Ġ( DA ĠAVG ĠL ST M ) Ġas Ġthe Ġaverage Ġvector Ġof Ġembed d ings Ġwere Ġquite Ġeffective Ġfor Ġrepresenting Ġthe Ġwhole Ġsequence ĠB IB REF 3 Ġ, ĠB IB REF 5 Ġ. ĠAs Ġwith Ġthe Ġfull Ġarchitectures , Ġfor ĠCP Ġboth Ġof Ġthese Ġvariants Ġends Ġwith Ġa Ġmany - to - one Ġclassification Ġper Ġmessage , Ġwhile Ġthe ĠCE I Ġmodel Ġends Ġwith Ġa Ġsequence Ġof Ġclass ifications . </s> Exper iment </s> We Ġexplored Ġthree Ġtypes Ġof Ġmodels Ġ( RB F ĠS VM , ĠLinear ĠS VM , Ġand ĠRandom ĠForest ĠClass ifier ) Ġwhich Ġhave Ġpreviously Ġbeen Ġshown Ġempir ically Ġuseful Ġfor Ġthe Ġlanguage Ġanalysis Ġin Ġsocial Ġmedia . ĠWe Ġfiltered Ġout Ġlow Ġfrequency ĠWord ĠP airs Ġfeatures Ġas Ġthey Ġtend Ġto Ġbe Ġnoisy Ġand Ġsparse ĠB IB REF 9 Ġ. ĠThen , Ġwe Ġconducted Ġun ivariate Ġfeature Ġselection Ġto Ġrestrict Ġall Ġremaining Ġfeatures Ġto Ġthose Ġshowing Ġat Ġleast Ġa Ġsmall Ġrelationship Ġwith Ġthe Ġoutcome . ĠSpecifically , Ġwe Ġkeep Ġall Ġfeatures Ġpassing Ġa Ġfamily - wise Ġerror Ġrate Ġof Ġ$\ alpha Ġ= Ġ60 $ Ġwith Ġthe Ġgiven Ġoutcome . ĠAfter Ġcomparing Ġthe Ġperformance Ġof Ġthe Ġoptimized Ġversion Ġof Ġeach Ġmodel , Ġwe Ġalso Ġconducted Ġa Ġfeature Ġab lation Ġtest Ġon Ġthe Ġbest Ġmodel Ġin Ġorder Ġto Ġsee Ġhow Ġmuch Ġeach Ġfeature Ġcontributes Ġto Ġthe Ġcaus ality Ġprediction . </s> We Ġused Ġbid irection al ĠL ST Ms Ġfor Ġcaus ality Ġclassification Ġand Ġcausal Ġexplanation Ġidentification Ġsince Ġthe Ġdiscourse Ġarguments Ġfor Ġcausal Ġexplanation Ġcan Ġshow Ġup Ġeither Ġbefore Ġand Ġafter Ġthe Ġeffected Ġevents Ġor Ġresults Ġand Ġwe Ġwant Ġour Ġmodel Ġto Ġbe Ġoptimized Ġfor Ġboth Ġcases . ĠHowever , Ġthere Ġis Ġa Ġrisk Ġof Ġover fitting Ġdue Ġto Ġthe Ġdataset Ġwhich Ġis Ġrelatively Ġsmall Ġfor Ġthe Ġhigh Ġcomplexity Ġof Ġthe Ġmodel , Ġso Ġwe Ġadded Ġa Ġdrop out Ġlayer Ġ( p = 0 . 3 ) Ġbetween Ġthe ĠWord - level ĠL ST M Ġand Ġthe ĠDA - level ĠL ST M . </s> For Ġtuning Ġour Ġmodel , Ġwe Ġexplore Ġthe Ġdimension ality Ġof Ġword Ġvector Ġand ĠL ST M Ġhidden Ġstate Ġvectors Ġof Ġdiscourse Ġarguments Ġof Ġ25 , Ġ50 , Ġ100 , Ġand Ġ200 Ġas Ġpret rained ĠGL O VE Ġvectors Ġwere Ġtrained Ġin Ġthis Ġsetting . ĠFor Ġoptimization , Ġwe Ġused ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ) Ġand ĠAdam ĠB IB REF 23 Ġwith Ġlearning Ġrates Ġ0 . 01 Ġand Ġ0 . 001 . </s> We Ġignore Ġmissing Ġword Ġembed d ings Ġbecause Ġour Ġdataset Ġis Ġquite Ġsmall Ġfor Ġret raining Ġnew Ġword Ġembed d ings . ĠHowever , Ġif Ġembed d ings Ġare Ġextracted Ġas Ġseparate Ġdiscourse Ġarguments , Ġwe Ġused Ġthe Ġaverage Ġof Ġall Ġvectors Ġof Ġall Ġdiscourse Ġarguments Ġin Ġthat Ġmessage . ĠAverage Ġembed d ings Ġhave Ġperformed Ġwell Ġfor Ġrepresenting Ġtext Ġsequences Ġin Ġother Ġtasks ĠB IB REF 5 Ġ. </s> We Ġfirst Ġuse Ġstate - of - the - art ĠPD TB Ġtag gers Ġfor Ġour Ġbaseline ĠB IB REF 13 Ġ, ĠB IB REF 12 Ġfor Ġthe Ġevaluation Ġof Ġthe Ġcaus ality Ġprediction Ġof Ġour Ġmodels Ġ( ĠB IB REF 12 Ġrequires Ġsentences Ġextracted Ġfrom Ġthe Ġtext Ġas Ġits Ġinput , Ġso Ġwe Ġused Ġour Ġparser Ġto Ġextract Ġsentences Ġfrom Ġthe Ġmessage ). ĠThen , Ġwe Ġcompare Ġhow Ġmodels Ġwork Ġfor Ġeach Ġtask Ġand Ġdis assembled Ġthem Ġto Ġinspect Ġhow Ġeach Ġpart Ġof Ġthe Ġmodels Ġcan Ġaffect Ġtheir Ġfinal Ġprediction Ġperformances . ĠWe Ġconducted ĠMcN em ar 's Ġtest Ġto Ġdetermine Ġwhether Ġthe Ġperformance Ġdifferences Ġare Ġstatistically Ġsignificant Ġat Ġ$ p Ġ< Ġ. 05 $ Ġ. </s> Results </s> We Ġinvestigated Ġvarious Ġmodels Ġfor Ġboth Ġcaus ality Ġdetection Ġand Ġexplanation Ġidentification . ĠBased Ġon Ġtheir Ġperformances Ġon Ġthe Ġtask , Ġwe Ġanalyzed Ġthe Ġrelationships Ġbetween Ġthe Ġtypes Ġof Ġmodels Ġand Ġthe Ġtasks , Ġand Ġscrutin ized Ġfurther Ġfor Ġthe Ġbest Ġperforming Ġmodels . ĠFor Ġperformance Ġanalysis , Ġwe Ġreported Ġweighted ĠF 1 Ġof Ġclasses . </s> C aus ality ĠPrediction </s> In Ġorder Ġto Ġclassify Ġwhether Ġa Ġmessage Ġcontains Ġcausal Ġrelation , Ġwe Ġcompared Ġoff - the - she lf ĠPD TB Ġpars ers , Ġlinear ĠS VM , ĠRB F ĠS VM , ĠRandom Ġforest Ġand ĠL ST M Ġclass ifiers . ĠThe Ġoff - the - she lf Ġpars ers Ġachieved Ġthe Ġlowest Ġaccur acies Ġ( ĠB IB REF 12 Ġand ĠB IB REF 13 Ġin ĠTable Ġ3 Ġ). ĠThis Ġresult Ġcan Ġbe Ġexpected Ġsince Ġ1 ) Ġthese Ġmodels Ġwere Ġtrained Ġwith Ġnews Ġarticles Ġand Ġ2 ) Ġthey Ġare Ġtrained Ġfor Ġall Ġpossible Ġdiscourse Ġrelations Ġin Ġaddition Ġto Ġcausal Ġrelations Ġ( e . g ., Ġcontrast , Ġcondition , Ġetc ). ĠAmong Ġour Ġsuggested Ġmodels , ĠS VM Ġand Ġrandom Ġforest Ġclass ifier Ġperformed Ġbetter Ġthan ĠL ST M Ġand , Ġin Ġthe Ġgeneral Ġtrend , Ġthe Ġmore Ġcomplex Ġthe Ġmodels Ġwere , Ġthe Ġworse Ġthey Ġperformed . ĠThis Ġsuggests Ġthat Ġthe Ġmodels Ġwith Ġmore Ġdirect Ġand Ġsimpler Ġlearning Ġmethods Ġwith Ġfeatures Ġmight Ġclassify Ġthe Ġcaus ality Ġmessages Ġbetter Ġthan Ġthe Ġones Ġmore Ġoptimized Ġfor Ġcapturing Ġdistribution al Ġinformation Ġor Ġnon - linear Ġrelationships Ġof Ġfeatures . </s> Table Ġ4 Ġshows Ġthe Ġresults Ġof Ġa Ġfeature Ġab lation Ġtest Ġto Ġsee Ġhow Ġeach Ġfeature Ġcontributes Ġto Ġcaus ality Ġclassification Ġperformance Ġof Ġthe Ġlinear ĠS VM Ġclass ifier . ĠPOS Ġtags Ġcaused Ġthe Ġlargest Ġdrop Ġin ĠF 1 . ĠWe Ġsuspect ĠPOS Ġtags Ġplayed Ġa Ġunique Ġrole Ġbecause Ġdiscourse Ġconnect ives Ġcan Ġhave Ġvarious Ġsurface Ġforms Ġ( e . g ., Ġbecause , Ġc uz , Ġbc uz , Ġetc ) Ġbut Ġstill Ġthe Ġsame ĠPOS Ġtag Ġ` P '. ĠAlso ĠPOS Ġtags Ġcan Ġcapture Ġthe Ġoccurrences Ġof Ġmod al Ġverbs , Ġa Ġfeature Ġpreviously Ġfound Ġto Ġbe Ġvery Ġuseful Ġfor Ġdetecting Ġsimilar Ġdiscourse Ġrelations ĠB IB REF 9 Ġ. ĠN - gram Ġfeatures Ġcaused Ġ0 . 022 ĠF 1 Ġdrop Ġwhile Ġsentiment Ġtags Ġdid Ġnot Ġaffect Ġthe Ġmodel Ġwhen Ġremoved . ĠUnlike Ġthe Ġprevious Ġwork Ġwhere ĠFirst - Last , ĠFirst 3 Ġand ĠWord Ġpairs Ġtended Ġto Ġgain Ġa Ġlarge ĠF 1 Ġincrease Ġfor Ġmultic lass Ġdiscourse Ġrelation Ġprediction , Ġin Ġour Ġcase , Ġthey Ġdid Ġnot Ġaffect Ġthe Ġprediction Ġperformance Ġcompared Ġto Ġother Ġfeature Ġtypes Ġsuch Ġas ĠPOS Ġtags Ġor ĠN - gram s . </s> Ca usal ĠExplan ation ĠIdentification </s> In Ġthis Ġtask , Ġthe Ġmodel Ġidentifies Ġcausal Ġexplanations Ġgiven Ġthe Ġdiscourse Ġarguments Ġof Ġthe Ġcaus ality Ġmessage . ĠWe Ġexplored Ġover Ġthe Ġsame Ġmodels Ġas Ġthose Ġwe Ġused Ġfor Ġcaus ality Ġ( s ans Ġthe Ġoutput Ġlayer ), Ġand Ġfound Ġthe Ġalmost Ġopposite Ġtrend Ġof Ġperformances Ġ( see ĠTable Ġ5 Ġ). ĠThe ĠLinear ĠS VM Ġobtained Ġlowest ĠF 1 Ġwhile Ġthe ĠL ST M Ġmodel Ġmade Ġthe Ġbest Ġidentification Ġperformance . ĠAs Ġopposed Ġto Ġthe Ġsimple Ġbinary Ġclassification Ġof Ġthe Ġcaus ality Ġmessages , Ġin Ġorder Ġto Ġdetect Ġcausal Ġexplanation , Ġit Ġis Ġmore Ġbeneficial Ġto Ġconsider Ġthe Ġrelation Ġacross Ġdiscourse Ġarguments Ġof Ġthe Ġwhole Ġmessage Ġand Ġimplicit Ġdistribution al Ġrepresentation Ġdue Ġto Ġthe Ġimplicit Ġcausal Ġrelations Ġbetween Ġtwo Ġdistant Ġarguments . </s> Arch itect ural ĠVari ants </s> For Ġcaus ality Ġprediction , Ġwe Ġexperimented Ġwith Ġonly Ġword Ġtokens Ġin Ġthe Ġwhole Ġmessage Ġwithout Ġhelp Ġof ĠWord - level ĠL ST M Ġlayer Ġ( Word ĠL ST M ), Ġand ĠF 1 Ġdropped Ġby Ġ0 . 0 64 Ġ( CP Ġin ĠTable Ġ6 Ġ). ĠAlso , Ġwhen Ġwe Ġused Ġthe Ġaverage Ġof Ġthe Ġsequence Ġof Ġword Ġembed d ings Ġof Ġeach Ġdiscourse Ġargument Ġas Ġan Ġinput Ġto Ġthe ĠDA - level ĠL ST M Ġand Ġit Ġcaused ĠF 1 Ġdrop Ġof Ġ0 . 07 3 . ĠThis Ġsuggests Ġthat Ġthe Ġinformation Ġgained Ġfrom Ġboth Ġthe Ġinteraction Ġof Ġwords Ġin Ġand Ġin Ġbetween Ġdiscourse Ġarguments Ġhelp Ġwhen Ġthe Ġmodel Ġutilizes Ġthe Ġdistribution al Ġrepresentation Ġof Ġthe Ġtexts . </s> For Ġcausal Ġexplanation Ġidentification , Ġin Ġorder Ġto Ġtest Ġhow Ġthe ĠL ST M Ġclass ifier Ġworks Ġwithout Ġits Ġcapability Ġof Ġcapturing Ġthe Ġrelations Ġbetween Ġdiscourse Ġarguments , Ġwe Ġremoved ĠDA - level ĠL ST M Ġlayer Ġand Ġran Ġthe ĠL ST M Ġdirectly Ġon Ġthe Ġword Ġembed ding Ġsequence Ġfor Ġeach Ġdiscourse Ġargument Ġfor Ġclass ifying Ġwhether Ġthe Ġargument Ġis Ġcausal Ġexplanation , Ġand Ġthe Ġmodel Ġhad Ġ0 . 06 1 ĠF 1 Ġdrop Ġ( Word ĠL ST M Ġin ĠCE I Ġin ĠTable Ġ6 Ġ). ĠAlso , Ġwhen Ġwe Ġran ĠDA - level ĠL ST M Ġon Ġthe Ġaverage Ġvectors Ġof Ġthe Ġword Ġsequences Ġof Ġeach Ġdiscourse Ġargument Ġof Ġmessages , ĠF 1 Ġdecreased Ġto Ġ0 . 8 18 . ĠThis Ġfollows Ġthe Ġsimilar Ġpattern Ġobserved Ġfrom Ġother Ġtypes Ġof Ġmodels Ġperformances Ġ( i . e ., ĠSV Ms Ġand ĠRandom ĠForest Ġclass ifiers ) Ġthat Ġthe Ġmodels Ġwith Ġhigher Ġcomplexity Ġfor Ġcapturing Ġthe Ġinteraction Ġof Ġdiscourse Ġarguments Ġtend Ġto Ġidentify Ġcausal Ġexplanation Ġwith Ġthe Ġhigher Ġaccur acies . </s> For ĠCE I Ġtask , Ġwe Ġfound Ġthat Ġwhen Ġthe Ġmodel Ġran Ġon Ġthe Ġsequence Ġrepresentation Ġof Ġdiscourse Ġargument Ġ( DA ĠAVG ĠL ST M ), Ġits Ġperformance Ġwas Ġhigher Ġthan Ġthe Ġplain Ġsequence Ġof Ġword Ġembed d ings Ġ( Word ĠL ST M ). ĠFinally , Ġin Ġboth Ġsubt asks , Ġwhen Ġthe Ġmodels Ġran Ġon Ġboth ĠWord - level Ġand ĠDA - Level Ġ( Full ĠL ST M ), Ġthey Ġobtained Ġthe Ġhighest Ġperformance . </s> Complete ĠPipeline </s> E val uations Ġthus Ġfar Ġzero ed - in Ġon Ġeach Ġsubt ask Ġof Ġcausal Ġexplanation Ġanalysis Ġ( i . e . ĠCE I Ġonly Ġfocused Ġon Ġdata Ġalready Ġidentified Ġto Ġcontain Ġcausal Ġexplanations ). ĠHere , Ġwe Ġseek Ġto Ġevaluate Ġthe Ġcomplete Ġpipeline Ġof ĠCP Ġand ĠCE I , Ġstarting Ġfrom Ġall Ġof Ġtest Ġdata Ġ( those Ġor Ġwithout Ġcaus ality ) Ġand Ġevaluating Ġthe Ġfinal Ġaccuracy Ġof ĠCE I Ġpredictions . ĠThis Ġis Ġintended Ġto Ġevaluate ĠCE I Ġperformance Ġunder Ġan Ġapplied Ġsetting Ġwhere Ġone Ġdoes Ġnot Ġalready Ġknow Ġwhether Ġa Ġdocument Ġhas Ġa Ġcausal Ġexplanation . </s> There Ġare Ġseveral Ġapproaches Ġwe Ġcould Ġtake Ġto Ġperform ĠCE I Ġstarting Ġfrom Ġun annot ated Ġdata . ĠWe Ġcould Ġsimply Ġrun ĠCE I Ġprediction Ġby Ġitself Ġ( CE I ĠOnly ) Ġor Ġthe Ġpipeline Ġof ĠCP Ġfirst Ġand Ġthen Ġonly Ġrun ĠCE I Ġon Ġdocuments Ġpredicted Ġas Ġcausal Ġ( CP Ġ+ ĠCE I ). ĠFurther , Ġthe ĠCE I Ġmodel Ġcould Ġbe Ġtrained Ġonly Ġon Ġthose Ġdocuments Ġannot ated Ġcausal Ġ( as Ġwas Ġdone Ġin Ġthe Ġprevious Ġexperiments ) Ġor Ġon Ġall Ġtraining Ġdocuments Ġincluding Ġmany Ġthat Ġare Ġnot Ġcausal . </s> Table Ġ7 Ġshow Ġresults Ġvarying Ġthe Ġpipeline Ġand Ġhow ĠCE I Ġwas Ġtrained . ĠThough Ġall Ġsetups Ġperformed Ġdecent Ġ( Ġ$ F 1 Ġ> Ġ0 . 81 $ Ġ) Ġwe Ġsee Ġthat Ġthe Ġpip el ined Ġapproach , Ġfirst Ġpredicting Ġcaus ality Ġ( with Ġthe Ġlinear ĠS VM ) Ġand Ġthen Ġpredicting Ġcausal Ġexplanations Ġonly Ġfor Ġthose Ġwith Ġmarked Ġcausal Ġ( CP Ġ+ ĠCE I Ġ$ _{ ca usal } $ Ġ) Ġyielded Ġthe Ġstrongest Ġresults . ĠThis Ġalso Ġutilized Ġthe ĠCE I Ġmodel Ġonly Ġtrained Ġon Ġthose Ġannot ated Ġcausal . ĠBesides Ġperformance , Ġan Ġadded Ġbenefit Ġfrom Ġthis Ġtwo Ġstep Ġapproach Ġis Ġthat Ġthe ĠCP Ġstep Ġis Ġless Ġcomputational Ġintensive Ġof Ġthe ĠCE I Ġstep Ġand Ġapproximately Ġ2 / 3 Ġof Ġdocuments Ġwill Ġnever Ġneed Ġthe ĠCE I Ġstep Ġapplied . </s> We Ġhad Ġan Ġinevitable Ġlimitation Ġon Ġthe Ġsize Ġof Ġour Ġdataset , Ġsince Ġthere Ġis Ġno Ġother Ġcaus ality Ġdataset Ġover Ġsocial Ġmedia Ġand Ġthe Ġannotation Ġrequired Ġan Ġintensive Ġiter ative Ġprocess . ĠThis Ġmight Ġhave Ġlimited Ġperformances Ġof Ġmore Ġcomplex Ġmodels , Ġbut Ġconsidering Ġthe Ġprocessing Ġtime Ġand Ġthe Ġcomputation Ġload , Ġthe Ġcombination Ġof Ġthe Ġlinear Ġmodel Ġand Ġthe ĠR NN - based Ġmodel Ġof Ġour Ġpipeline Ġobtained Ġboth Ġthe Ġhigh Ġperformance Ġand Ġefficiency Ġfor Ġthe Ġpractical Ġapplications Ġto Ġdownstream Ġtasks . ĠIn Ġother Ġwords , Ġit 's Ġpossible Ġthe Ġlinear Ġmodel Ġwill Ġnot Ġperform Ġas Ġwell Ġif Ġthe Ġtraining Ġsize Ġis Ġincreased Ġsubstantially . ĠHowever , Ġa Ġlinear Ġmodel Ġcould Ġstill Ġbe Ġused Ġto Ġdo Ġa Ġfirst - pass , Ġcomput ationally Ġefficient Ġlabeling , Ġin Ġorder Ġto Ġshort list Ġsocial Ġmedia Ġposts Ġfor Ġfurther Ġlabeling Ġfrom Ġan ĠL ST M Ġor Ġmore Ġcomplex Ġmodel . </s> Expl oration </s> Here , Ġwe Ġexplore Ġthe Ġuse Ġof Ġcausal Ġexplanation Ġanalysis Ġfor Ġdownstream Ġtasks . ĠFirst Ġwe Ġlook Ġat Ġthe Ġrelationship Ġbetween Ġuse Ġof Ġcausal Ġexplanation Ġand Ġone 's Ġdemographics : Ġage Ġand Ġgender . ĠThen , Ġwe Ġconsider Ġtheir Ġuse Ġin Ġsentiment Ġanalysis Ġfor Ġextracting Ġthe Ġcauses Ġof Ġpol arity Ġratings . ĠResearch Ġinvolving Ġhuman Ġsubjects Ġwas Ġapproved Ġby Ġthe ĠUniversity Ġof ĠPennsylvania ĠInst itutional ĠReview ĠBoard . </s> Conclusion </s> We Ġdeveloped Ġa Ġpipeline Ġfor Ġcausal Ġexplanation Ġanalysis Ġover Ġsocial Ġmedia Ġtext , Ġincluding Ġboth Ġcaus ality Ġprediction Ġand Ġcausal Ġexplanation Ġidentification . ĠWe Ġexamined Ġa Ġvariety Ġof Ġmodel Ġtypes Ġand ĠR NN Ġarchitectures Ġfor Ġeach Ġpart Ġof Ġthe Ġpipeline , Ġfinding Ġan ĠS VM Ġbest Ġfor Ġcaus ality Ġprediction Ġand Ġa Ġhierarchy Ġof ĠBi L ST Ms Ġfor Ġcausal Ġexplanation Ġidentification , Ġsuggesting Ġthe Ġlater Ġtask Ġrelies Ġmore Ġheavily Ġon Ġsequential Ġinformation . ĠIn Ġfact , Ġwe Ġfound Ġreplacing Ġeither Ġlayer Ġof Ġthe Ġhierarchical ĠL ST M Ġarchitecture Ġ( the Ġword - level Ġor Ġthe ĠDA - level ) Ġwith Ġa Ġan Ġequivalent ĠâĢ ľ bag Ġof Ġfeatures âĢ Ŀ Ġapproach Ġresulted Ġin Ġreduced Ġaccuracy . ĠResults Ġof Ġour Ġwhole Ġpipeline Ġof Ġcausal Ġexplanation Ġanalysis Ġwere Ġfound Ġquite Ġstrong , Ġachieving Ġan Ġ$ F 1 = 0 . 8 68 $ Ġat Ġidentifying Ġdiscourse Ġarguments Ġthat Ġare Ġcausal Ġexplanations . </s> Finally , Ġwe Ġdemonstrated Ġuse Ġof Ġour Ġmodels Ġin Ġapplications , Ġfinding Ġassociations Ġbetween Ġdemographics Ġand Ġrate Ġof Ġmentioning Ġcausal Ġexplanations , Ġas Ġwell Ġas Ġshowing Ġdifferences Ġin Ġthe Ġtop Ġwords Ġpredictive Ġof Ġnegative Ġratings Ġin ĠYelp Ġreviews . ĠUt il ization Ġof Ġdiscourse Ġstructure Ġin Ġsocial Ġmedia Ġanalysis Ġhas Ġbeen Ġa Ġlargely Ġunt apped Ġarea Ġof Ġexploration , Ġperhaps Ġdue Ġto Ġits Ġperceived Ġdifficulty . ĠWe Ġhope Ġthe Ġstrong Ġresults Ġof Ġcausal Ġexplanation Ġidentification Ġhere Ġleads Ġto Ġthe Ġintegration Ġof Ġmore Ġsyntax Ġand Ġdeeper Ġsemantics Ġinto Ġsocial Ġmedia Ġanalyses Ġand Ġultimately Ġenables Ġnew Ġapplications Ġbeyond Ġthe Ġcurrent Ġstate Ġof Ġthe Ġart . </s> Acknowled gments </s> This Ġwork Ġwas Ġsupported , Ġin Ġpart , Ġby Ġa Ġgrant Ġfrom Ġthe ĠTemple ton ĠReligion ĠTrust Ġ( ID Ġ# TR T 00 48 ). ĠThe Ġfund ers Ġhad Ġno Ġrole Ġin Ġstudy Ġdesign , Ġdata Ġcollection Ġand Ġanalysis , Ġdecision Ġto Ġpublish , Ġor Ġpreparation Ġof Ġthe Ġmanuscript . ĠWe Ġalso Ġthank ĠLaura ĠSmith , ĠY iy i ĠChen , ĠGret a ĠJaw el Ġand ĠVanessa ĠHernandez Ġfor Ġtheir Ġwork Ġin Ġidentifying Ġcausal Ġexplanations .