Irony Classifier, Sentiment Classifier for Irony, Sentiment Classifier for Non-irony, transformation from ironic sentences to non-ironic sentences
EXTRACTIVE
Irony Classifier: We implement a CNN classifier trained with our irony dataset. All the CNN classifiers we utilize in this paper use the same parameters as BIBREF20 . Sentiment Classifier for Irony: We first implement a one-layer LSTM network to classify ironic sentences in our dataset into positive and negative ironies. The LSTM network is trained with the dataset of Semeval 2015 Task 11 BIBREF0 which is used for the sentiment analysis of figurative language in twitter. Then, we use the positive ironies and negative ironies to train the CNN sentiment classifier for irony. Sentiment Classifier for Non-irony: Similar to the training process of the sentiment classifier for irony, we first implement a one-layer LSTM network trained with the dataset for the sentiment analysis of common twitters to classify the non-ironies into positive and negative non-ironies. Then we use the positive and negative non-ironies to train the sentiment classifier for non-irony. In this section, we describe some additional experiments on the transformation from ironic sentences to non-ironic sentences. Sometimes ironies are hard to understand and may cause misunderstanding, for which our task also explores the transformation from ironic sentences to non-ironic sentences.
1909.06200
Introduction </s> The Ġirony Ġis Ġa Ġkind Ġof Ġfigur ative Ġlanguage , Ġwhich Ġis Ġwidely Ġused Ġon Ġsocial Ġmedia ĠB IB REF 0 Ġ. ĠThe Ġirony Ġis Ġdefined Ġas Ġa Ġclash Ġbetween Ġthe Ġintended Ġmeaning Ġof Ġa Ġsentence Ġand Ġits Ġliteral Ġmeaning ĠB IB REF 1 Ġ. ĠAs Ġan Ġimportant Ġaspect Ġof Ġlanguage , Ġirony Ġplays Ġan Ġessential Ġrole Ġin Ġsentiment Ġanalysis ĠB IB REF 2 Ġ, ĠB IB REF 0 Ġand Ġopinion Ġmining ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ. </s> Although Ġsome Ġprevious Ġstudies Ġfocus Ġon Ġirony Ġdetection , Ġlittle Ġattention Ġis Ġpaid Ġto Ġirony Ġgeneration . ĠAs Ġiron ies Ġcan Ġstrengthen Ġsentiments Ġand Ġexpress Ġstronger Ġemotions , Ġwe Ġmainly Ġfocus Ġon Ġgenerating Ġironic Ġsentences . ĠGiven Ġa Ġnon - iron ic Ġsentence , Ġwe Ġimplement Ġa Ġneural Ġnetwork Ġto Ġtransfer Ġit Ġto Ġan Ġironic Ġsentence Ġand Ġconst rain Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġtwo Ġsentences Ġto Ġbe Ġthe Ġsame . ĠFor Ġexample , Ġthe Ġinput Ġis ĠâĢ ľ I Ġhate Ġit Ġwhen Ġmy Ġplans Ġget Ġruined " Ġwhich Ġis Ġnegative Ġin Ġsentiment Ġpol arity Ġand Ġthe Ġoutput Ġshould Ġbe Ġironic Ġand Ġnegative Ġin Ġsentiment Ġas Ġwell , Ġsuch Ġas ĠâĢ ľ I Ġlike Ġit Ġwhen Ġmy Ġplans Ġget Ġruined ". ĠThe Ġspeaker Ġuses ĠâĢ ľ like " Ġto Ġbe Ġironic Ġand Ġexpress Ġhis Ġor Ġher Ġnegative Ġsentiment . ĠAt Ġthe Ġsame Ġtime , Ġour Ġmodel Ġcan Ġpreserve Ġcontents Ġwhich Ġare Ġirrelevant Ġto Ġsentiment Ġpol arity Ġand Ġirony . ĠAccording Ġto Ġthe Ġcategories Ġmentioned Ġin ĠB IB REF 5 Ġ, Ġirony Ġcan Ġbe Ġclassified Ġinto Ġ3 Ġclasses : Ġverbal Ġirony Ġby Ġmeans Ġof Ġa Ġpol arity Ġcontrast , Ġthe Ġsentences Ġcontaining Ġexpression Ġwhose Ġpol arity Ġis Ġinverted Ġbetween Ġthe Ġintended Ġand Ġthe Ġliteral Ġevaluation ; Ġother Ġtypes Ġof Ġverbal Ġirony , Ġthe Ġsentences Ġthat Ġshow Ġno Ġpol arity Ġcontrast Ġbetween Ġthe Ġliteral Ġand Ġintended Ġmeaning Ġbut Ġare Ġstill Ġironic ; Ġand Ġsituational Ġirony , Ġthe Ġsentences Ġthat Ġdescribe Ġsituations Ġthat Ġfail Ġto Ġmeet Ġsome Ġexpectations . ĠAs Ġiron ies Ġin Ġthe Ġlatter Ġtwo Ġcategories Ġare Ġobscure Ġand Ġhard Ġto Ġunderstand , Ġwe Ġdecide Ġto Ġonly Ġfocus Ġon Ġiron ies Ġin Ġthe Ġfirst Ġcategory Ġin Ġthis Ġwork . ĠFor Ġexample , Ġour Ġwork Ġcan Ġbe Ġspecifically Ġdescribed Ġas : Ġgiven Ġa Ġsentence ĠâĢ ľ I Ġhate Ġto Ġbe Ġignored ", Ġwe Ġtrain Ġour Ġmodel Ġto Ġgenerate Ġan Ġironic Ġsentence Ġsuch Ġas ĠâĢ ľ I Ġlove Ġto Ġbe Ġignored ". ĠAlthough Ġthere Ġis ĠâĢ ľ love " Ġin Ġthe Ġgenerated Ġsentence , Ġthe Ġspeaker Ġstill Ġexpresses Ġhis Ġor Ġher Ġnegative Ġsentiment Ġby Ġirony . ĠWe Ġalso Ġmake Ġsome Ġexplor ations Ġin Ġthe Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences Ġat Ġthe Ġend Ġof Ġour Ġwork . ĠBecause Ġof Ġthe Ġlack Ġof Ġprevious Ġwork Ġand Ġbas elines Ġon Ġirony Ġgeneration , Ġwe Ġimplement Ġour Ġmodel Ġbased Ġon Ġstyle Ġtransfer . ĠOur Ġwork Ġwill Ġnot Ġonly Ġprovide Ġthe Ġfirst Ġlarge - scale Ġirony Ġdataset Ġbut Ġalso Ġmake Ġour Ġmodel Ġas Ġa Ġbenchmark Ġfor Ġthe Ġirony Ġgeneration . </s> Recently , Ġun super vised Ġstyle Ġtransfer Ġbecomes Ġa Ġvery Ġpopular Ġtopic . ĠMany Ġstate - of - the - art Ġstudies Ġtry Ġto Ġsolve Ġthe Ġtask Ġwith Ġsequence - to - sequence Ġ( seq 2 seq ) Ġframework . ĠThere Ġare Ġthree Ġmain Ġways Ġto Ġbuild Ġup Ġmodels . ĠThe Ġfirst Ġis Ġto Ġlearn Ġa Ġlatent Ġstyle - independent Ġcontent Ġrepresentation Ġand Ġgenerate Ġsentences Ġwith Ġthe Ġcontent Ġrepresentation Ġand Ġanother Ġstyle ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ. ĠThe Ġsecond Ġis Ġto Ġdirectly Ġtransfer Ġsentences Ġfrom Ġone Ġstyle Ġto Ġanother Ġunder Ġthe Ġcontrol Ġof Ġclass ifiers Ġand Ġreinforcement Ġlearning ĠB IB REF 8 Ġ. ĠThe Ġthird Ġis Ġto Ġremove Ġstyle Ġattribute Ġwords Ġfrom Ġthe Ġinput Ġsentence Ġand Ġcombine Ġthe Ġremaining Ġcontent Ġwith Ġnew Ġstyle Ġattribute Ġwords ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ. ĠThe Ġfirst Ġmethod Ġusually Ġob tains Ġbetter Ġperformances Ġvia Ġadvers arial Ġtraining Ġwith Ġdiscrim inators . ĠThe Ġstyle - independent Ġcontent Ġrepresentation , Ġnevertheless , Ġis Ġnot Ġeasily Ġobtained ĠB IB REF 11 Ġ, Ġwhich Ġresults Ġin Ġpoor Ġperformances . ĠThe Ġsecond Ġmethod Ġis Ġsuitable Ġfor Ġcomplex Ġstyles Ġwhich Ġare Ġdifficult Ġto Ġmodel Ġand Ġdescribe . ĠThe Ġmodel Ġcan Ġlearn Ġthe Ġdeep Ġsemantic Ġfeatures Ġby Ġitself Ġbut Ġsometimes Ġthe Ġmodel Ġis Ġsensitive Ġto Ġparameters Ġand Ġhard Ġto Ġtrain . ĠThe Ġthird Ġmethod Ġsucceeds Ġto Ġpreserve Ġcontent Ġbut Ġcannot Ġwork Ġfor Ġsome Ġcomplex Ġstyles Ġsuch Ġas Ġdemocratic Ġand Ġrepublican . ĠSent ences Ġwith Ġthose Ġstyles Ġusually Ġdo Ġnot Ġhave Ġspecific Ġstyle Ġattribute Ġwords . ĠUnfortunately , Ġdue Ġto Ġthe Ġlack Ġof Ġlarge Ġirony Ġdataset Ġand Ġdifficulties Ġof Ġmodeling Ġiron ies , Ġthere Ġhas Ġbeen Ġlittle Ġwork Ġtrying Ġto Ġgenerate Ġiron ies Ġbased Ġon Ġseq 2 seq Ġframework Ġas Ġfar Ġas Ġwe Ġknow . ĠInspired Ġby Ġmethods Ġfor Ġstyle Ġtransfer , Ġwe Ġdecide Ġto Ġimplement Ġa Ġspecifically Ġdesigned Ġmodel Ġbased Ġon Ġun super vised Ġstyle Ġtransfer Ġto Ġexplore Ġirony Ġgeneration . </s> In Ġthis Ġpaper , Ġin Ġorder Ġto Ġaddress Ġthe Ġlack Ġof Ġirony Ġdata , Ġwe Ġfirst Ġcrawl Ġover Ġ2 M Ġtweets Ġfrom Ġtwitter Ġto Ġbuild Ġa Ġdataset Ġwith Ġ262 , 755 Ġironic Ġand Ġ112 , 330 Ġnon - iron ic Ġtweets . ĠThen , Ġdue Ġto Ġthe Ġlack Ġof Ġparallel Ġdata , Ġwe Ġpropose Ġa Ġnovel Ġmodel Ġto Ġtransfer Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences Ġin Ġan Ġun super vised Ġway . ĠAs Ġironic Ġstyle Ġis Ġhard Ġto Ġmodel Ġand Ġdescribe , Ġwe Ġimplement Ġour Ġmodel Ġwith Ġthe Ġcontrol Ġof Ġclass ifiers Ġand Ġreinforcement Ġlearning . ĠDifferent Ġfrom Ġother Ġstudies Ġin Ġstyle Ġtransfer , Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġto Ġironic Ġsentences Ġhas Ġto Ġpreserve Ġsentiment Ġpol arity Ġas Ġmentioned Ġabove . ĠTherefore , Ġwe Ġnot Ġonly Ġdesign Ġan Ġirony Ġreward Ġto Ġcontrol Ġthe Ġirony Ġaccuracy Ġand Ġimplement Ġden o ising Ġauto - enc oder Ġand Ġback - translation Ġto Ġcontrol Ġcontent Ġpreservation Ġbut Ġalso Ġdesign Ġa Ġsentiment Ġreward Ġto Ġcontrol Ġsentiment Ġpreservation . </s> Exper imental Ġresults Ġdemonstrate Ġthat Ġour Ġmodel Ġachieves Ġa Ġhigh Ġirony Ġaccuracy Ġwith Ġwell - pres erved Ġsentiment Ġand Ġcontent . ĠThe Ġcontributions Ġof Ġour Ġwork Ġare Ġas Ġfollows : </s> Related ĠWork </s> Style ĠTransfer : ĠAs Ġirony Ġis Ġa Ġcomplicated Ġstyle Ġand Ġhard Ġto Ġmodel Ġwith Ġsome Ġspecific Ġstyle Ġattribute Ġwords , Ġwe Ġmainly Ġfocus Ġon Ġstudies Ġwithout Ġediting Ġstyle Ġattribute Ġwords . </s> Some Ġstudies Ġare Ġtrying Ġto Ġdis ent angle Ġstyle Ġrepresentation Ġfrom Ġcontent Ġrepresentation . ĠIn ĠB IB REF 12 Ġ, Ġauthors Ġleverage Ġadvers arial Ġnetworks Ġto Ġlearn Ġseparate Ġcontent Ġrepresentations Ġand Ġstyle Ġrepresentations . ĠIn ĠB IB REF 13 Ġand ĠB IB REF 6 Ġ, Ġresearchers Ġcombine Ġvari ational Ġauto - enc od ers Ġ( VA Es ) Ġwith Ġstyle Ġdiscrim inators . </s> However , Ġsome Ġrecent Ġstudies ĠB IB REF 11 Ġreveal Ġthat Ġthe Ġdis ent ang lement Ġof Ġcontent Ġand Ġstyle Ġrepresentations Ġmay Ġnot Ġbe Ġachieved Ġin Ġpractice . ĠTherefore , Ġsome Ġother Ġresearch Ġstudies ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġstrive Ġto Ġseparate Ġcontent Ġand Ġstyle Ġby Ġremoving Ġstyl istic Ġwords . ĠNonetheless , Ġmany Ġnon - iron ic Ġsentences Ġdo Ġnot Ġhave Ġspecific Ġstyl istic Ġwords Ġand Ġas Ġa Ġresult , Ġwe Ġfind Ġit Ġdifficult Ġto Ġtransfer Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences Ġthrough Ġthis Ġway Ġin Ġpractice . </s> Besides , Ġsome Ġother Ġresearch Ġstudies Ġdo Ġnot Ġdis ent angle Ġstyle Ġfrom Ġcontent Ġbut Ġdirectly Ġlearn Ġrepresentations Ġof Ġsentences . ĠIn ĠB IB REF 8 Ġ, Ġauthors Ġpropose Ġa Ġdual Ġreinforcement Ġlearning Ġframework Ġwithout Ġseparating Ġcontent Ġand Ġstyle Ġrepresentations . ĠIn ĠB IB REF 7 Ġ, Ġresearchers Ġutilize Ġa Ġmachine Ġtranslation Ġmodel Ġto Ġlearn Ġa Ġsentence Ġrepresentation Ġpreserving Ġthe Ġmeaning Ġof Ġthe Ġsentence Ġbut Ġreducing Ġstyl istic Ġproperties . ĠIn Ġthis Ġmethod , Ġthe Ġquality Ġof Ġgenerated Ġsentences Ġrelies Ġon Ġthe Ġperformance Ġof Ġclass ifiers Ġto Ġa Ġlarge Ġextent . ĠMeanwhile , Ġsuch Ġmodels Ġare Ġusually Ġsensitive Ġto Ġparameters Ġand Ġdifficult Ġto Ġtrain . ĠIn Ġcontrast , Ġwe Ġcombine Ġa Ġpre - training Ġprocess Ġwith Ġreinforcement Ġlearning Ġto Ġbuild Ġup Ġa Ġstable Ġlanguage Ġmodel Ġand Ġdesign Ġspecial Ġrewards Ġfor Ġour Ġtask . </s> Iron y ĠDetection : ĠWith Ġthe Ġdevelopment Ġof Ġsocial Ġmedia , Ġirony Ġdetection Ġbecomes Ġa Ġmore Ġimportant Ġtask . ĠMethods Ġfor Ġirony Ġdetection Ġcan Ġbe Ġmainly Ġdivided Ġinto Ġtwo Ġcategories : Ġmethods Ġbased Ġon Ġfeature Ġengineering Ġand Ġmethods Ġbased Ġon Ġneural Ġnetworks . </s> As Ġfor Ġmethods Ġbased Ġon Ġfeature Ġengineering , ĠIn ĠB IB REF 1 Ġ, Ġauthors Ġinvestigate Ġpragmatic Ġphenomena Ġand Ġvarious Ġirony Ġmarkers . ĠIn ĠB IB REF 14 Ġ, Ġresearchers Ġleverage Ġa Ġcombination Ġof Ġsentiment , Ġdistribution al Ġsemantic Ġand Ġtext Ġsurface Ġfeatures . ĠThose Ġmodels Ġrely Ġon Ġhand - crafted Ġfeatures Ġand Ġare Ġhard Ġto Ġimplement . </s> When Ġit Ġcomes Ġto Ġmethods Ġbased Ġon Ġneural Ġnetworks , Ġlong Ġshort - term Ġmemory Ġ( L ST M ) ĠB IB REF 15 Ġnetwork Ġis Ġwidely Ġused Ġand Ġis Ġvery Ġefficient Ġfor Ġirony Ġdetection . ĠIn ĠB IB REF 16 Ġ, Ġa Ġtweet Ġis Ġdivided Ġinto Ġtwo Ġsegments Ġand Ġa Ġsubtract Ġlayer Ġis Ġimplemented Ġto Ġcalculate Ġthe Ġdifference Ġbetween Ġtwo Ġsegments Ġin Ġorder Ġto Ġdetermine Ġwhether Ġthe Ġtweet Ġis Ġironic . ĠIn ĠB IB REF 17 Ġ, Ġauthors Ġutilize Ġa Ġrecurrent Ġneural Ġnetwork Ġwith ĠBi - L ST M Ġand Ġself - att ention Ġwithout Ġhand - crafted Ġfeatures . ĠIn ĠB IB REF 18 Ġ, Ġresearchers Ġpropose Ġa Ġsystem Ġbased Ġon Ġa Ġdensely Ġconnected ĠL ST M Ġnetwork . </s> Our ĠDat as et </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġhow Ġwe Ġbuild Ġour Ġdataset Ġwith Ġtweets . ĠFirst , Ġwe Ġcrawl Ġover Ġ2 M Ġtweets Ġfrom Ġtwitter Ġusing ĠGet Old Twe ets - python . ĠWe Ġcrawl ĠEnglish Ġtweets Ġfrom Ġ04 / 09 / 2012 Ġto Ġ/ 12 / 18 / 2018 . ĠWe Ġfirst Ġremove Ġall Ġre - t we ets Ġand Ġuse Ġlang det ect Ġto Ġremove Ġall Ġnon - English Ġsentences . ĠThen , Ġwe Ġremove Ġhasht ags Ġattached Ġat Ġthe Ġend Ġof Ġthe Ġtweets Ġbecause Ġthey Ġare Ġusually Ġnot Ġparts Ġof Ġsentences Ġand Ġwill Ġconfuse Ġour Ġlanguage Ġmodel . ĠAfter Ġthat , Ġwe Ġutilize ĠEk ph ras is Ġto Ġprocess Ġtweets . ĠWe Ġremove ĠURLs Ġand Ġrestore Ġremaining Ġhasht ags , Ġelong ated Ġwords , Ġrepeated Ġwords , Ġand Ġall - capital ized Ġwords . ĠTo Ġsimplify Ġour Ġdataset , ĠWe Ġreplace Ġall ĠâĢ ľ ĠIN LINE FORM 0 Ġmoney ĠIN LINE FORM 1 Ġ" Ġand ĠâĢ ľ ĠIN LINE FORM 2 Ġtime ĠIN LINE FORM 3 Ġ" Ġtokens Ġwith ĠâĢ ľ ĠIN LINE FORM 4 Ġnumber ĠIN LINE FORM 5 Ġ" Ġtoken Ġwhen Ġusing ĠEk ph ras is . ĠAnd Ġwe Ġdelete Ġsentences Ġwhose Ġlengths Ġare Ġless Ġthan Ġ10 Ġor Ġgreater Ġthan Ġ40 . ĠIn Ġorder Ġto Ġrestore Ġabbrevi ations , Ġwe Ġdownload Ġan Ġabbre viation Ġdictionary Ġfrom Ġweb op edia Ġand Ġrestore Ġabbrevi ations Ġto Ġnormal Ġwords Ġor Ġphrases Ġaccording Ġto Ġthe Ġdictionary . ĠFinally , Ġwe Ġremove Ġsentences Ġwhich Ġhave Ġmore Ġthan Ġtwo Ġrare Ġwords Ġ( app earing Ġless Ġthan Ġthree Ġtimes ) Ġin Ġorder Ġto Ġconst rain Ġthe Ġsize Ġof Ġvocabulary . ĠFinally , Ġwe Ġget Ġ6 62 , 530 Ġsentences Ġafter Ġpre - processing . </s> As Ġneural Ġnetworks Ġare Ġproved Ġeffective Ġin Ġirony Ġdetection , Ġwe Ġdecide Ġto Ġimplement Ġa Ġneural Ġclass ifier Ġin Ġorder Ġto Ġclassify Ġthe Ġsentences Ġinto Ġironic Ġand Ġnon - iron ic Ġsentences . ĠHowever , Ġthe Ġonly Ġhigh - quality Ġirony Ġdataset Ġwe Ġcan Ġobtain Ġis Ġthe Ġdataset Ġof ĠSem eval - 2018 ĠTask Ġ3 Ġand Ġthe Ġdataset Ġis Ġpretty Ġsmall , Ġwhich Ġwill Ġcause Ġover fitting Ġto Ġcomplex Ġmodels . ĠTherefore , Ġwe Ġjust Ġimplement Ġa Ġsimple Ġone - layer ĠR NN Ġwith ĠL ST M Ġcell Ġto Ġclassify Ġpre - process ed Ġsentences Ġinto Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġbecause ĠL ST M Ġnetworks Ġare Ġwidely Ġused Ġin Ġirony Ġdetection . ĠWe Ġtrain Ġthe Ġmodel Ġwith Ġthe Ġdataset Ġof ĠSem eval - 2018 ĠTask Ġ3 . ĠAfter Ġclassification , Ġwe Ġget Ġ262 , 755 Ġironic Ġsentences Ġand Ġ399 , 775 Ġnon - iron ic Ġsentences . ĠAccording Ġto Ġour Ġobservation , Ġnot Ġall Ġnon - iron ic Ġsentences Ġare Ġsuitable Ġto Ġbe Ġtransferred Ġinto Ġironic Ġsentences . ĠFor Ġexample , ĠâĢ ľ just Ġhanging Ġout Ġ. Ġwatching Ġ. Ġis Ġit Ġm onday Ġyet " Ġis Ġhard Ġto Ġtransfer Ġbecause Ġit Ġdoes Ġnot Ġhave Ġan Ġexplicit Ġsentiment Ġpol arity . ĠSo Ġwe Ġremove Ġall Ġinterrog ative Ġsentences Ġfrom Ġthe Ġnon - iron ic Ġsentences Ġand Ġonly Ġobtain Ġthe Ġsentences Ġwhich Ġhave Ġwords Ġexpressing Ġstrong Ġsentiments . ĠWe Ġevaluate Ġthe Ġsentiment Ġpol arity Ġof Ġeach Ġword Ġwith ĠText Bl ob Ġand Ġwe Ġview Ġthose Ġwords Ġwith Ġsentiment Ġscores Ġgreater Ġthan Ġ0 . 5 Ġor Ġless Ġthan Ġ- 0 . 5 Ġas Ġwords Ġexpressing Ġstrong Ġsentiments . ĠFinally , Ġwe Ġbuild Ġour Ġirony Ġdataset Ġwith Ġ262 , 755 Ġironic Ġsentences Ġand Ġ102 , 330 Ġnon - iron ic Ġsentences . </s> [ t ] ĠIron y ĠGeneration ĠAl gorithm </s> IN LINE FORM 0 Ġpre - train Ġwith Ġauto - enc oder ĠPre - train ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġwith ĠIN LINE FORM 3 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 16 ĠPre - train ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġwith ĠIN LINE FORM 6 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 17 ĠIN LINE FORM 7 Ġpre - train Ġwith Ġback - translation ĠPre - train ĠIN LINE FORM 8 Ġ, ĠIN LINE FORM 9 Ġ, ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġwith ĠIN LINE FORM 12 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 19 ĠPre - train ĠIN LINE FORM 13 Ġ, ĠIN LINE FORM 14 Ġ, ĠIN LINE FORM 15 Ġ, ĠIN LINE FORM 16 Ġwith ĠIN LINE FORM 17 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 20 </s> IN LINE FORM 0 Ġtrain Ġwith ĠRL Ġeach Ġepoch Ġe Ġ= Ġ1 , Ġ2 , Ġ... , ĠIN LINE FORM 1 ĠIN LINE FORM 2 Ġtrain Ġnon - ir ony 2 ir ony Ġwith ĠRL ĠIN LINE FORM 3 Ġin ĠN ĠIN LINE FORM 4 Ġupdate ĠIN LINE FORM 5 Ġ, ĠIN LINE FORM 6 Ġ, Ġusing ĠIN LINE FORM 7 Ġbased Ġon ĠE q . ĠEQ REF 29 ĠIN LINE FORM 8 Ġback - translation ĠIN LINE FORM 9 ĠIN LINE FORM 10 ĠIN LINE FORM 11 Ġupdate ĠIN LINE FORM 12 Ġ, ĠIN LINE FORM 13 Ġ, ĠIN LINE FORM 14 Ġ, ĠIN LINE FORM 15 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 19 ĠIN LINE FORM 16 Ġtrain Ġirony 2 non - ir ony Ġwith ĠRL ĠIN LINE FORM 17 Ġin ĠI ĠIN LINE FORM 18 Ġupdate ĠIN LINE FORM 19 Ġ, ĠIN LINE FORM 20 Ġ, Ġusing ĠIN LINE FORM 21 Ġsimilar Ġto ĠE q . ĠEQ REF 29 ĠIN LINE FORM 22 Ġback - translation ĠIN LINE FORM 23 ĠIN LINE FORM 24 ĠIN LINE FORM 25 Ġupdate ĠIN LINE FORM 26 Ġ, ĠIN LINE FORM 27 Ġ, ĠIN LINE FORM 28 Ġ, ĠIN LINE FORM 29 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 20 </s> Our ĠMethod </s> Given Ġtwo Ġnon - par allel Ġcorpor a : Ġnon - iron ic Ġcorpus ĠN ={ ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġ, Ġ... , ĠIN LINE FORM 2 Ġ} Ġand Ġironic Ġcorpus ĠI ={ ĠIN LINE FORM 3 Ġ, ĠIN LINE FORM 4 Ġ, Ġ... , ĠIN LINE FORM 5 Ġ}, Ġthe Ġgoal Ġof Ġour Ġirony Ġgeneration Ġmodel Ġis Ġto Ġgenerate Ġan Ġironic Ġsentence Ġfrom Ġa Ġnon - iron ic Ġsentence Ġwhile Ġpreserving Ġthe Ġcontent Ġand Ġsentiment Ġpol arity Ġof Ġthe Ġsource Ġinput Ġsentence . ĠWe Ġimplement Ġan Ġenc oder - dec oder Ġframework Ġwhere Ġtwo Ġenc od ers Ġare Ġutilized Ġto Ġencode Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġrespectively Ġand Ġtwo Ġdec od ers Ġare Ġutilized Ġto Ġdecode Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġfrom Ġlatent Ġrepresentations Ġrespectively . ĠIn Ġorder Ġto Ġenforce Ġa Ġshared Ġlatent Ġspace , Ġwe Ġshare Ġtwo Ġlayers Ġon Ġboth Ġthe Ġenc oder Ġside Ġand Ġthe Ġdec oder Ġside . ĠOur Ġmodel Ġarchitecture Ġis Ġillustrated Ġin ĠFigure ĠFIG REF 13 Ġ. ĠWe Ġdenote Ġirony Ġenc oder Ġas ĠIN LINE FORM 6 Ġ, Ġirony Ġdec oder Ġas ĠIN LINE FORM 7 Ġand Ġnon - ir ony Ġenc oder Ġas ĠIN LINE FORM 8 Ġ, Ġnon - ir ony Ġdec oder Ġas ĠIN LINE FORM 9 Ġ. ĠTheir Ġparameters Ġare ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġ, ĠIN LINE FORM 12 Ġand ĠIN LINE FORM 13 Ġ. </s> Our Ġirony Ġgeneration Ġalgorithm Ġis Ġshown Ġin ĠAl gorithm ĠSEC REF 3 Ġ. ĠWe Ġfirst Ġpre - train Ġour Ġmodel Ġusing Ġden o ising Ġauto - enc oder Ġand Ġback - translation Ġto Ġbuild Ġup Ġlanguage Ġmodels Ġfor Ġboth Ġstyles Ġ( section ĠSEC REF 14 Ġ). ĠThen Ġwe Ġimplement Ġreinforcement Ġlearning Ġto Ġtrain Ġthe Ġmodel Ġto Ġtransfer Ġsentences Ġfrom Ġone Ġstyle Ġto Ġanother Ġ( section ĠSEC REF 21 Ġ). ĠMeanwhile , Ġto Ġachieve Ġcontent Ġpreservation , Ġwe Ġutilize Ġback - translation Ġfor Ġone Ġtime Ġin Ġevery ĠIN LINE FORM 0 Ġtime Ġsteps . </s> P ret raining </s> In Ġorder Ġto Ġbuild Ġup Ġour Ġlanguage Ġmodel Ġand Ġpreserve Ġthe Ġcontent , Ġwe Ġapply Ġthe Ġauto - enc oder Ġmodel . ĠTo Ġprevent Ġthe Ġmodel Ġfrom Ġsimply Ġcopying Ġthe Ġinput Ġsentence , Ġwe Ġrandomly Ġadd Ġsome Ġnoises Ġin Ġthe Ġinput Ġsentence . ĠSpecifically , Ġfor Ġevery Ġword Ġin Ġthe Ġinput Ġsentence , Ġthere Ġis Ġ10 % Ġchance Ġthat Ġwe Ġdelete Ġit , Ġ10 Ġ% Ġchance Ġthat Ġwe Ġduplicate Ġit , Ġ10 % Ġchance Ġthat Ġwe Ġswap Ġit Ġwith Ġthe Ġnext Ġword , Ġor Ġit Ġremains Ġunchanged . ĠWe Ġfirst Ġencode Ġthe Ġinput Ġsentence ĠIN LINE FORM 0 Ġor ĠIN LINE FORM 1 Ġwith Ġrespective Ġenc oder ĠIN LINE FORM 2 Ġor ĠIN LINE FORM 3 Ġto Ġobtain Ġits Ġlatent Ġrepresentation ĠIN LINE FORM 4 Ġor ĠIN LINE FORM 5 Ġand Ġreconstruct Ġthe Ġinput Ġsentence Ġwith Ġthe Ġlatent Ġrepresentation Ġand Ġrespective Ġdec oder . ĠSo Ġwe Ġcan Ġget Ġthe Ġreconstruction Ġloss Ġfor Ġauto - enc oder ĠIN LINE FORM 6 Ġ: ĠDIS PLAY FORM 0 ĠDIS PLAY FORM 1 </s> In Ġaddition Ġto Ġden o ising Ġauto - enc oder , Ġwe Ġimplement Ġback - translation ĠB IB REF 19 Ġto Ġgenerate Ġa Ġpseudo - par allel Ġcorpus . ĠSuppose Ġour Ġmodel Ġtakes Ġnon - iron ic Ġsentence ĠIN LINE FORM 0 Ġas Ġinput . ĠWe Ġfirst Ġencode ĠIN LINE FORM 1 Ġwith ĠIN LINE FORM 2 Ġto Ġobtain Ġits Ġlatent Ġrepresentation ĠIN LINE FORM 3 Ġand Ġdecode Ġthe Ġlatent Ġrepresentation Ġwith ĠIN LINE FORM 4 Ġto Ġget Ġa Ġtransferred Ġsentence ĠIN LINE FORM 5 Ġ. ĠThen Ġwe Ġencode ĠIN LINE FORM 6 Ġwith ĠIN LINE FORM 7 Ġand Ġdecode Ġits Ġlatent Ġrepresentation Ġwith ĠIN LINE FORM 8 Ġto Ġreconstruct Ġthe Ġoriginal Ġinput Ġsentence ĠIN LINE FORM 9 Ġ. ĠTherefore , Ġour Ġreconstruction Ġloss Ġfor Ġback - translation ĠIN LINE FORM 10 Ġ: ĠDIS PLAY FORM 0 </s> And Ġif Ġour Ġmodel Ġtakes Ġironic Ġsentence ĠIN LINE FORM 0 Ġas Ġinput , Ġwe Ġcan Ġget Ġthe Ġreconstruction Ġloss Ġfor Ġback - translation Ġas : ĠDIS PLAY FORM 0 </s> Re in forcement ĠLearning </s> Since Ġthe Ġgold Ġtransferred Ġresult Ġof Ġinput Ġis Ġunavailable , Ġwe Ġcannot Ġevaluate Ġthe Ġquality Ġof Ġthe Ġgenerated Ġsentence Ġdirectly . ĠTherefore , Ġwe Ġimplement Ġreinforcement Ġlearning Ġand Ġelabor ately Ġdesign Ġtwo Ġrewards Ġto Ġdescribe Ġthe Ġirony Ġaccuracy Ġand Ġsentiment Ġpreservation , Ġrespectively . </s> A Ġpre - trained Ġbinary Ġirony Ġclass ifier Ġbased Ġon ĠCNN ĠB IB REF 20 Ġis Ġused Ġto Ġevaluate Ġhow Ġironic Ġa Ġsentence Ġis . ĠWe Ġdenote Ġthe Ġparameter Ġof Ġthe Ġclass ifier Ġas ĠIN LINE FORM 0 Ġand Ġit Ġis Ġfixed Ġduring Ġthe Ġtraining Ġprocess . </s> In Ġorder Ġto Ġfacilitate Ġthe Ġtransformation , Ġwe Ġdesign Ġthe Ġirony Ġreward Ġas Ġthe Ġdifference Ġbetween Ġthe Ġirony Ġscore Ġof Ġthe Ġinput Ġsentence Ġand Ġthat Ġof Ġthe Ġoutput Ġsentence . ĠForm ally , Ġwhen Ġwe Ġinput Ġa Ġnon - iron ic Ġsentence ĠIN LINE FORM 0 Ġand Ġtransfer Ġit Ġto Ġan Ġironic Ġsentence ĠIN LINE FORM 1 Ġ, Ġour Ġirony Ġreward Ġis Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġdenotes Ġironic Ġstyle Ġand ĠIN LINE FORM 1 Ġis Ġthe Ġprobability Ġof Ġthat Ġa Ġsentence ĠIN LINE FORM 2 Ġis Ġironic . </s> To Ġpreserve Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġinput Ġsentence , Ġwe Ġalso Ġneed Ġto Ġuse Ġclass ifiers Ġto Ġevaluate Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġsentences . ĠHowever , Ġthe Ġsentiment Ġanalysis Ġof Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġare Ġdifferent . ĠIn Ġthe Ġcase Ġof Ġfigur ative Ġlanguages Ġsuch Ġas Ġirony , Ġsarc asm Ġor Ġmetaphor , Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġliteral Ġmeaning Ġmay Ġdiffer Ġsignificantly Ġfrom Ġthat Ġof Ġthe Ġintended Ġfigur ative Ġmeaning ĠB IB REF 0 Ġ. ĠAs Ġwe Ġaim Ġto Ġtrain Ġour Ġmodel Ġto Ġtransfer Ġsentences Ġfrom Ġnon - iron ic Ġto Ġironic , Ġusing Ġonly Ġone Ġclass ifier Ġis Ġnot Ġenough . ĠAs Ġa Ġresult , Ġwe Ġimplement Ġtwo Ġpre - trained Ġsentiment Ġclass ifiers Ġfor Ġnon - iron ic Ġsentences Ġand Ġironic Ġsentences Ġrespectively . ĠWe Ġdenote Ġthe Ġparameter Ġof Ġthe Ġsentiment Ġclass ifier Ġfor Ġironic Ġsentences Ġas ĠIN LINE FORM 0 Ġand Ġthat Ġof Ġthe Ġsentiment Ġclass ifier Ġfor Ġnon - iron ic Ġsentences Ġas ĠIN LINE FORM 1 Ġ. </s> A Ġchallenge , Ġwhen Ġwe Ġimplement Ġtwo Ġclass ifiers Ġto Ġevaluate Ġthe Ġsentiment Ġpol arity , Ġis Ġthat Ġthe Ġtwo Ġclass ifiers Ġtrained Ġwith Ġdifferent Ġdatasets Ġmay Ġhave Ġdifferent Ġdistributions Ġof Ġscores . ĠThat Ġmeans Ġwe Ġcannot Ġdirectly Ġcalculate Ġthe Ġsentiment Ġreward Ġwith Ġscores Ġapplied Ġby Ġtwo Ġclass ifiers . ĠTo Ġalleviate Ġthis Ġproblem Ġand Ġstandard ize Ġthe Ġprediction Ġresults Ġof Ġtwo Ġclass ifiers , Ġwe Ġset Ġa Ġthreshold Ġfor Ġeach Ġclass ifier Ġand Ġsubtract Ġthe Ġrespective Ġthreshold Ġfrom Ġscores Ġapplied Ġby Ġthe Ġclass ifier Ġto Ġobtain Ġthe Ġcomparative Ġsentiment Ġpol arity Ġscore . ĠWe Ġget Ġthe Ġoptimal Ġthreshold Ġby Ġmaximizing Ġthe Ġability Ġof Ġthe Ġclass ifier Ġaccording Ġto Ġthe Ġdistribution Ġof Ġour Ġtraining Ġdata . </s> We Ġdenote Ġthe Ġthreshold Ġof Ġironic Ġsentiment Ġclass ifier Ġas ĠIN LINE FORM 0 Ġand Ġthe Ġthreshold Ġof Ġnon - iron ic Ġsentiment Ġclass ifier Ġas ĠIN LINE FORM 1 Ġ. ĠThe Ġstandardized Ġsentiment Ġscore Ġis Ġdefined Ġas ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġwhere ĠIN LINE FORM 4 Ġdenotes Ġthe Ġpositive Ġsentiment Ġpol arity Ġand ĠIN LINE FORM 5 Ġis Ġthe Ġprobability Ġof Ġthat Ġa Ġsentence Ġis Ġpositive Ġin Ġsentiment Ġpol arity . </s> As Ġmentioned Ġabove , Ġthe Ġinput Ġsentence Ġand Ġthe Ġgenerated Ġsentence Ġshould Ġexpress Ġthe Ġsame Ġsentiment . ĠFor Ġexample , Ġif Ġwe Ġinput Ġa Ġnon - iron ic Ġsentence ĠâĢ ľ I Ġhate Ġto Ġbe Ġignored " Ġwhich Ġis Ġnegative Ġin Ġsentiment Ġpol arity , Ġthe Ġgenerated Ġironic Ġsentence Ġshould Ġbe Ġalso Ġnegative , Ġsuch Ġas ĠâĢ ľ I Ġlove Ġto Ġbe Ġignored ". ĠTo Ġachieve Ġsentiment Ġpreservation , Ġwe Ġdesign Ġthe Ġsentiment Ġreward Ġas Ġthat Ġone Ġminus Ġthe Ġabsolute Ġvalue Ġof Ġthe Ġdifference Ġbetween Ġthe Ġstandardized Ġsentiment Ġscore Ġof Ġthe Ġinput Ġsentence Ġand Ġthat Ġof Ġthe Ġgenerated Ġsentence . ĠForm ally , Ġwhen Ġwe Ġinput Ġa Ġnon - iron ic Ġsentence ĠIN LINE FORM 0 Ġand Ġtransfer Ġit Ġto Ġan Ġironic Ġsentence ĠIN LINE FORM 1 Ġ, Ġour Ġsentiment Ġreward Ġis Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> To Ġencourage Ġour Ġmodel Ġto Ġfocus Ġon Ġboth Ġthe Ġirony Ġaccuracy Ġand Ġthe Ġsentiment Ġpreservation , Ġwe Ġapply Ġthe Ġharmonic Ġmean Ġof Ġirony Ġreward Ġand Ġsentiment Ġreward : ĠDIS PLAY FORM 0 </s> Policy ĠGrad ient </s> The Ġpolicy Ġgradient Ġalgorithm ĠB IB REF 21 Ġis Ġa Ġsimple Ġbut Ġwidely - used Ġalgorithm Ġin Ġreinforcement Ġlearning . ĠIt Ġis Ġused Ġto Ġmaximize Ġthe Ġexpected Ġreward ĠIN LINE FORM 0 Ġ. ĠThe Ġobjective Ġfunction Ġto Ġminimize Ġis Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġis Ġthe Ġreward Ġof ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġis Ġthe Ġinput Ġsize . </s> Training ĠDetails </s> IN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġin Ġour Ġmodel Ġare ĠTransformers ĠB IB REF 22 Ġwith Ġ4 Ġlayers Ġand Ġ2 Ġshared Ġlayers . ĠThe Ġword Ġembed d ings Ġof Ġ128 Ġdimensions Ġare Ġlearned Ġduring Ġthe Ġtraining Ġprocess . ĠOur Ġmaximum Ġsentence Ġlength Ġis Ġset Ġas Ġ40 . ĠThe Ġoptim izer Ġis ĠAdam ĠB IB REF 23 Ġand Ġthe Ġlearning Ġrate Ġis ĠIN LINE FORM 4 Ġ. ĠThe Ġbatch Ġsize Ġis Ġ32 Ġand Ġharmonic Ġweight ĠIN LINE FORM 5 Ġin ĠE q . 9 Ġis Ġ0 . 5 . ĠWe Ġset Ġthe Ġinterval ĠIN LINE FORM 6 Ġas Ġ200 . ĠThe Ġmodel Ġis Ġpre - trained Ġfor Ġ6 Ġepoch s Ġand Ġtrained Ġfor Ġ15 Ġepoch s Ġfor Ġreinforcement Ġlearning . </s> Iron y ĠClass ifier : ĠWe Ġimplement Ġa ĠCNN Ġclass ifier Ġtrained Ġwith Ġour Ġirony Ġdataset . ĠAll Ġthe ĠCNN Ġclass ifiers Ġwe Ġutilize Ġin Ġthis Ġpaper Ġuse Ġthe Ġsame Ġparameters Ġas ĠB IB REF 20 Ġ. </s> Sent iment ĠClass ifier Ġfor ĠIron y : ĠWe Ġfirst Ġimplement Ġa Ġone - layer ĠL ST M Ġnetwork Ġto Ġclassify Ġironic Ġsentences Ġin Ġour Ġdataset Ġinto Ġpositive Ġand Ġnegative Ġiron ies . ĠThe ĠL ST M Ġnetwork Ġis Ġtrained Ġwith Ġthe Ġdataset Ġof ĠSem eval Ġ2015 ĠTask Ġ11 ĠB IB REF 0 Ġwhich Ġis Ġused Ġfor Ġthe Ġsentiment Ġanalysis Ġof Ġfigur ative Ġlanguage Ġin Ġtwitter . ĠThen , Ġwe Ġuse Ġthe Ġpositive Ġiron ies Ġand Ġnegative Ġiron ies Ġto Ġtrain Ġthe ĠCNN Ġsentiment Ġclass ifier Ġfor Ġirony . </s> Sent iment ĠClass ifier Ġfor ĠNon - ir ony : ĠSimilar Ġto Ġthe Ġtraining Ġprocess Ġof Ġthe Ġsentiment Ġclass ifier Ġfor Ġirony , Ġwe Ġfirst Ġimplement Ġa Ġone - layer ĠL ST M Ġnetwork Ġtrained Ġwith Ġthe Ġdataset Ġfor Ġthe Ġsentiment Ġanalysis Ġof Ġcommon Ġtw itters Ġto Ġclassify Ġthe Ġnon - iron ies Ġinto Ġpositive Ġand Ġnegative Ġnon - iron ies . ĠThen Ġwe Ġuse Ġthe Ġpositive Ġand Ġnegative Ġnon - iron ies Ġto Ġtrain Ġthe Ġsentiment Ġclass ifier Ġfor Ġnon - ir ony . </s> Bas elines </s> We Ġcompare Ġour Ġmodel Ġwith Ġthe Ġfollowing Ġstate - of - art Ġgener ative Ġmodels : </s> Back Trans ĠB IB REF 7 Ġ: ĠIn ĠB IB REF 7 Ġ, Ġauthors Ġpropose Ġa Ġmodel Ġusing Ġmachine Ġtranslation Ġin Ġorder Ġto Ġpreserve Ġthe Ġmeaning Ġof Ġthe Ġsentence Ġwhile Ġreducing Ġstyl istic Ġproperties . </s> Un pa ired ĠB IB REF 10 Ġ: ĠIn ĠB IB REF 10 Ġ, Ġresearchers Ġimplement Ġa Ġmethod Ġto Ġremove Ġemotional Ġwords Ġand Ġadd Ġdesired Ġsentiment Ġcontrolled Ġby Ġreinforcement Ġlearning . </s> Cross Al ign ĠB IB REF 6 Ġ: ĠIn ĠB IB REF 6 Ġ, Ġauthors Ġleverage Ġrefined Ġalignment Ġof Ġlatent Ġrepresentations Ġto Ġperform Ġstyle Ġtransfer Ġand Ġa Ġcross - aligned Ġauto - enc oder Ġis Ġimplemented . </s> CP TG ĠB IB REF 24 Ġ: ĠAn Ġinterpol ated Ġreconstruction Ġloss Ġis Ġintroduced Ġin ĠB IB REF 24 Ġand Ġa Ġdiscrim inator Ġis Ġimplemented Ġto Ġcontrol Ġattributes Ġin Ġthis Ġwork . </s> Dual RL ĠB IB REF 8 Ġ: ĠIn ĠB IB REF 8 Ġ, Ġresearchers Ġuse Ġtwo Ġreinforcement Ġrewards Ġsimultaneously Ġto Ġcontrol Ġstyle Ġaccuracy Ġand Ġcontent Ġpreservation . </s> E val uation ĠMet rics </s> In Ġorder Ġto Ġevaluate Ġsentiment Ġpreservation , Ġwe Ġuse Ġthe Ġabsolute Ġvalue Ġof Ġthe Ġdifference Ġbetween Ġthe Ġstandardized Ġsentiment Ġscore Ġof Ġthe Ġinput Ġsentence Ġand Ġthat Ġof Ġthe Ġgenerated Ġsentence . ĠWe Ġcall Ġthe Ġvalue Ġas Ġsentiment Ġdelta Ġ( sent i Ġdelta ). ĠBesides , Ġwe Ġreport Ġthe Ġsentiment Ġaccuracy Ġ( Sent i ĠACC ) Ġwhich Ġmeasures Ġwhether Ġthe Ġoutput Ġsentence Ġhas Ġthe Ġsame Ġsentiment Ġpol arity Ġas Ġthe Ġinput Ġsentence Ġbased Ġon Ġour Ġstandardized Ġsentiment Ġclass ifiers . ĠThe ĠB LE U Ġscore ĠB IB REF 25 Ġbetween Ġthe Ġinput Ġsentences Ġand Ġthe Ġoutput Ġsentences Ġis Ġcalculated Ġto Ġevaluate Ġthe Ġcontent Ġpreservation Ġperformance . ĠIn Ġorder Ġto Ġevaluate Ġthe Ġoverall Ġperformance Ġof Ġdifferent Ġmodels , Ġwe Ġalso Ġreport Ġthe Ġgeometric Ġmean Ġ( G 2 ) Ġand Ġharmonic Ġmean Ġ( H 2 ) Ġof Ġthe Ġsentiment Ġaccuracy Ġand Ġthe ĠB LE U Ġscore . ĠAs Ġfor Ġthe Ġirony Ġaccuracy , Ġwe Ġonly Ġreport Ġit Ġin Ġhuman Ġevaluation Ġresults Ġbecause Ġit Ġis Ġmore Ġaccurate Ġfor Ġthe Ġhuman Ġto Ġevaluate Ġthe Ġquality Ġof Ġirony Ġas Ġit Ġis Ġvery Ġcomplicated . </s> We Ġfirst Ġsample Ġ50 Ġnon - iron ic Ġinput Ġsentences Ġand Ġtheir Ġcorresponding Ġoutput Ġsentences Ġof Ġdifferent Ġmodels . ĠThen , Ġwe Ġask Ġfour Ġannot ators Ġwho Ġare Ġproficient Ġin ĠEnglish Ġto Ġevaluate Ġthe Ġqualities Ġof Ġthe Ġgenerated Ġsentences Ġof Ġdifferent Ġmodels . ĠThey Ġare Ġrequired Ġto Ġrank Ġthe Ġoutput Ġsentences Ġof Ġour Ġmodel Ġand Ġbas elines Ġfrom Ġthe Ġbest Ġto Ġthe Ġworst Ġin Ġterms Ġof Ġirony Ġaccuracy Ġ( Iron y ), ĠSent iment Ġpreservation Ġ( Sent i ) Ġand Ġcontent Ġpreservation Ġ( Content ). ĠThe Ġbest Ġoutput Ġis Ġranked Ġwith Ġ1 Ġand Ġthe Ġworst Ġoutput Ġis Ġranked Ġwith Ġ6 . ĠThat Ġmeans Ġthat Ġthe Ġsmaller Ġour Ġhuman Ġevaluation Ġvalue Ġis , Ġthe Ġbetter Ġthe Ġcorresponding Ġmodel Ġis . </s> Results Ġand ĠDisc ussions </s> Table ĠT AB REF 35 Ġshows Ġthe Ġautomatic Ġevaluation Ġresults Ġof Ġthe Ġmodels Ġin Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences . ĠFrom Ġthe Ġresults , Ġour Ġmodel Ġob tains Ġthe Ġbest Ġresult Ġin Ġsentiment Ġdelta . ĠThe ĠDual RL Ġmodel Ġachieves Ġthe Ġhighest Ġresult Ġin Ġother Ġmetrics , Ġbut Ġmost Ġof Ġits Ġoutputs Ġare Ġthe Ġalmost Ġsame Ġas Ġthe Ġinput Ġsentences . ĠSo Ġit Ġis Ġreasonable Ġthat ĠDual RL Ġsystem Ġoutper forms Ġours Ġin Ġthese Ġmetrics Ġbut Ġit Ġactually Ġdoes Ġnot Ġtransfer Ġthe Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences Ġat Ġall . ĠFrom Ġthis Ġperspective , Ġwe Ġcannot Ġview ĠDual RL Ġas Ġan Ġeffective Ġmodel Ġfor Ġirony Ġgeneration . ĠIn Ġcontrast , Ġour Ġmodel Ġgets Ġresults Ġclose Ġto Ġthose Ġof ĠDual RL Ġand Ġob tains Ġa Ġbalance Ġbetween Ġirony Ġaccuracy , Ġsentiment Ġpreservation , Ġand Ġcontent Ġpreservation Ġif Ġwe Ġalso Ġconsider Ġthe Ġirony Ġaccuracy Ġdiscussed Ġbelow . </s> And Ġfrom Ġhuman Ġevaluation Ġresults Ġshown Ġin ĠTable ĠT AB REF 36 Ġ, Ġour Ġmodel Ġgets Ġthe Ġbest Ġaverage Ġrank Ġin Ġirony Ġaccuracy . ĠAnd Ġas Ġmentioned Ġabove , Ġthe ĠDual RL Ġmodel Ġusually Ġdoes Ġnot Ġchange Ġthe Ġinput Ġsentence Ġand Ġoutputs Ġthe Ġsame Ġsentence . ĠTherefore , Ġit Ġis Ġreasonable Ġthat Ġit Ġob tains Ġthe Ġbest Ġrank Ġin Ġsentiment Ġand Ġcontent Ġpreservation Ġand Ġours Ġis Ġthe Ġsecond . ĠHowever , Ġit Ġstill Ġdemonstrates Ġthat Ġour Ġmodel , Ġinstead Ġof Ġchanging Ġnothing , Ġtransfers Ġthe Ġstyle Ġof Ġthe Ġinput Ġsentence Ġwith Ġcontent Ġand Ġsentiment Ġpreservation Ġat Ġthe Ġsame Ġtime . </s> Case ĠStudy </s> In Ġthe Ġsection , Ġwe Ġpresent Ġsome Ġexample Ġoutputs Ġof Ġdifferent Ġmodels . ĠTable ĠT AB REF 37 Ġshows Ġthe Ġresults Ġof Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences . ĠWe Ġcan Ġobserve Ġthat : Ġ( 1 ) ĠThe ĠBack Trans Ġsystem , Ġthe ĠUn pa ired Ġsystem , Ġthe ĠCross Al ign Ġsystem Ġand Ġthe ĠC PT G Ġsystem Ġtends Ġto Ġgenerate Ġsentences Ġwhich Ġare Ġtowards Ġirony Ġbut Ġdo Ġnot Ġpreserve Ġcontent . Ġ( 2 ) ĠThe ĠDual RL Ġsystem Ġpreserves Ġcontent Ġand Ġsentiment Ġvery Ġwell Ġbut Ġeven Ġdoes Ġnot Ġchange Ġthe Ġinput Ġsentence . Ġ( 3 ) ĠOur Ġmodel Ġconsiders Ġboth Ġaspects Ġand Ġachieves Ġa Ġbetter Ġbalance Ġamong Ġirony Ġaccuracy , Ġsentiment Ġand Ġcontent Ġpreservation . </s> Error ĠAnalysis </s> Although Ġour Ġmodel Ġoutper forms Ġother Ġstyle Ġtransfer Ġbas elines Ġaccording Ġto Ġautomatic Ġand Ġhuman Ġevaluation Ġresults , Ġthere Ġare Ġstill Ġsome Ġfailure Ġcases Ġbecause Ġirony Ġgeneration Ġis Ġstill Ġa Ġvery Ġchallenging Ġtask . ĠWe Ġwould Ġlike Ġto Ġshare Ġthe Ġissues Ġwe Ġmeet Ġduring Ġour Ġexperiments Ġand Ġour Ġsolutions Ġto Ġsome Ġof Ġthem Ġin Ġthis Ġsection . </s> No ĠChange : ĠAs Ġmentioned Ġabove , Ġmany Ġstyle Ġtransfer Ġmodels , Ġsuch Ġas ĠDual RL , Ġtend Ġto Ġmake Ġfew Ġchanges Ġto Ġthe Ġinput Ġsentence Ġand Ġoutput Ġthe Ġsame Ġsentence . ĠActually , Ġthis Ġis Ġa Ġcommon Ġissue Ġfor Ġun super vised Ġstyle Ġtransfer Ġsystems Ġand Ġwe Ġalso Ġmeet Ġit Ġduring Ġour Ġexperiments . ĠThe Ġmain Ġreason Ġfor Ġthe Ġissue Ġis Ġthat Ġrewards Ġfor Ġcontent Ġpreservation Ġare Ġtoo Ġprominent Ġand Ġrewards Ġfor Ġstyle Ġaccuracy Ġcannot Ġwork Ġwell . ĠIn Ġcontrast , Ġin Ġorder Ġto Ġguarantee Ġthe Ġread ability Ġand Ġflu ency Ġof Ġthe Ġoutput Ġsentence , Ġwe Ġalso Ġcannot Ġemphasize Ġtoo Ġmuch Ġon Ġrewards Ġfor Ġstyle Ġaccuracy Ġbecause Ġit Ġmay Ġcause Ġsome Ġother Ġissues Ġsuch Ġas Ġword Ġrepetition Ġmentioned Ġbelow . ĠA Ġmethod Ġto Ġsolve Ġthe Ġproblem Ġis Ġtuning Ġhyper param eters Ġand Ġthis Ġis Ġalso Ġthe Ġmethod Ġwe Ġimplement Ġin Ġthis Ġwork . ĠAs Ġfor Ġcontent Ġpreservation , Ġmaybe ĠM LE Ġmethods Ġsuch Ġas Ġback - translation Ġare Ġnot Ġenough Ġbecause Ġthey Ġtend Ġto Ġforce Ġmodels Ġto Ġgenerate Ġspecific Ġwords . ĠIn Ġthe Ġfuture , Ġwe Ġshould Ġfurther Ġdesign Ġsome Ġmore Ġsuitable Ġmethods Ġto Ġcontrol Ġcontent Ġpreservation Ġfor Ġmodels Ġwithout Ġdis ent angling Ġstyle Ġand Ġcontent Ġrepresentations , Ġsuch Ġas ĠDual RL Ġand Ġours . </s> Word ĠRep etition : ĠDuring Ġour Ġexperiments , Ġwe Ġobserve Ġthat Ġsome Ġof Ġthe Ġoutputs Ġprefer Ġto Ġrepeat Ġthe Ġsame Ġword Ġas Ġshown Ġin ĠTable ĠT AB REF 38 Ġ. ĠThis Ġis Ġbecause Ġreinforcement Ġlearning Ġrewards Ġencourage Ġthe Ġmodel Ġto Ġgenerate Ġwords Ġwhich Ġcan Ġget Ġhigh Ġscores Ġfrom Ġclass ifiers Ġand Ġeven Ġback - translation Ġcannot Ġstop Ġit . ĠOur Ġsolution Ġis Ġthat Ġwe Ġcan Ġlower Ġthe Ġprobability Ġof Ġdecoding Ġa Ġword Ġin Ġdec od ers Ġif Ġthe Ġword Ġhas Ġbeen Ġgenerated Ġin Ġthe Ġprevious Ġtime Ġsteps Ġduring Ġtesting . ĠWe Ġalso Ġtry Ġto Ġimplement Ġthis Ġmethod Ġduring Ġtraining Ġtime Ġbut Ġobtain Ġworse Ġperformances Ġbecause Ġit Ġmay Ġlimit Ġthe Ġeffects Ġof Ġtraining . ĠSome Ġprevious Ġstudies Ġutilize Ġlanguage Ġmodels Ġto Ġcontrol Ġthe Ġflu ency Ġof Ġthe Ġoutput Ġsentence Ġand Ġwe Ġalso Ġtry Ġthis Ġmethod . ĠNonetheless , Ġpre - training Ġa Ġlanguage Ġmodel Ġwith Ġtweets Ġand Ġusing Ġit Ġto Ġgenerate Ġrewards Ġis Ġdifficult Ġbecause Ġtweets Ġare Ġmore Ġcasual Ġand Ġhave Ġmore Ġnoise . ĠRewards Ġfrom Ġthat Ġkind Ġof Ġlanguage Ġmodel Ġare Ġusually Ġnot Ġaccurate Ġand Ġmay Ġconfuse Ġthe Ġmodel . ĠIn Ġthe Ġfuture , Ġwe Ġshould Ġcome Ġup Ġwith Ġbetter Ġmethods Ġto Ġmodel Ġlanguage Ġflu ency Ġwith Ġthe Ġconsideration Ġof Ġirony Ġaccuracy , Ġsentiment Ġand Ġcontent Ġpreservation , Ġespecially Ġfor Ġtweets . </s> Impro per ĠWords : ĠAs Ġironic Ġstyle Ġis Ġhard Ġfor Ġour Ġmodel Ġto Ġlearn , Ġit Ġmay Ġgenerate Ġsome Ġimproper Ġwords Ġwhich Ġmake Ġthe Ġsentence Ġstrange . ĠAs Ġthe Ġexample Ġshown Ġin Ġthe ĠTable ĠT AB REF 38 Ġ, Ġthe Ġsentiment Ġword Ġin Ġthe Ġinput Ġsentence Ġis ĠâĢ ľ w onder ful " Ġand Ġthe Ġmodel Ġshould Ġchange Ġit Ġinto Ġa Ġnegative Ġword Ġsuch Ġas ĠâĢ ľ s ad " Ġto Ġmake Ġthe Ġoutput Ġsentence Ġironic . ĠHowever , Ġthe Ġmodel Ġchanges ĠâĢ ľ fr iday " Ġand ĠâĢ ľ f ifa " Ġwhich Ġare Ġnot Ġrelated Ġto Ġironic Ġstyles . ĠWe Ġhave Ġnot Ġfound Ġa Ġvery Ġeffective Ġmethod Ġto Ġaddress Ġthis Ġissue Ġand Ġmaybe Ġwe Ġshould Ġfurther Ġexplore Ġstronger Ġmodels Ġto Ġlearn Ġironic Ġstyles Ġbetter . </s> Additional ĠExper iments </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġsome Ġadditional Ġexperiments Ġon Ġthe Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences . ĠSometimes Ġiron ies Ġare Ġhard Ġto Ġunderstand Ġand Ġmay Ġcause Ġmisunderstanding , Ġfor Ġwhich Ġour Ġtask Ġalso Ġexplores Ġthe Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences . </s> As Ġshown Ġin ĠTable ĠT AB REF 46 Ġ, Ġwe Ġalso Ġconduct Ġautomatic Ġevaluations Ġand Ġthe Ġconclusions Ġare Ġsimilar Ġto Ġthose Ġof Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences . ĠAs Ġfor Ġhuman Ġevaluation Ġresults Ġin ĠTable ĠT AB REF 47 Ġ, Ġour Ġmodel Ġstill Ġcan Ġachieve Ġthe Ġsecond - best Ġresults Ġin Ġsentiment Ġand Ġcontent Ġpreservation . ĠNevertheless , ĠDual RL Ġsystem Ġand Ġours Ġget Ġpoor Ġperformances Ġin Ġirony Ġaccuracy . ĠThe Ġreason Ġmay Ġbe Ġthat Ġthe Ġother Ġfour Ġbas elines Ġtend Ġto Ġgenerate Ġcommon Ġand Ġeven Ġnot Ġfluent Ġsentences Ġwhich Ġare Ġirrelevant Ġto Ġthe Ġinput Ġsentences Ġand Ġare Ġhard Ġto Ġbe Ġidentified Ġas Ġiron ies . ĠSo Ġannot ators Ġusually Ġmark Ġthese Ġoutput Ġsentences Ġas Ġnon - iron ic Ġsentences , Ġwhich Ġcauses Ġthese Ġmodels Ġto Ġobtain Ġbetter Ġperformances Ġthan ĠDual RL Ġand Ġours Ġbut Ġmuch Ġpoorer Ġresults Ġin Ġsentiment Ġand Ġcontent Ġpreservation . ĠSome Ġexamples Ġare Ġshown Ġin ĠTable ĠT AB REF 52 Ġ. </s> Conclusion Ġand ĠFuture ĠWork </s> In Ġthis Ġpaper , Ġwe Ġfirst Ġsystematically Ġdefine Ġirony Ġgeneration Ġbased Ġon Ġstyle Ġtransfer . ĠBecause Ġof Ġthe Ġlack Ġof Ġirony Ġdata , Ġwe Ġmake Ġuse Ġof Ġtwitter Ġand Ġbuild Ġa Ġlarge - scale Ġdataset . ĠIn Ġorder Ġto Ġcontrol Ġirony Ġaccuracy , Ġsentiment Ġpreservation Ġand Ġcontent Ġpreservation Ġat Ġthe Ġsame Ġtime , Ġwe Ġalso Ġdesign Ġa Ġcombination Ġof Ġrewards Ġfor Ġreinforcement Ġlearning Ġand Ġincorporate Ġreinforcement Ġlearning Ġwith Ġa Ġpre - training Ġprocess . ĠExperimental Ġresults Ġdemonstrate Ġthat Ġour Ġmodel Ġoutper forms Ġother Ġgener ative Ġmodels Ġand Ġour Ġrewards Ġare Ġeffective . ĠAlthough Ġour Ġmodel Ġdesign Ġis Ġeffective , Ġthere Ġare Ġstill Ġmany Ġerrors Ġand Ġwe Ġsystematically Ġanalyze Ġthem . ĠIn Ġthe Ġfuture , Ġwe Ġare Ġinterested Ġin Ġexploring Ġthese Ġdirections Ġand Ġour Ġwork Ġmay Ġextend Ġto Ġother Ġkinds Ġof Ġiron ies Ġwhich Ġare Ġmore Ġdifficult Ġto Ġmodel .
Introduction The irony is a kind of figurative language, which is widely used on social media BIBREF0 . The irony is defined as a clash between the intended meaning of a sentence and its literal meaning BIBREF1 . As an important aspect of language, irony plays an essential role in sentiment analysis BIBREF2 , BIBREF0 and opinion mining BIBREF3 , BIBREF4 . Although some previous studies focus on irony detection, little attention is paid to irony generation. As ironies can strengthen sentiments and express stronger emotions, we mainly focus on generating ironic sentences. Given a non-ironic sentence, we implement a neural network to transfer it to an ironic sentence and constrain the sentiment polarity of the two sentences to be the same. For example, the input is “I hate it when my plans get ruined" which is negative in sentiment polarity and the output should be ironic and negative in sentiment as well, such as “I like it when my plans get ruined". The speaker uses “like" to be ironic and express his or her negative sentiment. At the same time, our model can preserve contents which are irrelevant to sentiment polarity and irony. According to the categories mentioned in BIBREF5 , irony can be classified into 3 classes: verbal irony by means of a polarity contrast, the sentences containing expression whose polarity is inverted between the intended and the literal evaluation; other types of verbal irony, the sentences that show no polarity contrast between the literal and intended meaning but are still ironic; and situational irony, the sentences that describe situations that fail to meet some expectations. As ironies in the latter two categories are obscure and hard to understand, we decide to only focus on ironies in the first category in this work. For example, our work can be specifically described as: given a sentence “I hate to be ignored", we train our model to generate an ironic sentence such as “I love to be ignored". Although there is “love" in the generated sentence, the speaker still expresses his or her negative sentiment by irony. We also make some explorations in the transformation from ironic sentences to non-ironic sentences at the end of our work. Because of the lack of previous work and baselines on irony generation, we implement our model based on style transfer. Our work will not only provide the first large-scale irony dataset but also make our model as a benchmark for the irony generation. Recently, unsupervised style transfer becomes a very popular topic. Many state-of-the-art studies try to solve the task with sequence-to-sequence (seq2seq) framework. There are three main ways to build up models. The first is to learn a latent style-independent content representation and generate sentences with the content representation and another style BIBREF6 , BIBREF7 . The second is to directly transfer sentences from one style to another under the control of classifiers and reinforcement learning BIBREF8 . The third is to remove style attribute words from the input sentence and combine the remaining content with new style attribute words BIBREF9 , BIBREF10 . The first method usually obtains better performances via adversarial training with discriminators. The style-independent content representation, nevertheless, is not easily obtained BIBREF11 , which results in poor performances. The second method is suitable for complex styles which are difficult to model and describe. The model can learn the deep semantic features by itself but sometimes the model is sensitive to parameters and hard to train. The third method succeeds to preserve content but cannot work for some complex styles such as democratic and republican. Sentences with those styles usually do not have specific style attribute words. Unfortunately, due to the lack of large irony dataset and difficulties of modeling ironies, there has been little work trying to generate ironies based on seq2seq framework as far as we know. Inspired by methods for style transfer, we decide to implement a specifically designed model based on unsupervised style transfer to explore irony generation. In this paper, in order to address the lack of irony data, we first crawl over 2M tweets from twitter to build a dataset with 262,755 ironic and 112,330 non-ironic tweets. Then, due to the lack of parallel data, we propose a novel model to transfer non-ironic sentences to ironic sentences in an unsupervised way. As ironic style is hard to model and describe, we implement our model with the control of classifiers and reinforcement learning. Different from other studies in style transfer, the transformation from non-ironic to ironic sentences has to preserve sentiment polarity as mentioned above. Therefore, we not only design an irony reward to control the irony accuracy and implement denoising auto-encoder and back-translation to control content preservation but also design a sentiment reward to control sentiment preservation. Experimental results demonstrate that our model achieves a high irony accuracy with well-preserved sentiment and content. The contributions of our work are as follows: Related Work Style Transfer: As irony is a complicated style and hard to model with some specific style attribute words, we mainly focus on studies without editing style attribute words. Some studies are trying to disentangle style representation from content representation. In BIBREF12 , authors leverage adversarial networks to learn separate content representations and style representations. In BIBREF13 and BIBREF6 , researchers combine variational auto-encoders (VAEs) with style discriminators. However, some recent studies BIBREF11 reveal that the disentanglement of content and style representations may not be achieved in practice. Therefore, some other research studies BIBREF9 , BIBREF10 strive to separate content and style by removing stylistic words. Nonetheless, many non-ironic sentences do not have specific stylistic words and as a result, we find it difficult to transfer non-ironic sentences to ironic sentences through this way in practice. Besides, some other research studies do not disentangle style from content but directly learn representations of sentences. In BIBREF8 , authors propose a dual reinforcement learning framework without separating content and style representations. In BIBREF7 , researchers utilize a machine translation model to learn a sentence representation preserving the meaning of the sentence but reducing stylistic properties. In this method, the quality of generated sentences relies on the performance of classifiers to a large extent. Meanwhile, such models are usually sensitive to parameters and difficult to train. In contrast, we combine a pre-training process with reinforcement learning to build up a stable language model and design special rewards for our task. Irony Detection: With the development of social media, irony detection becomes a more important task. Methods for irony detection can be mainly divided into two categories: methods based on feature engineering and methods based on neural networks. As for methods based on feature engineering, In BIBREF1 , authors investigate pragmatic phenomena and various irony markers. In BIBREF14 , researchers leverage a combination of sentiment, distributional semantic and text surface features. Those models rely on hand-crafted features and are hard to implement. When it comes to methods based on neural networks, long short-term memory (LSTM) BIBREF15 network is widely used and is very efficient for irony detection. In BIBREF16 , a tweet is divided into two segments and a subtract layer is implemented to calculate the difference between two segments in order to determine whether the tweet is ironic. In BIBREF17 , authors utilize a recurrent neural network with Bi-LSTM and self-attention without hand-crafted features. In BIBREF18 , researchers propose a system based on a densely connected LSTM network. Our Dataset In this section, we describe how we build our dataset with tweets. First, we crawl over 2M tweets from twitter using GetOldTweets-python. We crawl English tweets from 04/09/2012 to /12/18/2018. We first remove all re-tweets and use langdetect to remove all non-English sentences. Then, we remove hashtags attached at the end of the tweets because they are usually not parts of sentences and will confuse our language model. After that, we utilize Ekphrasis to process tweets. We remove URLs and restore remaining hashtags, elongated words, repeated words, and all-capitalized words. To simplify our dataset, We replace all “ INLINEFORM0 money INLINEFORM1 " and “ INLINEFORM2 time INLINEFORM3 " tokens with “ INLINEFORM4 number INLINEFORM5 " token when using Ekphrasis. And we delete sentences whose lengths are less than 10 or greater than 40. In order to restore abbreviations, we download an abbreviation dictionary from webopedia and restore abbreviations to normal words or phrases according to the dictionary. Finally, we remove sentences which have more than two rare words (appearing less than three times) in order to constrain the size of vocabulary. Finally, we get 662,530 sentences after pre-processing. As neural networks are proved effective in irony detection, we decide to implement a neural classifier in order to classify the sentences into ironic and non-ironic sentences. However, the only high-quality irony dataset we can obtain is the dataset of Semeval-2018 Task 3 and the dataset is pretty small, which will cause overfitting to complex models. Therefore, we just implement a simple one-layer RNN with LSTM cell to classify pre-processed sentences into ironic sentences and non-ironic sentences because LSTM networks are widely used in irony detection. We train the model with the dataset of Semeval-2018 Task 3. After classification, we get 262,755 ironic sentences and 399,775 non-ironic sentences. According to our observation, not all non-ironic sentences are suitable to be transferred into ironic sentences. For example, “just hanging out . watching . is it monday yet" is hard to transfer because it does not have an explicit sentiment polarity. So we remove all interrogative sentences from the non-ironic sentences and only obtain the sentences which have words expressing strong sentiments. We evaluate the sentiment polarity of each word with TextBlob and we view those words with sentiment scores greater than 0.5 or less than -0.5 as words expressing strong sentiments. Finally, we build our irony dataset with 262,755 ironic sentences and 102,330 non-ironic sentences. [t] Irony Generation Algorithm INLINEFORM0 pre-train with auto-encoder Pre-train INLINEFORM1 , INLINEFORM2 with INLINEFORM3 using MLE based on Eq. EQREF16 Pre-train INLINEFORM4 , INLINEFORM5 with INLINEFORM6 using MLE based on Eq. EQREF17 INLINEFORM7 pre-train with back-translation Pre-train INLINEFORM8 , INLINEFORM9 , INLINEFORM10 , INLINEFORM11 with INLINEFORM12 using MLE based on Eq. EQREF19 Pre-train INLINEFORM13 , INLINEFORM14 , INLINEFORM15 , INLINEFORM16 with INLINEFORM17 using MLE based on Eq. EQREF20 INLINEFORM0 train with RL each epoch e = 1, 2, ..., INLINEFORM1 INLINEFORM2 train non-irony2irony with RL INLINEFORM3 in N INLINEFORM4 update INLINEFORM5 , INLINEFORM6 , using INLINEFORM7 based on Eq. EQREF29 INLINEFORM8 back-translation INLINEFORM9 INLINEFORM10 INLINEFORM11 update INLINEFORM12 , INLINEFORM13 , INLINEFORM14 , INLINEFORM15 using MLE based on Eq. EQREF19 INLINEFORM16 train irony2non-irony with RL INLINEFORM17 in I INLINEFORM18 update INLINEFORM19 , INLINEFORM20 , using INLINEFORM21 similar to Eq. EQREF29 INLINEFORM22 back-translation INLINEFORM23 INLINEFORM24 INLINEFORM25 update INLINEFORM26 , INLINEFORM27 , INLINEFORM28 , INLINEFORM29 using MLE based on Eq. EQREF20 Our Method Given two non-parallel corpora: non-ironic corpus N={ INLINEFORM0 , INLINEFORM1 , ..., INLINEFORM2 } and ironic corpus I={ INLINEFORM3 , INLINEFORM4 , ..., INLINEFORM5 }, the goal of our irony generation model is to generate an ironic sentence from a non-ironic sentence while preserving the content and sentiment polarity of the source input sentence. We implement an encoder-decoder framework where two encoders are utilized to encode ironic sentences and non-ironic sentences respectively and two decoders are utilized to decode ironic sentences and non-ironic sentences from latent representations respectively. In order to enforce a shared latent space, we share two layers on both the encoder side and the decoder side. Our model architecture is illustrated in Figure FIGREF13 . We denote irony encoder as INLINEFORM6 , irony decoder as INLINEFORM7 and non-irony encoder as INLINEFORM8 , non-irony decoder as INLINEFORM9 . Their parameters are INLINEFORM10 , INLINEFORM11 , INLINEFORM12 and INLINEFORM13 . Our irony generation algorithm is shown in Algorithm SECREF3 . We first pre-train our model using denoising auto-encoder and back-translation to build up language models for both styles (section SECREF14 ). Then we implement reinforcement learning to train the model to transfer sentences from one style to another (section SECREF21 ). Meanwhile, to achieve content preservation, we utilize back-translation for one time in every INLINEFORM0 time steps. Pretraining In order to build up our language model and preserve the content, we apply the auto-encoder model. To prevent the model from simply copying the input sentence, we randomly add some noises in the input sentence. Specifically, for every word in the input sentence, there is 10% chance that we delete it, 10 % chance that we duplicate it, 10% chance that we swap it with the next word, or it remains unchanged. We first encode the input sentence INLINEFORM0 or INLINEFORM1 with respective encoder INLINEFORM2 or INLINEFORM3 to obtain its latent representation INLINEFORM4 or INLINEFORM5 and reconstruct the input sentence with the latent representation and respective decoder. So we can get the reconstruction loss for auto-encoder INLINEFORM6 : DISPLAYFORM0 DISPLAYFORM1 In addition to denoising auto-encoder, we implement back-translation BIBREF19 to generate a pseudo-parallel corpus. Suppose our model takes non-ironic sentence INLINEFORM0 as input. We first encode INLINEFORM1 with INLINEFORM2 to obtain its latent representation INLINEFORM3 and decode the latent representation with INLINEFORM4 to get a transferred sentence INLINEFORM5 . Then we encode INLINEFORM6 with INLINEFORM7 and decode its latent representation with INLINEFORM8 to reconstruct the original input sentence INLINEFORM9 . Therefore, our reconstruction loss for back-translation INLINEFORM10 : DISPLAYFORM0 And if our model takes ironic sentence INLINEFORM0 as input, we can get the reconstruction loss for back-translation as: DISPLAYFORM0 Reinforcement Learning Since the gold transferred result of input is unavailable, we cannot evaluate the quality of the generated sentence directly. Therefore, we implement reinforcement learning and elaborately design two rewards to describe the irony accuracy and sentiment preservation, respectively. A pre-trained binary irony classifier based on CNN BIBREF20 is used to evaluate how ironic a sentence is. We denote the parameter of the classifier as INLINEFORM0 and it is fixed during the training process. In order to facilitate the transformation, we design the irony reward as the difference between the irony score of the input sentence and that of the output sentence. Formally, when we input a non-ironic sentence INLINEFORM0 and transfer it to an ironic sentence INLINEFORM1 , our irony reward is defined as: DISPLAYFORM0 where INLINEFORM0 denotes ironic style and INLINEFORM1 is the probability of that a sentence INLINEFORM2 is ironic. To preserve the sentiment polarity of the input sentence, we also need to use classifiers to evaluate the sentiment polarity of the sentences. However, the sentiment analysis of ironic sentences and non-ironic sentences are different. In the case of figurative languages such as irony, sarcasm or metaphor, the sentiment polarity of the literal meaning may differ significantly from that of the intended figurative meaning BIBREF0 . As we aim to train our model to transfer sentences from non-ironic to ironic, using only one classifier is not enough. As a result, we implement two pre-trained sentiment classifiers for non-ironic sentences and ironic sentences respectively. We denote the parameter of the sentiment classifier for ironic sentences as INLINEFORM0 and that of the sentiment classifier for non-ironic sentences as INLINEFORM1 . A challenge, when we implement two classifiers to evaluate the sentiment polarity, is that the two classifiers trained with different datasets may have different distributions of scores. That means we cannot directly calculate the sentiment reward with scores applied by two classifiers. To alleviate this problem and standardize the prediction results of two classifiers, we set a threshold for each classifier and subtract the respective threshold from scores applied by the classifier to obtain the comparative sentiment polarity score. We get the optimal threshold by maximizing the ability of the classifier according to the distribution of our training data. We denote the threshold of ironic sentiment classifier as INLINEFORM0 and the threshold of non-ironic sentiment classifier as INLINEFORM1 . The standardized sentiment score is defined as INLINEFORM2 and INLINEFORM3 where INLINEFORM4 denotes the positive sentiment polarity and INLINEFORM5 is the probability of that a sentence is positive in sentiment polarity. As mentioned above, the input sentence and the generated sentence should express the same sentiment. For example, if we input a non-ironic sentence “I hate to be ignored" which is negative in sentiment polarity, the generated ironic sentence should be also negative, such as “I love to be ignored". To achieve sentiment preservation, we design the sentiment reward as that one minus the absolute value of the difference between the standardized sentiment score of the input sentence and that of the generated sentence. Formally, when we input a non-ironic sentence INLINEFORM0 and transfer it to an ironic sentence INLINEFORM1 , our sentiment reward is defined as: DISPLAYFORM0 To encourage our model to focus on both the irony accuracy and the sentiment preservation, we apply the harmonic mean of irony reward and sentiment reward: DISPLAYFORM0 Policy Gradient The policy gradient algorithm BIBREF21 is a simple but widely-used algorithm in reinforcement learning. It is used to maximize the expected reward INLINEFORM0 . The objective function to minimize is defined as: DISPLAYFORM0 where INLINEFORM0 , INLINEFORM1 is the reward of INLINEFORM2 and INLINEFORM3 is the input size. Training Details INLINEFORM0 , INLINEFORM1 , INLINEFORM2 and INLINEFORM3 in our model are Transformers BIBREF22 with 4 layers and 2 shared layers. The word embeddings of 128 dimensions are learned during the training process. Our maximum sentence length is set as 40. The optimizer is Adam BIBREF23 and the learning rate is INLINEFORM4 . The batch size is 32 and harmonic weight INLINEFORM5 in Eq.9 is 0.5. We set the interval INLINEFORM6 as 200. The model is pre-trained for 6 epochs and trained for 15 epochs for reinforcement learning. Irony Classifier: We implement a CNN classifier trained with our irony dataset. All the CNN classifiers we utilize in this paper use the same parameters as BIBREF20 . Sentiment Classifier for Irony: We first implement a one-layer LSTM network to classify ironic sentences in our dataset into positive and negative ironies. The LSTM network is trained with the dataset of Semeval 2015 Task 11 BIBREF0 which is used for the sentiment analysis of figurative language in twitter. Then, we use the positive ironies and negative ironies to train the CNN sentiment classifier for irony. Sentiment Classifier for Non-irony: Similar to the training process of the sentiment classifier for irony, we first implement a one-layer LSTM network trained with the dataset for the sentiment analysis of common twitters to classify the non-ironies into positive and negative non-ironies. Then we use the positive and negative non-ironies to train the sentiment classifier for non-irony. Baselines We compare our model with the following state-of-art generative models: BackTrans BIBREF7 : In BIBREF7 , authors propose a model using machine translation in order to preserve the meaning of the sentence while reducing stylistic properties. Unpaired BIBREF10 : In BIBREF10 , researchers implement a method to remove emotional words and add desired sentiment controlled by reinforcement learning. CrossAlign BIBREF6 : In BIBREF6 , authors leverage refined alignment of latent representations to perform style transfer and a cross-aligned auto-encoder is implemented. CPTG BIBREF24 : An interpolated reconstruction loss is introduced in BIBREF24 and a discriminator is implemented to control attributes in this work. DualRL BIBREF8 : In BIBREF8 , researchers use two reinforcement rewards simultaneously to control style accuracy and content preservation. Evaluation Metrics In order to evaluate sentiment preservation, we use the absolute value of the difference between the standardized sentiment score of the input sentence and that of the generated sentence. We call the value as sentiment delta (senti delta). Besides, we report the sentiment accuracy (Senti ACC) which measures whether the output sentence has the same sentiment polarity as the input sentence based on our standardized sentiment classifiers. The BLEU score BIBREF25 between the input sentences and the output sentences is calculated to evaluate the content preservation performance. In order to evaluate the overall performance of different models, we also report the geometric mean (G2) and harmonic mean (H2) of the sentiment accuracy and the BLEU score. As for the irony accuracy, we only report it in human evaluation results because it is more accurate for the human to evaluate the quality of irony as it is very complicated. We first sample 50 non-ironic input sentences and their corresponding output sentences of different models. Then, we ask four annotators who are proficient in English to evaluate the qualities of the generated sentences of different models. They are required to rank the output sentences of our model and baselines from the best to the worst in terms of irony accuracy (Irony), Sentiment preservation (Senti) and content preservation (Content). The best output is ranked with 1 and the worst output is ranked with 6. That means that the smaller our human evaluation value is, the better the corresponding model is. Results and Discussions Table TABREF35 shows the automatic evaluation results of the models in the transformation from non-ironic sentences to ironic sentences. From the results, our model obtains the best result in sentiment delta. The DualRL model achieves the highest result in other metrics, but most of its outputs are the almost same as the input sentences. So it is reasonable that DualRL system outperforms ours in these metrics but it actually does not transfer the non-ironic sentences to ironic sentences at all. From this perspective, we cannot view DualRL as an effective model for irony generation. In contrast, our model gets results close to those of DualRL and obtains a balance between irony accuracy, sentiment preservation, and content preservation if we also consider the irony accuracy discussed below. And from human evaluation results shown in Table TABREF36 , our model gets the best average rank in irony accuracy. And as mentioned above, the DualRL model usually does not change the input sentence and outputs the same sentence. Therefore, it is reasonable that it obtains the best rank in sentiment and content preservation and ours is the second. However, it still demonstrates that our model, instead of changing nothing, transfers the style of the input sentence with content and sentiment preservation at the same time. Case Study In the section, we present some example outputs of different models. Table TABREF37 shows the results of the transformation from non-ironic sentences to ironic sentences. We can observe that: (1) The BackTrans system, the Unpaired system, the CrossAlign system and the CPTG system tends to generate sentences which are towards irony but do not preserve content. (2) The DualRL system preserves content and sentiment very well but even does not change the input sentence. (3) Our model considers both aspects and achieves a better balance among irony accuracy, sentiment and content preservation. Error Analysis Although our model outperforms other style transfer baselines according to automatic and human evaluation results, there are still some failure cases because irony generation is still a very challenging task. We would like to share the issues we meet during our experiments and our solutions to some of them in this section. No Change: As mentioned above, many style transfer models, such as DualRL, tend to make few changes to the input sentence and output the same sentence. Actually, this is a common issue for unsupervised style transfer systems and we also meet it during our experiments. The main reason for the issue is that rewards for content preservation are too prominent and rewards for style accuracy cannot work well. In contrast, in order to guarantee the readability and fluency of the output sentence, we also cannot emphasize too much on rewards for style accuracy because it may cause some other issues such as word repetition mentioned below. A method to solve the problem is tuning hyperparameters and this is also the method we implement in this work. As for content preservation, maybe MLE methods such as back-translation are not enough because they tend to force models to generate specific words. In the future, we should further design some more suitable methods to control content preservation for models without disentangling style and content representations, such as DualRL and ours. Word Repetition: During our experiments, we observe that some of the outputs prefer to repeat the same word as shown in Table TABREF38 . This is because reinforcement learning rewards encourage the model to generate words which can get high scores from classifiers and even back-translation cannot stop it. Our solution is that we can lower the probability of decoding a word in decoders if the word has been generated in the previous time steps during testing. We also try to implement this method during training time but obtain worse performances because it may limit the effects of training. Some previous studies utilize language models to control the fluency of the output sentence and we also try this method. Nonetheless, pre-training a language model with tweets and using it to generate rewards is difficult because tweets are more casual and have more noise. Rewards from that kind of language model are usually not accurate and may confuse the model. In the future, we should come up with better methods to model language fluency with the consideration of irony accuracy, sentiment and content preservation, especially for tweets. Improper Words: As ironic style is hard for our model to learn, it may generate some improper words which make the sentence strange. As the example shown in the Table TABREF38 , the sentiment word in the input sentence is “wonderful" and the model should change it into a negative word such as “sad" to make the output sentence ironic. However, the model changes “friday" and “fifa" which are not related to ironic styles. We have not found a very effective method to address this issue and maybe we should further explore stronger models to learn ironic styles better. Additional Experiments In this section, we describe some additional experiments on the transformation from ironic sentences to non-ironic sentences. Sometimes ironies are hard to understand and may cause misunderstanding, for which our task also explores the transformation from ironic sentences to non-ironic sentences. As shown in Table TABREF46 , we also conduct automatic evaluations and the conclusions are similar to those of the transformation from non-ironic sentences to ironic sentences. As for human evaluation results in Table TABREF47 , our model still can achieve the second-best results in sentiment and content preservation. Nevertheless, DualRL system and ours get poor performances in irony accuracy. The reason may be that the other four baselines tend to generate common and even not fluent sentences which are irrelevant to the input sentences and are hard to be identified as ironies. So annotators usually mark these output sentences as non-ironic sentences, which causes these models to obtain better performances than DualRL and ours but much poorer results in sentiment and content preservation. Some examples are shown in Table TABREF52 . Conclusion and Future Work In this paper, we first systematically define irony generation based on style transfer. Because of the lack of irony data, we make use of twitter and build a large-scale dataset. In order to control irony accuracy, sentiment preservation and content preservation at the same time, we also design a combination of rewards for reinforcement learning and incorporate reinforcement learning with a pre-training process. Experimental results demonstrate that our model outperforms other generative models and our rewards are effective. Although our model design is effective, there are still many errors and we systematically analyze them. In the future, we are interested in exploring these directions and our work may extend to other kinds of ironies which are more difficult to model.
What experiments are conducted?
5c3e98e3cebaecd5d3e75ec2c9fc3dd267ac3c83
What Ġexperiments Ġare Ġconducted ?
Iron y ĠClass ifier , ĠSent iment ĠClass ifier Ġfor ĠIron y , ĠSent iment ĠClass ifier Ġfor ĠNon - ir ony , Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences
<s> What Ġexperiments Ġare Ġconducted ? </s> Introduction </s> The Ġirony Ġis Ġa Ġkind Ġof Ġfigur ative Ġlanguage , Ġwhich Ġis Ġwidely Ġused Ġon Ġsocial Ġmedia ĠB IB REF 0 Ġ. ĠThe Ġirony Ġis Ġdefined Ġas Ġa Ġclash Ġbetween Ġthe Ġintended Ġmeaning Ġof Ġa Ġsentence Ġand Ġits Ġliteral Ġmeaning ĠB IB REF 1 Ġ. ĠAs Ġan Ġimportant Ġaspect Ġof Ġlanguage , Ġirony Ġplays Ġan Ġessential Ġrole Ġin Ġsentiment Ġanalysis ĠB IB REF 2 Ġ, ĠB IB REF 0 Ġand Ġopinion Ġmining ĠB IB REF 3 Ġ, ĠB IB REF 4 Ġ. </s> Although Ġsome Ġprevious Ġstudies Ġfocus Ġon Ġirony Ġdetection , Ġlittle Ġattention Ġis Ġpaid Ġto Ġirony Ġgeneration . ĠAs Ġiron ies Ġcan Ġstrengthen Ġsentiments Ġand Ġexpress Ġstronger Ġemotions , Ġwe Ġmainly Ġfocus Ġon Ġgenerating Ġironic Ġsentences . ĠGiven Ġa Ġnon - iron ic Ġsentence , Ġwe Ġimplement Ġa Ġneural Ġnetwork Ġto Ġtransfer Ġit Ġto Ġan Ġironic Ġsentence Ġand Ġconst rain Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġtwo Ġsentences Ġto Ġbe Ġthe Ġsame . ĠFor Ġexample , Ġthe Ġinput Ġis ĠâĢ ľ I Ġhate Ġit Ġwhen Ġmy Ġplans Ġget Ġruined " Ġwhich Ġis Ġnegative Ġin Ġsentiment Ġpol arity Ġand Ġthe Ġoutput Ġshould Ġbe Ġironic Ġand Ġnegative Ġin Ġsentiment Ġas Ġwell , Ġsuch Ġas ĠâĢ ľ I Ġlike Ġit Ġwhen Ġmy Ġplans Ġget Ġruined ". ĠThe Ġspeaker Ġuses ĠâĢ ľ like " Ġto Ġbe Ġironic Ġand Ġexpress Ġhis Ġor Ġher Ġnegative Ġsentiment . ĠAt Ġthe Ġsame Ġtime , Ġour Ġmodel Ġcan Ġpreserve Ġcontents Ġwhich Ġare Ġirrelevant Ġto Ġsentiment Ġpol arity Ġand Ġirony . ĠAccording Ġto Ġthe Ġcategories Ġmentioned Ġin ĠB IB REF 5 Ġ, Ġirony Ġcan Ġbe Ġclassified Ġinto Ġ3 Ġclasses : Ġverbal Ġirony Ġby Ġmeans Ġof Ġa Ġpol arity Ġcontrast , Ġthe Ġsentences Ġcontaining Ġexpression Ġwhose Ġpol arity Ġis Ġinverted Ġbetween Ġthe Ġintended Ġand Ġthe Ġliteral Ġevaluation ; Ġother Ġtypes Ġof Ġverbal Ġirony , Ġthe Ġsentences Ġthat Ġshow Ġno Ġpol arity Ġcontrast Ġbetween Ġthe Ġliteral Ġand Ġintended Ġmeaning Ġbut Ġare Ġstill Ġironic ; Ġand Ġsituational Ġirony , Ġthe Ġsentences Ġthat Ġdescribe Ġsituations Ġthat Ġfail Ġto Ġmeet Ġsome Ġexpectations . ĠAs Ġiron ies Ġin Ġthe Ġlatter Ġtwo Ġcategories Ġare Ġobscure Ġand Ġhard Ġto Ġunderstand , Ġwe Ġdecide Ġto Ġonly Ġfocus Ġon Ġiron ies Ġin Ġthe Ġfirst Ġcategory Ġin Ġthis Ġwork . ĠFor Ġexample , Ġour Ġwork Ġcan Ġbe Ġspecifically Ġdescribed Ġas : Ġgiven Ġa Ġsentence ĠâĢ ľ I Ġhate Ġto Ġbe Ġignored ", Ġwe Ġtrain Ġour Ġmodel Ġto Ġgenerate Ġan Ġironic Ġsentence Ġsuch Ġas ĠâĢ ľ I Ġlove Ġto Ġbe Ġignored ". ĠAlthough Ġthere Ġis ĠâĢ ľ love " Ġin Ġthe Ġgenerated Ġsentence , Ġthe Ġspeaker Ġstill Ġexpresses Ġhis Ġor Ġher Ġnegative Ġsentiment Ġby Ġirony . ĠWe Ġalso Ġmake Ġsome Ġexplor ations Ġin Ġthe Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences Ġat Ġthe Ġend Ġof Ġour Ġwork . ĠBecause Ġof Ġthe Ġlack Ġof Ġprevious Ġwork Ġand Ġbas elines Ġon Ġirony Ġgeneration , Ġwe Ġimplement Ġour Ġmodel Ġbased Ġon Ġstyle Ġtransfer . ĠOur Ġwork Ġwill Ġnot Ġonly Ġprovide Ġthe Ġfirst Ġlarge - scale Ġirony Ġdataset Ġbut Ġalso Ġmake Ġour Ġmodel Ġas Ġa Ġbenchmark Ġfor Ġthe Ġirony Ġgeneration . </s> Recently , Ġun super vised Ġstyle Ġtransfer Ġbecomes Ġa Ġvery Ġpopular Ġtopic . ĠMany Ġstate - of - the - art Ġstudies Ġtry Ġto Ġsolve Ġthe Ġtask Ġwith Ġsequence - to - sequence Ġ( seq 2 seq ) Ġframework . ĠThere Ġare Ġthree Ġmain Ġways Ġto Ġbuild Ġup Ġmodels . ĠThe Ġfirst Ġis Ġto Ġlearn Ġa Ġlatent Ġstyle - independent Ġcontent Ġrepresentation Ġand Ġgenerate Ġsentences Ġwith Ġthe Ġcontent Ġrepresentation Ġand Ġanother Ġstyle ĠB IB REF 6 Ġ, ĠB IB REF 7 Ġ. ĠThe Ġsecond Ġis Ġto Ġdirectly Ġtransfer Ġsentences Ġfrom Ġone Ġstyle Ġto Ġanother Ġunder Ġthe Ġcontrol Ġof Ġclass ifiers Ġand Ġreinforcement Ġlearning ĠB IB REF 8 Ġ. ĠThe Ġthird Ġis Ġto Ġremove Ġstyle Ġattribute Ġwords Ġfrom Ġthe Ġinput Ġsentence Ġand Ġcombine Ġthe Ġremaining Ġcontent Ġwith Ġnew Ġstyle Ġattribute Ġwords ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġ. ĠThe Ġfirst Ġmethod Ġusually Ġob tains Ġbetter Ġperformances Ġvia Ġadvers arial Ġtraining Ġwith Ġdiscrim inators . ĠThe Ġstyle - independent Ġcontent Ġrepresentation , Ġnevertheless , Ġis Ġnot Ġeasily Ġobtained ĠB IB REF 11 Ġ, Ġwhich Ġresults Ġin Ġpoor Ġperformances . ĠThe Ġsecond Ġmethod Ġis Ġsuitable Ġfor Ġcomplex Ġstyles Ġwhich Ġare Ġdifficult Ġto Ġmodel Ġand Ġdescribe . ĠThe Ġmodel Ġcan Ġlearn Ġthe Ġdeep Ġsemantic Ġfeatures Ġby Ġitself Ġbut Ġsometimes Ġthe Ġmodel Ġis Ġsensitive Ġto Ġparameters Ġand Ġhard Ġto Ġtrain . ĠThe Ġthird Ġmethod Ġsucceeds Ġto Ġpreserve Ġcontent Ġbut Ġcannot Ġwork Ġfor Ġsome Ġcomplex Ġstyles Ġsuch Ġas Ġdemocratic Ġand Ġrepublican . ĠSent ences Ġwith Ġthose Ġstyles Ġusually Ġdo Ġnot Ġhave Ġspecific Ġstyle Ġattribute Ġwords . ĠUnfortunately , Ġdue Ġto Ġthe Ġlack Ġof Ġlarge Ġirony Ġdataset Ġand Ġdifficulties Ġof Ġmodeling Ġiron ies , Ġthere Ġhas Ġbeen Ġlittle Ġwork Ġtrying Ġto Ġgenerate Ġiron ies Ġbased Ġon Ġseq 2 seq Ġframework Ġas Ġfar Ġas Ġwe Ġknow . ĠInspired Ġby Ġmethods Ġfor Ġstyle Ġtransfer , Ġwe Ġdecide Ġto Ġimplement Ġa Ġspecifically Ġdesigned Ġmodel Ġbased Ġon Ġun super vised Ġstyle Ġtransfer Ġto Ġexplore Ġirony Ġgeneration . </s> In Ġthis Ġpaper , Ġin Ġorder Ġto Ġaddress Ġthe Ġlack Ġof Ġirony Ġdata , Ġwe Ġfirst Ġcrawl Ġover Ġ2 M Ġtweets Ġfrom Ġtwitter Ġto Ġbuild Ġa Ġdataset Ġwith Ġ262 , 755 Ġironic Ġand Ġ112 , 330 Ġnon - iron ic Ġtweets . ĠThen , Ġdue Ġto Ġthe Ġlack Ġof Ġparallel Ġdata , Ġwe Ġpropose Ġa Ġnovel Ġmodel Ġto Ġtransfer Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences Ġin Ġan Ġun super vised Ġway . ĠAs Ġironic Ġstyle Ġis Ġhard Ġto Ġmodel Ġand Ġdescribe , Ġwe Ġimplement Ġour Ġmodel Ġwith Ġthe Ġcontrol Ġof Ġclass ifiers Ġand Ġreinforcement Ġlearning . ĠDifferent Ġfrom Ġother Ġstudies Ġin Ġstyle Ġtransfer , Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġto Ġironic Ġsentences Ġhas Ġto Ġpreserve Ġsentiment Ġpol arity Ġas Ġmentioned Ġabove . ĠTherefore , Ġwe Ġnot Ġonly Ġdesign Ġan Ġirony Ġreward Ġto Ġcontrol Ġthe Ġirony Ġaccuracy Ġand Ġimplement Ġden o ising Ġauto - enc oder Ġand Ġback - translation Ġto Ġcontrol Ġcontent Ġpreservation Ġbut Ġalso Ġdesign Ġa Ġsentiment Ġreward Ġto Ġcontrol Ġsentiment Ġpreservation . </s> Exper imental Ġresults Ġdemonstrate Ġthat Ġour Ġmodel Ġachieves Ġa Ġhigh Ġirony Ġaccuracy Ġwith Ġwell - pres erved Ġsentiment Ġand Ġcontent . ĠThe Ġcontributions Ġof Ġour Ġwork Ġare Ġas Ġfollows : </s> Related ĠWork </s> Style ĠTransfer : ĠAs Ġirony Ġis Ġa Ġcomplicated Ġstyle Ġand Ġhard Ġto Ġmodel Ġwith Ġsome Ġspecific Ġstyle Ġattribute Ġwords , Ġwe Ġmainly Ġfocus Ġon Ġstudies Ġwithout Ġediting Ġstyle Ġattribute Ġwords . </s> Some Ġstudies Ġare Ġtrying Ġto Ġdis ent angle Ġstyle Ġrepresentation Ġfrom Ġcontent Ġrepresentation . ĠIn ĠB IB REF 12 Ġ, Ġauthors Ġleverage Ġadvers arial Ġnetworks Ġto Ġlearn Ġseparate Ġcontent Ġrepresentations Ġand Ġstyle Ġrepresentations . ĠIn ĠB IB REF 13 Ġand ĠB IB REF 6 Ġ, Ġresearchers Ġcombine Ġvari ational Ġauto - enc od ers Ġ( VA Es ) Ġwith Ġstyle Ġdiscrim inators . </s> However , Ġsome Ġrecent Ġstudies ĠB IB REF 11 Ġreveal Ġthat Ġthe Ġdis ent ang lement Ġof Ġcontent Ġand Ġstyle Ġrepresentations Ġmay Ġnot Ġbe Ġachieved Ġin Ġpractice . ĠTherefore , Ġsome Ġother Ġresearch Ġstudies ĠB IB REF 9 Ġ, ĠB IB REF 10 Ġstrive Ġto Ġseparate Ġcontent Ġand Ġstyle Ġby Ġremoving Ġstyl istic Ġwords . ĠNonetheless , Ġmany Ġnon - iron ic Ġsentences Ġdo Ġnot Ġhave Ġspecific Ġstyl istic Ġwords Ġand Ġas Ġa Ġresult , Ġwe Ġfind Ġit Ġdifficult Ġto Ġtransfer Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences Ġthrough Ġthis Ġway Ġin Ġpractice . </s> Besides , Ġsome Ġother Ġresearch Ġstudies Ġdo Ġnot Ġdis ent angle Ġstyle Ġfrom Ġcontent Ġbut Ġdirectly Ġlearn Ġrepresentations Ġof Ġsentences . ĠIn ĠB IB REF 8 Ġ, Ġauthors Ġpropose Ġa Ġdual Ġreinforcement Ġlearning Ġframework Ġwithout Ġseparating Ġcontent Ġand Ġstyle Ġrepresentations . ĠIn ĠB IB REF 7 Ġ, Ġresearchers Ġutilize Ġa Ġmachine Ġtranslation Ġmodel Ġto Ġlearn Ġa Ġsentence Ġrepresentation Ġpreserving Ġthe Ġmeaning Ġof Ġthe Ġsentence Ġbut Ġreducing Ġstyl istic Ġproperties . ĠIn Ġthis Ġmethod , Ġthe Ġquality Ġof Ġgenerated Ġsentences Ġrelies Ġon Ġthe Ġperformance Ġof Ġclass ifiers Ġto Ġa Ġlarge Ġextent . ĠMeanwhile , Ġsuch Ġmodels Ġare Ġusually Ġsensitive Ġto Ġparameters Ġand Ġdifficult Ġto Ġtrain . ĠIn Ġcontrast , Ġwe Ġcombine Ġa Ġpre - training Ġprocess Ġwith Ġreinforcement Ġlearning Ġto Ġbuild Ġup Ġa Ġstable Ġlanguage Ġmodel Ġand Ġdesign Ġspecial Ġrewards Ġfor Ġour Ġtask . </s> Iron y ĠDetection : ĠWith Ġthe Ġdevelopment Ġof Ġsocial Ġmedia , Ġirony Ġdetection Ġbecomes Ġa Ġmore Ġimportant Ġtask . ĠMethods Ġfor Ġirony Ġdetection Ġcan Ġbe Ġmainly Ġdivided Ġinto Ġtwo Ġcategories : Ġmethods Ġbased Ġon Ġfeature Ġengineering Ġand Ġmethods Ġbased Ġon Ġneural Ġnetworks . </s> As Ġfor Ġmethods Ġbased Ġon Ġfeature Ġengineering , ĠIn ĠB IB REF 1 Ġ, Ġauthors Ġinvestigate Ġpragmatic Ġphenomena Ġand Ġvarious Ġirony Ġmarkers . ĠIn ĠB IB REF 14 Ġ, Ġresearchers Ġleverage Ġa Ġcombination Ġof Ġsentiment , Ġdistribution al Ġsemantic Ġand Ġtext Ġsurface Ġfeatures . ĠThose Ġmodels Ġrely Ġon Ġhand - crafted Ġfeatures Ġand Ġare Ġhard Ġto Ġimplement . </s> When Ġit Ġcomes Ġto Ġmethods Ġbased Ġon Ġneural Ġnetworks , Ġlong Ġshort - term Ġmemory Ġ( L ST M ) ĠB IB REF 15 Ġnetwork Ġis Ġwidely Ġused Ġand Ġis Ġvery Ġefficient Ġfor Ġirony Ġdetection . ĠIn ĠB IB REF 16 Ġ, Ġa Ġtweet Ġis Ġdivided Ġinto Ġtwo Ġsegments Ġand Ġa Ġsubtract Ġlayer Ġis Ġimplemented Ġto Ġcalculate Ġthe Ġdifference Ġbetween Ġtwo Ġsegments Ġin Ġorder Ġto Ġdetermine Ġwhether Ġthe Ġtweet Ġis Ġironic . ĠIn ĠB IB REF 17 Ġ, Ġauthors Ġutilize Ġa Ġrecurrent Ġneural Ġnetwork Ġwith ĠBi - L ST M Ġand Ġself - att ention Ġwithout Ġhand - crafted Ġfeatures . ĠIn ĠB IB REF 18 Ġ, Ġresearchers Ġpropose Ġa Ġsystem Ġbased Ġon Ġa Ġdensely Ġconnected ĠL ST M Ġnetwork . </s> Our ĠDat as et </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġhow Ġwe Ġbuild Ġour Ġdataset Ġwith Ġtweets . ĠFirst , Ġwe Ġcrawl Ġover Ġ2 M Ġtweets Ġfrom Ġtwitter Ġusing ĠGet Old Twe ets - python . ĠWe Ġcrawl ĠEnglish Ġtweets Ġfrom Ġ04 / 09 / 2012 Ġto Ġ/ 12 / 18 / 2018 . ĠWe Ġfirst Ġremove Ġall Ġre - t we ets Ġand Ġuse Ġlang det ect Ġto Ġremove Ġall Ġnon - English Ġsentences . ĠThen , Ġwe Ġremove Ġhasht ags Ġattached Ġat Ġthe Ġend Ġof Ġthe Ġtweets Ġbecause Ġthey Ġare Ġusually Ġnot Ġparts Ġof Ġsentences Ġand Ġwill Ġconfuse Ġour Ġlanguage Ġmodel . ĠAfter Ġthat , Ġwe Ġutilize ĠEk ph ras is Ġto Ġprocess Ġtweets . ĠWe Ġremove ĠURLs Ġand Ġrestore Ġremaining Ġhasht ags , Ġelong ated Ġwords , Ġrepeated Ġwords , Ġand Ġall - capital ized Ġwords . ĠTo Ġsimplify Ġour Ġdataset , ĠWe Ġreplace Ġall ĠâĢ ľ ĠIN LINE FORM 0 Ġmoney ĠIN LINE FORM 1 Ġ" Ġand ĠâĢ ľ ĠIN LINE FORM 2 Ġtime ĠIN LINE FORM 3 Ġ" Ġtokens Ġwith ĠâĢ ľ ĠIN LINE FORM 4 Ġnumber ĠIN LINE FORM 5 Ġ" Ġtoken Ġwhen Ġusing ĠEk ph ras is . ĠAnd Ġwe Ġdelete Ġsentences Ġwhose Ġlengths Ġare Ġless Ġthan Ġ10 Ġor Ġgreater Ġthan Ġ40 . ĠIn Ġorder Ġto Ġrestore Ġabbrevi ations , Ġwe Ġdownload Ġan Ġabbre viation Ġdictionary Ġfrom Ġweb op edia Ġand Ġrestore Ġabbrevi ations Ġto Ġnormal Ġwords Ġor Ġphrases Ġaccording Ġto Ġthe Ġdictionary . ĠFinally , Ġwe Ġremove Ġsentences Ġwhich Ġhave Ġmore Ġthan Ġtwo Ġrare Ġwords Ġ( app earing Ġless Ġthan Ġthree Ġtimes ) Ġin Ġorder Ġto Ġconst rain Ġthe Ġsize Ġof Ġvocabulary . ĠFinally , Ġwe Ġget Ġ6 62 , 530 Ġsentences Ġafter Ġpre - processing . </s> As Ġneural Ġnetworks Ġare Ġproved Ġeffective Ġin Ġirony Ġdetection , Ġwe Ġdecide Ġto Ġimplement Ġa Ġneural Ġclass ifier Ġin Ġorder Ġto Ġclassify Ġthe Ġsentences Ġinto Ġironic Ġand Ġnon - iron ic Ġsentences . ĠHowever , Ġthe Ġonly Ġhigh - quality Ġirony Ġdataset Ġwe Ġcan Ġobtain Ġis Ġthe Ġdataset Ġof ĠSem eval - 2018 ĠTask Ġ3 Ġand Ġthe Ġdataset Ġis Ġpretty Ġsmall , Ġwhich Ġwill Ġcause Ġover fitting Ġto Ġcomplex Ġmodels . ĠTherefore , Ġwe Ġjust Ġimplement Ġa Ġsimple Ġone - layer ĠR NN Ġwith ĠL ST M Ġcell Ġto Ġclassify Ġpre - process ed Ġsentences Ġinto Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġbecause ĠL ST M Ġnetworks Ġare Ġwidely Ġused Ġin Ġirony Ġdetection . ĠWe Ġtrain Ġthe Ġmodel Ġwith Ġthe Ġdataset Ġof ĠSem eval - 2018 ĠTask Ġ3 . ĠAfter Ġclassification , Ġwe Ġget Ġ262 , 755 Ġironic Ġsentences Ġand Ġ399 , 775 Ġnon - iron ic Ġsentences . ĠAccording Ġto Ġour Ġobservation , Ġnot Ġall Ġnon - iron ic Ġsentences Ġare Ġsuitable Ġto Ġbe Ġtransferred Ġinto Ġironic Ġsentences . ĠFor Ġexample , ĠâĢ ľ just Ġhanging Ġout Ġ. Ġwatching Ġ. Ġis Ġit Ġm onday Ġyet " Ġis Ġhard Ġto Ġtransfer Ġbecause Ġit Ġdoes Ġnot Ġhave Ġan Ġexplicit Ġsentiment Ġpol arity . ĠSo Ġwe Ġremove Ġall Ġinterrog ative Ġsentences Ġfrom Ġthe Ġnon - iron ic Ġsentences Ġand Ġonly Ġobtain Ġthe Ġsentences Ġwhich Ġhave Ġwords Ġexpressing Ġstrong Ġsentiments . ĠWe Ġevaluate Ġthe Ġsentiment Ġpol arity Ġof Ġeach Ġword Ġwith ĠText Bl ob Ġand Ġwe Ġview Ġthose Ġwords Ġwith Ġsentiment Ġscores Ġgreater Ġthan Ġ0 . 5 Ġor Ġless Ġthan Ġ- 0 . 5 Ġas Ġwords Ġexpressing Ġstrong Ġsentiments . ĠFinally , Ġwe Ġbuild Ġour Ġirony Ġdataset Ġwith Ġ262 , 755 Ġironic Ġsentences Ġand Ġ102 , 330 Ġnon - iron ic Ġsentences . </s> [ t ] ĠIron y ĠGeneration ĠAl gorithm </s> IN LINE FORM 0 Ġpre - train Ġwith Ġauto - enc oder ĠPre - train ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġwith ĠIN LINE FORM 3 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 16 ĠPre - train ĠIN LINE FORM 4 Ġ, ĠIN LINE FORM 5 Ġwith ĠIN LINE FORM 6 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 17 ĠIN LINE FORM 7 Ġpre - train Ġwith Ġback - translation ĠPre - train ĠIN LINE FORM 8 Ġ, ĠIN LINE FORM 9 Ġ, ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġwith ĠIN LINE FORM 12 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 19 ĠPre - train ĠIN LINE FORM 13 Ġ, ĠIN LINE FORM 14 Ġ, ĠIN LINE FORM 15 Ġ, ĠIN LINE FORM 16 Ġwith ĠIN LINE FORM 17 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 20 </s> IN LINE FORM 0 Ġtrain Ġwith ĠRL Ġeach Ġepoch Ġe Ġ= Ġ1 , Ġ2 , Ġ... , ĠIN LINE FORM 1 ĠIN LINE FORM 2 Ġtrain Ġnon - ir ony 2 ir ony Ġwith ĠRL ĠIN LINE FORM 3 Ġin ĠN ĠIN LINE FORM 4 Ġupdate ĠIN LINE FORM 5 Ġ, ĠIN LINE FORM 6 Ġ, Ġusing ĠIN LINE FORM 7 Ġbased Ġon ĠE q . ĠEQ REF 29 ĠIN LINE FORM 8 Ġback - translation ĠIN LINE FORM 9 ĠIN LINE FORM 10 ĠIN LINE FORM 11 Ġupdate ĠIN LINE FORM 12 Ġ, ĠIN LINE FORM 13 Ġ, ĠIN LINE FORM 14 Ġ, ĠIN LINE FORM 15 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 19 ĠIN LINE FORM 16 Ġtrain Ġirony 2 non - ir ony Ġwith ĠRL ĠIN LINE FORM 17 Ġin ĠI ĠIN LINE FORM 18 Ġupdate ĠIN LINE FORM 19 Ġ, ĠIN LINE FORM 20 Ġ, Ġusing ĠIN LINE FORM 21 Ġsimilar Ġto ĠE q . ĠEQ REF 29 ĠIN LINE FORM 22 Ġback - translation ĠIN LINE FORM 23 ĠIN LINE FORM 24 ĠIN LINE FORM 25 Ġupdate ĠIN LINE FORM 26 Ġ, ĠIN LINE FORM 27 Ġ, ĠIN LINE FORM 28 Ġ, ĠIN LINE FORM 29 Ġusing ĠM LE Ġbased Ġon ĠE q . ĠEQ REF 20 </s> Our ĠMethod </s> Given Ġtwo Ġnon - par allel Ġcorpor a : Ġnon - iron ic Ġcorpus ĠN ={ ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġ, Ġ... , ĠIN LINE FORM 2 Ġ} Ġand Ġironic Ġcorpus ĠI ={ ĠIN LINE FORM 3 Ġ, ĠIN LINE FORM 4 Ġ, Ġ... , ĠIN LINE FORM 5 Ġ}, Ġthe Ġgoal Ġof Ġour Ġirony Ġgeneration Ġmodel Ġis Ġto Ġgenerate Ġan Ġironic Ġsentence Ġfrom Ġa Ġnon - iron ic Ġsentence Ġwhile Ġpreserving Ġthe Ġcontent Ġand Ġsentiment Ġpol arity Ġof Ġthe Ġsource Ġinput Ġsentence . ĠWe Ġimplement Ġan Ġenc oder - dec oder Ġframework Ġwhere Ġtwo Ġenc od ers Ġare Ġutilized Ġto Ġencode Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġrespectively Ġand Ġtwo Ġdec od ers Ġare Ġutilized Ġto Ġdecode Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġfrom Ġlatent Ġrepresentations Ġrespectively . ĠIn Ġorder Ġto Ġenforce Ġa Ġshared Ġlatent Ġspace , Ġwe Ġshare Ġtwo Ġlayers Ġon Ġboth Ġthe Ġenc oder Ġside Ġand Ġthe Ġdec oder Ġside . ĠOur Ġmodel Ġarchitecture Ġis Ġillustrated Ġin ĠFigure ĠFIG REF 13 Ġ. ĠWe Ġdenote Ġirony Ġenc oder Ġas ĠIN LINE FORM 6 Ġ, Ġirony Ġdec oder Ġas ĠIN LINE FORM 7 Ġand Ġnon - ir ony Ġenc oder Ġas ĠIN LINE FORM 8 Ġ, Ġnon - ir ony Ġdec oder Ġas ĠIN LINE FORM 9 Ġ. ĠTheir Ġparameters Ġare ĠIN LINE FORM 10 Ġ, ĠIN LINE FORM 11 Ġ, ĠIN LINE FORM 12 Ġand ĠIN LINE FORM 13 Ġ. </s> Our Ġirony Ġgeneration Ġalgorithm Ġis Ġshown Ġin ĠAl gorithm ĠSEC REF 3 Ġ. ĠWe Ġfirst Ġpre - train Ġour Ġmodel Ġusing Ġden o ising Ġauto - enc oder Ġand Ġback - translation Ġto Ġbuild Ġup Ġlanguage Ġmodels Ġfor Ġboth Ġstyles Ġ( section ĠSEC REF 14 Ġ). ĠThen Ġwe Ġimplement Ġreinforcement Ġlearning Ġto Ġtrain Ġthe Ġmodel Ġto Ġtransfer Ġsentences Ġfrom Ġone Ġstyle Ġto Ġanother Ġ( section ĠSEC REF 21 Ġ). ĠMeanwhile , Ġto Ġachieve Ġcontent Ġpreservation , Ġwe Ġutilize Ġback - translation Ġfor Ġone Ġtime Ġin Ġevery ĠIN LINE FORM 0 Ġtime Ġsteps . </s> P ret raining </s> In Ġorder Ġto Ġbuild Ġup Ġour Ġlanguage Ġmodel Ġand Ġpreserve Ġthe Ġcontent , Ġwe Ġapply Ġthe Ġauto - enc oder Ġmodel . ĠTo Ġprevent Ġthe Ġmodel Ġfrom Ġsimply Ġcopying Ġthe Ġinput Ġsentence , Ġwe Ġrandomly Ġadd Ġsome Ġnoises Ġin Ġthe Ġinput Ġsentence . ĠSpecifically , Ġfor Ġevery Ġword Ġin Ġthe Ġinput Ġsentence , Ġthere Ġis Ġ10 % Ġchance Ġthat Ġwe Ġdelete Ġit , Ġ10 Ġ% Ġchance Ġthat Ġwe Ġduplicate Ġit , Ġ10 % Ġchance Ġthat Ġwe Ġswap Ġit Ġwith Ġthe Ġnext Ġword , Ġor Ġit Ġremains Ġunchanged . ĠWe Ġfirst Ġencode Ġthe Ġinput Ġsentence ĠIN LINE FORM 0 Ġor ĠIN LINE FORM 1 Ġwith Ġrespective Ġenc oder ĠIN LINE FORM 2 Ġor ĠIN LINE FORM 3 Ġto Ġobtain Ġits Ġlatent Ġrepresentation ĠIN LINE FORM 4 Ġor ĠIN LINE FORM 5 Ġand Ġreconstruct Ġthe Ġinput Ġsentence Ġwith Ġthe Ġlatent Ġrepresentation Ġand Ġrespective Ġdec oder . ĠSo Ġwe Ġcan Ġget Ġthe Ġreconstruction Ġloss Ġfor Ġauto - enc oder ĠIN LINE FORM 6 Ġ: ĠDIS PLAY FORM 0 ĠDIS PLAY FORM 1 </s> In Ġaddition Ġto Ġden o ising Ġauto - enc oder , Ġwe Ġimplement Ġback - translation ĠB IB REF 19 Ġto Ġgenerate Ġa Ġpseudo - par allel Ġcorpus . ĠSuppose Ġour Ġmodel Ġtakes Ġnon - iron ic Ġsentence ĠIN LINE FORM 0 Ġas Ġinput . ĠWe Ġfirst Ġencode ĠIN LINE FORM 1 Ġwith ĠIN LINE FORM 2 Ġto Ġobtain Ġits Ġlatent Ġrepresentation ĠIN LINE FORM 3 Ġand Ġdecode Ġthe Ġlatent Ġrepresentation Ġwith ĠIN LINE FORM 4 Ġto Ġget Ġa Ġtransferred Ġsentence ĠIN LINE FORM 5 Ġ. ĠThen Ġwe Ġencode ĠIN LINE FORM 6 Ġwith ĠIN LINE FORM 7 Ġand Ġdecode Ġits Ġlatent Ġrepresentation Ġwith ĠIN LINE FORM 8 Ġto Ġreconstruct Ġthe Ġoriginal Ġinput Ġsentence ĠIN LINE FORM 9 Ġ. ĠTherefore , Ġour Ġreconstruction Ġloss Ġfor Ġback - translation ĠIN LINE FORM 10 Ġ: ĠDIS PLAY FORM 0 </s> And Ġif Ġour Ġmodel Ġtakes Ġironic Ġsentence ĠIN LINE FORM 0 Ġas Ġinput , Ġwe Ġcan Ġget Ġthe Ġreconstruction Ġloss Ġfor Ġback - translation Ġas : ĠDIS PLAY FORM 0 </s> Re in forcement ĠLearning </s> Since Ġthe Ġgold Ġtransferred Ġresult Ġof Ġinput Ġis Ġunavailable , Ġwe Ġcannot Ġevaluate Ġthe Ġquality Ġof Ġthe Ġgenerated Ġsentence Ġdirectly . ĠTherefore , Ġwe Ġimplement Ġreinforcement Ġlearning Ġand Ġelabor ately Ġdesign Ġtwo Ġrewards Ġto Ġdescribe Ġthe Ġirony Ġaccuracy Ġand Ġsentiment Ġpreservation , Ġrespectively . </s> A Ġpre - trained Ġbinary Ġirony Ġclass ifier Ġbased Ġon ĠCNN ĠB IB REF 20 Ġis Ġused Ġto Ġevaluate Ġhow Ġironic Ġa Ġsentence Ġis . ĠWe Ġdenote Ġthe Ġparameter Ġof Ġthe Ġclass ifier Ġas ĠIN LINE FORM 0 Ġand Ġit Ġis Ġfixed Ġduring Ġthe Ġtraining Ġprocess . </s> In Ġorder Ġto Ġfacilitate Ġthe Ġtransformation , Ġwe Ġdesign Ġthe Ġirony Ġreward Ġas Ġthe Ġdifference Ġbetween Ġthe Ġirony Ġscore Ġof Ġthe Ġinput Ġsentence Ġand Ġthat Ġof Ġthe Ġoutput Ġsentence . ĠForm ally , Ġwhen Ġwe Ġinput Ġa Ġnon - iron ic Ġsentence ĠIN LINE FORM 0 Ġand Ġtransfer Ġit Ġto Ġan Ġironic Ġsentence ĠIN LINE FORM 1 Ġ, Ġour Ġirony Ġreward Ġis Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġdenotes Ġironic Ġstyle Ġand ĠIN LINE FORM 1 Ġis Ġthe Ġprobability Ġof Ġthat Ġa Ġsentence ĠIN LINE FORM 2 Ġis Ġironic . </s> To Ġpreserve Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġinput Ġsentence , Ġwe Ġalso Ġneed Ġto Ġuse Ġclass ifiers Ġto Ġevaluate Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġsentences . ĠHowever , Ġthe Ġsentiment Ġanalysis Ġof Ġironic Ġsentences Ġand Ġnon - iron ic Ġsentences Ġare Ġdifferent . ĠIn Ġthe Ġcase Ġof Ġfigur ative Ġlanguages Ġsuch Ġas Ġirony , Ġsarc asm Ġor Ġmetaphor , Ġthe Ġsentiment Ġpol arity Ġof Ġthe Ġliteral Ġmeaning Ġmay Ġdiffer Ġsignificantly Ġfrom Ġthat Ġof Ġthe Ġintended Ġfigur ative Ġmeaning ĠB IB REF 0 Ġ. ĠAs Ġwe Ġaim Ġto Ġtrain Ġour Ġmodel Ġto Ġtransfer Ġsentences Ġfrom Ġnon - iron ic Ġto Ġironic , Ġusing Ġonly Ġone Ġclass ifier Ġis Ġnot Ġenough . ĠAs Ġa Ġresult , Ġwe Ġimplement Ġtwo Ġpre - trained Ġsentiment Ġclass ifiers Ġfor Ġnon - iron ic Ġsentences Ġand Ġironic Ġsentences Ġrespectively . ĠWe Ġdenote Ġthe Ġparameter Ġof Ġthe Ġsentiment Ġclass ifier Ġfor Ġironic Ġsentences Ġas ĠIN LINE FORM 0 Ġand Ġthat Ġof Ġthe Ġsentiment Ġclass ifier Ġfor Ġnon - iron ic Ġsentences Ġas ĠIN LINE FORM 1 Ġ. </s> A Ġchallenge , Ġwhen Ġwe Ġimplement Ġtwo Ġclass ifiers Ġto Ġevaluate Ġthe Ġsentiment Ġpol arity , Ġis Ġthat Ġthe Ġtwo Ġclass ifiers Ġtrained Ġwith Ġdifferent Ġdatasets Ġmay Ġhave Ġdifferent Ġdistributions Ġof Ġscores . ĠThat Ġmeans Ġwe Ġcannot Ġdirectly Ġcalculate Ġthe Ġsentiment Ġreward Ġwith Ġscores Ġapplied Ġby Ġtwo Ġclass ifiers . ĠTo Ġalleviate Ġthis Ġproblem Ġand Ġstandard ize Ġthe Ġprediction Ġresults Ġof Ġtwo Ġclass ifiers , Ġwe Ġset Ġa Ġthreshold Ġfor Ġeach Ġclass ifier Ġand Ġsubtract Ġthe Ġrespective Ġthreshold Ġfrom Ġscores Ġapplied Ġby Ġthe Ġclass ifier Ġto Ġobtain Ġthe Ġcomparative Ġsentiment Ġpol arity Ġscore . ĠWe Ġget Ġthe Ġoptimal Ġthreshold Ġby Ġmaximizing Ġthe Ġability Ġof Ġthe Ġclass ifier Ġaccording Ġto Ġthe Ġdistribution Ġof Ġour Ġtraining Ġdata . </s> We Ġdenote Ġthe Ġthreshold Ġof Ġironic Ġsentiment Ġclass ifier Ġas ĠIN LINE FORM 0 Ġand Ġthe Ġthreshold Ġof Ġnon - iron ic Ġsentiment Ġclass ifier Ġas ĠIN LINE FORM 1 Ġ. ĠThe Ġstandardized Ġsentiment Ġscore Ġis Ġdefined Ġas ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġwhere ĠIN LINE FORM 4 Ġdenotes Ġthe Ġpositive Ġsentiment Ġpol arity Ġand ĠIN LINE FORM 5 Ġis Ġthe Ġprobability Ġof Ġthat Ġa Ġsentence Ġis Ġpositive Ġin Ġsentiment Ġpol arity . </s> As Ġmentioned Ġabove , Ġthe Ġinput Ġsentence Ġand Ġthe Ġgenerated Ġsentence Ġshould Ġexpress Ġthe Ġsame Ġsentiment . ĠFor Ġexample , Ġif Ġwe Ġinput Ġa Ġnon - iron ic Ġsentence ĠâĢ ľ I Ġhate Ġto Ġbe Ġignored " Ġwhich Ġis Ġnegative Ġin Ġsentiment Ġpol arity , Ġthe Ġgenerated Ġironic Ġsentence Ġshould Ġbe Ġalso Ġnegative , Ġsuch Ġas ĠâĢ ľ I Ġlove Ġto Ġbe Ġignored ". ĠTo Ġachieve Ġsentiment Ġpreservation , Ġwe Ġdesign Ġthe Ġsentiment Ġreward Ġas Ġthat Ġone Ġminus Ġthe Ġabsolute Ġvalue Ġof Ġthe Ġdifference Ġbetween Ġthe Ġstandardized Ġsentiment Ġscore Ġof Ġthe Ġinput Ġsentence Ġand Ġthat Ġof Ġthe Ġgenerated Ġsentence . ĠForm ally , Ġwhen Ġwe Ġinput Ġa Ġnon - iron ic Ġsentence ĠIN LINE FORM 0 Ġand Ġtransfer Ġit Ġto Ġan Ġironic Ġsentence ĠIN LINE FORM 1 Ġ, Ġour Ġsentiment Ġreward Ġis Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> To Ġencourage Ġour Ġmodel Ġto Ġfocus Ġon Ġboth Ġthe Ġirony Ġaccuracy Ġand Ġthe Ġsentiment Ġpreservation , Ġwe Ġapply Ġthe Ġharmonic Ġmean Ġof Ġirony Ġreward Ġand Ġsentiment Ġreward : ĠDIS PLAY FORM 0 </s> Policy ĠGrad ient </s> The Ġpolicy Ġgradient Ġalgorithm ĠB IB REF 21 Ġis Ġa Ġsimple Ġbut Ġwidely - used Ġalgorithm Ġin Ġreinforcement Ġlearning . ĠIt Ġis Ġused Ġto Ġmaximize Ġthe Ġexpected Ġreward ĠIN LINE FORM 0 Ġ. ĠThe Ġobjective Ġfunction Ġto Ġminimize Ġis Ġdefined Ġas : ĠDIS PLAY FORM 0 </s> where ĠIN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġis Ġthe Ġreward Ġof ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġis Ġthe Ġinput Ġsize . </s> Training ĠDetails </s> IN LINE FORM 0 Ġ, ĠIN LINE FORM 1 Ġ, ĠIN LINE FORM 2 Ġand ĠIN LINE FORM 3 Ġin Ġour Ġmodel Ġare ĠTransformers ĠB IB REF 22 Ġwith Ġ4 Ġlayers Ġand Ġ2 Ġshared Ġlayers . ĠThe Ġword Ġembed d ings Ġof Ġ128 Ġdimensions Ġare Ġlearned Ġduring Ġthe Ġtraining Ġprocess . ĠOur Ġmaximum Ġsentence Ġlength Ġis Ġset Ġas Ġ40 . ĠThe Ġoptim izer Ġis ĠAdam ĠB IB REF 23 Ġand Ġthe Ġlearning Ġrate Ġis ĠIN LINE FORM 4 Ġ. ĠThe Ġbatch Ġsize Ġis Ġ32 Ġand Ġharmonic Ġweight ĠIN LINE FORM 5 Ġin ĠE q . 9 Ġis Ġ0 . 5 . ĠWe Ġset Ġthe Ġinterval ĠIN LINE FORM 6 Ġas Ġ200 . ĠThe Ġmodel Ġis Ġpre - trained Ġfor Ġ6 Ġepoch s Ġand Ġtrained Ġfor Ġ15 Ġepoch s Ġfor Ġreinforcement Ġlearning . </s> Iron y ĠClass ifier : ĠWe Ġimplement Ġa ĠCNN Ġclass ifier Ġtrained Ġwith Ġour Ġirony Ġdataset . ĠAll Ġthe ĠCNN Ġclass ifiers Ġwe Ġutilize Ġin Ġthis Ġpaper Ġuse Ġthe Ġsame Ġparameters Ġas ĠB IB REF 20 Ġ. </s> Sent iment ĠClass ifier Ġfor ĠIron y : ĠWe Ġfirst Ġimplement Ġa Ġone - layer ĠL ST M Ġnetwork Ġto Ġclassify Ġironic Ġsentences Ġin Ġour Ġdataset Ġinto Ġpositive Ġand Ġnegative Ġiron ies . ĠThe ĠL ST M Ġnetwork Ġis Ġtrained Ġwith Ġthe Ġdataset Ġof ĠSem eval Ġ2015 ĠTask Ġ11 ĠB IB REF 0 Ġwhich Ġis Ġused Ġfor Ġthe Ġsentiment Ġanalysis Ġof Ġfigur ative Ġlanguage Ġin Ġtwitter . ĠThen , Ġwe Ġuse Ġthe Ġpositive Ġiron ies Ġand Ġnegative Ġiron ies Ġto Ġtrain Ġthe ĠCNN Ġsentiment Ġclass ifier Ġfor Ġirony . </s> Sent iment ĠClass ifier Ġfor ĠNon - ir ony : ĠSimilar Ġto Ġthe Ġtraining Ġprocess Ġof Ġthe Ġsentiment Ġclass ifier Ġfor Ġirony , Ġwe Ġfirst Ġimplement Ġa Ġone - layer ĠL ST M Ġnetwork Ġtrained Ġwith Ġthe Ġdataset Ġfor Ġthe Ġsentiment Ġanalysis Ġof Ġcommon Ġtw itters Ġto Ġclassify Ġthe Ġnon - iron ies Ġinto Ġpositive Ġand Ġnegative Ġnon - iron ies . ĠThen Ġwe Ġuse Ġthe Ġpositive Ġand Ġnegative Ġnon - iron ies Ġto Ġtrain Ġthe Ġsentiment Ġclass ifier Ġfor Ġnon - ir ony . </s> Bas elines </s> We Ġcompare Ġour Ġmodel Ġwith Ġthe Ġfollowing Ġstate - of - art Ġgener ative Ġmodels : </s> Back Trans ĠB IB REF 7 Ġ: ĠIn ĠB IB REF 7 Ġ, Ġauthors Ġpropose Ġa Ġmodel Ġusing Ġmachine Ġtranslation Ġin Ġorder Ġto Ġpreserve Ġthe Ġmeaning Ġof Ġthe Ġsentence Ġwhile Ġreducing Ġstyl istic Ġproperties . </s> Un pa ired ĠB IB REF 10 Ġ: ĠIn ĠB IB REF 10 Ġ, Ġresearchers Ġimplement Ġa Ġmethod Ġto Ġremove Ġemotional Ġwords Ġand Ġadd Ġdesired Ġsentiment Ġcontrolled Ġby Ġreinforcement Ġlearning . </s> Cross Al ign ĠB IB REF 6 Ġ: ĠIn ĠB IB REF 6 Ġ, Ġauthors Ġleverage Ġrefined Ġalignment Ġof Ġlatent Ġrepresentations Ġto Ġperform Ġstyle Ġtransfer Ġand Ġa Ġcross - aligned Ġauto - enc oder Ġis Ġimplemented . </s> CP TG ĠB IB REF 24 Ġ: ĠAn Ġinterpol ated Ġreconstruction Ġloss Ġis Ġintroduced Ġin ĠB IB REF 24 Ġand Ġa Ġdiscrim inator Ġis Ġimplemented Ġto Ġcontrol Ġattributes Ġin Ġthis Ġwork . </s> Dual RL ĠB IB REF 8 Ġ: ĠIn ĠB IB REF 8 Ġ, Ġresearchers Ġuse Ġtwo Ġreinforcement Ġrewards Ġsimultaneously Ġto Ġcontrol Ġstyle Ġaccuracy Ġand Ġcontent Ġpreservation . </s> E val uation ĠMet rics </s> In Ġorder Ġto Ġevaluate Ġsentiment Ġpreservation , Ġwe Ġuse Ġthe Ġabsolute Ġvalue Ġof Ġthe Ġdifference Ġbetween Ġthe Ġstandardized Ġsentiment Ġscore Ġof Ġthe Ġinput Ġsentence Ġand Ġthat Ġof Ġthe Ġgenerated Ġsentence . ĠWe Ġcall Ġthe Ġvalue Ġas Ġsentiment Ġdelta Ġ( sent i Ġdelta ). ĠBesides , Ġwe Ġreport Ġthe Ġsentiment Ġaccuracy Ġ( Sent i ĠACC ) Ġwhich Ġmeasures Ġwhether Ġthe Ġoutput Ġsentence Ġhas Ġthe Ġsame Ġsentiment Ġpol arity Ġas Ġthe Ġinput Ġsentence Ġbased Ġon Ġour Ġstandardized Ġsentiment Ġclass ifiers . ĠThe ĠB LE U Ġscore ĠB IB REF 25 Ġbetween Ġthe Ġinput Ġsentences Ġand Ġthe Ġoutput Ġsentences Ġis Ġcalculated Ġto Ġevaluate Ġthe Ġcontent Ġpreservation Ġperformance . ĠIn Ġorder Ġto Ġevaluate Ġthe Ġoverall Ġperformance Ġof Ġdifferent Ġmodels , Ġwe Ġalso Ġreport Ġthe Ġgeometric Ġmean Ġ( G 2 ) Ġand Ġharmonic Ġmean Ġ( H 2 ) Ġof Ġthe Ġsentiment Ġaccuracy Ġand Ġthe ĠB LE U Ġscore . ĠAs Ġfor Ġthe Ġirony Ġaccuracy , Ġwe Ġonly Ġreport Ġit Ġin Ġhuman Ġevaluation Ġresults Ġbecause Ġit Ġis Ġmore Ġaccurate Ġfor Ġthe Ġhuman Ġto Ġevaluate Ġthe Ġquality Ġof Ġirony Ġas Ġit Ġis Ġvery Ġcomplicated . </s> We Ġfirst Ġsample Ġ50 Ġnon - iron ic Ġinput Ġsentences Ġand Ġtheir Ġcorresponding Ġoutput Ġsentences Ġof Ġdifferent Ġmodels . ĠThen , Ġwe Ġask Ġfour Ġannot ators Ġwho Ġare Ġproficient Ġin ĠEnglish Ġto Ġevaluate Ġthe Ġqualities Ġof Ġthe Ġgenerated Ġsentences Ġof Ġdifferent Ġmodels . ĠThey Ġare Ġrequired Ġto Ġrank Ġthe Ġoutput Ġsentences Ġof Ġour Ġmodel Ġand Ġbas elines Ġfrom Ġthe Ġbest Ġto Ġthe Ġworst Ġin Ġterms Ġof Ġirony Ġaccuracy Ġ( Iron y ), ĠSent iment Ġpreservation Ġ( Sent i ) Ġand Ġcontent Ġpreservation Ġ( Content ). ĠThe Ġbest Ġoutput Ġis Ġranked Ġwith Ġ1 Ġand Ġthe Ġworst Ġoutput Ġis Ġranked Ġwith Ġ6 . ĠThat Ġmeans Ġthat Ġthe Ġsmaller Ġour Ġhuman Ġevaluation Ġvalue Ġis , Ġthe Ġbetter Ġthe Ġcorresponding Ġmodel Ġis . </s> Results Ġand ĠDisc ussions </s> Table ĠT AB REF 35 Ġshows Ġthe Ġautomatic Ġevaluation Ġresults Ġof Ġthe Ġmodels Ġin Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences . ĠFrom Ġthe Ġresults , Ġour Ġmodel Ġob tains Ġthe Ġbest Ġresult Ġin Ġsentiment Ġdelta . ĠThe ĠDual RL Ġmodel Ġachieves Ġthe Ġhighest Ġresult Ġin Ġother Ġmetrics , Ġbut Ġmost Ġof Ġits Ġoutputs Ġare Ġthe Ġalmost Ġsame Ġas Ġthe Ġinput Ġsentences . ĠSo Ġit Ġis Ġreasonable Ġthat ĠDual RL Ġsystem Ġoutper forms Ġours Ġin Ġthese Ġmetrics Ġbut Ġit Ġactually Ġdoes Ġnot Ġtransfer Ġthe Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences Ġat Ġall . ĠFrom Ġthis Ġperspective , Ġwe Ġcannot Ġview ĠDual RL Ġas Ġan Ġeffective Ġmodel Ġfor Ġirony Ġgeneration . ĠIn Ġcontrast , Ġour Ġmodel Ġgets Ġresults Ġclose Ġto Ġthose Ġof ĠDual RL Ġand Ġob tains Ġa Ġbalance Ġbetween Ġirony Ġaccuracy , Ġsentiment Ġpreservation , Ġand Ġcontent Ġpreservation Ġif Ġwe Ġalso Ġconsider Ġthe Ġirony Ġaccuracy Ġdiscussed Ġbelow . </s> And Ġfrom Ġhuman Ġevaluation Ġresults Ġshown Ġin ĠTable ĠT AB REF 36 Ġ, Ġour Ġmodel Ġgets Ġthe Ġbest Ġaverage Ġrank Ġin Ġirony Ġaccuracy . ĠAnd Ġas Ġmentioned Ġabove , Ġthe ĠDual RL Ġmodel Ġusually Ġdoes Ġnot Ġchange Ġthe Ġinput Ġsentence Ġand Ġoutputs Ġthe Ġsame Ġsentence . ĠTherefore , Ġit Ġis Ġreasonable Ġthat Ġit Ġob tains Ġthe Ġbest Ġrank Ġin Ġsentiment Ġand Ġcontent Ġpreservation Ġand Ġours Ġis Ġthe Ġsecond . ĠHowever , Ġit Ġstill Ġdemonstrates Ġthat Ġour Ġmodel , Ġinstead Ġof Ġchanging Ġnothing , Ġtransfers Ġthe Ġstyle Ġof Ġthe Ġinput Ġsentence Ġwith Ġcontent Ġand Ġsentiment Ġpreservation Ġat Ġthe Ġsame Ġtime . </s> Case ĠStudy </s> In Ġthe Ġsection , Ġwe Ġpresent Ġsome Ġexample Ġoutputs Ġof Ġdifferent Ġmodels . ĠTable ĠT AB REF 37 Ġshows Ġthe Ġresults Ġof Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences . ĠWe Ġcan Ġobserve Ġthat : Ġ( 1 ) ĠThe ĠBack Trans Ġsystem , Ġthe ĠUn pa ired Ġsystem , Ġthe ĠCross Al ign Ġsystem Ġand Ġthe ĠC PT G Ġsystem Ġtends Ġto Ġgenerate Ġsentences Ġwhich Ġare Ġtowards Ġirony Ġbut Ġdo Ġnot Ġpreserve Ġcontent . Ġ( 2 ) ĠThe ĠDual RL Ġsystem Ġpreserves Ġcontent Ġand Ġsentiment Ġvery Ġwell Ġbut Ġeven Ġdoes Ġnot Ġchange Ġthe Ġinput Ġsentence . Ġ( 3 ) ĠOur Ġmodel Ġconsiders Ġboth Ġaspects Ġand Ġachieves Ġa Ġbetter Ġbalance Ġamong Ġirony Ġaccuracy , Ġsentiment Ġand Ġcontent Ġpreservation . </s> Error ĠAnalysis </s> Although Ġour Ġmodel Ġoutper forms Ġother Ġstyle Ġtransfer Ġbas elines Ġaccording Ġto Ġautomatic Ġand Ġhuman Ġevaluation Ġresults , Ġthere Ġare Ġstill Ġsome Ġfailure Ġcases Ġbecause Ġirony Ġgeneration Ġis Ġstill Ġa Ġvery Ġchallenging Ġtask . ĠWe Ġwould Ġlike Ġto Ġshare Ġthe Ġissues Ġwe Ġmeet Ġduring Ġour Ġexperiments Ġand Ġour Ġsolutions Ġto Ġsome Ġof Ġthem Ġin Ġthis Ġsection . </s> No ĠChange : ĠAs Ġmentioned Ġabove , Ġmany Ġstyle Ġtransfer Ġmodels , Ġsuch Ġas ĠDual RL , Ġtend Ġto Ġmake Ġfew Ġchanges Ġto Ġthe Ġinput Ġsentence Ġand Ġoutput Ġthe Ġsame Ġsentence . ĠActually , Ġthis Ġis Ġa Ġcommon Ġissue Ġfor Ġun super vised Ġstyle Ġtransfer Ġsystems Ġand Ġwe Ġalso Ġmeet Ġit Ġduring Ġour Ġexperiments . ĠThe Ġmain Ġreason Ġfor Ġthe Ġissue Ġis Ġthat Ġrewards Ġfor Ġcontent Ġpreservation Ġare Ġtoo Ġprominent Ġand Ġrewards Ġfor Ġstyle Ġaccuracy Ġcannot Ġwork Ġwell . ĠIn Ġcontrast , Ġin Ġorder Ġto Ġguarantee Ġthe Ġread ability Ġand Ġflu ency Ġof Ġthe Ġoutput Ġsentence , Ġwe Ġalso Ġcannot Ġemphasize Ġtoo Ġmuch Ġon Ġrewards Ġfor Ġstyle Ġaccuracy Ġbecause Ġit Ġmay Ġcause Ġsome Ġother Ġissues Ġsuch Ġas Ġword Ġrepetition Ġmentioned Ġbelow . ĠA Ġmethod Ġto Ġsolve Ġthe Ġproblem Ġis Ġtuning Ġhyper param eters Ġand Ġthis Ġis Ġalso Ġthe Ġmethod Ġwe Ġimplement Ġin Ġthis Ġwork . ĠAs Ġfor Ġcontent Ġpreservation , Ġmaybe ĠM LE Ġmethods Ġsuch Ġas Ġback - translation Ġare Ġnot Ġenough Ġbecause Ġthey Ġtend Ġto Ġforce Ġmodels Ġto Ġgenerate Ġspecific Ġwords . ĠIn Ġthe Ġfuture , Ġwe Ġshould Ġfurther Ġdesign Ġsome Ġmore Ġsuitable Ġmethods Ġto Ġcontrol Ġcontent Ġpreservation Ġfor Ġmodels Ġwithout Ġdis ent angling Ġstyle Ġand Ġcontent Ġrepresentations , Ġsuch Ġas ĠDual RL Ġand Ġours . </s> Word ĠRep etition : ĠDuring Ġour Ġexperiments , Ġwe Ġobserve Ġthat Ġsome Ġof Ġthe Ġoutputs Ġprefer Ġto Ġrepeat Ġthe Ġsame Ġword Ġas Ġshown Ġin ĠTable ĠT AB REF 38 Ġ. ĠThis Ġis Ġbecause Ġreinforcement Ġlearning Ġrewards Ġencourage Ġthe Ġmodel Ġto Ġgenerate Ġwords Ġwhich Ġcan Ġget Ġhigh Ġscores Ġfrom Ġclass ifiers Ġand Ġeven Ġback - translation Ġcannot Ġstop Ġit . ĠOur Ġsolution Ġis Ġthat Ġwe Ġcan Ġlower Ġthe Ġprobability Ġof Ġdecoding Ġa Ġword Ġin Ġdec od ers Ġif Ġthe Ġword Ġhas Ġbeen Ġgenerated Ġin Ġthe Ġprevious Ġtime Ġsteps Ġduring Ġtesting . ĠWe Ġalso Ġtry Ġto Ġimplement Ġthis Ġmethod Ġduring Ġtraining Ġtime Ġbut Ġobtain Ġworse Ġperformances Ġbecause Ġit Ġmay Ġlimit Ġthe Ġeffects Ġof Ġtraining . ĠSome Ġprevious Ġstudies Ġutilize Ġlanguage Ġmodels Ġto Ġcontrol Ġthe Ġflu ency Ġof Ġthe Ġoutput Ġsentence Ġand Ġwe Ġalso Ġtry Ġthis Ġmethod . ĠNonetheless , Ġpre - training Ġa Ġlanguage Ġmodel Ġwith Ġtweets Ġand Ġusing Ġit Ġto Ġgenerate Ġrewards Ġis Ġdifficult Ġbecause Ġtweets Ġare Ġmore Ġcasual Ġand Ġhave Ġmore Ġnoise . ĠRewards Ġfrom Ġthat Ġkind Ġof Ġlanguage Ġmodel Ġare Ġusually Ġnot Ġaccurate Ġand Ġmay Ġconfuse Ġthe Ġmodel . ĠIn Ġthe Ġfuture , Ġwe Ġshould Ġcome Ġup Ġwith Ġbetter Ġmethods Ġto Ġmodel Ġlanguage Ġflu ency Ġwith Ġthe Ġconsideration Ġof Ġirony Ġaccuracy , Ġsentiment Ġand Ġcontent Ġpreservation , Ġespecially Ġfor Ġtweets . </s> Impro per ĠWords : ĠAs Ġironic Ġstyle Ġis Ġhard Ġfor Ġour Ġmodel Ġto Ġlearn , Ġit Ġmay Ġgenerate Ġsome Ġimproper Ġwords Ġwhich Ġmake Ġthe Ġsentence Ġstrange . ĠAs Ġthe Ġexample Ġshown Ġin Ġthe ĠTable ĠT AB REF 38 Ġ, Ġthe Ġsentiment Ġword Ġin Ġthe Ġinput Ġsentence Ġis ĠâĢ ľ w onder ful " Ġand Ġthe Ġmodel Ġshould Ġchange Ġit Ġinto Ġa Ġnegative Ġword Ġsuch Ġas ĠâĢ ľ s ad " Ġto Ġmake Ġthe Ġoutput Ġsentence Ġironic . ĠHowever , Ġthe Ġmodel Ġchanges ĠâĢ ľ fr iday " Ġand ĠâĢ ľ f ifa " Ġwhich Ġare Ġnot Ġrelated Ġto Ġironic Ġstyles . ĠWe Ġhave Ġnot Ġfound Ġa Ġvery Ġeffective Ġmethod Ġto Ġaddress Ġthis Ġissue Ġand Ġmaybe Ġwe Ġshould Ġfurther Ġexplore Ġstronger Ġmodels Ġto Ġlearn Ġironic Ġstyles Ġbetter . </s> Additional ĠExper iments </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġsome Ġadditional Ġexperiments Ġon Ġthe Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences . ĠSometimes Ġiron ies Ġare Ġhard Ġto Ġunderstand Ġand Ġmay Ġcause Ġmisunderstanding , Ġfor Ġwhich Ġour Ġtask Ġalso Ġexplores Ġthe Ġtransformation Ġfrom Ġironic Ġsentences Ġto Ġnon - iron ic Ġsentences . </s> As Ġshown Ġin ĠTable ĠT AB REF 46 Ġ, Ġwe Ġalso Ġconduct Ġautomatic Ġevaluations Ġand Ġthe Ġconclusions Ġare Ġsimilar Ġto Ġthose Ġof Ġthe Ġtransformation Ġfrom Ġnon - iron ic Ġsentences Ġto Ġironic Ġsentences . ĠAs Ġfor Ġhuman Ġevaluation Ġresults Ġin ĠTable ĠT AB REF 47 Ġ, Ġour Ġmodel Ġstill Ġcan Ġachieve Ġthe Ġsecond - best Ġresults Ġin Ġsentiment Ġand Ġcontent Ġpreservation . ĠNevertheless , ĠDual RL Ġsystem Ġand Ġours Ġget Ġpoor Ġperformances Ġin Ġirony Ġaccuracy . ĠThe Ġreason Ġmay Ġbe Ġthat Ġthe Ġother Ġfour Ġbas elines Ġtend Ġto Ġgenerate Ġcommon Ġand Ġeven Ġnot Ġfluent Ġsentences Ġwhich Ġare Ġirrelevant Ġto Ġthe Ġinput Ġsentences Ġand Ġare Ġhard Ġto Ġbe Ġidentified Ġas Ġiron ies . ĠSo Ġannot ators Ġusually Ġmark Ġthese Ġoutput Ġsentences Ġas Ġnon - iron ic Ġsentences , Ġwhich Ġcauses Ġthese Ġmodels Ġto Ġobtain Ġbetter Ġperformances Ġthan ĠDual RL Ġand Ġours Ġbut Ġmuch Ġpoorer Ġresults Ġin Ġsentiment Ġand Ġcontent Ġpreservation . ĠSome Ġexamples Ġare Ġshown Ġin ĠTable ĠT AB REF 52 Ġ. </s> Conclusion Ġand ĠFuture ĠWork </s> In Ġthis Ġpaper , Ġwe Ġfirst Ġsystematically Ġdefine Ġirony Ġgeneration Ġbased Ġon Ġstyle Ġtransfer . ĠBecause Ġof Ġthe Ġlack Ġof Ġirony Ġdata , Ġwe Ġmake Ġuse Ġof Ġtwitter Ġand Ġbuild Ġa Ġlarge - scale Ġdataset . ĠIn Ġorder Ġto Ġcontrol Ġirony Ġaccuracy , Ġsentiment Ġpreservation Ġand Ġcontent Ġpreservation Ġat Ġthe Ġsame Ġtime , Ġwe Ġalso Ġdesign Ġa Ġcombination Ġof Ġrewards Ġfor Ġreinforcement Ġlearning Ġand Ġincorporate Ġreinforcement Ġlearning Ġwith Ġa Ġpre - training Ġprocess . ĠExperimental Ġresults Ġdemonstrate Ġthat Ġour Ġmodel Ġoutper forms Ġother Ġgener ative Ġmodels Ġand Ġour Ġrewards Ġare Ġeffective . ĠAlthough Ġour Ġmodel Ġdesign Ġis Ġeffective , Ġthere Ġare Ġstill Ġmany Ġerrors Ġand Ġwe Ġsystematically Ġanalyze Ġthem . ĠIn Ġthe Ġfuture , Ġwe Ġare Ġinterested Ġin Ġexploring Ġthese Ġdirections Ġand Ġour Ġwork Ġmay Ġextend Ġto Ġother Ġkinds Ġof Ġiron ies Ġwhich Ġare Ġmore Ġdifficult Ġto Ġmodel .