three datasets based on IMDB reviews and Yelp reviews
EXTRACTIVE
1 IMDB dataset and 2 Yelp datasets
ABSTRACTIVE
We construct three datasets based on IMDB reviews and Yelp reviews. The IMDB dataset is binarised and split into a training and test set, each with 25K reviews (2K reviews from the training set are reserved for development). We filter out any review that has more than 400 tokens, producing the final dataset (imdb400). For Yelp, we binarise the ratings, and create 2 datasets, where we keep only reviews with $\le $ 50 tokens (yelp50) and $\le $200 tokens (yelp200). We randomly partition both datasets into train/dev/test sets (90/5/5 for yelp50; 99/0.5/0.5 for yelp200). For all datasets, we use spaCy for tokenisation. We train and tune target classifiers (see Section SECREF8) using the training and development sets; and evaluate their performance on the original examples in the test sets as well as the adversarial examples generated by attacking methods for the test sets. Note that AutoEncoder also involves a training process, for which we train and tune AutoEncoder using the training and development sets in yelp50, yelp200 and imdb400. Statistics of the three datasets are presented in Table TABREF22. These datasets present a variation in the text lengths (e.g. the average number of words for yelp50, yelp200 and imdb400 is 34, 82 and 195 words respectively), training data size (e.g. the number of training examples for target classifiers for imdb400, yelp50 and yelp200 are 18K, 407K and 2M, respectively) and input domain (e.g. restaurant vs. movie reviews).
We construct three datasets based on IMDB reviews and Yelp reviews. The IMDB dataset is binarised and split into a training and test set, each with 25K reviews (2K reviews from the training set are reserved for development). We filter out any review that has more than 400 tokens, producing the final dataset (imdb400). For Yelp, we binarise the ratings, and create 2 datasets, where we keep only reviews with $\le $ 50 tokens (yelp50) and $\le $200 tokens (yelp200). We randomly partition both datasets into train/dev/test sets (90/5/5 for yelp50; 99/0.5/0.5 for yelp200). For all datasets, we use spaCy for tokenisation. We train and tune target classifiers (see Section SECREF8) using the training and development sets; and evaluate their performance on the original examples in the test sets as well as the adversarial examples generated by attacking methods for the test sets. Note that AutoEncoder also involves a training process, for which we train and tune AutoEncoder using the training and development sets in yelp50, yelp200 and imdb400. Statistics of the three datasets are presented in Table TABREF22. These datasets present a variation in the text lengths (e.g. the average number of words for yelp50, yelp200 and imdb400 is 34, 82 and 195 words respectively), training data size (e.g. the number of training examples for target classifiers for imdb400, yelp50 and yelp200 are 18K, 407K and 2M, respectively) and input domain (e.g. restaurant vs. movie reviews).
2001.07820
Introduction </s> Ad vers arial Ġexamples , Ġa Ġterm Ġintroduced Ġin ĠB IB REF 0 , Ġare Ġinputs Ġtransformed Ġby Ġsmall Ġpert urb ations Ġthat Ġmachine Ġlearning Ġmodels Ġconsistently Ġmis class ify . ĠThe Ġexperiments Ġare Ġconducted Ġin Ġthe Ġcontext Ġof Ġcomputer Ġvision Ġ( CV ), Ġand Ġthe Ġcore Ġidea Ġis Ġencaps ulated Ġby Ġan Ġillust rative Ġexample : Ġafter Ġimper cept ible Ġnoises Ġare Ġadded Ġto Ġa Ġp anda Ġimage , Ġan Ġimage Ġclass ifier Ġpredicts , Ġwith Ġhigh Ġconfidence , Ġthat Ġit Ġis Ġa Ġgib bon . ĠInterestingly , Ġthese Ġadvers arial Ġexamples Ġcan Ġalso Ġbe Ġused Ġto Ġimprove Ġthe Ġclass ifier ĠâĢĶ Ġeither Ġas Ġadditional Ġtraining Ġdata ĠB IB REF 0 Ġor Ġas Ġa Ġregular isation Ġobjective ĠB IB REF 1 ĠâĢĶ Ġthus Ġproviding Ġmotivation Ġfor Ġgenerating Ġeffective Ġadvers arial Ġexamples . </s> The Ġgerm Ġof Ġthis Ġpaper Ġcomes Ġfrom Ġour Ġinvestigation Ġof Ġadvers arial Ġattack Ġmethods Ġfor Ġnatural Ġlanguage Ġprocessing Ġ( N LP ) Ġtasks , Ġe . g . Ġsentiment Ġclassification , Ġwhich Ġdrives Ġus Ġto Ġquantify Ġwhat Ġis Ġan ĠâĢ ľ effective âĢ Ŀ Ġor ĠâĢ ľ good âĢ Ŀ Ġadvers arial Ġexample . ĠIn Ġthe Ġcontext Ġof Ġimages , Ġa Ġgood Ġadvers arial Ġexample Ġis Ġtypically Ġdefined Ġaccording Ġtwo Ġcriteria : </s> it Ġhas Ġsuccessfully Ġfooled Ġthe Ġclass ifier ; </s> it Ġis Ġvisually Ġsimilar Ġto Ġthe Ġoriginal Ġexample . </s> In ĠN LP , Ġdefining Ġa Ġgood Ġadvers arial Ġexample Ġis Ġa Ġlittle Ġmore Ġinvolving , Ġbecause Ġwhile Ġcriterion Ġ( b ) Ġcan Ġbe Ġmeasured Ġwith Ġa Ġcomparable Ġtext Ġsimilarity Ġmetric Ġ( e . g . ĠB LE U Ġor Ġedit Ġdistance ), Ġan Ġadvers arial Ġexample Ġshould Ġalso : </s> be Ġfluent Ġor Ġnatural ; </s> pres erve Ġits Ġoriginal Ġlabel . </s> These Ġtwo Ġadditional Ġcriteria Ġare Ġgenerally Ġirrelevant Ġfor Ġimages , Ġas Ġadding Ġminor Ġpert urb ations Ġto Ġan Ġimage Ġis Ġunlikely Ġto : Ġ( 1 ) Ġcreate Ġan Ġun interpret able Ġimage Ġ( where Ġelse Ġchanging Ġone Ġword Ġin Ġa Ġsentence Ġcan Ġrender Ġa Ġsentence Ġinco herent ), Ġor Ġ( 2 ) Ġchange Ġhow Ġwe Ġperceive Ġthe Ġimage , Ġsay Ġfrom Ġseeing Ġa Ġp anda Ġto Ġa Ġgib bon Ġ( but Ġa Ġsentence 's Ġsentiment Ġcan Ġbe Ġreversed Ġby Ġsimply Ġadding Ġa Ġnegative Ġad verb Ġsuch Ġas Ġnot ). ĠWithout Ġconsidering Ġcriterion Ġ( d ), Ġgenerating Ġadvers arial Ġexamples Ġin ĠN LP Ġwould Ġbe Ġtrivial , Ġas Ġthe Ġmodel Ġcan Ġlearn Ġto Ġsimply Ġreplace Ġa Ġpositive Ġadjective Ġ( am azing ) Ġwith Ġa Ġnegative Ġone Ġ( aw ful ) Ġto Ġattack Ġa Ġsentiment Ġclass ifier . </s> To Ġthe Ġbest Ġof Ġour Ġknowledge , Ġmost Ġstudies Ġon Ġadvers arial Ġexample Ġgeneration Ġin ĠN LP Ġhave Ġlargely Ġignored Ġthese Ġadditional Ġcriteria ĠB IB REF 2 , ĠB IB REF 3 , ĠB IB REF 4 , ĠB IB REF 5 . ĠWe Ġbelieve Ġthe Ġlack Ġof Ġa Ġrigorous Ġevaluation Ġframework Ġpartially Ġexplains Ġwhy Ġadvers arial Ġtraining Ġfor ĠN LP Ġmodels Ġhas Ġnot Ġseen Ġthe Ġsame Ġextent Ġof Ġimprovement Ġcompared Ġto ĠCV Ġmodels . ĠAs Ġour Ġexperiments Ġreveal , Ġexamples Ġgenerated Ġfrom Ġmost Ġattacking Ġmethods Ġare Ġsuccessful Ġin Ġfool ing Ġthe Ġclass ifier , Ġbut Ġtheir Ġlanguage Ġis Ġoften Ġunnatural Ġand Ġthe Ġoriginal Ġlabel Ġis Ġnot Ġproperly Ġpreserved . </s> The Ġcore Ġcontribution Ġof Ġour Ġpaper Ġis Ġto Ġintroduce Ġa Ġsystematic , Ġrigorous Ġevaluation Ġframework Ġto Ġassess Ġthe Ġquality Ġof Ġadvers arial Ġexamples Ġfor ĠN LP . ĠWe Ġfocus Ġon Ġsentiment Ġclassification Ġas Ġthe Ġtarget Ġtask , Ġas Ġit Ġis Ġa Ġpopular Ġapplication Ġthat Ġhighlights Ġthe Ġimportance Ġof Ġcriteria Ġdiscussed Ġabove . ĠWe Ġtest Ġa Ġnumber Ġof Ġattacking Ġmethods Ġand Ġalso Ġpropose Ġan Ġalternative Ġapproach Ġ( based Ġon Ġan Ġauto - enc oder ) Ġfor Ġgenerating Ġadvers arial Ġexamples . ĠWe Ġlearn Ġthat Ġa Ġnumber Ġof Ġfactors Ġcan Ġinfluence Ġthe Ġperformance Ġof Ġadvers arial Ġattacks , Ġincluding Ġarchitecture Ġof Ġthe Ġclass ifier , Ġsentence Ġlength Ġand Ġinput Ġdomain . </s> Related ĠWork </s> Most Ġexisting Ġadvers arial Ġattack Ġmethods Ġfor Ġtext Ġinputs Ġare Ġderived Ġfrom Ġthose Ġfor Ġimage Ġinputs . ĠThese Ġmethods Ġcan Ġbe Ġcategor ised Ġinto Ġthree Ġtypes Ġincluding Ġgradient - based Ġattacks , Ġoptim isation - based Ġattacks Ġand Ġmodel - based Ġattacks . </s> Grad ient - based Ġattacks Ġare Ġmainly Ġwhite - box Ġattacks Ġthat Ġrely Ġon Ġcalculating Ġthe Ġgrad ients Ġof Ġthe Ġtarget Ġclass ifier Ġwith Ġrespect Ġto Ġthe Ġinput Ġrepresentation . ĠThis Ġclass Ġof Ġattacking Ġmethods ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 6 Ġare Ġmainly Ġderived Ġfrom Ġthe Ġfast Ġgradient Ġsign Ġmethod Ġ( FG SM ) ĠB IB REF 1 , Ġand Ġit Ġhas Ġbeen Ġshown Ġto Ġbe Ġeffective Ġin Ġattacking ĠCV Ġclass ifiers . ĠHowever , Ġthese Ġgradient - based Ġmethods Ġcould Ġnot Ġbe Ġapplied Ġto Ġtext Ġdirectly Ġbecause Ġpert urbed Ġword Ġembed d ings Ġdo Ġnot Ġnecessarily Ġmap Ġto Ġvalid Ġwords . ĠOther Ġmethods Ġsuch Ġas ĠDeep F ool ĠB IB REF 8 Ġthat Ġrely Ġon Ġpert ur bing Ġthe Ġword Ġembed ding Ġspace Ġface Ġsimilar Ġroad blocks . ĠB IB REF 5 Ġpropose Ġto Ġuse Ġnearest Ġneighbour Ġsearch Ġto Ġfind Ġthe Ġclosest Ġword Ġto Ġthe Ġpert urbed Ġembed ding . </s> Both Ġoptim isation - based Ġand Ġmodel - based Ġattacks Ġtreat Ġadvers arial Ġattack Ġas Ġan Ġoptim isation Ġproblem Ġwhere Ġthe Ġconstraints Ġare Ġto Ġmaxim ise Ġthe Ġloss Ġof Ġtarget Ġclass ifiers Ġand Ġto Ġminim ise Ġthe Ġdifference Ġbetween Ġoriginal Ġand Ġadvers arial Ġexamples . ĠBetween Ġthese Ġtwo , Ġthe Ġformer Ġuses Ġoptim isation Ġalgorithms Ġdirectly ; Ġwhile Ġthe Ġlatter Ġtrains Ġa Ġse perate Ġmodel Ġto Ġgenerate Ġadvers arial Ġexamples Ġand Ġtherefore Ġinvolves Ġa Ġtraining Ġprocess . ĠSome Ġof Ġthe Ġmost Ġeffective Ġattacks Ġfor Ġimages Ġare Ġachieved Ġby Ġoptim isation - based Ġmethods , Ġsuch Ġas Ġthe ĠL - BF GS Ġattack ĠB IB REF 1 Ġand Ġthe ĠC & W Ġattack ĠB IB REF 9 Ġin Ġwhite - box Ġattacks Ġand Ġthe ĠZ OO Ġmethod ĠB IB REF 10 Ġin Ġblack - box Ġattacks . ĠFor Ġtexts , Ġthe Ġwhite - box Ġattack ĠHot Fl ip ĠB IB REF 3 Ġand Ġblack - box Ġattack ĠDeep Word Bug ĠB IB REF 11 Ġand ĠText Bug ger ĠB IB REF 12 Ġare Ġproposed Ġin Ġthis Ġcategory . </s> In Ġa Ġsimilar Ġvein , Ġa Ġfew Ġmodel - based Ġattacks Ġhave Ġbeen Ġproposed Ġfor Ġimages , Ġe . g . ĠB IB REF 13 Ġdesign Ġa Ġgener ative Ġadvers arial Ġnetwork Ġ( GAN ) Ġto Ġgenerate Ġthe Ġimage Ġpert urb ation Ġfrom Ġa Ġnoise Ġmap . ĠThe Ġattacking Ġmethod Ġand Ġtarget Ġclass ifier Ġtypically Ġform Ġa Ġsingle Ġlarge Ġnetwork Ġand Ġthe Ġattacking Ġmethod Ġis Ġtrained Ġusing Ġthe Ġloss Ġfrom Ġthe Ġtarget Ġclass ifier . ĠFor Ġthis Ġreason , Ġit Ġis Ġnot Ġvery Ġstraightforward Ġto Ġuse Ġthese Ġmodel - based Ġtechniques Ġfor Ġtext Ġbecause Ġthere Ġis Ġa Ġdiscontin uity Ġin Ġthe Ġnetwork Ġ( since Ġwords Ġin Ġthe Ġadvers arial Ġexamples Ġare Ġdiscrete ) Ġand Ġso Ġit Ġis Ġnot Ġfully Ġdifferent iable . </s> Method ology Ġ: :: ĠSent iment ĠClass ifiers </s> There Ġare Ġa Ġnumber Ġof Ġoff - the - she lf Ġneural Ġmodels Ġfor Ġsentiment Ġclassification ĠB IB REF 14 , ĠB IB REF 15 , Ġmost Ġof Ġwhich Ġare Ġbased Ġon Ġlong - short Ġterm Ġmemory Ġnetworks Ġ( L ST M ) ĠB IB REF 16 Ġor Ġconv olution al Ġneural Ġnetworks Ġ( CNN ) ĠB IB REF 14 . ĠIn Ġthis Ġpaper , Ġwe Ġpre - train Ġthree Ġsentiment Ġclass ifiers : ĠBi L ST M , ĠBi L ST M $ + $ A , Ġand ĠCNN . ĠThese Ġclass ifiers Ġare Ġtargeted Ġby Ġwhite - box Ġattacking Ġmethods Ġto Ġgenerate Ġadvers arial Ġexamples Ġ( det ailed Ġin ĠSection ĠSEC REF 9 ). ĠBi L ST M Ġis Ġcomposed Ġof Ġan Ġembed ding Ġlayer Ġthat Ġmaps Ġindividual Ġwords Ġto Ġpre - trained Ġword Ġembed d ings ; Ġa Ġnumber Ġof Ġbi - direction al ĠL ST Ms Ġthat Ġcapture Ġsequential Ġcontexts ; Ġand Ġan Ġoutput Ġlayer Ġthat Ġmaps Ġthe Ġaveraged ĠL ST M Ġhidden Ġstates Ġto Ġa Ġbinary Ġoutput . ĠBi L ST M $ + $ A Ġis Ġsimilar Ġto ĠBi L ST M Ġexcept Ġit Ġhas Ġan Ġextra Ġself - att ention Ġlayer Ġwhich Ġlearns Ġto Ġattend Ġto Ġsalient Ġwords Ġfor Ġsentiment Ġclassification , Ġand Ġwe Ġcompute Ġa Ġweighted Ġmean Ġof Ġthe ĠL ST M Ġhidden Ġstates Ġprior Ġto Ġthe Ġoutput Ġlayer . ĠManual Ġinspection Ġof Ġthe Ġattention Ġweights Ġshow Ġthat Ġpol arity Ġwords Ġsuch Ġas Ġawesome Ġand Ġdisappointed Ġare Ġassigned Ġwith Ġhigher Ġweights . ĠFinally , ĠCNN Ġhas Ġa Ġnumber Ġof Ġconv olution al Ġfilters Ġof Ġvarying Ġsizes , Ġand Ġtheir Ġoutputs Ġare Ġconc aten ated , Ġpooled Ġand Ġfed Ġto Ġa Ġfully - connected Ġlayer Ġfollowed Ġby Ġa Ġbinary Ġoutput Ġlayer . </s> Recent Ġdevelopment Ġin Ġtransformer - based Ġpre - trained Ġmodels Ġhave Ġproduced Ġstate - of - the - art Ġperformance Ġon Ġa Ġrange Ġof ĠN LP Ġtasks ĠB IB REF 17 , ĠB IB REF 18 . ĠTo Ġvalidate Ġthe Ġtransfer ability Ġof Ġthe Ġattacking Ġmethods , Ġwe Ġalso Ġfine - t une Ġa ĠB ERT Ġclass ifier Ġfor Ġblack - box Ġtests . ĠThat Ġis , Ġwe Ġuse Ġthe Ġadvers arial Ġexamples Ġgenerated Ġfor Ġattacking Ġthe Ġthree Ġprevious Ġclass ifiers Ġ( Bi L ST M , ĠBi L ST M $ + $ A Ġand ĠCNN ) Ġas Ġtest Ġdata Ġfor ĠB ERT Ġto Ġmeasure Ġits Ġclassification Ġperformance Ġto Ġunderstand Ġwhether Ġthese Ġadvers arial Ġexamples Ġcan Ġfool ĠB ERT . </s> Method ology Ġ: :: ĠBench mark ĠAtt acking ĠMethods </s> We Ġexperiment Ġwith Ġfive Ġbenchmark Ġattacking Ġmethods Ġfor Ġtexts : ĠF GM , ĠFG VM , ĠDeep F ool ĠB IB REF 5 , ĠHot Fl ip ĠB IB REF 3 ) Ġand ĠTY C ĠB IB REF 4 . </s> To Ġpert urb Ġthe Ġdiscrete Ġinputs , Ġboth ĠF GM Ġand ĠFG VM Ġintroduce Ġnoises Ġin Ġthe Ġword Ġembed ding Ġspace Ġvia Ġthe Ġfast Ġgradient Ġmethod ĠB IB REF 1 Ġand Ġreconstruct Ġthe Ġinput Ġby Ġmapping Ġpert urbed Ġword Ġembed d ings Ġto Ġvalid Ġwords Ġvia Ġnearest Ġneighbour Ġsearch . ĠBetween ĠF GM Ġand ĠFG VM , Ġthe Ġformer Ġintroduce Ġnoises Ġthat Ġis Ġproportional Ġto Ġthe Ġsign Ġof Ġthe Ġgrad ients Ġwhile Ġthe Ġlatter Ġintroduce Ġpert urb ations Ġproportional Ġto Ġthe Ġgrad ients Ġdirectly . ĠThe Ġproportion Ġis Ġknown Ġas Ġthe Ġover shoot Ġvalue Ġand Ġden oted Ġby Ġ$\ ep sil on Ġ$ . ĠDeep F ool Ġuses Ġthe Ġsame Ġtrick Ġto Ġdeal Ġwith Ġdiscrete Ġinputs Ġexcept Ġthat , Ġinstead Ġof Ġusing Ġthe Ġfast Ġgradient Ġmethod , Ġit Ġuses Ġthe ĠDeep F ool Ġmethod Ġintroduced Ġin ĠB IB REF 8 Ġfor Ġimage Ġto Ġsearch Ġfor Ġan Ġoptimal Ġdirection Ġto Ġpert urb Ġthe Ġword Ġembed d ings . </s> Unlike Ġthe Ġprevious Ġmethods , ĠHot Fl ip Ġand ĠTY C Ġrely Ġon Ġperforming Ġone Ġor Ġmore Ġatomic Ġflip Ġoperations Ġto Ġreplace Ġwords Ġwhile Ġmonitoring Ġthe Ġlabel Ġchange Ġgiven Ġby Ġthe Ġtarget Ġclass ifier . ĠIn ĠHot Fl ip , Ġthe Ġdirectional Ġderivatives Ġw . r . t . Ġflip Ġoperations Ġare Ġcalculated Ġand Ġthe Ġflip Ġoperation Ġthat Ġresults Ġin Ġthe Ġlargest Ġincrease Ġin Ġloss Ġis Ġselected . ĠTY C Ġis Ġsimilar Ġto ĠF GM , ĠFG VM Ġand ĠDeep F ool Ġin Ġthat Ġit Ġalso Ġuses Ġnearest Ġneighbour Ġsearch Ġto Ġmap Ġthe Ġpert urbed Ġembed d ings Ġto Ġvalid Ġwords , Ġbut Ġinstead Ġof Ġusing Ġthe Ġpert urbed Ġtokens Ġdirectly , Ġit Ġuses Ġgreedy Ġsearch Ġor Ġbeam Ġsearch Ġto Ġflip Ġoriginal Ġtokens Ġto Ġpert urbed Ġones Ġone Ġat Ġa Ġtime Ġin Ġorder Ġof Ġtheir Ġvulnerability . </s> Method ology Ġ: :: ĠModel - based ĠAtt acking ĠMethod </s> The Ġbenchmark Ġmethods Ġwe Ġtest Ġ( Section ĠSEC REF 9 ) Ġare Ġgradient - based Ġand Ġoptim isation - based Ġattacks Ġ( Section ĠSEC REF 2 ). ĠWe Ġpropose Ġan Ġalternative Ġmodel - based Ġmethod Ġto Ġtrain Ġa Ġse perate Ġgener ative Ġmodel Ġto Ġgenerate Ġtext Ġadvers arial Ġexamples . ĠDen oting Ġthe Ġgener ative Ġmodel Ġas Ġ$\ math cal Ġ{ G } $, Ġwe Ġtrain Ġit Ġto Ġgenerate Ġan Ġadvers arial Ġexample Ġ$ X ^ {\ prime Ġ} $ Ġgiven Ġan Ġinput Ġexample Ġ$ X $ Ġsuch Ġthat Ġ$ X ^ {\ prime Ġ} $ Ġis Ġsimilar Ġto Ġ$ X $ Ġbut Ġthat Ġit Ġchanges Ġthe Ġprediction Ġof Ġthe Ġtarget Ġclass ifier Ġ$\ math cal Ġ{ D } $, Ġi . e . Ġ$ X ^ {\ prime Ġ} Ġ\ sim ĠX $ Ġand Ġ$\ math cal Ġ{ D } ( X ^ {\ prime Ġ}) Ġ\ ne Ġ\ math cal Ġ{ D } ( X ) $. </s> For Ġimages , Ġit Ġis Ġstraightforward Ġto Ġcombine Ġ$\ math cal Ġ{ G } $ Ġand Ġ$\ math cal Ġ{ D } $ Ġin Ġthe Ġsame Ġnetwork Ġand Ġuse Ġthe Ġloss Ġof Ġ$\ math cal Ġ{ D } $ Ġto Ġupdate Ġ$\ math cal Ġ{ G } $ Ġbecause Ġ$ X ^ {\ prime Ġ} $ Ġis Ġcontinuous Ġand Ġcan Ġbe Ġfed Ġto Ġ$\ math cal Ġ{ D } $ Ġwhile Ġkeeping Ġthe Ġwhole Ġnetwork Ġdifferent iable . ĠFor Ġtexts , Ġ$ X ^ {\ prime Ġ} $ Ġare Ġdiscrete Ġsymbols Ġ( typically Ġdec oded Ġwith Ġarg max Ġor Ġbeam - search ) Ġand Ġso Ġthe Ġnetwork Ġis Ġnot Ġfully Ġdifferent iable . ĠTo Ġcreate Ġa Ġfully Ġdifferent iable Ġnetwork Ġwith Ġ$\ math cal Ġ{ G } + \ math cal Ġ{ D } $, Ġwe Ġpropose Ġto Ġuse ĠG umb el - Soft max Ġto Ġimitate Ġthe Ġcateg orical Ġdistribution ĠB IB REF 19 . ĠAs Ġthe Ġtemperature Ġ($ \ t au Ġ$ ) Ġof ĠG umb el - Soft max Ġsampling Ġapproaches Ġ0 , Ġthe Ġdistribution Ġof Ġthe Ġsampled Ġoutput Ġ$ X ^ * $ Ġis Ġidentical Ġto Ġ$ X ^ {\ prime Ġ} $. ĠWe Ġillustrate Ġthis Ġin ĠFigure ĠFIG REF 12 , Ġwhere Ġ$ X ^ * $ Ġis Ġfed Ġto Ġ$\ math cal Ġ{ D } $ Ġduring Ġtraining Ġwhile Ġ$ X ^ {\ prime Ġ} $ Ġis Ġgenerated Ġas Ġthe Ġoutput Ġ( i . e . Ġthe Ġadvers arial Ġexample ) Ġat Ġtest Ġtime . </s> $ \ math cal Ġ{ G } $ Ġis Ġdesigned Ġas Ġan Ġauto - enc oder Ġto Ġreconstruct Ġthe Ġinput Ġ( den oted Ġas ĠAuto Enc oder ), Ġwith Ġthe Ġfollowing Ġobjectives Ġto Ġcapture Ġthe Ġevaluation Ġcriteria Ġdescribed Ġin ĠSection ĠSEC REF 1 : </s> $ L _{ adv } $, Ġthe Ġadvers arial Ġloss Ġthat Ġmaxim ises Ġthe Ġcross - ent ropy Ġof Ġ$\ math cal Ġ{ D } $ ; </s> $ L _{ seq } $, Ġthe Ġauto - enc oder Ġloss Ġto Ġreconstruct Ġ$ X ^ {\ prime Ġ} $ Ġfrom Ġ$ X ^ {\ prime Ġ} $ ; </s> $ L _{ sem } $, Ġthe Ġcos ine Ġdistance Ġbetween Ġthe Ġmean Ġembed d ings Ġof Ġ$ X $ Ġand Ġ$ X ^ * $. </s> $ L _{ adv } $ Ġensures Ġthat Ġthe Ġadvers arial Ġexamples Ġare Ġfool ing Ġthe Ġclass ifier Ġ( crit erion Ġ( a )); Ġ$ L _{ seq } $ Ġregulates Ġthe Ġadvers arial Ġexample Ġso Ġthat Ġit Ġisn 't Ġtoo Ġdifferent Ġfrom Ġthe Ġoriginal Ġinput Ġand Ġhas Ġa Ġreasonable Ġlanguage Ġ( crit eria Ġ( b ) Ġand Ġ( c )); Ġand Ġ$ L _{ sem } $ Ġconst r ains Ġthe Ġunderlying Ġsemantic Ġcontent Ġof Ġthe Ġoriginal Ġand Ġadvers arial Ġsentences Ġto Ġbe Ġsimilar Ġ( crit erion Ġ( b )); Ġso Ġreduces Ġthe Ġlikelihood Ġof Ġa Ġsentiment Ġflip Ġ( ind irect ly Ġfor Ġcriteria Ġ( d )). ĠNote Ġthat Ġcriteria Ġ( d ) Ġis Ġthe Ġperhaps Ġthe Ġmost Ġdifficult Ġaspect Ġto Ġhandle , Ġas Ġits Ġinterpretation Ġis Ġultimately Ġa Ġhuman Ġjudgement . </s> We Ġuse Ġtwo Ġscaling Ġhyper - param eters Ġ$\ lambda Ġ_ { ae } $ Ġand Ġ$\ lambda Ġ_ { seq } $ Ġto Ġweigh Ġthe Ġthree Ġobjectives : Ġ$\ lambda Ġ_ { ae } (\ lambda Ġ_ { seq } Ġ* ĠL _{ seq } Ġ+ Ġ( 1 - \ lambda Ġ_ { seq } )* L _{ sem }) Ġ+ Ġ( 1 - \ lambda Ġ_ { ae } )* L _{ adv } $. ĠWe Ġintroduce Ġattention Ġlayers Ġto ĠAuto Enc oder Ġand Ġtest Ġboth Ġgreedy Ġand Ġbeam Ġsearch Ġdecoding Ġto Ġimprove Ġthe Ġquality Ġof Ġgenerated Ġadvers arial Ġexamples . ĠDuring Ġtraining Ġwe Ġalternate Ġbetween Ġpositive - to - negative Ġand Ġnegative - to - positive Ġattack Ġfor Ġdifferent Ġmini - bat ches . </s> Exper iments Ġ: :: ĠDat as ets </s> We Ġconstruct Ġthree Ġdatasets Ġbased Ġon ĠIM DB Ġreviews Ġand ĠYelp Ġreviews . ĠThe ĠIM DB Ġdataset Ġis Ġbin ar ised Ġand Ġsplit Ġinto Ġa Ġtraining Ġand Ġtest Ġset , Ġeach Ġwith Ġ25 K Ġreviews Ġ( 2 K Ġreviews Ġfrom Ġthe Ġtraining Ġset Ġare Ġreserved Ġfor Ġdevelopment ). ĠWe Ġfilter Ġout Ġany Ġreview Ġthat Ġhas Ġmore Ġthan Ġ400 Ġtokens , Ġproducing Ġthe Ġfinal Ġdataset Ġ( im db 400 ). ĠFor ĠYelp , Ġwe Ġbin ar ise Ġthe Ġratings , Ġand Ġcreate Ġ2 Ġdatasets , Ġwhere Ġwe Ġkeep Ġonly Ġreviews Ġwith Ġ$\ le Ġ$ Ġ50 Ġtokens Ġ( y el p 50 ) Ġand Ġ$\ le Ġ$ 200 Ġtokens Ġ( y el p 200 ). ĠWe Ġrandomly Ġpartition Ġboth Ġdatasets Ġinto Ġtrain / dev / test Ġsets Ġ( 90 / 5 / 5 Ġfor Ġy el p 50 ; Ġ99 / 0 . 5 / 0 . 5 Ġfor Ġy el p 200 ). ĠFor Ġall Ġdatasets , Ġwe Ġuse Ġspa Cy Ġfor Ġtoken isation . ĠWe Ġtrain Ġand Ġtune Ġtarget Ġclass ifiers Ġ( see ĠSection ĠSEC REF 8 ) Ġusing Ġthe Ġtraining Ġand Ġdevelopment Ġsets ; Ġand Ġevaluate Ġtheir Ġperformance Ġon Ġthe Ġoriginal Ġexamples Ġin Ġthe Ġtest Ġsets Ġas Ġwell Ġas Ġthe Ġadvers arial Ġexamples Ġgenerated Ġby Ġattacking Ġmethods Ġfor Ġthe Ġtest Ġsets . ĠNote Ġthat ĠAuto Enc oder Ġalso Ġinvolves Ġa Ġtraining Ġprocess , Ġfor Ġwhich Ġwe Ġtrain Ġand Ġtune ĠAuto Enc oder Ġusing Ġthe Ġtraining Ġand Ġdevelopment Ġsets Ġin Ġy el p 50 , Ġy el p 200 Ġand Ġim db 400 . ĠStatistics Ġof Ġthe Ġthree Ġdatasets Ġare Ġpresented Ġin ĠTable ĠT AB REF 22 . ĠThese Ġdatasets Ġpresent Ġa Ġvariation Ġin Ġthe Ġtext Ġlengths Ġ( e . g . Ġthe Ġaverage Ġnumber Ġof Ġwords Ġfor Ġy el p 50 , Ġy el p 200 Ġand Ġim db 400 Ġis Ġ34 , Ġ82 Ġand Ġ195 Ġwords Ġrespectively ), Ġtraining Ġdata Ġsize Ġ( e . g . Ġthe Ġnumber Ġof Ġtraining Ġexamples Ġfor Ġtarget Ġclass ifiers Ġfor Ġim db 400 , Ġy el p 50 Ġand Ġy el p 200 Ġare Ġ18 K , Ġ407 K Ġand Ġ2 M , Ġrespectively ) Ġand Ġinput Ġdomain Ġ( e . g . Ġrestaurant Ġvs . Ġmovie Ġreviews ). </s> Exper iments Ġ: :: ĠImplementation ĠDetails </s> We Ġuse Ġthe Ġpre - trained Ġglove . 840 B . 300 d Ġembed d ings ĠB IB REF 20 Ġfor Ġall Ġ6 Ġattacking Ġmethods . ĠFor ĠF GM , ĠFG VM Ġand ĠDeep F ool , Ġwe Ġtune Ġ$\ ep sil on Ġ$ , Ġthe Ġover shoot Ġhyper - param eter Ġ( Section ĠSEC REF 9 ) Ġand Ġkeep Ġthe Ġiter ative Ġstep Ġ$ n $ Ġstatic Ġ( 5 ). ĠFor ĠTY C , Ġbesides Ġ$\ ep sil on Ġ$ Ġwe Ġalso Ġtune Ġthe Ġupper Ġlimit Ġof Ġflipped Ġwords , Ġranging Ġfrom Ġ10 % âĢĵ 100 % Ġof Ġthe Ġmaximum Ġlength . ĠFor ĠHot Fl ip , Ġwe Ġtune Ġonly Ġthe Ġupper Ġlimit Ġof Ġflipped Ġwords , Ġin Ġthe Ġrange Ġof Ġ$ [ 1 , Ġ7 ] $. </s> We Ġpre - train ĠAuto Enc oder Ġto Ġreconstruct Ġsentences Ġin Ġdifferent Ġdatasets Ġas Ġwe Ġfound Ġthat Ġthis Ġimproves Ġthe Ġquality Ġof Ġthe Ġgenerated Ġadvers arial Ġexamples . ĠDuring Ġpre - training , Ġwe Ġtune Ġbatch Ġsize , Ġnumber Ġof Ġlayers Ġand Ġnumber Ġof Ġunits , Ġand Ġstop Ġthe Ġtraining Ġafter Ġthe Ġperformance Ġon Ġthe Ġdevelopment Ġsets Ġstops Ġimproving Ġfor Ġ20 K Ġsteps . ĠThe Ġmodel Ġis Ġthen Ġinitial ised Ġwith Ġthe Ġpre - trained Ġweights Ġand Ġtrained Ġbased Ġon Ġobjectives Ġdefined Ġin ĠSection ĠSEC REF 11 . ĠDuring Ġthe Ġtraining Ġprocess Ġwe Ġtune Ġ$\ lambda Ġ_ { ae } $ Ġand Ġ$\ lambda Ġ_ { seq } $ Ġwhile Ġkeeping Ġthe Ġbatch Ġsize Ġ( 32 ) Ġand Ġlearning Ġrate Ġ($ 1 e ^{ - 4 } $ ) Ġfixed . ĠAs Ġpart Ġof Ġour Ġpreliminary Ġstudy , Ġwe Ġalso Ġtested Ġdifferent Ġvalues Ġfor Ġthe ĠG umb el - soft max Ġtemperature Ġ$\ t au Ġ$ Ġand Ġfind Ġthat Ġ$\ t au Ġ= 0 . 1 $ Ġperforms Ġthe Ġbest . ĠEmb edd ings Ġare Ġfixed Ġthroughout Ġall Ġtraining Ġprocesses . </s> For Ġtarget Ġclass ifiers , Ġwe Ġtune Ġbatch Ġsize , Ġlearning Ġrate , Ġnumber Ġof Ġlayers , Ġnumber Ġof Ġunits , Ġattention Ġsize Ġ( Bi L ST M $ + $ A ), Ġfilter Ġsizes Ġand Ġdrop out Ġprobability Ġ( CNN ). ĠFor ĠB ERT , Ġwe Ġuse Ġthe Ġdefault Ġfine - tun ing Ġhyper - param eter Ġvalues Ġexcept Ġfor Ġbatch Ġsize , Ġwhere Ġwe Ġadjust Ġbased Ġon Ġmemory Ġconsumption . ĠNote Ġthat Ġafter Ġthe Ġtarget Ġclass ifiers Ġare Ġtrained Ġtheir Ġweights Ġare Ġnot Ġupdated Ġwhen Ġtraining Ġor Ġtesting Ġthe Ġattacking Ġmethods . </s> E val uation </s> We Ġpropose Ġboth Ġautomatic Ġmetrics Ġand Ġmanual Ġevaluation Ġstrategies Ġto Ġassess Ġthe Ġquality Ġof Ġadvers arial Ġexamples , Ġbased Ġon Ġfour Ġcriteria Ġdefined Ġin ĠSection ĠSEC REF 1 : Ġ( a ) Ġattacking Ġperformance Ġ( i . e . Ġhow Ġwell Ġthey Ġfool Ġthe Ġclass ifier ); Ġ( b ) Ġtextual Ġsimilarity Ġbetween Ġthe Ġoriginal Ġinput Ġand Ġthe Ġadvers arial Ġinput ; Ġ( c ) Ġflu ency Ġof Ġthe Ġadvers arial Ġexample ; Ġand Ġ( d ) Ġlabel Ġpreservation . ĠNote Ġthat Ġthe Ġautomatic Ġmetrics Ġonly Ġaddress Ġthe Ġfirst Ġ3 Ġcriteria Ġ( a , Ġb Ġand Ġc ); Ġwe Ġcontend Ġthat Ġcriterion Ġ( d ) Ġrequires Ġmanual Ġevaluation , Ġas Ġthe Ġjudgement Ġof Ġwhether Ġthe Ġoriginal Ġlabel Ġis Ġpreserved Ġis Ġinherently Ġa Ġhuman Ġdecision . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation : ĠMet rics </s> As Ġsentiment Ġclassification Ġis Ġour Ġtarget Ġtask , Ġwe Ġuse Ġthe Ġstandard Ġclassification Ġaccuracy Ġ( ACC ) Ġto Ġevaluate Ġthe Ġattacking Ġperformance Ġof Ġadvers arial Ġexamples Ġ( crit erion Ġ( a )). </s> To Ġassess Ġthe Ġsimilarity Ġbetween Ġthe Ġoriginal Ġand Ġ( trans formed ) Ġadvers arial Ġexamples Ġ( crit eria Ġ( b )), Ġwe Ġcompute ĠB LE U Ġscores ĠB IB REF 21 . </s> To Ġmeasure Ġflu ency , Ġwe Ġfirst Ġexplore Ġa Ġsupervised ĠB ERT Ġmodel Ġfine - tun ed Ġto Ġpredict Ġlinguistic Ġaccept ability ĠB IB REF 17 . ĠHowever , Ġin Ġpreliminary Ġexperiments Ġwe Ġfound Ġthat ĠB ERT Ġperforms Ġvery Ġpoorly Ġat Ġpredicting Ġthe Ġaccept ability Ġof Ġadvers arial Ġexamples Ġ( e . g . Ġit Ġpredicts Ġword - sal ad - like Ġsentences Ġgenerated Ġby ĠFG VM Ġas Ġvery Ġacceptable ), Ġrevealing Ġthe Ġbr itt leness Ġof Ġthese Ġsupervised Ġmodels . ĠWe Ġnext Ġexplore Ġan Ġun super vised Ġapproach ĠB IB REF 22 , ĠB IB REF 23 , Ġusing Ġnormal ised Ġsentence Ġprobabilities Ġestimated Ġby Ġpre - trained Ġlanguage Ġmodels Ġfor Ġmeasuring Ġaccept ability . ĠIn Ġthe Ġoriginal Ġpapers , Ġthe Ġauthors Ġtested Ġsimple Ġrecurrent Ġlanguage Ġmodels ; Ġhere Ġwe Ġuse Ġmodern Ġpre - trained Ġlanguage Ġmodels Ġsuch Ġas ĠG PT - 2 ĠB IB REF 24 Ġand ĠXL Net ĠB IB REF 18 . ĠOur Ġfinal Ġaccept ability Ġmetric Ġ( AC PT ) Ġis Ġbased Ġon Ġnormal ised ĠXL Net Ġsentence Ġprobabilities : Ġ${ \ log ĠP ( s )} Ġ/ Ġ({ (( 5 + | s | )/ ( 5 + 1 )) ^ \ alpha Ġ}) $, Ġwhere Ġ$ s $ Ġis Ġthe Ġsentence , Ġand Ġ$\ alpha Ġ$ Ġis Ġa Ġhyper - param eter Ġ( set Ġto Ġ0 . 8 ) Ġto Ġdamp en Ġthe Ġimpact Ġof Ġlarge Ġvalues ĠB IB REF 25 . </s> We Ġonly Ġcomputed ĠB LE U Ġand ĠAC PT Ġscores Ġfor Ġadvers arial Ġexamples Ġthat Ġhave Ġsuccessfully Ġfooled Ġthe Ġclass ifier . ĠOur Ġrationale Ġis Ġthat Ġunsuccessful Ġexamples Ġcan Ġartificially Ġboost Ġthese Ġscores Ġby Ġnot Ġmaking Ġany Ġmodifications , Ġand Ġso Ġthe Ġbetter Ġapproach Ġis Ġto Ġonly Ġconsider Ġsuccessful Ġexamples . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation : ĠResults </s> We Ġpresent Ġthe Ġperformance Ġof Ġthe Ġattacking Ġmethods Ġagainst Ġ3 Ġtarget Ġclass ifiers Ġ( Table ĠT AB REF 23 A ; Ġtop ) Ġand Ġon Ġ3 Ġdatasets Ġ( Table ĠT AB REF 23 B ; Ġbottom ). ĠWe Ġchoose Ġ3 ĠACC Ġthresholds Ġfor Ġthe Ġattacking Ġperformance : ĠT 0 , ĠT 1 Ġand ĠT 2 , Ġwhich Ġcorrespond Ġapproximately Ġto Ġaccuracy Ġscores Ġof Ġ90 %, Ġ80 % Ġand Ġ70 % Ġfor Ġthe ĠYelp Ġdatasets Ġ( y el p 50 , Ġy el p 200 ); Ġand Ġ80 %, Ġ70 % Ġand Ġ50 % Ġfor Ġthe ĠIM DB Ġdatasets Ġ( im db 400 ). ĠEach Ġmethod Ġis Ġtuned Ġaccordingly Ġto Ġachieve Ġa Ġparticular Ġaccuracy . ĠMissing Ġnumbers Ġ( d ashed Ġlines ) Ġindicate Ġthe Ġmethod Ġis Ġunable Ġto Ġproduce Ġthe Ġdesired Ġaccuracy , Ġe . g . ĠHot Fl ip Ġwith Ġonly Ġ1 Ġword Ġflip Ġproduces Ġ81 . 5 % Ġaccuracy Ġ( T 1 ) Ġwhen Ġattacking ĠCNN Ġon Ġy el p 50 , Ġand Ġso ĠT 0 Ġaccuracy Ġis Ġun ach iev able . </s> Looking Ġat ĠB LE U Ġand ĠAC PT , ĠHot Fl ip Ġis Ġthe Ġmost Ġconsistent Ġmethod Ġover Ġmultiple Ġdatasets Ġand Ġclass ifiers . ĠAuto Enc oder Ġis Ġalso Ġfairly Ġcompetitive , Ġproducing Ġlargely Ġcomparable Ġperformance Ġ( except Ġfor Ġthe Ġy el p 200 Ġdataset ). ĠGrad ient - based Ġmethods ĠF GM Ġand ĠFG VM Ġperform Ġvery Ġpoorly . ĠIn Ġgeneral , Ġthey Ġtend Ġto Ġproduce Ġword Ġsalad Ġadvers arial Ġexamples , Ġas Ġindicated Ġby Ġtheir Ġpoor ĠB LE U Ġscores . ĠDeep F ool Ġsimilarly Ġgenerates Ġinco herent Ġsentences Ġwith Ġlow ĠB LE U Ġscores , Ġbut Ġoccasionally Ġproduces Ġgood ĠAC PT Ġ( Bi L ST M Ġat ĠT 1 Ġand ĠT 2 ), Ġsuggesting Ġpotential Ġbr itt leness Ġof Ġthe Ġun super vised Ġapproach Ġfor Ġevaluating Ġaccept ability . </s> Comp aring Ġthe Ġperformance Ġacross Ġdifferent ĠACC Ġthresholds , Ġwe Ġobserve Ġa Ġconsistent Ġpattern Ġof Ġdecreasing Ġperformance Ġover Ġall Ġmetrics Ġas Ġthe Ġattacking Ġperformance Ġincreases Ġfrom ĠT 0 Ġto ĠT 2 . ĠThese Ġobservations Ġsuggest Ġthat Ġall Ġmethods Ġare Ġtrading Ġoff Ġflu ency Ġand Ġcontent Ġpreservation Ġas Ġthey Ġattempt Ġto Ġgenerate Ġstronger Ġadvers arial Ġexamples . </s> We Ġnow Ġfocus Ġon ĠTable ĠT AB REF 23 A , Ġto Ġunderstand Ġthe Ġimpact Ġof Ġmodel Ġarchitecture Ġfor Ġthe Ġtarget Ġclass ifier . ĠWith Ġ1 Ġword Ġflip Ġas Ġthe Ġupper Ġlimit Ġfor ĠHot Fl ip , Ġthe Ġaccuracy Ġof ĠBi L ST M $ + $ A Ġand ĠBi L ST M Ġdrops Ġto ĠT 0 Ġ( approximately Ġ4 % Ġaccuracy Ġdecrease ) Ġwhile Ġthe Ġaccuracy Ġof ĠCNN Ġdrops Ġto ĠT 1 Ġ( approximately Ġ13 % Ġaccuracy Ġdecrease ), Ġsuggesting Ġthat Ġconv olution al Ġnetworks Ġare Ġmore Ġvulnerable Ġto Ġattacks Ġ( not ing Ġthat Ġit Ġis Ġthe Ġpredominant Ġarchitecture Ġfor ĠCV ). ĠLooking Ġat ĠTable ĠT AB REF 23 B , Ġwe Ġalso Ġfind Ġthat Ġthe Ġattacking Ġperformance Ġis Ġinfluenced Ġby Ġthe Ġinput Ġtext Ġlength Ġand Ġthe Ġnumber Ġof Ġtraining Ġexamples Ġfor Ġtarget Ġclass ifiers . ĠFor ĠHot Fl ip , Ġwe Ġsee Ġimprovements Ġover ĠB LE U Ġand ĠAC PT Ġas Ġtext Ġlength Ġincreases Ġfrom Ġy el p 50 Ġto Ġy el p 200 Ġto Ġim db 400 , Ġindicating Ġthe Ġperformance Ġof ĠHot Fl ip Ġis Ġmore Ġaffected Ġby Ġinput Ġlengths . ĠWe Ġthink Ġthis Ġis Ġbecause Ġwith Ġmore Ġwords Ġit Ġis Ġmore Ġlikely Ġfor ĠHot Fl ip Ġto Ġfind Ġa Ġvulnerable Ġspot Ġto Ġtarget . ĠWhile Ġfor ĠTY C Ġand ĠAuto Enc oder , Ġwe Ġsee Ġimprovements Ġover ĠB LE U Ġand ĠAC PT Ġas Ġthe Ġnumber Ġof Ġtraining Ġexamples Ġfor Ġtarget Ġclass ifier Ġdecreases Ġfrom Ġy el p 200 Ġ( 2 M ) Ġto Ġy el p 50 Ġ( 407 K ) Ġto Ġim db 400 Ġ( 22 K ), Ġindicating Ġthese Ġtwo Ġmethods Ġare Ġless Ġeffective Ġfor Ġattacking Ġtarget Ġclass ifiers Ġthat Ġare Ġtrained Ġwith Ġmore Ġdata . ĠTherefore , Ġincreasing Ġthe Ġtraining Ġdata Ġfor Ġtarget Ġclass ifier Ġmay Ġimprove Ġtheir Ġrobust ness Ġagainst Ġadvers arial Ġexamples Ġthat Ġare Ġgenerated Ġby Ġcertain Ġmethods . </s> As Ġa Ġblack - box Ġtest Ġto Ġcheck Ġhow Ġwell Ġthese Ġadvers arial Ġexamples Ġgeneral ise Ġto Ġfool ing Ġother Ġclass ifiers , Ġalso Ġknown Ġas Ġtransfer ability , Ġwe Ġfeed Ġthe Ġadvers arial Ġexamples Ġfrom Ġthe Ġ3 Ġbest Ġmethods , Ġi . e . ĠTY C , ĠHot Fl ip Ġand ĠAuto Enc oder , Ġto Ġa Ġpre - trained ĠB ERT Ġtrained Ġfor Ġsentiment Ġclassification Ġ( Figure ĠFIG REF 31 ) Ġand Ġmeasure Ġits Ġaccuracy . ĠUn surprisingly , Ġwe Ġobserve Ġthat Ġthe Ġattacking Ġperformance Ġ( i . e . Ġthe Ġdrop Ġin ĠACC ) Ġis Ġnot Ġas Ġgood Ġas Ġthose Ġreported Ġfor Ġthe Ġwhite - box Ġtests . ĠInterestingly , Ġwe Ġfind Ġthat ĠHot Fl ip , Ġthe Ġbest Ġmethod , Ġproduces Ġthe Ġleast Ġeffective Ġadvers arial Ġexamples Ġfor ĠB ERT . ĠBoth ĠTY C Ġand ĠAuto Enc oder Ġperform Ġbetter Ġhere , Ġas Ġtheir Ġgenerated Ġadvers arial Ġexamples Ġdo Ġwell Ġin Ġfool ing ĠB ERT . </s> To Ġsummar ise , Ġour Ġresults Ġdemonstrate Ġthat Ġthe Ġbest Ġwhite - box Ġmethods Ġ( e . g . ĠHot Fl ip ) Ġmay Ġnot Ġproduce Ġadvers arial Ġexamples Ġthat Ġgeneral ise Ġto Ġfool ing Ġother Ġclass ifiers . ĠWe Ġalso Ġsaw Ġthat Ġconv olution al Ġnetworks Ġare Ġmore Ġvulnerable Ġthan Ġrecurrent Ġnetworks , Ġand Ġthat Ġdataset Ġfeatures Ġsuch Ġas Ġtext Ġlengths Ġand Ġtraining Ġdata Ġsize Ġ( for Ġtarget Ġclass ifiers ) Ġcan Ġinfluence Ġhow Ġdifficult Ġit Ġis Ġto Ġperform Ġadvers arial Ġattack . </s> E val uation Ġ: :: ĠHuman ĠEvaluation : ĠDesign </s> Aut omatic Ġmetrics Ġprovide Ġa Ġproxy Ġto Ġquantify Ġthe Ġquality Ġof Ġthe Ġadvers arial Ġexamples . ĠTo Ġvalidate Ġthat Ġthese Ġmetrics Ġwork , Ġwe Ġconduct Ġa Ġcrowds ourcing Ġexperiment Ġon ĠFigure - Eight . ĠRecall Ġthat Ġthe Ġautomatic Ġmetrics Ġdo Ġnot Ġassess Ġsentiment Ġpreservation Ġ( crit erion Ġ( d )); Ġwe Ġevaluate Ġthat Ġaspect Ġhere . </s> We Ġexperiment Ġwith Ġthe Ġ3 Ġbest Ġmethods Ġ( TY C , ĠAuto Enc oder Ġand ĠHot Fl ip ) Ġon Ġ2 Ġaccuracy Ġthresholds Ġ( T 0 Ġand ĠT 2 ), Ġusing ĠBi L ST M $ + $ A Ġas Ġthe Ġclass ifier . ĠFor Ġeach Ġmethod Ġand Ġthreshold , Ġwe Ġrandomly Ġsample Ġ25 Ġpositive - to - negative Ġand Ġ25 Ġnegative - to - positive Ġexamples . ĠTo Ġcontrol Ġfor Ġquality , Ġwe Ġreserve Ġand Ġannot ate Ġ10 % Ġof Ġthe Ġsamples Ġourselves Ġas Ġcontrol Ġquestions . ĠWorkers Ġare Ġfirst Ġpresented Ġwith Ġ10 Ġcontrol Ġquestions Ġas Ġa Ġquiz , Ġand Ġonly Ġthose Ġwho Ġpass Ġthe Ġquiz Ġwith Ġat Ġleast Ġ80 % Ġaccuracy Ġcan Ġcontinue Ġto Ġwork Ġon Ġthe Ġtask . ĠWe Ġdisplay Ġ10 Ġquestions Ġper Ġpage , Ġwhere Ġone Ġcontrol Ġquestion Ġis Ġembedded Ġto Ġcontinue Ġmonitor Ġworker Ġperformance . ĠThe Ġtask Ġis Ġdesigned Ġsuch Ġthat Ġeach Ġcontrol Ġquestion Ġcan Ġonly Ġbe Ġseen Ġonce Ġper Ġworker . ĠWe Ġrestrict Ġour Ġjobs Ġto Ġworkers Ġin ĠUnited ĠStates , ĠUnited ĠKingdoms , ĠAustralia , Ġand ĠCanada . </s> To Ġevaluate Ġthe Ġcriteria Ġdiscussed Ġin ĠSection ĠSEC REF 1 : Ġ( b ) Ġtextual Ġsimilarity , Ġ( c ) Ġflu ency , Ġand Ġ( d ) Ġsentiment Ġpreservation , Ġwe Ġask Ġthe Ġannot ators Ġthree Ġquestions : </s> Is Ġsnippet ĠB Ġa Ġgood Ġparaph rase Ġof Ġsnippet ĠA ? </s> $ \ c ir cled circ Ġ$ ĠYes Ġ$\ c ir cled circ Ġ$ ĠSomew hat Ġyes Ġ$\ c ir cled circ Ġ$ ĠNo </s> How Ġnatural Ġdoes Ġthe Ġtext Ġread ? </s> $ \ c ir cled circ Ġ$ ĠVery Ġunnatural Ġ$\ c ir cled circ Ġ$ ĠSomew hat Ġnatural Ġ$\ c ir cled circ Ġ$ ĠNatural </s> What Ġis Ġthe Ġsentiment Ġof Ġthe Ġtext ? </s> $ \ c ir cled circ Ġ$ ĠPositive Ġ$\ c ir cled circ Ġ$ ĠNegative Ġ$\ c ir cled circ Ġ$ ĠCannot Ġtell </s> For Ġquestion Ġ1 , Ġwe Ġdisplay Ġboth Ġthe Ġadvers arial Ġinput Ġand Ġthe Ġoriginal Ġinput , Ġwhile Ġfor Ġquestion Ġ2 Ġand Ġ3 Ġwe Ġpresent Ġonly Ġthe Ġadvers arial Ġexample . ĠAs Ġan Ġupper - bound , Ġwe Ġalso Ġrun Ġa Ġsurvey Ġon Ġquestion Ġ2 Ġand Ġ3 Ġfor Ġ50 Ġrandom Ġoriginal Ġsamples . </s> E val uation Ġ: :: ĠHuman ĠEvaluation : ĠResults </s> We Ġpresent Ġthe Ġpercentage Ġof Ġanswers Ġto Ġeach Ġquestion Ġin ĠFigure ĠFIG REF 38 . ĠThe Ġgreen Ġbars Ġillustrate Ġhow Ġwell Ġthe Ġadvers arial Ġexamples Ġparaph rase Ġthe Ġoriginal Ġones ; Ġblue Ġhow Ġnatural Ġthe Ġadvers arial Ġexamples Ġread ; Ġand Ġred Ġwhether Ġthe Ġsentiment Ġof Ġthe Ġadvers arial Ġexamples Ġis Ġconsistent Ġcompared Ġto Ġthe Ġoriginal . </s> Looking Ġat Ġthe Ġperformance Ġof Ġthe Ġoriginal Ġsentences Ġ( âĢ ľ ( a ) ĠOriginal Ġsamples âĢ Ŀ ), Ġwe Ġsee Ġthat Ġtheir Ġlanguage Ġis Ġlargely Ġfluent Ġand Ġtheir Ġsentiment Ġis Ġgenerally Ġconsistent Ġto Ġthe Ġoriginal Ġexamples ', Ġalthough Ġit 's Ġworth Ġnoting Ġthat Ġthe Ġreview Ġsentiment Ġof Ġim db 400 Ġcan Ġbe Ġsomewhat Ġambiguous Ġ( 63 % Ġagreement ). ĠWe Ġthink Ġthis Ġis Ġdue Ġto Ġmovie Ġreviews Ġbeing Ġmore Ġdescriptive Ġand Ġtherefore Ġcreating Ġpotential Ġambiguity Ġin Ġtheir Ġexpression Ġof Ġsentiment . </s> On Ġcontent Ġpreservation Ġ( crit erion Ġ( b ); Ġgreen Ġbars ), Ġall Ġmethods Ġproduce Ġpoor Ġparaph r ases Ġon Ġy el p 50 . ĠFor Ġim db 400 , Ġhowever , Ġthe Ġresults Ġare Ġmore Ġpromising ; Ġadvers arial Ġexamples Ġgenerated Ġby ĠHot Fl ip , Ġin Ġparticular , Ġare Ġgood , Ġeven Ġat ĠT 2 . </s> Next Ġwe Ġlook Ġat Ġflu ency Ġ( crit erion Ġ( c ); Ġblue Ġbars ). ĠWe Ġsee Ġa Ġsimilar Ġtrend : Ġperformance Ġin Ġim db 400 Ġis Ġsubstantially Ġbetter Ġthan Ġy el p 50 . ĠIn Ġfact Ġwe Ġsee Ġalmost Ġno Ġdecrease Ġin Ġflu ency Ġin Ġthe Ġadvers arial Ġexamples Ġcompared Ġto Ġthe Ġoriginal Ġin Ġim db 400 . ĠIn Ġy el p 50 , ĠHot Fl ip Ġand ĠAuto Enc oder Ġare Ġfairly Ġcompetitive , Ġproducing Ġadvers arial Ġexamples Ġthat Ġare Ġonly Ġmarginally Ġless Ġfluent Ġcompared Ġto Ġthe Ġoriginal Ġat ĠT 0 . ĠAt ĠT 2 , Ġhowever , Ġthese Ġmethods Ġbegin Ġto Ġtrade Ġoff Ġflu ency . ĠAll Ġin Ġall , Ġthe Ġparaph ras ability Ġand Ġflu ency Ġsurveys Ġsuggest Ġthat Ġim db 400 Ġis Ġan Ġeasier Ġdataset Ġfor Ġadvers arial Ġexperiments , Ġand Ġit Ġis Ġthe Ġpredominant Ġdataset Ġused Ġby Ġmost Ġstudies . </s> Lastly , Ġwe Ġconsider Ġsentiment Ġpreservation Ġ( crit erion Ġ( d ); Ġred Ġbars ). ĠAll Ġmethods Ġperform Ġpoorly Ġat Ġpreserving Ġthe Ġoriginal Ġsentiment Ġon Ġboth Ġy el p 50 Ġand Ġim db 400 Ġdatasets . ĠThe Ġartifact Ġis Ġarguably Ġmore Ġprofound Ġon Ġshorter Ġinputs , Ġas Ġthe Ġoriginal Ġexamples Ġin Ġim db 400 Ġhave Ġlower Ġagreement Ġin Ġthe Ġfirst Ġplace Ġ( y el p 50 Ġvs . Ġim db 400 : Ġ86 % Ġvs . Ġ63 %). ĠAgain Ġboth ĠHot Fl ip Ġand ĠAuto Enc oder Ġare Ġthe Ġbetter Ġmethods Ġhere Ġ( interest ingly , Ġwe Ġobserve Ġan Ġincrease Ġin Ġagreement Ġas Ġtheir Ġattacking Ġperformance Ġincreases Ġfrom ĠT 0 Ġand ĠT 2 ). </s> Sum mar ising Ġour Ġfindings , ĠHot Fl ip Ġis Ġgenerally Ġthe Ġbest Ġmethod Ġacross Ġall Ġcriteria , Ġnoting Ġthat Ġits Ġadvers arial Ġexamples , Ġhowever , Ġhave Ġpoor Ġtransfer ability . ĠTY C Ġgenerates Ġgood Ġblack - box Ġadvers arial Ġexamples Ġbut Ġdo Ġnot Ġdo Ġwell Ġin Ġterms Ġof Ġcontent Ġpreservation Ġand Ġflu ency . ĠAuto Enc oder Ġproduces Ġcomparable Ġresults Ġwith ĠHot Fl ip Ġfor Ġmeeting Ġthe Ġfour Ġcriteria Ġand Ġgenerates Ġexamples Ġthat Ġgeneral ise Ġreasonably , Ġbut Ġit Ġis Ġvery Ġsensitive Ġto Ġthe Ġincrease Ġof Ġtraining Ġexamples Ġfor Ġthe Ġtarget Ġclass ifier . ĠThe ĠAC PT Ġmetric Ġappears Ġto Ġbe Ġeffective Ġin Ġevaluating Ġflu ency , Ġas Ġwe Ġsee Ġgood Ġagreement Ġwith Ġhuman Ġevaluation . ĠAll Ġsaid , Ġwe Ġfound Ġthat Ġall Ġmethods Ġtend Ġto Ġproduce Ġadvers arial Ġexamples Ġthat Ġdo Ġnot Ġpreserve Ġtheir Ġoriginal Ġsentiments , Ġrevealing Ġthat Ġthese Ġmethods Ġin Ġa Ġway ĠâĢ ľ cheat âĢ Ŀ Ġby Ġsimply Ġflipping Ġthe Ġsentiments Ġof Ġthe Ġoriginal Ġsentences Ġto Ġfool Ġthe Ġclass ifier , Ġand Ġtherefore Ġthe Ġadvers arial Ġexamples Ġmight Ġbe Ġineffective Ġfor Ġadvers arial Ġtraining , Ġas Ġthey Ġare Ġnot Ġexamples Ġthat Ġreveal Ġpotential Ġvulnerabilities Ġin Ġthe Ġclass ifier . </s> Conclusion </s> We Ġpropose Ġan Ġevaluation Ġframework Ġfor Ġassessing Ġthe Ġquality Ġof Ġadvers arial Ġexamples Ġin ĠN LP , Ġbased Ġon Ġfour Ġcriteria : Ġ( a ) Ġattacking Ġperformance , Ġ( b ) Ġtextual Ġsimilarity ; Ġ( c ) Ġflu ency ; Ġ( d ) Ġlabel Ġpreservation . ĠOur Ġframework Ġinvolves Ġboth Ġautomatic Ġand Ġhuman Ġevaluation , Ġand Ġwe Ġtest Ġ5 Ġbenchmark Ġmethods Ġand Ġa Ġnovel Ġauto - enc oder Ġapproach . ĠWe Ġfound Ġthat Ġthe Ġarchitecture Ġof Ġthe Ġtarget Ġclass ifier Ġis Ġan Ġimportant Ġfactor Ġwhen Ġit Ġcomes Ġto Ġattacking Ġperformance , Ġe . g . ĠCNN s Ġare Ġmore Ġvulnerable Ġthan ĠL ST Ms ; Ġdataset Ġfeatures Ġsuch Ġas Ġlength Ġof Ġtext , Ġtraining Ġdata Ġsize Ġ( for Ġtarget Ġclass ifiers ) Ġand Ġinput Ġdomains Ġare Ġalso Ġinfluencing Ġfactors Ġthat Ġaffect Ġhow Ġdifficulty Ġit Ġis Ġto Ġperform Ġadvers arial Ġattack ; Ġand Ġthe Ġpredominant Ġdataset Ġ( IM DB ) Ġused Ġby Ġmost Ġstudies Ġis Ġcomparatively Ġeasy Ġfor Ġadvers arial Ġattack . ĠLastly , Ġwe Ġalso Ġobserve Ġin Ġour Ġhuman Ġevaluation Ġthat Ġon Ġshorter Ġtexts Ġ( Y el p ) Ġthese Ġmethods Ġproduce Ġadvers arial Ġexamples Ġthat Ġtend Ġnot Ġto Ġpreserve Ġtheir Ġsemantic Ġcontent Ġand Ġhave Ġlow Ġread ability . ĠMore Ġimportantly , Ġthese Ġmethods Ġalso ĠâĢ ľ cheat âĢ Ŀ Ġby Ġsimply Ġflipping Ġthe Ġsentiment Ġin Ġthe Ġadvers arial Ġexamples , Ġand Ġthis Ġbehaviour Ġis Ġevident Ġon Ġboth Ġdatasets , Ġsuggesting Ġthey Ġcould Ġbe Ġineffective Ġfor Ġadvers arial Ġtraining .
Introduction Adversarial examples, a term introduced in BIBREF0, are inputs transformed by small perturbations that machine learning models consistently misclassify. The experiments are conducted in the context of computer vision (CV), and the core idea is encapsulated by an illustrative example: after imperceptible noises are added to a panda image, an image classifier predicts, with high confidence, that it is a gibbon. Interestingly, these adversarial examples can also be used to improve the classifier — either as additional training data BIBREF0 or as a regularisation objective BIBREF1 — thus providing motivation for generating effective adversarial examples. The germ of this paper comes from our investigation of adversarial attack methods for natural language processing (NLP) tasks, e.g. sentiment classification, which drives us to quantify what is an “effective” or “good” adversarial example. In the context of images, a good adversarial example is typically defined according two criteria: it has successfully fooled the classifier; it is visually similar to the original example. In NLP, defining a good adversarial example is a little more involving, because while criterion (b) can be measured with a comparable text similarity metric (e.g. BLEU or edit distance), an adversarial example should also: be fluent or natural; preserve its original label. These two additional criteria are generally irrelevant for images, as adding minor perturbations to an image is unlikely to: (1) create an uninterpretable image (where else changing one word in a sentence can render a sentence incoherent), or (2) change how we perceive the image, say from seeing a panda to a gibbon (but a sentence's sentiment can be reversed by simply adding a negative adverb such as not). Without considering criterion (d), generating adversarial examples in NLP would be trivial, as the model can learn to simply replace a positive adjective (amazing) with a negative one (awful) to attack a sentiment classifier. To the best of our knowledge, most studies on adversarial example generation in NLP have largely ignored these additional criteria BIBREF2, BIBREF3, BIBREF4, BIBREF5. We believe the lack of a rigorous evaluation framework partially explains why adversarial training for NLP models has not seen the same extent of improvement compared to CV models. As our experiments reveal, examples generated from most attacking methods are successful in fooling the classifier, but their language is often unnatural and the original label is not properly preserved. The core contribution of our paper is to introduce a systematic, rigorous evaluation framework to assess the quality of adversarial examples for NLP. We focus on sentiment classification as the target task, as it is a popular application that highlights the importance of criteria discussed above. We test a number of attacking methods and also propose an alternative approach (based on an auto-encoder) for generating adversarial examples. We learn that a number of factors can influence the performance of adversarial attacks, including architecture of the classifier, sentence length and input domain. Related Work Most existing adversarial attack methods for text inputs are derived from those for image inputs. These methods can be categorised into three types including gradient-based attacks, optimisation-based attacks and model-based attacks. Gradient-based attacks are mainly white-box attacks that rely on calculating the gradients of the target classifier with respect to the input representation. This class of attacking methods BIBREF6, BIBREF7, BIBREF6 are mainly derived from the fast gradient sign method (FGSM) BIBREF1, and it has been shown to be effective in attacking CV classifiers. However, these gradient-based methods could not be applied to text directly because perturbed word embeddings do not necessarily map to valid words. Other methods such as DeepFool BIBREF8 that rely on perturbing the word embedding space face similar roadblocks. BIBREF5 propose to use nearest neighbour search to find the closest word to the perturbed embedding. Both optimisation-based and model-based attacks treat adversarial attack as an optimisation problem where the constraints are to maximise the loss of target classifiers and to minimise the difference between original and adversarial examples. Between these two, the former uses optimisation algorithms directly; while the latter trains a seperate model to generate adversarial examples and therefore involves a training process. Some of the most effective attacks for images are achieved by optimisation-based methods, such as the L-BFGS attack BIBREF1 and the C&W attack BIBREF9 in white-box attacks and the ZOO method BIBREF10 in black-box attacks. For texts, the white-box attack HotFlip BIBREF3 and black-box attack DeepWordBug BIBREF11 and TextBugger BIBREF12 are proposed in this category. In a similar vein, a few model-based attacks have been proposed for images, e.g. BIBREF13 design a generative adversarial network (GAN) to generate the image perturbation from a noise map. The attacking method and target classifier typically form a single large network and the attacking method is trained using the loss from the target classifier. For this reason, it is not very straightforward to use these model-based techniques for text because there is a discontinuity in the network (since words in the adversarial examples are discrete) and so it is not fully differentiable. Methodology ::: Sentiment Classifiers There are a number of off-the-shelf neural models for sentiment classification BIBREF14, BIBREF15, most of which are based on long-short term memory networks (LSTM) BIBREF16 or convolutional neural networks (CNN) BIBREF14. In this paper, we pre-train three sentiment classifiers: BiLSTM, BiLSTM$+$A, and CNN. These classifiers are targeted by white-box attacking methods to generate adversarial examples (detailed in Section SECREF9). BiLSTM is composed of an embedding layer that maps individual words to pre-trained word embeddings; a number of bi-directional LSTMs that capture sequential contexts; and an output layer that maps the averaged LSTM hidden states to a binary output. BiLSTM$+$A is similar to BiLSTM except it has an extra self-attention layer which learns to attend to salient words for sentiment classification, and we compute a weighted mean of the LSTM hidden states prior to the output layer. Manual inspection of the attention weights show that polarity words such as awesome and disappointed are assigned with higher weights. Finally, CNN has a number of convolutional filters of varying sizes, and their outputs are concatenated, pooled and fed to a fully-connected layer followed by a binary output layer. Recent development in transformer-based pre-trained models have produced state-of-the-art performance on a range of NLP tasks BIBREF17, BIBREF18. To validate the transferability of the attacking methods, we also fine-tune a BERT classifier for black-box tests. That is, we use the adversarial examples generated for attacking the three previous classifiers (BiLSTM, BiLSTM$+$A and CNN) as test data for BERT to measure its classification performance to understand whether these adversarial examples can fool BERT. Methodology ::: Benchmark Attacking Methods We experiment with five benchmark attacking methods for texts: FGM, FGVM, DeepFool BIBREF5, HotFlip BIBREF3) and TYC BIBREF4. To perturb the discrete inputs, both FGM and FGVM introduce noises in the word embedding space via the fast gradient method BIBREF1 and reconstruct the input by mapping perturbed word embeddings to valid words via nearest neighbour search. Between FGM and FGVM, the former introduce noises that is proportional to the sign of the gradients while the latter introduce perturbations proportional to the gradients directly. The proportion is known as the overshoot value and denoted by $\epsilon $. DeepFool uses the same trick to deal with discrete inputs except that, instead of using the fast gradient method, it uses the DeepFool method introduced in BIBREF8 for image to search for an optimal direction to perturb the word embeddings. Unlike the previous methods, HotFlip and TYC rely on performing one or more atomic flip operations to replace words while monitoring the label change given by the target classifier. In HotFlip, the directional derivatives w.r.t. flip operations are calculated and the flip operation that results in the largest increase in loss is selected. TYC is similar to FGM, FGVM and DeepFool in that it also uses nearest neighbour search to map the perturbed embeddings to valid words, but instead of using the perturbed tokens directly, it uses greedy search or beam search to flip original tokens to perturbed ones one at a time in order of their vulnerability. Methodology ::: Model-based Attacking Method The benchmark methods we test (Section SECREF9) are gradient-based and optimisation-based attacks (Section SECREF2). We propose an alternative model-based method to train a seperate generative model to generate text adversarial examples. Denoting the generative model as $\mathcal {G}$, we train it to generate an adversarial example $X^{\prime }$ given an input example $X$ such that $X^{\prime }$ is similar to $X$ but that it changes the prediction of the target classifier $\mathcal {D}$, i.e. $X^{\prime } \sim X$ and $\mathcal {D}(X^{\prime }) \ne \mathcal {D}(X)$. For images, it is straightforward to combine $\mathcal {G}$ and $\mathcal {D}$ in the same network and use the loss of $\mathcal {D}$ to update $\mathcal {G}$ because $X^{\prime }$ is continuous and can be fed to $\mathcal {D}$ while keeping the whole network differentiable. For texts, $X^{\prime }$ are discrete symbols (typically decoded with argmax or beam-search) and so the network is not fully differentiable. To create a fully differentiable network with $\mathcal {G}+\mathcal {D}$, we propose to use Gumbel-Softmax to imitate the categorical distribution BIBREF19. As the temperature ($\tau $) of Gumbel-Softmax sampling approaches 0, the distribution of the sampled output $X^*$ is identical to $X^{\prime }$. We illustrate this in Figure FIGREF12, where $X^*$ is fed to $\mathcal {D}$ during training while $X^{\prime }$ is generated as the output (i.e. the adversarial example) at test time. $\mathcal {G}$ is designed as an auto-encoder to reconstruct the input (denoted as AutoEncoder), with the following objectives to capture the evaluation criteria described in Section SECREF1: $L_{adv}$, the adversarial loss that maximises the cross-entropy of $\mathcal {D}$; $L_{seq}$, the auto-encoder loss to reconstruct $X^{\prime }$ from $X^{\prime }$; $L_{sem}$, the cosine distance between the mean embeddings of $X$ and $X^*$. $L_{adv}$ ensures that the adversarial examples are fooling the classifier (criterion (a)); $L_{seq}$ regulates the adversarial example so that it isn't too different from the original input and has a reasonable language (criteria (b) and (c)); and $L_{sem}$ constrains the underlying semantic content of the original and adversarial sentences to be similar (criterion (b)); so reduces the likelihood of a sentiment flip (indirectly for criteria (d)). Note that criteria (d) is the perhaps the most difficult aspect to handle, as its interpretation is ultimately a human judgement. We use two scaling hyper-parameters $\lambda _{ae}$ and $\lambda _{seq}$ to weigh the three objectives: $\lambda _{ae}(\lambda _{seq} * L_{seq} + (1-\lambda _{seq})*L_{sem}) + (1-\lambda _{ae})*L_{adv}$. We introduce attention layers to AutoEncoder and test both greedy and beam search decoding to improve the quality of generated adversarial examples. During training we alternate between positive-to-negative and negative-to-positive attack for different mini-batches. Experiments ::: Datasets We construct three datasets based on IMDB reviews and Yelp reviews. The IMDB dataset is binarised and split into a training and test set, each with 25K reviews (2K reviews from the training set are reserved for development). We filter out any review that has more than 400 tokens, producing the final dataset (imdb400). For Yelp, we binarise the ratings, and create 2 datasets, where we keep only reviews with $\le $ 50 tokens (yelp50) and $\le $200 tokens (yelp200). We randomly partition both datasets into train/dev/test sets (90/5/5 for yelp50; 99/0.5/0.5 for yelp200). For all datasets, we use spaCy for tokenisation. We train and tune target classifiers (see Section SECREF8) using the training and development sets; and evaluate their performance on the original examples in the test sets as well as the adversarial examples generated by attacking methods for the test sets. Note that AutoEncoder also involves a training process, for which we train and tune AutoEncoder using the training and development sets in yelp50, yelp200 and imdb400. Statistics of the three datasets are presented in Table TABREF22. These datasets present a variation in the text lengths (e.g. the average number of words for yelp50, yelp200 and imdb400 is 34, 82 and 195 words respectively), training data size (e.g. the number of training examples for target classifiers for imdb400, yelp50 and yelp200 are 18K, 407K and 2M, respectively) and input domain (e.g. restaurant vs. movie reviews). Experiments ::: Implementation Details We use the pre-trained glove.840B.300d embeddings BIBREF20 for all 6 attacking methods. For FGM, FGVM and DeepFool, we tune $\epsilon $, the overshoot hyper-parameter (Section SECREF9) and keep the iterative step $n$ static (5). For TYC, besides $\epsilon $ we also tune the upper limit of flipped words, ranging from 10%–100% of the maximum length. For HotFlip, we tune only the upper limit of flipped words, in the range of $[1, 7]$. We pre-train AutoEncoder to reconstruct sentences in different datasets as we found that this improves the quality of the generated adversarial examples. During pre-training, we tune batch size, number of layers and number of units, and stop the training after the performance on the development sets stops improving for 20K steps. The model is then initialised with the pre-trained weights and trained based on objectives defined in Section SECREF11. During the training process we tune $\lambda _{ae}$ and $\lambda _{seq}$ while keeping the batch size (32) and learning rate ($1e^{-4}$) fixed. As part of our preliminary study, we also tested different values for the Gumbel-softmax temperature $\tau $ and find that $\tau =0.1$ performs the best. Embeddings are fixed throughout all training processes. For target classifiers, we tune batch size, learning rate, number of layers, number of units, attention size (BiLSTM$+$A), filter sizes and dropout probability (CNN). For BERT, we use the default fine-tuning hyper-parameter values except for batch size, where we adjust based on memory consumption. Note that after the target classifiers are trained their weights are not updated when training or testing the attacking methods. Evaluation We propose both automatic metrics and manual evaluation strategies to assess the quality of adversarial examples, based on four criteria defined in Section SECREF1: (a) attacking performance (i.e. how well they fool the classifier); (b) textual similarity between the original input and the adversarial input; (c) fluency of the adversarial example; and (d) label preservation. Note that the automatic metrics only address the first 3 criteria (a, b and c); we contend that criterion (d) requires manual evaluation, as the judgement of whether the original label is preserved is inherently a human decision. Evaluation ::: Automatic Evaluation: Metrics As sentiment classification is our target task, we use the standard classification accuracy (ACC) to evaluate the attacking performance of adversarial examples (criterion (a)). To assess the similarity between the original and (transformed) adversarial examples (criteria (b)), we compute BLEU scores BIBREF21. To measure fluency, we first explore a supervised BERT model fine-tuned to predict linguistic acceptability BIBREF17. However, in preliminary experiments we found that BERT performs very poorly at predicting the acceptability of adversarial examples (e.g. it predicts word-salad-like sentences generated by FGVM as very acceptable), revealing the brittleness of these supervised models. We next explore an unsupervised approach BIBREF22, BIBREF23, using normalised sentence probabilities estimated by pre-trained language models for measuring acceptability. In the original papers, the authors tested simple recurrent language models; here we use modern pre-trained language models such as GPT-2 BIBREF24 and XLNet BIBREF18. Our final acceptability metric (ACPT) is based on normalised XLNet sentence probabilities: ${\log P(s)} / ({((5+|s|)/(5+1))^\alpha })$, where $s$ is the sentence, and $\alpha $ is a hyper-parameter (set to 0.8) to dampen the impact of large values BIBREF25. We only computed BLEU and ACPT scores for adversarial examples that have successfully fooled the classifier. Our rationale is that unsuccessful examples can artificially boost these scores by not making any modifications, and so the better approach is to only consider successful examples. Evaluation ::: Automatic Evaluation: Results We present the performance of the attacking methods against 3 target classifiers (Table TABREF23A; top) and on 3 datasets (Table TABREF23B; bottom). We choose 3 ACC thresholds for the attacking performance: T0, T1 and T2, which correspond approximately to accuracy scores of 90%, 80% and 70% for the Yelp datasets (yelp50, yelp200); and 80%, 70% and 50% for the IMDB datasets (imdb400). Each method is tuned accordingly to achieve a particular accuracy. Missing numbers (dashed lines) indicate the method is unable to produce the desired accuracy, e.g. HotFlip with only 1 word flip produces 81.5% accuracy (T1) when attacking CNN on yelp50, and so T0 accuracy is unachievable. Looking at BLEU and ACPT, HotFlip is the most consistent method over multiple datasets and classifiers. AutoEncoder is also fairly competitive, producing largely comparable performance (except for the yelp200 dataset). Gradient-based methods FGM and FGVM perform very poorly. In general, they tend to produce word salad adversarial examples, as indicated by their poor BLEU scores. DeepFool similarly generates incoherent sentences with low BLEU scores, but occasionally produces good ACPT (BiLSTM at T1 and T2), suggesting potential brittleness of the unsupervised approach for evaluating acceptability. Comparing the performance across different ACC thresholds, we observe a consistent pattern of decreasing performance over all metrics as the attacking performance increases from T0 to T2. These observations suggest that all methods are trading off fluency and content preservation as they attempt to generate stronger adversarial examples. We now focus on Table TABREF23A, to understand the impact of model architecture for the target classifier. With 1 word flip as the upper limit for HotFlip, the accuracy of BiLSTM$+$A and BiLSTM drops to T0 (approximately 4% accuracy decrease) while the accuracy of CNN drops to T1 (approximately 13% accuracy decrease), suggesting that convolutional networks are more vulnerable to attacks (noting that it is the predominant architecture for CV). Looking at Table TABREF23B, we also find that the attacking performance is influenced by the input text length and the number of training examples for target classifiers. For HotFlip, we see improvements over BLEU and ACPT as text length increases from yelp50 to yelp200 to imdb400, indicating the performance of HotFlip is more affected by input lengths. We think this is because with more words it is more likely for HotFlip to find a vulnerable spot to target. While for TYC and AutoEncoder, we see improvements over BLEU and ACPT as the number of training examples for target classifier decreases from yelp200 (2M) to yelp50 (407K) to imdb400 (22K), indicating these two methods are less effective for attacking target classifiers that are trained with more data. Therefore, increasing the training data for target classifier may improve their robustness against adversarial examples that are generated by certain methods. As a black-box test to check how well these adversarial examples generalise to fooling other classifiers, also known as transferability, we feed the adversarial examples from the 3 best methods, i.e. TYC, HotFlip and AutoEncoder, to a pre-trained BERT trained for sentiment classification (Figure FIGREF31) and measure its accuracy. Unsurprisingly, we observe that the attacking performance (i.e. the drop in ACC) is not as good as those reported for the white-box tests. Interestingly, we find that HotFlip, the best method, produces the least effective adversarial examples for BERT. Both TYC and AutoEncoder perform better here, as their generated adversarial examples do well in fooling BERT. To summarise, our results demonstrate that the best white-box methods (e.g. HotFlip) may not produce adversarial examples that generalise to fooling other classifiers. We also saw that convolutional networks are more vulnerable than recurrent networks, and that dataset features such as text lengths and training data size (for target classifiers) can influence how difficult it is to perform adversarial attack. Evaluation ::: Human Evaluation: Design Automatic metrics provide a proxy to quantify the quality of the adversarial examples. To validate that these metrics work, we conduct a crowdsourcing experiment on Figure-Eight. Recall that the automatic metrics do not assess sentiment preservation (criterion (d)); we evaluate that aspect here. We experiment with the 3 best methods (TYC, AutoEncoder and HotFlip) on 2 accuracy thresholds (T0 and T2), using BiLSTM$+$A as the classifier. For each method and threshold, we randomly sample 25 positive-to-negative and 25 negative-to-positive examples. To control for quality, we reserve and annotate 10% of the samples ourselves as control questions. Workers are first presented with 10 control questions as a quiz, and only those who pass the quiz with at least 80% accuracy can continue to work on the task. We display 10 questions per page, where one control question is embedded to continue monitor worker performance. The task is designed such that each control question can only be seen once per worker. We restrict our jobs to workers in United States, United Kingdoms, Australia, and Canada. To evaluate the criteria discussed in Section SECREF1: (b) textual similarity, (c) fluency, and (d) sentiment preservation, we ask the annotators three questions: Is snippet B a good paraphrase of snippet A? $\circledcirc $ Yes $\circledcirc $ Somewhat yes $\circledcirc $ No How natural does the text read? $\circledcirc $ Very unnatural $\circledcirc $ Somewhat natural $\circledcirc $ Natural What is the sentiment of the text? $\circledcirc $ Positive $\circledcirc $ Negative $\circledcirc $ Cannot tell For question 1, we display both the adversarial input and the original input, while for question 2 and 3 we present only the adversarial example. As an upper-bound, we also run a survey on question 2 and 3 for 50 random original samples. Evaluation ::: Human Evaluation: Results We present the percentage of answers to each question in Figure FIGREF38. The green bars illustrate how well the adversarial examples paraphrase the original ones; blue how natural the adversarial examples read; and red whether the sentiment of the adversarial examples is consistent compared to the original. Looking at the performance of the original sentences (“(a) Original samples”), we see that their language is largely fluent and their sentiment is generally consistent to the original examples', although it's worth noting that the review sentiment of imdb400 can be somewhat ambiguous (63% agreement). We think this is due to movie reviews being more descriptive and therefore creating potential ambiguity in their expression of sentiment. On content preservation (criterion (b); green bars), all methods produce poor paraphrases on yelp50. For imdb400, however, the results are more promising; adversarial examples generated by HotFlip, in particular, are good, even at T2. Next we look at fluency (criterion (c); blue bars). We see a similar trend: performance in imdb400 is substantially better than yelp50. In fact we see almost no decrease in fluency in the adversarial examples compared to the original in imdb400. In yelp50, HotFlip and AutoEncoder are fairly competitive, producing adversarial examples that are only marginally less fluent compared to the original at T0. At T2, however, these methods begin to trade off fluency. All in all, the paraphrasability and fluency surveys suggest that imdb400 is an easier dataset for adversarial experiments, and it is the predominant dataset used by most studies. Lastly, we consider sentiment preservation (criterion (d); red bars). All methods perform poorly at preserving the original sentiment on both yelp50 and imdb400 datasets. The artifact is arguably more profound on shorter inputs, as the original examples in imdb400 have lower agreement in the first place (yelp50 vs. imdb400: 86% vs. 63%). Again both HotFlip and AutoEncoder are the better methods here (interestingly, we observe an increase in agreement as their attacking performance increases from T0 and T2). Summarising our findings, HotFlip is generally the best method across all criteria, noting that its adversarial examples, however, have poor transferability. TYC generates good black-box adversarial examples but do not do well in terms of content preservation and fluency. AutoEncoder produces comparable results with HotFlip for meeting the four criteria and generates examples that generalise reasonably, but it is very sensitive to the increase of training examples for the target classifier. The ACPT metric appears to be effective in evaluating fluency, as we see good agreement with human evaluation. All said, we found that all methods tend to produce adversarial examples that do not preserve their original sentiments, revealing that these methods in a way “cheat” by simply flipping the sentiments of the original sentences to fool the classifier, and therefore the adversarial examples might be ineffective for adversarial training, as they are not examples that reveal potential vulnerabilities in the classifier. Conclusion We propose an evaluation framework for assessing the quality of adversarial examples in NLP, based on four criteria: (a) attacking performance, (b) textual similarity; (c) fluency; (d) label preservation. Our framework involves both automatic and human evaluation, and we test 5 benchmark methods and a novel auto-encoder approach. We found that the architecture of the target classifier is an important factor when it comes to attacking performance, e.g. CNNs are more vulnerable than LSTMs; dataset features such as length of text, training data size (for target classifiers) and input domains are also influencing factors that affect how difficulty it is to perform adversarial attack; and the predominant dataset (IMDB) used by most studies is comparatively easy for adversarial attack. Lastly, we also observe in our human evaluation that on shorter texts (Yelp) these methods produce adversarial examples that tend not to preserve their semantic content and have low readability. More importantly, these methods also “cheat” by simply flipping the sentiment in the adversarial examples, and this behaviour is evident on both datasets, suggesting they could be ineffective for adversarial training.
What datasets do they use?
395b61d368e8766014aa960fde0192e4196bcb85
What Ġdatasets Ġdo Ġthey Ġuse ?
three Ġdatasets Ġbased Ġon ĠIM DB Ġreviews Ġand ĠYelp Ġreviews
<s> What Ġdatasets Ġdo Ġthey Ġuse ? </s> Introduction </s> Ad vers arial Ġexamples , Ġa Ġterm Ġintroduced Ġin ĠB IB REF 0 , Ġare Ġinputs Ġtransformed Ġby Ġsmall Ġpert urb ations Ġthat Ġmachine Ġlearning Ġmodels Ġconsistently Ġmis class ify . ĠThe Ġexperiments Ġare Ġconducted Ġin Ġthe Ġcontext Ġof Ġcomputer Ġvision Ġ( CV ), Ġand Ġthe Ġcore Ġidea Ġis Ġencaps ulated Ġby Ġan Ġillust rative Ġexample : Ġafter Ġimper cept ible Ġnoises Ġare Ġadded Ġto Ġa Ġp anda Ġimage , Ġan Ġimage Ġclass ifier Ġpredicts , Ġwith Ġhigh Ġconfidence , Ġthat Ġit Ġis Ġa Ġgib bon . ĠInterestingly , Ġthese Ġadvers arial Ġexamples Ġcan Ġalso Ġbe Ġused Ġto Ġimprove Ġthe Ġclass ifier ĠâĢĶ Ġeither Ġas Ġadditional Ġtraining Ġdata ĠB IB REF 0 Ġor Ġas Ġa Ġregular isation Ġobjective ĠB IB REF 1 ĠâĢĶ Ġthus Ġproviding Ġmotivation Ġfor Ġgenerating Ġeffective Ġadvers arial Ġexamples . </s> The Ġgerm Ġof Ġthis Ġpaper Ġcomes Ġfrom Ġour Ġinvestigation Ġof Ġadvers arial Ġattack Ġmethods Ġfor Ġnatural Ġlanguage Ġprocessing Ġ( N LP ) Ġtasks , Ġe . g . Ġsentiment Ġclassification , Ġwhich Ġdrives Ġus Ġto Ġquantify Ġwhat Ġis Ġan ĠâĢ ľ effective âĢ Ŀ Ġor ĠâĢ ľ good âĢ Ŀ Ġadvers arial Ġexample . ĠIn Ġthe Ġcontext Ġof Ġimages , Ġa Ġgood Ġadvers arial Ġexample Ġis Ġtypically Ġdefined Ġaccording Ġtwo Ġcriteria : </s> it Ġhas Ġsuccessfully Ġfooled Ġthe Ġclass ifier ; </s> it Ġis Ġvisually Ġsimilar Ġto Ġthe Ġoriginal Ġexample . </s> In ĠN LP , Ġdefining Ġa Ġgood Ġadvers arial Ġexample Ġis Ġa Ġlittle Ġmore Ġinvolving , Ġbecause Ġwhile Ġcriterion Ġ( b ) Ġcan Ġbe Ġmeasured Ġwith Ġa Ġcomparable Ġtext Ġsimilarity Ġmetric Ġ( e . g . ĠB LE U Ġor Ġedit Ġdistance ), Ġan Ġadvers arial Ġexample Ġshould Ġalso : </s> be Ġfluent Ġor Ġnatural ; </s> pres erve Ġits Ġoriginal Ġlabel . </s> These Ġtwo Ġadditional Ġcriteria Ġare Ġgenerally Ġirrelevant Ġfor Ġimages , Ġas Ġadding Ġminor Ġpert urb ations Ġto Ġan Ġimage Ġis Ġunlikely Ġto : Ġ( 1 ) Ġcreate Ġan Ġun interpret able Ġimage Ġ( where Ġelse Ġchanging Ġone Ġword Ġin Ġa Ġsentence Ġcan Ġrender Ġa Ġsentence Ġinco herent ), Ġor Ġ( 2 ) Ġchange Ġhow Ġwe Ġperceive Ġthe Ġimage , Ġsay Ġfrom Ġseeing Ġa Ġp anda Ġto Ġa Ġgib bon Ġ( but Ġa Ġsentence 's Ġsentiment Ġcan Ġbe Ġreversed Ġby Ġsimply Ġadding Ġa Ġnegative Ġad verb Ġsuch Ġas Ġnot ). ĠWithout Ġconsidering Ġcriterion Ġ( d ), Ġgenerating Ġadvers arial Ġexamples Ġin ĠN LP Ġwould Ġbe Ġtrivial , Ġas Ġthe Ġmodel Ġcan Ġlearn Ġto Ġsimply Ġreplace Ġa Ġpositive Ġadjective Ġ( am azing ) Ġwith Ġa Ġnegative Ġone Ġ( aw ful ) Ġto Ġattack Ġa Ġsentiment Ġclass ifier . </s> To Ġthe Ġbest Ġof Ġour Ġknowledge , Ġmost Ġstudies Ġon Ġadvers arial Ġexample Ġgeneration Ġin ĠN LP Ġhave Ġlargely Ġignored Ġthese Ġadditional Ġcriteria ĠB IB REF 2 , ĠB IB REF 3 , ĠB IB REF 4 , ĠB IB REF 5 . ĠWe Ġbelieve Ġthe Ġlack Ġof Ġa Ġrigorous Ġevaluation Ġframework Ġpartially Ġexplains Ġwhy Ġadvers arial Ġtraining Ġfor ĠN LP Ġmodels Ġhas Ġnot Ġseen Ġthe Ġsame Ġextent Ġof Ġimprovement Ġcompared Ġto ĠCV Ġmodels . ĠAs Ġour Ġexperiments Ġreveal , Ġexamples Ġgenerated Ġfrom Ġmost Ġattacking Ġmethods Ġare Ġsuccessful Ġin Ġfool ing Ġthe Ġclass ifier , Ġbut Ġtheir Ġlanguage Ġis Ġoften Ġunnatural Ġand Ġthe Ġoriginal Ġlabel Ġis Ġnot Ġproperly Ġpreserved . </s> The Ġcore Ġcontribution Ġof Ġour Ġpaper Ġis Ġto Ġintroduce Ġa Ġsystematic , Ġrigorous Ġevaluation Ġframework Ġto Ġassess Ġthe Ġquality Ġof Ġadvers arial Ġexamples Ġfor ĠN LP . ĠWe Ġfocus Ġon Ġsentiment Ġclassification Ġas Ġthe Ġtarget Ġtask , Ġas Ġit Ġis Ġa Ġpopular Ġapplication Ġthat Ġhighlights Ġthe Ġimportance Ġof Ġcriteria Ġdiscussed Ġabove . ĠWe Ġtest Ġa Ġnumber Ġof Ġattacking Ġmethods Ġand Ġalso Ġpropose Ġan Ġalternative Ġapproach Ġ( based Ġon Ġan Ġauto - enc oder ) Ġfor Ġgenerating Ġadvers arial Ġexamples . ĠWe Ġlearn Ġthat Ġa Ġnumber Ġof Ġfactors Ġcan Ġinfluence Ġthe Ġperformance Ġof Ġadvers arial Ġattacks , Ġincluding Ġarchitecture Ġof Ġthe Ġclass ifier , Ġsentence Ġlength Ġand Ġinput Ġdomain . </s> Related ĠWork </s> Most Ġexisting Ġadvers arial Ġattack Ġmethods Ġfor Ġtext Ġinputs Ġare Ġderived Ġfrom Ġthose Ġfor Ġimage Ġinputs . ĠThese Ġmethods Ġcan Ġbe Ġcategor ised Ġinto Ġthree Ġtypes Ġincluding Ġgradient - based Ġattacks , Ġoptim isation - based Ġattacks Ġand Ġmodel - based Ġattacks . </s> Grad ient - based Ġattacks Ġare Ġmainly Ġwhite - box Ġattacks Ġthat Ġrely Ġon Ġcalculating Ġthe Ġgrad ients Ġof Ġthe Ġtarget Ġclass ifier Ġwith Ġrespect Ġto Ġthe Ġinput Ġrepresentation . ĠThis Ġclass Ġof Ġattacking Ġmethods ĠB IB REF 6 , ĠB IB REF 7 , ĠB IB REF 6 Ġare Ġmainly Ġderived Ġfrom Ġthe Ġfast Ġgradient Ġsign Ġmethod Ġ( FG SM ) ĠB IB REF 1 , Ġand Ġit Ġhas Ġbeen Ġshown Ġto Ġbe Ġeffective Ġin Ġattacking ĠCV Ġclass ifiers . ĠHowever , Ġthese Ġgradient - based Ġmethods Ġcould Ġnot Ġbe Ġapplied Ġto Ġtext Ġdirectly Ġbecause Ġpert urbed Ġword Ġembed d ings Ġdo Ġnot Ġnecessarily Ġmap Ġto Ġvalid Ġwords . ĠOther Ġmethods Ġsuch Ġas ĠDeep F ool ĠB IB REF 8 Ġthat Ġrely Ġon Ġpert ur bing Ġthe Ġword Ġembed ding Ġspace Ġface Ġsimilar Ġroad blocks . ĠB IB REF 5 Ġpropose Ġto Ġuse Ġnearest Ġneighbour Ġsearch Ġto Ġfind Ġthe Ġclosest Ġword Ġto Ġthe Ġpert urbed Ġembed ding . </s> Both Ġoptim isation - based Ġand Ġmodel - based Ġattacks Ġtreat Ġadvers arial Ġattack Ġas Ġan Ġoptim isation Ġproblem Ġwhere Ġthe Ġconstraints Ġare Ġto Ġmaxim ise Ġthe Ġloss Ġof Ġtarget Ġclass ifiers Ġand Ġto Ġminim ise Ġthe Ġdifference Ġbetween Ġoriginal Ġand Ġadvers arial Ġexamples . ĠBetween Ġthese Ġtwo , Ġthe Ġformer Ġuses Ġoptim isation Ġalgorithms Ġdirectly ; Ġwhile Ġthe Ġlatter Ġtrains Ġa Ġse perate Ġmodel Ġto Ġgenerate Ġadvers arial Ġexamples Ġand Ġtherefore Ġinvolves Ġa Ġtraining Ġprocess . ĠSome Ġof Ġthe Ġmost Ġeffective Ġattacks Ġfor Ġimages Ġare Ġachieved Ġby Ġoptim isation - based Ġmethods , Ġsuch Ġas Ġthe ĠL - BF GS Ġattack ĠB IB REF 1 Ġand Ġthe ĠC & W Ġattack ĠB IB REF 9 Ġin Ġwhite - box Ġattacks Ġand Ġthe ĠZ OO Ġmethod ĠB IB REF 10 Ġin Ġblack - box Ġattacks . ĠFor Ġtexts , Ġthe Ġwhite - box Ġattack ĠHot Fl ip ĠB IB REF 3 Ġand Ġblack - box Ġattack ĠDeep Word Bug ĠB IB REF 11 Ġand ĠText Bug ger ĠB IB REF 12 Ġare Ġproposed Ġin Ġthis Ġcategory . </s> In Ġa Ġsimilar Ġvein , Ġa Ġfew Ġmodel - based Ġattacks Ġhave Ġbeen Ġproposed Ġfor Ġimages , Ġe . g . ĠB IB REF 13 Ġdesign Ġa Ġgener ative Ġadvers arial Ġnetwork Ġ( GAN ) Ġto Ġgenerate Ġthe Ġimage Ġpert urb ation Ġfrom Ġa Ġnoise Ġmap . ĠThe Ġattacking Ġmethod Ġand Ġtarget Ġclass ifier Ġtypically Ġform Ġa Ġsingle Ġlarge Ġnetwork Ġand Ġthe Ġattacking Ġmethod Ġis Ġtrained Ġusing Ġthe Ġloss Ġfrom Ġthe Ġtarget Ġclass ifier . ĠFor Ġthis Ġreason , Ġit Ġis Ġnot Ġvery Ġstraightforward Ġto Ġuse Ġthese Ġmodel - based Ġtechniques Ġfor Ġtext Ġbecause Ġthere Ġis Ġa Ġdiscontin uity Ġin Ġthe Ġnetwork Ġ( since Ġwords Ġin Ġthe Ġadvers arial Ġexamples Ġare Ġdiscrete ) Ġand Ġso Ġit Ġis Ġnot Ġfully Ġdifferent iable . </s> Method ology Ġ: :: ĠSent iment ĠClass ifiers </s> There Ġare Ġa Ġnumber Ġof Ġoff - the - she lf Ġneural Ġmodels Ġfor Ġsentiment Ġclassification ĠB IB REF 14 , ĠB IB REF 15 , Ġmost Ġof Ġwhich Ġare Ġbased Ġon Ġlong - short Ġterm Ġmemory Ġnetworks Ġ( L ST M ) ĠB IB REF 16 Ġor Ġconv olution al Ġneural Ġnetworks Ġ( CNN ) ĠB IB REF 14 . ĠIn Ġthis Ġpaper , Ġwe Ġpre - train Ġthree Ġsentiment Ġclass ifiers : ĠBi L ST M , ĠBi L ST M $ + $ A , Ġand ĠCNN . ĠThese Ġclass ifiers Ġare Ġtargeted Ġby Ġwhite - box Ġattacking Ġmethods Ġto Ġgenerate Ġadvers arial Ġexamples Ġ( det ailed Ġin ĠSection ĠSEC REF 9 ). ĠBi L ST M Ġis Ġcomposed Ġof Ġan Ġembed ding Ġlayer Ġthat Ġmaps Ġindividual Ġwords Ġto Ġpre - trained Ġword Ġembed d ings ; Ġa Ġnumber Ġof Ġbi - direction al ĠL ST Ms Ġthat Ġcapture Ġsequential Ġcontexts ; Ġand Ġan Ġoutput Ġlayer Ġthat Ġmaps Ġthe Ġaveraged ĠL ST M Ġhidden Ġstates Ġto Ġa Ġbinary Ġoutput . ĠBi L ST M $ + $ A Ġis Ġsimilar Ġto ĠBi L ST M Ġexcept Ġit Ġhas Ġan Ġextra Ġself - att ention Ġlayer Ġwhich Ġlearns Ġto Ġattend Ġto Ġsalient Ġwords Ġfor Ġsentiment Ġclassification , Ġand Ġwe Ġcompute Ġa Ġweighted Ġmean Ġof Ġthe ĠL ST M Ġhidden Ġstates Ġprior Ġto Ġthe Ġoutput Ġlayer . ĠManual Ġinspection Ġof Ġthe Ġattention Ġweights Ġshow Ġthat Ġpol arity Ġwords Ġsuch Ġas Ġawesome Ġand Ġdisappointed Ġare Ġassigned Ġwith Ġhigher Ġweights . ĠFinally , ĠCNN Ġhas Ġa Ġnumber Ġof Ġconv olution al Ġfilters Ġof Ġvarying Ġsizes , Ġand Ġtheir Ġoutputs Ġare Ġconc aten ated , Ġpooled Ġand Ġfed Ġto Ġa Ġfully - connected Ġlayer Ġfollowed Ġby Ġa Ġbinary Ġoutput Ġlayer . </s> Recent Ġdevelopment Ġin Ġtransformer - based Ġpre - trained Ġmodels Ġhave Ġproduced Ġstate - of - the - art Ġperformance Ġon Ġa Ġrange Ġof ĠN LP Ġtasks ĠB IB REF 17 , ĠB IB REF 18 . ĠTo Ġvalidate Ġthe Ġtransfer ability Ġof Ġthe Ġattacking Ġmethods , Ġwe Ġalso Ġfine - t une Ġa ĠB ERT Ġclass ifier Ġfor Ġblack - box Ġtests . ĠThat Ġis , Ġwe Ġuse Ġthe Ġadvers arial Ġexamples Ġgenerated Ġfor Ġattacking Ġthe Ġthree Ġprevious Ġclass ifiers Ġ( Bi L ST M , ĠBi L ST M $ + $ A Ġand ĠCNN ) Ġas Ġtest Ġdata Ġfor ĠB ERT Ġto Ġmeasure Ġits Ġclassification Ġperformance Ġto Ġunderstand Ġwhether Ġthese Ġadvers arial Ġexamples Ġcan Ġfool ĠB ERT . </s> Method ology Ġ: :: ĠBench mark ĠAtt acking ĠMethods </s> We Ġexperiment Ġwith Ġfive Ġbenchmark Ġattacking Ġmethods Ġfor Ġtexts : ĠF GM , ĠFG VM , ĠDeep F ool ĠB IB REF 5 , ĠHot Fl ip ĠB IB REF 3 ) Ġand ĠTY C ĠB IB REF 4 . </s> To Ġpert urb Ġthe Ġdiscrete Ġinputs , Ġboth ĠF GM Ġand ĠFG VM Ġintroduce Ġnoises Ġin Ġthe Ġword Ġembed ding Ġspace Ġvia Ġthe Ġfast Ġgradient Ġmethod ĠB IB REF 1 Ġand Ġreconstruct Ġthe Ġinput Ġby Ġmapping Ġpert urbed Ġword Ġembed d ings Ġto Ġvalid Ġwords Ġvia Ġnearest Ġneighbour Ġsearch . ĠBetween ĠF GM Ġand ĠFG VM , Ġthe Ġformer Ġintroduce Ġnoises Ġthat Ġis Ġproportional Ġto Ġthe Ġsign Ġof Ġthe Ġgrad ients Ġwhile Ġthe Ġlatter Ġintroduce Ġpert urb ations Ġproportional Ġto Ġthe Ġgrad ients Ġdirectly . ĠThe Ġproportion Ġis Ġknown Ġas Ġthe Ġover shoot Ġvalue Ġand Ġden oted Ġby Ġ$\ ep sil on Ġ$ . ĠDeep F ool Ġuses Ġthe Ġsame Ġtrick Ġto Ġdeal Ġwith Ġdiscrete Ġinputs Ġexcept Ġthat , Ġinstead Ġof Ġusing Ġthe Ġfast Ġgradient Ġmethod , Ġit Ġuses Ġthe ĠDeep F ool Ġmethod Ġintroduced Ġin ĠB IB REF 8 Ġfor Ġimage Ġto Ġsearch Ġfor Ġan Ġoptimal Ġdirection Ġto Ġpert urb Ġthe Ġword Ġembed d ings . </s> Unlike Ġthe Ġprevious Ġmethods , ĠHot Fl ip Ġand ĠTY C Ġrely Ġon Ġperforming Ġone Ġor Ġmore Ġatomic Ġflip Ġoperations Ġto Ġreplace Ġwords Ġwhile Ġmonitoring Ġthe Ġlabel Ġchange Ġgiven Ġby Ġthe Ġtarget Ġclass ifier . ĠIn ĠHot Fl ip , Ġthe Ġdirectional Ġderivatives Ġw . r . t . Ġflip Ġoperations Ġare Ġcalculated Ġand Ġthe Ġflip Ġoperation Ġthat Ġresults Ġin Ġthe Ġlargest Ġincrease Ġin Ġloss Ġis Ġselected . ĠTY C Ġis Ġsimilar Ġto ĠF GM , ĠFG VM Ġand ĠDeep F ool Ġin Ġthat Ġit Ġalso Ġuses Ġnearest Ġneighbour Ġsearch Ġto Ġmap Ġthe Ġpert urbed Ġembed d ings Ġto Ġvalid Ġwords , Ġbut Ġinstead Ġof Ġusing Ġthe Ġpert urbed Ġtokens Ġdirectly , Ġit Ġuses Ġgreedy Ġsearch Ġor Ġbeam Ġsearch Ġto Ġflip Ġoriginal Ġtokens Ġto Ġpert urbed Ġones Ġone Ġat Ġa Ġtime Ġin Ġorder Ġof Ġtheir Ġvulnerability . </s> Method ology Ġ: :: ĠModel - based ĠAtt acking ĠMethod </s> The Ġbenchmark Ġmethods Ġwe Ġtest Ġ( Section ĠSEC REF 9 ) Ġare Ġgradient - based Ġand Ġoptim isation - based Ġattacks Ġ( Section ĠSEC REF 2 ). ĠWe Ġpropose Ġan Ġalternative Ġmodel - based Ġmethod Ġto Ġtrain Ġa Ġse perate Ġgener ative Ġmodel Ġto Ġgenerate Ġtext Ġadvers arial Ġexamples . ĠDen oting Ġthe Ġgener ative Ġmodel Ġas Ġ$\ math cal Ġ{ G } $, Ġwe Ġtrain Ġit Ġto Ġgenerate Ġan Ġadvers arial Ġexample Ġ$ X ^ {\ prime Ġ} $ Ġgiven Ġan Ġinput Ġexample Ġ$ X $ Ġsuch Ġthat Ġ$ X ^ {\ prime Ġ} $ Ġis Ġsimilar Ġto Ġ$ X $ Ġbut Ġthat Ġit Ġchanges Ġthe Ġprediction Ġof Ġthe Ġtarget Ġclass ifier Ġ$\ math cal Ġ{ D } $, Ġi . e . Ġ$ X ^ {\ prime Ġ} Ġ\ sim ĠX $ Ġand Ġ$\ math cal Ġ{ D } ( X ^ {\ prime Ġ}) Ġ\ ne Ġ\ math cal Ġ{ D } ( X ) $. </s> For Ġimages , Ġit Ġis Ġstraightforward Ġto Ġcombine Ġ$\ math cal Ġ{ G } $ Ġand Ġ$\ math cal Ġ{ D } $ Ġin Ġthe Ġsame Ġnetwork Ġand Ġuse Ġthe Ġloss Ġof Ġ$\ math cal Ġ{ D } $ Ġto Ġupdate Ġ$\ math cal Ġ{ G } $ Ġbecause Ġ$ X ^ {\ prime Ġ} $ Ġis Ġcontinuous Ġand Ġcan Ġbe Ġfed Ġto Ġ$\ math cal Ġ{ D } $ Ġwhile Ġkeeping Ġthe Ġwhole Ġnetwork Ġdifferent iable . ĠFor Ġtexts , Ġ$ X ^ {\ prime Ġ} $ Ġare Ġdiscrete Ġsymbols Ġ( typically Ġdec oded Ġwith Ġarg max Ġor Ġbeam - search ) Ġand Ġso Ġthe Ġnetwork Ġis Ġnot Ġfully Ġdifferent iable . ĠTo Ġcreate Ġa Ġfully Ġdifferent iable Ġnetwork Ġwith Ġ$\ math cal Ġ{ G } + \ math cal Ġ{ D } $, Ġwe Ġpropose Ġto Ġuse ĠG umb el - Soft max Ġto Ġimitate Ġthe Ġcateg orical Ġdistribution ĠB IB REF 19 . ĠAs Ġthe Ġtemperature Ġ($ \ t au Ġ$ ) Ġof ĠG umb el - Soft max Ġsampling Ġapproaches Ġ0 , Ġthe Ġdistribution Ġof Ġthe Ġsampled Ġoutput Ġ$ X ^ * $ Ġis Ġidentical Ġto Ġ$ X ^ {\ prime Ġ} $. ĠWe Ġillustrate Ġthis Ġin ĠFigure ĠFIG REF 12 , Ġwhere Ġ$ X ^ * $ Ġis Ġfed Ġto Ġ$\ math cal Ġ{ D } $ Ġduring Ġtraining Ġwhile Ġ$ X ^ {\ prime Ġ} $ Ġis Ġgenerated Ġas Ġthe Ġoutput Ġ( i . e . Ġthe Ġadvers arial Ġexample ) Ġat Ġtest Ġtime . </s> $ \ math cal Ġ{ G } $ Ġis Ġdesigned Ġas Ġan Ġauto - enc oder Ġto Ġreconstruct Ġthe Ġinput Ġ( den oted Ġas ĠAuto Enc oder ), Ġwith Ġthe Ġfollowing Ġobjectives Ġto Ġcapture Ġthe Ġevaluation Ġcriteria Ġdescribed Ġin ĠSection ĠSEC REF 1 : </s> $ L _{ adv } $, Ġthe Ġadvers arial Ġloss Ġthat Ġmaxim ises Ġthe Ġcross - ent ropy Ġof Ġ$\ math cal Ġ{ D } $ ; </s> $ L _{ seq } $, Ġthe Ġauto - enc oder Ġloss Ġto Ġreconstruct Ġ$ X ^ {\ prime Ġ} $ Ġfrom Ġ$ X ^ {\ prime Ġ} $ ; </s> $ L _{ sem } $, Ġthe Ġcos ine Ġdistance Ġbetween Ġthe Ġmean Ġembed d ings Ġof Ġ$ X $ Ġand Ġ$ X ^ * $. </s> $ L _{ adv } $ Ġensures Ġthat Ġthe Ġadvers arial Ġexamples Ġare Ġfool ing Ġthe Ġclass ifier Ġ( crit erion Ġ( a )); Ġ$ L _{ seq } $ Ġregulates Ġthe Ġadvers arial Ġexample Ġso Ġthat Ġit Ġisn 't Ġtoo Ġdifferent Ġfrom Ġthe Ġoriginal Ġinput Ġand Ġhas Ġa Ġreasonable Ġlanguage Ġ( crit eria Ġ( b ) Ġand Ġ( c )); Ġand Ġ$ L _{ sem } $ Ġconst r ains Ġthe Ġunderlying Ġsemantic Ġcontent Ġof Ġthe Ġoriginal Ġand Ġadvers arial Ġsentences Ġto Ġbe Ġsimilar Ġ( crit erion Ġ( b )); Ġso Ġreduces Ġthe Ġlikelihood Ġof Ġa Ġsentiment Ġflip Ġ( ind irect ly Ġfor Ġcriteria Ġ( d )). ĠNote Ġthat Ġcriteria Ġ( d ) Ġis Ġthe Ġperhaps Ġthe Ġmost Ġdifficult Ġaspect Ġto Ġhandle , Ġas Ġits Ġinterpretation Ġis Ġultimately Ġa Ġhuman Ġjudgement . </s> We Ġuse Ġtwo Ġscaling Ġhyper - param eters Ġ$\ lambda Ġ_ { ae } $ Ġand Ġ$\ lambda Ġ_ { seq } $ Ġto Ġweigh Ġthe Ġthree Ġobjectives : Ġ$\ lambda Ġ_ { ae } (\ lambda Ġ_ { seq } Ġ* ĠL _{ seq } Ġ+ Ġ( 1 - \ lambda Ġ_ { seq } )* L _{ sem }) Ġ+ Ġ( 1 - \ lambda Ġ_ { ae } )* L _{ adv } $. ĠWe Ġintroduce Ġattention Ġlayers Ġto ĠAuto Enc oder Ġand Ġtest Ġboth Ġgreedy Ġand Ġbeam Ġsearch Ġdecoding Ġto Ġimprove Ġthe Ġquality Ġof Ġgenerated Ġadvers arial Ġexamples . ĠDuring Ġtraining Ġwe Ġalternate Ġbetween Ġpositive - to - negative Ġand Ġnegative - to - positive Ġattack Ġfor Ġdifferent Ġmini - bat ches . </s> Exper iments Ġ: :: ĠDat as ets </s> We Ġconstruct Ġthree Ġdatasets Ġbased Ġon ĠIM DB Ġreviews Ġand ĠYelp Ġreviews . ĠThe ĠIM DB Ġdataset Ġis Ġbin ar ised Ġand Ġsplit Ġinto Ġa Ġtraining Ġand Ġtest Ġset , Ġeach Ġwith Ġ25 K Ġreviews Ġ( 2 K Ġreviews Ġfrom Ġthe Ġtraining Ġset Ġare Ġreserved Ġfor Ġdevelopment ). ĠWe Ġfilter Ġout Ġany Ġreview Ġthat Ġhas Ġmore Ġthan Ġ400 Ġtokens , Ġproducing Ġthe Ġfinal Ġdataset Ġ( im db 400 ). ĠFor ĠYelp , Ġwe Ġbin ar ise Ġthe Ġratings , Ġand Ġcreate Ġ2 Ġdatasets , Ġwhere Ġwe Ġkeep Ġonly Ġreviews Ġwith Ġ$\ le Ġ$ Ġ50 Ġtokens Ġ( y el p 50 ) Ġand Ġ$\ le Ġ$ 200 Ġtokens Ġ( y el p 200 ). ĠWe Ġrandomly Ġpartition Ġboth Ġdatasets Ġinto Ġtrain / dev / test Ġsets Ġ( 90 / 5 / 5 Ġfor Ġy el p 50 ; Ġ99 / 0 . 5 / 0 . 5 Ġfor Ġy el p 200 ). ĠFor Ġall Ġdatasets , Ġwe Ġuse Ġspa Cy Ġfor Ġtoken isation . ĠWe Ġtrain Ġand Ġtune Ġtarget Ġclass ifiers Ġ( see ĠSection ĠSEC REF 8 ) Ġusing Ġthe Ġtraining Ġand Ġdevelopment Ġsets ; Ġand Ġevaluate Ġtheir Ġperformance Ġon Ġthe Ġoriginal Ġexamples Ġin Ġthe Ġtest Ġsets Ġas Ġwell Ġas Ġthe Ġadvers arial Ġexamples Ġgenerated Ġby Ġattacking Ġmethods Ġfor Ġthe Ġtest Ġsets . ĠNote Ġthat ĠAuto Enc oder Ġalso Ġinvolves Ġa Ġtraining Ġprocess , Ġfor Ġwhich Ġwe Ġtrain Ġand Ġtune ĠAuto Enc oder Ġusing Ġthe Ġtraining Ġand Ġdevelopment Ġsets Ġin Ġy el p 50 , Ġy el p 200 Ġand Ġim db 400 . ĠStatistics Ġof Ġthe Ġthree Ġdatasets Ġare Ġpresented Ġin ĠTable ĠT AB REF 22 . ĠThese Ġdatasets Ġpresent Ġa Ġvariation Ġin Ġthe Ġtext Ġlengths Ġ( e . g . Ġthe Ġaverage Ġnumber Ġof Ġwords Ġfor Ġy el p 50 , Ġy el p 200 Ġand Ġim db 400 Ġis Ġ34 , Ġ82 Ġand Ġ195 Ġwords Ġrespectively ), Ġtraining Ġdata Ġsize Ġ( e . g . Ġthe Ġnumber Ġof Ġtraining Ġexamples Ġfor Ġtarget Ġclass ifiers Ġfor Ġim db 400 , Ġy el p 50 Ġand Ġy el p 200 Ġare Ġ18 K , Ġ407 K Ġand Ġ2 M , Ġrespectively ) Ġand Ġinput Ġdomain Ġ( e . g . Ġrestaurant Ġvs . Ġmovie Ġreviews ). </s> Exper iments Ġ: :: ĠImplementation ĠDetails </s> We Ġuse Ġthe Ġpre - trained Ġglove . 840 B . 300 d Ġembed d ings ĠB IB REF 20 Ġfor Ġall Ġ6 Ġattacking Ġmethods . ĠFor ĠF GM , ĠFG VM Ġand ĠDeep F ool , Ġwe Ġtune Ġ$\ ep sil on Ġ$ , Ġthe Ġover shoot Ġhyper - param eter Ġ( Section ĠSEC REF 9 ) Ġand Ġkeep Ġthe Ġiter ative Ġstep Ġ$ n $ Ġstatic Ġ( 5 ). ĠFor ĠTY C , Ġbesides Ġ$\ ep sil on Ġ$ Ġwe Ġalso Ġtune Ġthe Ġupper Ġlimit Ġof Ġflipped Ġwords , Ġranging Ġfrom Ġ10 % âĢĵ 100 % Ġof Ġthe Ġmaximum Ġlength . ĠFor ĠHot Fl ip , Ġwe Ġtune Ġonly Ġthe Ġupper Ġlimit Ġof Ġflipped Ġwords , Ġin Ġthe Ġrange Ġof Ġ$ [ 1 , Ġ7 ] $. </s> We Ġpre - train ĠAuto Enc oder Ġto Ġreconstruct Ġsentences Ġin Ġdifferent Ġdatasets Ġas Ġwe Ġfound Ġthat Ġthis Ġimproves Ġthe Ġquality Ġof Ġthe Ġgenerated Ġadvers arial Ġexamples . ĠDuring Ġpre - training , Ġwe Ġtune Ġbatch Ġsize , Ġnumber Ġof Ġlayers Ġand Ġnumber Ġof Ġunits , Ġand Ġstop Ġthe Ġtraining Ġafter Ġthe Ġperformance Ġon Ġthe Ġdevelopment Ġsets Ġstops Ġimproving Ġfor Ġ20 K Ġsteps . ĠThe Ġmodel Ġis Ġthen Ġinitial ised Ġwith Ġthe Ġpre - trained Ġweights Ġand Ġtrained Ġbased Ġon Ġobjectives Ġdefined Ġin ĠSection ĠSEC REF 11 . ĠDuring Ġthe Ġtraining Ġprocess Ġwe Ġtune Ġ$\ lambda Ġ_ { ae } $ Ġand Ġ$\ lambda Ġ_ { seq } $ Ġwhile Ġkeeping Ġthe Ġbatch Ġsize Ġ( 32 ) Ġand Ġlearning Ġrate Ġ($ 1 e ^{ - 4 } $ ) Ġfixed . ĠAs Ġpart Ġof Ġour Ġpreliminary Ġstudy , Ġwe Ġalso Ġtested Ġdifferent Ġvalues Ġfor Ġthe ĠG umb el - soft max Ġtemperature Ġ$\ t au Ġ$ Ġand Ġfind Ġthat Ġ$\ t au Ġ= 0 . 1 $ Ġperforms Ġthe Ġbest . ĠEmb edd ings Ġare Ġfixed Ġthroughout Ġall Ġtraining Ġprocesses . </s> For Ġtarget Ġclass ifiers , Ġwe Ġtune Ġbatch Ġsize , Ġlearning Ġrate , Ġnumber Ġof Ġlayers , Ġnumber Ġof Ġunits , Ġattention Ġsize Ġ( Bi L ST M $ + $ A ), Ġfilter Ġsizes Ġand Ġdrop out Ġprobability Ġ( CNN ). ĠFor ĠB ERT , Ġwe Ġuse Ġthe Ġdefault Ġfine - tun ing Ġhyper - param eter Ġvalues Ġexcept Ġfor Ġbatch Ġsize , Ġwhere Ġwe Ġadjust Ġbased Ġon Ġmemory Ġconsumption . ĠNote Ġthat Ġafter Ġthe Ġtarget Ġclass ifiers Ġare Ġtrained Ġtheir Ġweights Ġare Ġnot Ġupdated Ġwhen Ġtraining Ġor Ġtesting Ġthe Ġattacking Ġmethods . </s> E val uation </s> We Ġpropose Ġboth Ġautomatic Ġmetrics Ġand Ġmanual Ġevaluation Ġstrategies Ġto Ġassess Ġthe Ġquality Ġof Ġadvers arial Ġexamples , Ġbased Ġon Ġfour Ġcriteria Ġdefined Ġin ĠSection ĠSEC REF 1 : Ġ( a ) Ġattacking Ġperformance Ġ( i . e . Ġhow Ġwell Ġthey Ġfool Ġthe Ġclass ifier ); Ġ( b ) Ġtextual Ġsimilarity Ġbetween Ġthe Ġoriginal Ġinput Ġand Ġthe Ġadvers arial Ġinput ; Ġ( c ) Ġflu ency Ġof Ġthe Ġadvers arial Ġexample ; Ġand Ġ( d ) Ġlabel Ġpreservation . ĠNote Ġthat Ġthe Ġautomatic Ġmetrics Ġonly Ġaddress Ġthe Ġfirst Ġ3 Ġcriteria Ġ( a , Ġb Ġand Ġc ); Ġwe Ġcontend Ġthat Ġcriterion Ġ( d ) Ġrequires Ġmanual Ġevaluation , Ġas Ġthe Ġjudgement Ġof Ġwhether Ġthe Ġoriginal Ġlabel Ġis Ġpreserved Ġis Ġinherently Ġa Ġhuman Ġdecision . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation : ĠMet rics </s> As Ġsentiment Ġclassification Ġis Ġour Ġtarget Ġtask , Ġwe Ġuse Ġthe Ġstandard Ġclassification Ġaccuracy Ġ( ACC ) Ġto Ġevaluate Ġthe Ġattacking Ġperformance Ġof Ġadvers arial Ġexamples Ġ( crit erion Ġ( a )). </s> To Ġassess Ġthe Ġsimilarity Ġbetween Ġthe Ġoriginal Ġand Ġ( trans formed ) Ġadvers arial Ġexamples Ġ( crit eria Ġ( b )), Ġwe Ġcompute ĠB LE U Ġscores ĠB IB REF 21 . </s> To Ġmeasure Ġflu ency , Ġwe Ġfirst Ġexplore Ġa Ġsupervised ĠB ERT Ġmodel Ġfine - tun ed Ġto Ġpredict Ġlinguistic Ġaccept ability ĠB IB REF 17 . ĠHowever , Ġin Ġpreliminary Ġexperiments Ġwe Ġfound Ġthat ĠB ERT Ġperforms Ġvery Ġpoorly Ġat Ġpredicting Ġthe Ġaccept ability Ġof Ġadvers arial Ġexamples Ġ( e . g . Ġit Ġpredicts Ġword - sal ad - like Ġsentences Ġgenerated Ġby ĠFG VM Ġas Ġvery Ġacceptable ), Ġrevealing Ġthe Ġbr itt leness Ġof Ġthese Ġsupervised Ġmodels . ĠWe Ġnext Ġexplore Ġan Ġun super vised Ġapproach ĠB IB REF 22 , ĠB IB REF 23 , Ġusing Ġnormal ised Ġsentence Ġprobabilities Ġestimated Ġby Ġpre - trained Ġlanguage Ġmodels Ġfor Ġmeasuring Ġaccept ability . ĠIn Ġthe Ġoriginal Ġpapers , Ġthe Ġauthors Ġtested Ġsimple Ġrecurrent Ġlanguage Ġmodels ; Ġhere Ġwe Ġuse Ġmodern Ġpre - trained Ġlanguage Ġmodels Ġsuch Ġas ĠG PT - 2 ĠB IB REF 24 Ġand ĠXL Net ĠB IB REF 18 . ĠOur Ġfinal Ġaccept ability Ġmetric Ġ( AC PT ) Ġis Ġbased Ġon Ġnormal ised ĠXL Net Ġsentence Ġprobabilities : Ġ${ \ log ĠP ( s )} Ġ/ Ġ({ (( 5 + | s | )/ ( 5 + 1 )) ^ \ alpha Ġ}) $, Ġwhere Ġ$ s $ Ġis Ġthe Ġsentence , Ġand Ġ$\ alpha Ġ$ Ġis Ġa Ġhyper - param eter Ġ( set Ġto Ġ0 . 8 ) Ġto Ġdamp en Ġthe Ġimpact Ġof Ġlarge Ġvalues ĠB IB REF 25 . </s> We Ġonly Ġcomputed ĠB LE U Ġand ĠAC PT Ġscores Ġfor Ġadvers arial Ġexamples Ġthat Ġhave Ġsuccessfully Ġfooled Ġthe Ġclass ifier . ĠOur Ġrationale Ġis Ġthat Ġunsuccessful Ġexamples Ġcan Ġartificially Ġboost Ġthese Ġscores Ġby Ġnot Ġmaking Ġany Ġmodifications , Ġand Ġso Ġthe Ġbetter Ġapproach Ġis Ġto Ġonly Ġconsider Ġsuccessful Ġexamples . </s> E val uation Ġ: :: ĠAutomatic ĠEvaluation : ĠResults </s> We Ġpresent Ġthe Ġperformance Ġof Ġthe Ġattacking Ġmethods Ġagainst Ġ3 Ġtarget Ġclass ifiers Ġ( Table ĠT AB REF 23 A ; Ġtop ) Ġand Ġon Ġ3 Ġdatasets Ġ( Table ĠT AB REF 23 B ; Ġbottom ). ĠWe Ġchoose Ġ3 ĠACC Ġthresholds Ġfor Ġthe Ġattacking Ġperformance : ĠT 0 , ĠT 1 Ġand ĠT 2 , Ġwhich Ġcorrespond Ġapproximately Ġto Ġaccuracy Ġscores Ġof Ġ90 %, Ġ80 % Ġand Ġ70 % Ġfor Ġthe ĠYelp Ġdatasets Ġ( y el p 50 , Ġy el p 200 ); Ġand Ġ80 %, Ġ70 % Ġand Ġ50 % Ġfor Ġthe ĠIM DB Ġdatasets Ġ( im db 400 ). ĠEach Ġmethod Ġis Ġtuned Ġaccordingly Ġto Ġachieve Ġa Ġparticular Ġaccuracy . ĠMissing Ġnumbers Ġ( d ashed Ġlines ) Ġindicate Ġthe Ġmethod Ġis Ġunable Ġto Ġproduce Ġthe Ġdesired Ġaccuracy , Ġe . g . ĠHot Fl ip Ġwith Ġonly Ġ1 Ġword Ġflip Ġproduces Ġ81 . 5 % Ġaccuracy Ġ( T 1 ) Ġwhen Ġattacking ĠCNN Ġon Ġy el p 50 , Ġand Ġso ĠT 0 Ġaccuracy Ġis Ġun ach iev able . </s> Looking Ġat ĠB LE U Ġand ĠAC PT , ĠHot Fl ip Ġis Ġthe Ġmost Ġconsistent Ġmethod Ġover Ġmultiple Ġdatasets Ġand Ġclass ifiers . ĠAuto Enc oder Ġis Ġalso Ġfairly Ġcompetitive , Ġproducing Ġlargely Ġcomparable Ġperformance Ġ( except Ġfor Ġthe Ġy el p 200 Ġdataset ). ĠGrad ient - based Ġmethods ĠF GM Ġand ĠFG VM Ġperform Ġvery Ġpoorly . ĠIn Ġgeneral , Ġthey Ġtend Ġto Ġproduce Ġword Ġsalad Ġadvers arial Ġexamples , Ġas Ġindicated Ġby Ġtheir Ġpoor ĠB LE U Ġscores . ĠDeep F ool Ġsimilarly Ġgenerates Ġinco herent Ġsentences Ġwith Ġlow ĠB LE U Ġscores , Ġbut Ġoccasionally Ġproduces Ġgood ĠAC PT Ġ( Bi L ST M Ġat ĠT 1 Ġand ĠT 2 ), Ġsuggesting Ġpotential Ġbr itt leness Ġof Ġthe Ġun super vised Ġapproach Ġfor Ġevaluating Ġaccept ability . </s> Comp aring Ġthe Ġperformance Ġacross Ġdifferent ĠACC Ġthresholds , Ġwe Ġobserve Ġa Ġconsistent Ġpattern Ġof Ġdecreasing Ġperformance Ġover Ġall Ġmetrics Ġas Ġthe Ġattacking Ġperformance Ġincreases Ġfrom ĠT 0 Ġto ĠT 2 . ĠThese Ġobservations Ġsuggest Ġthat Ġall Ġmethods Ġare Ġtrading Ġoff Ġflu ency Ġand Ġcontent Ġpreservation Ġas Ġthey Ġattempt Ġto Ġgenerate Ġstronger Ġadvers arial Ġexamples . </s> We Ġnow Ġfocus Ġon ĠTable ĠT AB REF 23 A , Ġto Ġunderstand Ġthe Ġimpact Ġof Ġmodel Ġarchitecture Ġfor Ġthe Ġtarget Ġclass ifier . ĠWith Ġ1 Ġword Ġflip Ġas Ġthe Ġupper Ġlimit Ġfor ĠHot Fl ip , Ġthe Ġaccuracy Ġof ĠBi L ST M $ + $ A Ġand ĠBi L ST M Ġdrops Ġto ĠT 0 Ġ( approximately Ġ4 % Ġaccuracy Ġdecrease ) Ġwhile Ġthe Ġaccuracy Ġof ĠCNN Ġdrops Ġto ĠT 1 Ġ( approximately Ġ13 % Ġaccuracy Ġdecrease ), Ġsuggesting Ġthat Ġconv olution al Ġnetworks Ġare Ġmore Ġvulnerable Ġto Ġattacks Ġ( not ing Ġthat Ġit Ġis Ġthe Ġpredominant Ġarchitecture Ġfor ĠCV ). ĠLooking Ġat ĠTable ĠT AB REF 23 B , Ġwe Ġalso Ġfind Ġthat Ġthe Ġattacking Ġperformance Ġis Ġinfluenced Ġby Ġthe Ġinput Ġtext Ġlength Ġand Ġthe Ġnumber Ġof Ġtraining Ġexamples Ġfor Ġtarget Ġclass ifiers . ĠFor ĠHot Fl ip , Ġwe Ġsee Ġimprovements Ġover ĠB LE U Ġand ĠAC PT Ġas Ġtext Ġlength Ġincreases Ġfrom Ġy el p 50 Ġto Ġy el p 200 Ġto Ġim db 400 , Ġindicating Ġthe Ġperformance Ġof ĠHot Fl ip Ġis Ġmore Ġaffected Ġby Ġinput Ġlengths . ĠWe Ġthink Ġthis Ġis Ġbecause Ġwith Ġmore Ġwords Ġit Ġis Ġmore Ġlikely Ġfor ĠHot Fl ip Ġto Ġfind Ġa Ġvulnerable Ġspot Ġto Ġtarget . ĠWhile Ġfor ĠTY C Ġand ĠAuto Enc oder , Ġwe Ġsee Ġimprovements Ġover ĠB LE U Ġand ĠAC PT Ġas Ġthe Ġnumber Ġof Ġtraining Ġexamples Ġfor Ġtarget Ġclass ifier Ġdecreases Ġfrom Ġy el p 200 Ġ( 2 M ) Ġto Ġy el p 50 Ġ( 407 K ) Ġto Ġim db 400 Ġ( 22 K ), Ġindicating Ġthese Ġtwo Ġmethods Ġare Ġless Ġeffective Ġfor Ġattacking Ġtarget Ġclass ifiers Ġthat Ġare Ġtrained Ġwith Ġmore Ġdata . ĠTherefore , Ġincreasing Ġthe Ġtraining Ġdata Ġfor Ġtarget Ġclass ifier Ġmay Ġimprove Ġtheir Ġrobust ness Ġagainst Ġadvers arial Ġexamples Ġthat Ġare Ġgenerated Ġby Ġcertain Ġmethods . </s> As Ġa Ġblack - box Ġtest Ġto Ġcheck Ġhow Ġwell Ġthese Ġadvers arial Ġexamples Ġgeneral ise Ġto Ġfool ing Ġother Ġclass ifiers , Ġalso Ġknown Ġas Ġtransfer ability , Ġwe Ġfeed Ġthe Ġadvers arial Ġexamples Ġfrom Ġthe Ġ3 Ġbest Ġmethods , Ġi . e . ĠTY C , ĠHot Fl ip Ġand ĠAuto Enc oder , Ġto Ġa Ġpre - trained ĠB ERT Ġtrained Ġfor Ġsentiment Ġclassification Ġ( Figure ĠFIG REF 31 ) Ġand Ġmeasure Ġits Ġaccuracy . ĠUn surprisingly , Ġwe Ġobserve Ġthat Ġthe Ġattacking Ġperformance Ġ( i . e . Ġthe Ġdrop Ġin ĠACC ) Ġis Ġnot Ġas Ġgood Ġas Ġthose Ġreported Ġfor Ġthe Ġwhite - box Ġtests . ĠInterestingly , Ġwe Ġfind Ġthat ĠHot Fl ip , Ġthe Ġbest Ġmethod , Ġproduces Ġthe Ġleast Ġeffective Ġadvers arial Ġexamples Ġfor ĠB ERT . ĠBoth ĠTY C Ġand ĠAuto Enc oder Ġperform Ġbetter Ġhere , Ġas Ġtheir Ġgenerated Ġadvers arial Ġexamples Ġdo Ġwell Ġin Ġfool ing ĠB ERT . </s> To Ġsummar ise , Ġour Ġresults Ġdemonstrate Ġthat Ġthe Ġbest Ġwhite - box Ġmethods Ġ( e . g . ĠHot Fl ip ) Ġmay Ġnot Ġproduce Ġadvers arial Ġexamples Ġthat Ġgeneral ise Ġto Ġfool ing Ġother Ġclass ifiers . ĠWe Ġalso Ġsaw Ġthat Ġconv olution al Ġnetworks Ġare Ġmore Ġvulnerable Ġthan Ġrecurrent Ġnetworks , Ġand Ġthat Ġdataset Ġfeatures Ġsuch Ġas Ġtext Ġlengths Ġand Ġtraining Ġdata Ġsize Ġ( for Ġtarget Ġclass ifiers ) Ġcan Ġinfluence Ġhow Ġdifficult Ġit Ġis Ġto Ġperform Ġadvers arial Ġattack . </s> E val uation Ġ: :: ĠHuman ĠEvaluation : ĠDesign </s> Aut omatic Ġmetrics Ġprovide Ġa Ġproxy Ġto Ġquantify Ġthe Ġquality Ġof Ġthe Ġadvers arial Ġexamples . ĠTo Ġvalidate Ġthat Ġthese Ġmetrics Ġwork , Ġwe Ġconduct Ġa Ġcrowds ourcing Ġexperiment Ġon ĠFigure - Eight . ĠRecall Ġthat Ġthe Ġautomatic Ġmetrics Ġdo Ġnot Ġassess Ġsentiment Ġpreservation Ġ( crit erion Ġ( d )); Ġwe Ġevaluate Ġthat Ġaspect Ġhere . </s> We Ġexperiment Ġwith Ġthe Ġ3 Ġbest Ġmethods Ġ( TY C , ĠAuto Enc oder Ġand ĠHot Fl ip ) Ġon Ġ2 Ġaccuracy Ġthresholds Ġ( T 0 Ġand ĠT 2 ), Ġusing ĠBi L ST M $ + $ A Ġas Ġthe Ġclass ifier . ĠFor Ġeach Ġmethod Ġand Ġthreshold , Ġwe Ġrandomly Ġsample Ġ25 Ġpositive - to - negative Ġand Ġ25 Ġnegative - to - positive Ġexamples . ĠTo Ġcontrol Ġfor Ġquality , Ġwe Ġreserve Ġand Ġannot ate Ġ10 % Ġof Ġthe Ġsamples Ġourselves Ġas Ġcontrol Ġquestions . ĠWorkers Ġare Ġfirst Ġpresented Ġwith Ġ10 Ġcontrol Ġquestions Ġas Ġa Ġquiz , Ġand Ġonly Ġthose Ġwho Ġpass Ġthe Ġquiz Ġwith Ġat Ġleast Ġ80 % Ġaccuracy Ġcan Ġcontinue Ġto Ġwork Ġon Ġthe Ġtask . ĠWe Ġdisplay Ġ10 Ġquestions Ġper Ġpage , Ġwhere Ġone Ġcontrol Ġquestion Ġis Ġembedded Ġto Ġcontinue Ġmonitor Ġworker Ġperformance . ĠThe Ġtask Ġis Ġdesigned Ġsuch Ġthat Ġeach Ġcontrol Ġquestion Ġcan Ġonly Ġbe Ġseen Ġonce Ġper Ġworker . ĠWe Ġrestrict Ġour Ġjobs Ġto Ġworkers Ġin ĠUnited ĠStates , ĠUnited ĠKingdoms , ĠAustralia , Ġand ĠCanada . </s> To Ġevaluate Ġthe Ġcriteria Ġdiscussed Ġin ĠSection ĠSEC REF 1 : Ġ( b ) Ġtextual Ġsimilarity , Ġ( c ) Ġflu ency , Ġand Ġ( d ) Ġsentiment Ġpreservation , Ġwe Ġask Ġthe Ġannot ators Ġthree Ġquestions : </s> Is Ġsnippet ĠB Ġa Ġgood Ġparaph rase Ġof Ġsnippet ĠA ? </s> $ \ c ir cled circ Ġ$ ĠYes Ġ$\ c ir cled circ Ġ$ ĠSomew hat Ġyes Ġ$\ c ir cled circ Ġ$ ĠNo </s> How Ġnatural Ġdoes Ġthe Ġtext Ġread ? </s> $ \ c ir cled circ Ġ$ ĠVery Ġunnatural Ġ$\ c ir cled circ Ġ$ ĠSomew hat Ġnatural Ġ$\ c ir cled circ Ġ$ ĠNatural </s> What Ġis Ġthe Ġsentiment Ġof Ġthe Ġtext ? </s> $ \ c ir cled circ Ġ$ ĠPositive Ġ$\ c ir cled circ Ġ$ ĠNegative Ġ$\ c ir cled circ Ġ$ ĠCannot Ġtell </s> For Ġquestion Ġ1 , Ġwe Ġdisplay Ġboth Ġthe Ġadvers arial Ġinput Ġand Ġthe Ġoriginal Ġinput , Ġwhile Ġfor Ġquestion Ġ2 Ġand Ġ3 Ġwe Ġpresent Ġonly Ġthe Ġadvers arial Ġexample . ĠAs Ġan Ġupper - bound , Ġwe Ġalso Ġrun Ġa Ġsurvey Ġon Ġquestion Ġ2 Ġand Ġ3 Ġfor Ġ50 Ġrandom Ġoriginal Ġsamples . </s> E val uation Ġ: :: ĠHuman ĠEvaluation : ĠResults </s> We Ġpresent Ġthe Ġpercentage Ġof Ġanswers Ġto Ġeach Ġquestion Ġin ĠFigure ĠFIG REF 38 . ĠThe Ġgreen Ġbars Ġillustrate Ġhow Ġwell Ġthe Ġadvers arial Ġexamples Ġparaph rase Ġthe Ġoriginal Ġones ; Ġblue Ġhow Ġnatural Ġthe Ġadvers arial Ġexamples Ġread ; Ġand Ġred Ġwhether Ġthe Ġsentiment Ġof Ġthe Ġadvers arial Ġexamples Ġis Ġconsistent Ġcompared Ġto Ġthe Ġoriginal . </s> Looking Ġat Ġthe Ġperformance Ġof Ġthe Ġoriginal Ġsentences Ġ( âĢ ľ ( a ) ĠOriginal Ġsamples âĢ Ŀ ), Ġwe Ġsee Ġthat Ġtheir Ġlanguage Ġis Ġlargely Ġfluent Ġand Ġtheir Ġsentiment Ġis Ġgenerally Ġconsistent Ġto Ġthe Ġoriginal Ġexamples ', Ġalthough Ġit 's Ġworth Ġnoting Ġthat Ġthe Ġreview Ġsentiment Ġof Ġim db 400 Ġcan Ġbe Ġsomewhat Ġambiguous Ġ( 63 % Ġagreement ). ĠWe Ġthink Ġthis Ġis Ġdue Ġto Ġmovie Ġreviews Ġbeing Ġmore Ġdescriptive Ġand Ġtherefore Ġcreating Ġpotential Ġambiguity Ġin Ġtheir Ġexpression Ġof Ġsentiment . </s> On Ġcontent Ġpreservation Ġ( crit erion Ġ( b ); Ġgreen Ġbars ), Ġall Ġmethods Ġproduce Ġpoor Ġparaph r ases Ġon Ġy el p 50 . ĠFor Ġim db 400 , Ġhowever , Ġthe Ġresults Ġare Ġmore Ġpromising ; Ġadvers arial Ġexamples Ġgenerated Ġby ĠHot Fl ip , Ġin Ġparticular , Ġare Ġgood , Ġeven Ġat ĠT 2 . </s> Next Ġwe Ġlook Ġat Ġflu ency Ġ( crit erion Ġ( c ); Ġblue Ġbars ). ĠWe Ġsee Ġa Ġsimilar Ġtrend : Ġperformance Ġin Ġim db 400 Ġis Ġsubstantially Ġbetter Ġthan Ġy el p 50 . ĠIn Ġfact Ġwe Ġsee Ġalmost Ġno Ġdecrease Ġin Ġflu ency Ġin Ġthe Ġadvers arial Ġexamples Ġcompared Ġto Ġthe Ġoriginal Ġin Ġim db 400 . ĠIn Ġy el p 50 , ĠHot Fl ip Ġand ĠAuto Enc oder Ġare Ġfairly Ġcompetitive , Ġproducing Ġadvers arial Ġexamples Ġthat Ġare Ġonly Ġmarginally Ġless Ġfluent Ġcompared Ġto Ġthe Ġoriginal Ġat ĠT 0 . ĠAt ĠT 2 , Ġhowever , Ġthese Ġmethods Ġbegin Ġto Ġtrade Ġoff Ġflu ency . ĠAll Ġin Ġall , Ġthe Ġparaph ras ability Ġand Ġflu ency Ġsurveys Ġsuggest Ġthat Ġim db 400 Ġis Ġan Ġeasier Ġdataset Ġfor Ġadvers arial Ġexperiments , Ġand Ġit Ġis Ġthe Ġpredominant Ġdataset Ġused Ġby Ġmost Ġstudies . </s> Lastly , Ġwe Ġconsider Ġsentiment Ġpreservation Ġ( crit erion Ġ( d ); Ġred Ġbars ). ĠAll Ġmethods Ġperform Ġpoorly Ġat Ġpreserving Ġthe Ġoriginal Ġsentiment Ġon Ġboth Ġy el p 50 Ġand Ġim db 400 Ġdatasets . ĠThe Ġartifact Ġis Ġarguably Ġmore Ġprofound Ġon Ġshorter Ġinputs , Ġas Ġthe Ġoriginal Ġexamples Ġin Ġim db 400 Ġhave Ġlower Ġagreement Ġin Ġthe Ġfirst Ġplace Ġ( y el p 50 Ġvs . Ġim db 400 : Ġ86 % Ġvs . Ġ63 %). ĠAgain Ġboth ĠHot Fl ip Ġand ĠAuto Enc oder Ġare Ġthe Ġbetter Ġmethods Ġhere Ġ( interest ingly , Ġwe Ġobserve Ġan Ġincrease Ġin Ġagreement Ġas Ġtheir Ġattacking Ġperformance Ġincreases Ġfrom ĠT 0 Ġand ĠT 2 ). </s> Sum mar ising Ġour Ġfindings , ĠHot Fl ip Ġis Ġgenerally Ġthe Ġbest Ġmethod Ġacross Ġall Ġcriteria , Ġnoting Ġthat Ġits Ġadvers arial Ġexamples , Ġhowever , Ġhave Ġpoor Ġtransfer ability . ĠTY C Ġgenerates Ġgood Ġblack - box Ġadvers arial Ġexamples Ġbut Ġdo Ġnot Ġdo Ġwell Ġin Ġterms Ġof Ġcontent Ġpreservation Ġand Ġflu ency . ĠAuto Enc oder Ġproduces Ġcomparable Ġresults Ġwith ĠHot Fl ip Ġfor Ġmeeting Ġthe Ġfour Ġcriteria Ġand Ġgenerates Ġexamples Ġthat Ġgeneral ise Ġreasonably , Ġbut Ġit Ġis Ġvery Ġsensitive Ġto Ġthe Ġincrease Ġof Ġtraining Ġexamples Ġfor Ġthe Ġtarget Ġclass ifier . ĠThe ĠAC PT Ġmetric Ġappears Ġto Ġbe Ġeffective Ġin Ġevaluating Ġflu ency , Ġas Ġwe Ġsee Ġgood Ġagreement Ġwith Ġhuman Ġevaluation . ĠAll Ġsaid , Ġwe Ġfound Ġthat Ġall Ġmethods Ġtend Ġto Ġproduce Ġadvers arial Ġexamples Ġthat Ġdo Ġnot Ġpreserve Ġtheir Ġoriginal Ġsentiments , Ġrevealing Ġthat Ġthese Ġmethods Ġin Ġa Ġway ĠâĢ ľ cheat âĢ Ŀ Ġby Ġsimply Ġflipping Ġthe Ġsentiments Ġof Ġthe Ġoriginal Ġsentences Ġto Ġfool Ġthe Ġclass ifier , Ġand Ġtherefore Ġthe Ġadvers arial Ġexamples Ġmight Ġbe Ġineffective Ġfor Ġadvers arial Ġtraining , Ġas Ġthey Ġare Ġnot Ġexamples Ġthat Ġreveal Ġpotential Ġvulnerabilities Ġin Ġthe Ġclass ifier . </s> Conclusion </s> We Ġpropose Ġan Ġevaluation Ġframework Ġfor Ġassessing Ġthe Ġquality Ġof Ġadvers arial Ġexamples Ġin ĠN LP , Ġbased Ġon Ġfour Ġcriteria : Ġ( a ) Ġattacking Ġperformance , Ġ( b ) Ġtextual Ġsimilarity ; Ġ( c ) Ġflu ency ; Ġ( d ) Ġlabel Ġpreservation . ĠOur Ġframework Ġinvolves Ġboth Ġautomatic Ġand Ġhuman Ġevaluation , Ġand Ġwe Ġtest Ġ5 Ġbenchmark Ġmethods Ġand Ġa Ġnovel Ġauto - enc oder Ġapproach . ĠWe Ġfound Ġthat Ġthe Ġarchitecture Ġof Ġthe Ġtarget Ġclass ifier Ġis Ġan Ġimportant Ġfactor Ġwhen Ġit Ġcomes Ġto Ġattacking Ġperformance , Ġe . g . ĠCNN s Ġare Ġmore Ġvulnerable Ġthan ĠL ST Ms ; Ġdataset Ġfeatures Ġsuch Ġas Ġlength Ġof Ġtext , Ġtraining Ġdata Ġsize Ġ( for Ġtarget Ġclass ifiers ) Ġand Ġinput Ġdomains Ġare Ġalso Ġinfluencing Ġfactors Ġthat Ġaffect Ġhow Ġdifficulty Ġit Ġis Ġto Ġperform Ġadvers arial Ġattack ; Ġand Ġthe Ġpredominant Ġdataset Ġ( IM DB ) Ġused Ġby Ġmost Ġstudies Ġis Ġcomparatively Ġeasy Ġfor Ġadvers arial Ġattack . ĠLastly , Ġwe Ġalso Ġobserve Ġin Ġour Ġhuman Ġevaluation Ġthat Ġon Ġshorter Ġtexts Ġ( Y el p ) Ġthese Ġmethods Ġproduce Ġadvers arial Ġexamples Ġthat Ġtend Ġnot Ġto Ġpreserve Ġtheir Ġsemantic Ġcontent Ġand Ġhave Ġlow Ġread ability . ĠMore Ġimportantly , Ġthese Ġmethods Ġalso ĠâĢ ľ cheat âĢ Ŀ Ġby Ġsimply Ġflipping Ġthe Ġsentiment Ġin Ġthe Ġadvers arial Ġexamples , Ġand Ġthis Ġbehaviour Ġis Ġevident Ġon Ġboth Ġdatasets , Ġsuggesting Ġthey Ġcould Ġbe Ġineffective Ġfor Ġadvers arial Ġtraining .