Yes
BOOLEAN
Yes
BOOLEAN
We utilize AWD-LSTM BIBREF20 and TransformerXL BIBREF21 based language models. For model hyperparameters please to refer to Supplementary Section Table TABREF5. We use Adam optimizer BIBREF22 with $\beta 1= 0.7$ and $\beta 2= 0.8$ similar to BIBREF19 and use a batch size of 50. Other practices for LM training were the same as BIBREF21 and BIBREF20 for Transformer-XL and AWD-LSTM respectively. We refer to our proposed GAN as Creative-GAN and compare it to a baseline (a language model equivalent to our pre-trained generator) and a GumbelGAN model BIBREF14 across all proposed datasets. We use three creative English datasets with distinct linguistic characteristics: (1) A corpus of 740 classical and contemporary English poems, (2) a corpus of 14950 metaphor sentences retrieved from a metaphor database website and (3) a corpus of 1500 song lyrics ranging across genres. The mix of linguistic styles within this corpus offers the potential for interesting variation during the generation phase. We use the same pre-processing as in earlier work BIBREF19, BIBREF23. We reserve 10% of our data for test set and another 10% for our validation set.
We utilize AWD-LSTM BIBREF20 and TransformerXL BIBREF21 based language models. For model hyperparameters please to refer to Supplementary Section Table TABREF5. We use Adam optimizer BIBREF22 with $\beta 1= 0.7$ and $\beta 2= 0.8$ similar to BIBREF19 and use a batch size of 50. Other practices for LM training were the same as BIBREF21 and BIBREF20 for Transformer-XL and AWD-LSTM respectively. We refer to our proposed GAN as Creative-GAN and compare it to a baseline (a language model equivalent to our pre-trained generator) and a GumbelGAN model BIBREF14 across all proposed datasets. We use three creative English datasets with distinct linguistic characteristics: (1) A corpus of 740 classical and contemporary English poems, (2) a corpus of 14950 metaphor sentences retrieved from a metaphor database website and (3) a corpus of 1500 song lyrics ranging across genres. The mix of linguistic styles within this corpus offers the potential for interesting variation during the generation phase. We use the same pre-processing as in earlier work BIBREF19, BIBREF23. We reserve 10% of our data for test set and another 10% for our validation set.
1909.09534
Introduction Ġand Ġrelated Ġwork </s> Language Ġmodels Ġcan Ġbe Ġoptimized Ġto Ġrecognize Ġsyntax Ġand Ġsemantics Ġwith Ġgreat Ġaccuracy ĠB IB REF 0 . ĠHowever , Ġthe Ġoutput Ġgenerated Ġcan Ġbe Ġrepetitive Ġand Ġgeneric Ġleading Ġto Ġmon oton ous Ġor Ġun interesting Ġresponses Ġ( e . g ĠâĢ ľ I Ġdon 't Ġknow âĢ Ŀ ) Ġregardless Ġof Ġthe Ġinput ĠB IB REF 1 . ĠWhile Ġapplication Ġof Ġattention ĠB IB REF 2 , ĠB IB REF 3 Ġand Ġadvanced Ġdecoding Ġmechanisms Ġlike Ġbeam Ġsearch Ġand Ġvariation Ġsampling ĠB IB REF 4 Ġhave Ġshown Ġimprovements , Ġit Ġdoes Ġnot Ġsolve Ġthe Ġunderlying Ġproblem . ĠIn Ġcreative Ġtext Ġgeneration , Ġthe Ġobjective Ġis Ġnot Ġstrongly Ġbound Ġto Ġthe Ġground Ġtruth âĢĶ instead Ġthe Ġobjective Ġis Ġto Ġgenerate Ġdiverse , Ġunique Ġor Ġoriginal Ġsamples . ĠWe Ġattempt Ġto Ġdo Ġthis Ġthrough Ġa Ġdiscrim inator Ġwhich Ġcan Ġgive Ġfeedback Ġto Ġthe Ġgener ative Ġmodel Ġthrough Ġa Ġcost Ġfunction Ġthat Ġencourages Ġsampling Ġof Ġcreative Ġtokens . ĠThe Ġcontributions Ġof Ġthis Ġpaper Ġare Ġin Ġthe Ġusage Ġof Ġa ĠG AN Ġframework Ġto Ġgenerate Ġcreative Ġpieces Ġof Ġwriting . ĠOur Ġexperiments Ġsuggest Ġthat Ġgener ative Ġtext Ġmodels , Ġwhile Ġvery Ġgood Ġat Ġencaps ulating Ġsemantic , Ġsynt actic Ġand Ġdomain Ġinformation , Ġperform Ġbetter Ġwith Ġexternal Ġfeedback Ġfrom Ġa Ġdiscrim inator Ġfor Ġfine - tun ing Ġobjective less Ġdecoding Ġtasks Ġlike Ġthat Ġof Ġcreative Ġtext . ĠWe Ġshow Ġthis Ġby Ġevaluating Ġour Ġmodel Ġon Ġthree Ġvery Ġdifferent Ġcreative Ġdatasets Ġcontaining Ġpoetry , Ġmetaphors Ġand Ġlyrics . </s> Previous Ġwork Ġon Ġhandling Ġthe Ġshortcomings Ġof ĠM LE Ġinclude Ġlength - normal izing Ġsentence Ġprobability ĠB IB REF 5 , Ġfuture Ġcost Ġestimation ĠB IB REF 6 , Ġdiversity - boost ing Ġobjective Ġfunction ĠB IB REF 7 , ĠB IB REF 1 Ġor Ġpenal izing Ġrepeating Ġtokens ĠB IB REF 8 . ĠWhen Ġit Ġcomes Ġto Ġpoetry Ġgeneration Ġusing Ġgener ative Ġtext Ġmodels , ĠZhang Ġand ĠLap ata ĠB IB REF 9 , ĠYi Ġet Ġal . ĠB IB REF 10 Ġand ĠWang Ġet Ġal . ĠB IB REF 11 Ġuse Ġlanguage Ġmodeling Ġto Ġgenerate ĠChinese Ġpoems . ĠHowever , Ġnone Ġof Ġthese Ġmethods Ġprovide Ġfeedback Ġon Ġthe Ġquality Ġof Ġthe Ġgenerated Ġsample Ġand Ġhence , Ġdo Ġnot Ġaddress Ġthe Ġqualitative Ġobjective Ġrequired Ġfor Ġcreative Ġdecoding . ĠFor Ġthe Ġtask Ġof Ġtext Ġgeneration , ĠMask GAN ĠB IB REF 12 Ġuses Ġa ĠRein forcement ĠLearning Ġsignal Ġfrom Ġthe Ġdiscrim inator , ĠF MD - GAN ĠB IB REF 13 Ġuses Ġan Ġoptimal Ġtransport Ġmechanism Ġas Ġan Ġobjective Ġfunction . ĠG umb el GAN ĠB IB REF 14 Ġuses ĠG umb el - Soft max Ġdistribution Ġthat Ġreplaces Ġthe Ġnon - different iable Ġsample Ġfrom Ġa Ġcateg orical Ġdistribution Ġwith Ġa Ġdifferent iable Ġsample Ġto Ġpropagate Ġstronger Ġgrad ients . ĠLi Ġet Ġal . ĠB IB REF 1 Ġuse Ġa Ġdiscrim inator Ġfor Ġa Ġdiversity Ġpromoting Ġobjective . ĠYu Ġet Ġal . ĠB IB REF 15 Ġuse ĠSe q GAN Ġto Ġgenerate Ġpoetry Ġand Ġcomment Ġon Ġthe Ġperformance Ġof ĠSe q GAN Ġover ĠM LE Ġin Ġhuman Ġevaluations , Ġencouraging Ġour Ġstudy Ġof ĠG AN s Ġfor Ġcreative Ġtext Ġgeneration . ĠHowever , Ġthese Ġstudies Ġdo Ġnot Ġfocus Ġsolely Ġon Ġcreative Ġtext . </s> GAN s Ġfor Ġcreative Ġtext Ġgeneration </s> Using ĠG AN s , Ġwe Ġcan Ġtrain Ġgener ative Ġmodels Ġin Ġa Ġtwo - player Ġgame Ġsetting Ġbetween Ġa Ġdiscrim inator Ġand Ġa Ġgenerator , Ġwhere Ġthe Ġdiscrim inator Ġ( a Ġbinary Ġclass ifier ) Ġlearns Ġto Ġdistinguish Ġbetween Ġreal Ġand Ġfake Ġdata Ġsamples Ġand Ġthe Ġgenerator Ġtries Ġto Ġfool Ġthe Ġdiscrim inator Ġby Ġgenerating Ġauthentic Ġand Ġhigh Ġquality Ġoutput ĠB IB REF 16 . ĠG AN s Ġhave Ġshown Ġto Ġbe Ġsuccessful Ġin Ġimage Ġgeneration Ġtasks ĠB IB REF 17 Ġand Ġrecently , Ġsome Ġprogress Ġhas Ġbeen Ġobserved Ġin Ġtext Ġgeneration ĠB IB REF 13 , ĠB IB REF 12 , ĠB IB REF 15 . ĠOur Ġgenerator Ġis Ġa Ġlanguage Ġmodel Ġtrained Ġusing Ġback prop ag ation Ġthrough Ġtime ĠB IB REF 18 . ĠDuring Ġthe Ġpre - training Ġphase Ġwe Ġoptimize Ġfor ĠM LE Ġand Ġduring Ġthe ĠG AN Ġtraining Ġphase , Ġwe Ġoptimize Ġon Ġthe Ġcreativity Ġreward Ġfrom Ġthe Ġdiscrim inator . ĠThe Ġdiscrim inator 's Ġenc oder Ġhas Ġthe Ġsame Ġarchitecture Ġas Ġthe Ġgenerator Ġenc oder Ġmodule Ġwith Ġthe Ġaddition Ġof Ġa Ġpooled Ġdec oder Ġlayer . ĠThe Ġdec oder Ġcontains Ġ3 Ġ$ [ D ense ĠB atch ĠNormal ization , Re LU ] $ Ġblocks Ġand Ġan Ġadd t ional Ġ$ S igm oid $ Ġlayer . ĠThe Ġdiscrim inator Ġdec oder Ġtakes Ġthe Ġhidden Ġstate Ġat Ġthe Ġlast Ġtime Ġstep Ġof Ġa Ġsequence Ġconc aten ated Ġwith Ġboth Ġthe Ġmax - pool ed Ġand Ġmean - pool ed Ġrepresentation Ġof Ġthe Ġhidden Ġstates ĠB IB REF 19 Ġand Ġoutputs Ġa Ġnumber Ġin Ġthe Ġrange Ġ$ [ 0 , 1 ] $. ĠThe Ġdifficulty Ġof Ġusing ĠG AN s Ġin Ġtext Ġgeneration Ġcomes Ġfrom Ġthe Ġdiscrete Ġnature Ġof Ġtext , Ġmaking Ġthe Ġmodel Ġnon - different iable Ġhence , Ġwe Ġupdate Ġparameters Ġfor Ġthe Ġgenerator Ġmodel Ġwith Ġpolicy Ġgrad ients Ġas Ġdescribed Ġin ĠYu ĠB IB REF 15 . </s> We Ġutilize ĠAW D - L ST M ĠB IB REF 20 Ġand ĠTrans former XL ĠB IB REF 21 Ġbased Ġlanguage Ġmodels . ĠFor Ġmodel Ġhyper param eters Ġplease Ġto Ġrefer Ġto ĠSupplementary ĠSection ĠTable ĠT AB REF 5 . ĠWe Ġuse ĠAdam Ġoptim izer ĠB IB REF 22 Ġwith Ġ$\ beta Ġ1 = Ġ0 . 7 $ Ġand Ġ$\ beta Ġ2 = Ġ0 . 8 $ Ġsimilar Ġto ĠB IB REF 19 Ġand Ġuse Ġa Ġbatch Ġsize Ġof Ġ50 . ĠOther Ġpractices Ġfor ĠLM Ġtraining Ġwere Ġthe Ġsame Ġas ĠB IB REF 21 Ġand ĠB IB REF 20 Ġfor ĠTrans former - XL Ġand ĠAW D - L ST M Ġrespectively . ĠWe Ġrefer Ġto Ġour Ġproposed ĠG AN Ġas ĠCreative - GAN Ġand Ġcompare Ġit Ġto Ġa Ġbaseline Ġ( a Ġlanguage Ġmodel Ġequivalent Ġto Ġour Ġpre - trained Ġgenerator ) Ġand Ġa ĠG umb el GAN Ġmodel ĠB IB REF 14 Ġacross Ġall Ġproposed Ġdatasets . ĠWe Ġuse Ġthree Ġcreative ĠEnglish Ġdatasets Ġwith Ġdistinct Ġlinguistic Ġcharacteristics : Ġ( 1 ) ĠA Ġcorpus Ġof Ġ7 40 Ġclassical Ġand Ġcontemporary ĠEnglish Ġpoems , Ġ( 2 ) Ġa Ġcorpus Ġof Ġ149 50 Ġmetaphor Ġsentences Ġretrieved Ġfrom Ġa Ġmetaphor Ġdatabase Ġwebsite Ġand Ġ( 3 ) Ġa Ġcorpus Ġof Ġ1500 Ġsong Ġlyrics Ġranging Ġacross Ġgenres . ĠThe Ġmix Ġof Ġlinguistic Ġstyles Ġwithin Ġthis Ġcorpus Ġoffers Ġthe Ġpotential Ġfor Ġinteresting Ġvariation Ġduring Ġthe Ġgeneration Ġphase . ĠWe Ġuse Ġthe Ġsame Ġpre - processing Ġas Ġin Ġearlier Ġwork ĠB IB REF 19 , ĠB IB REF 23 . ĠWe Ġreserve Ġ10 % Ġof Ġour Ġdata Ġfor Ġtest Ġset Ġand Ġanother Ġ10 % Ġfor Ġour Ġvalidation Ġset . </s> We Ġfirst Ġpre - train Ġour Ġgenerator Ġon Ġthe ĠGutenberg Ġdataset ĠB IB REF 24 Ġfor Ġ20 Ġepoch s Ġand Ġthen Ġfine - t une ĠB IB REF 19 Ġthem Ġto Ġour Ġtarget Ġdatasets Ġwith Ġa Ġlanguage Ġmodeling Ġobjective . ĠThe Ġdiscrim inator 's Ġenc oder Ġis Ġinitialized Ġto Ġthe Ġsame Ġweights Ġas Ġour Ġfine - tun ed Ġlanguage Ġmodel . ĠOnce Ġwe Ġhave Ġour Ġfine - tun ed Ġenc od ers Ġfor Ġeach Ġtarget Ġdataset , Ġwe Ġtrain Ġin Ġan Ġadvers arial Ġmanner . ĠThe Ġdiscrim inator Ġobjective Ġhere Ġis Ġto Ġscore Ġthe Ġquality Ġof Ġthe Ġcreative Ġtext . ĠThe Ġdiscrim inator Ġis Ġtrained Ġfor Ġ3 Ġiterations Ġfor Ġevery Ġiteration Ġof Ġthe Ġgenerator , Ġa Ġpractice Ġseen Ġin Ġprevious Ġwork ĠB IB REF 25 . ĠCreative - GAN Ġrelies Ġon Ġusing Ġthe Ġreward Ġfrom Ġthe Ġdiscrim inator ĠB IB REF 12 , ĠB IB REF 15 Ġfor Ġback prop ag ation . ĠWe Ġfollow Ġa Ġsimilar Ġtraining Ġprocedure Ġfor ĠG umb el GAN . ĠOutput s Ġare Ġgenerated Ġthrough Ġsampling Ġover Ġa Ġmult in omial Ġdistribution Ġfor Ġall Ġmethods , Ġinstead Ġof Ġ$ arg max $ Ġon Ġthe Ġlog - like lihood Ġprobabilities , Ġas Ġsampling Ġhas Ġshown Ġto Ġproduce Ġbetter Ġoutput Ġquality ĠB IB REF 4 . ĠPlease Ġrefer Ġto ĠSupplementary ĠSection ĠTable ĠT AB REF 6 Ġfor Ġtraining Ġparameters Ġof Ġeach Ġdataset Ġand ĠTable ĠT AB REF 5 Ġfor Ġhyper param eters Ġof Ġeach Ġenc oder . ĠWe Ġpick Ġthese Ġvalues Ġafter Ġexperimentation Ġwith Ġour Ġvalidation Ġset . ĠTraining Ġand Ġoutput Ġgeneration Ġcode Ġcan Ġbe Ġfound Ġonline . </s> E val uation Ġand Ġconclusion </s> E val uating Ġcreative Ġgeneration Ġtasks Ġis Ġboth Ġcritical Ġand Ġcomplex ĠB IB REF 26 . ĠAlong Ġthe Ġlines Ġof Ġprevious Ġresearch Ġon Ġevaluating Ġtext Ġgeneration Ġtasks ĠB IB REF 26 , Ġwe Ġreport Ġthe Ġperplex ity Ġscores Ġof Ġour Ġtest Ġset Ġon Ġthe Ġevaluated Ġmodels Ġin Ġthe ĠSupplementary ĠSection , ĠTable ĠT AB REF 4 ĠOur Ġmodel Ġshows Ġimprovements Ġover Ġbaseline Ġand ĠG umb el GAN . ĠCommon Ġcomputational Ġmethods Ġlike ĠB LE U ĠB IB REF 27 Ġand Ġperplex ity Ġare Ġat Ġbest Ġa Ġhe uristic Ġand Ġnot Ġstrong Ġindicators Ġof Ġgood Ġperformance Ġin Ġtext Ġgeneration Ġmodels ĠB IB REF 28 . ĠParticularly , Ġsince Ġthese Ġscores Ġuse Ġtarget Ġsequences Ġas Ġa Ġreference , Ġit Ġhas Ġthe Ġsame Ġpitfalls Ġas Ġrelying Ġon ĠM LE . ĠThe Ġadvantages Ġin Ġthis Ġapproach Ġlie Ġin Ġthe Ġdiscrim inator 's Ġability Ġto Ġinfluence Ġthe Ġgenerator Ġto Ġexplore Ġother Ġpossibilities . ĠSample Ġoutputs Ġfor Ġour Ġmodel Ġcan Ġbe Ġfound Ġon Ġour Ġwebsite Ġ. </s> Supplement ary ĠMaterial </s> In Ġthis Ġsection , Ġwe Ġreport Ġour Ġresults Ġon Ġcomputational Ġmetrics , Ġhyper param eters Ġand Ġtraining Ġconfigurations Ġfor Ġour Ġmodels . ĠTable ĠT AB REF 4 Ġshows Ġthe Ġresults Ġof Ġthe Ġperplex ity Ġscore Ġevaluation Ġof Ġthe Ġevaluated Ġmodels , ĠTable ĠT AB REF 5 Ġshows Ġhyper param eters Ġfor Ġeach Ġencoding Ġmethod Ġand ĠTable ĠT AB REF 6 Ġshows Ġour Ġtraining Ġparameters . ĠIn ĠTable ĠT AB REF 6 , Ġthe Ġvalues Ġfor ĠGutenberg Ġdataset Ġin Ġcolumns , ĠG umb el GAN Ġand ĠCreative - GAN Ġare Ġempty Ġas Ġwe Ġonly Ġpret rain Ġour ĠL Ms Ġwith Ġthe ĠGutenberg Ġdataset
Introduction and related work Language models can be optimized to recognize syntax and semantics with great accuracy BIBREF0. However, the output generated can be repetitive and generic leading to monotonous or uninteresting responses (e.g “I don't know”) regardless of the input BIBREF1. While application of attention BIBREF2, BIBREF3 and advanced decoding mechanisms like beam search and variation sampling BIBREF4 have shown improvements, it does not solve the underlying problem. In creative text generation, the objective is not strongly bound to the ground truth—instead the objective is to generate diverse, unique or original samples. We attempt to do this through a discriminator which can give feedback to the generative model through a cost function that encourages sampling of creative tokens. The contributions of this paper are in the usage of a GAN framework to generate creative pieces of writing. Our experiments suggest that generative text models, while very good at encapsulating semantic, syntactic and domain information, perform better with external feedback from a discriminator for fine-tuning objectiveless decoding tasks like that of creative text. We show this by evaluating our model on three very different creative datasets containing poetry, metaphors and lyrics. Previous work on handling the shortcomings of MLE include length-normalizing sentence probability BIBREF5, future cost estimation BIBREF6, diversity-boosting objective function BIBREF7, BIBREF1 or penalizing repeating tokens BIBREF8. When it comes to poetry generation using generative text models, Zhang and Lapata BIBREF9, Yi et al. BIBREF10 and Wang et al. BIBREF11 use language modeling to generate Chinese poems. However, none of these methods provide feedback on the quality of the generated sample and hence, do not address the qualitative objective required for creative decoding. For the task of text generation, MaskGAN BIBREF12 uses a Reinforcement Learning signal from the discriminator, FMD-GAN BIBREF13 uses an optimal transport mechanism as an objective function. GumbelGAN BIBREF14 uses Gumbel-Softmax distribution that replaces the non-differentiable sample from a categorical distribution with a differentiable sample to propagate stronger gradients. Li et al. BIBREF1 use a discriminator for a diversity promoting objective. Yu et al. BIBREF15 use SeqGAN to generate poetry and comment on the performance of SeqGAN over MLE in human evaluations, encouraging our study of GANs for creative text generation. However, these studies do not focus solely on creative text. GANs for creative text generation Using GANs, we can train generative models in a two-player game setting between a discriminator and a generator, where the discriminator (a binary classifier) learns to distinguish between real and fake data samples and the generator tries to fool the discriminator by generating authentic and high quality output BIBREF16. GANs have shown to be successful in image generation tasks BIBREF17 and recently, some progress has been observed in text generation BIBREF13, BIBREF12, BIBREF15. Our generator is a language model trained using backpropagation through time BIBREF18. During the pre-training phase we optimize for MLE and during the GAN training phase, we optimize on the creativity reward from the discriminator. The discriminator's encoder has the same architecture as the generator encoder module with the addition of a pooled decoder layer. The decoder contains 3 $[Dense Batch Normalization,ReLU]$ blocks and an addtional $Sigmoid$ layer. The discriminator decoder takes the hidden state at the last time step of a sequence concatenated with both the max-pooled and mean-pooled representation of the hidden states BIBREF19 and outputs a number in the range $[0,1]$. The difficulty of using GANs in text generation comes from the discrete nature of text, making the model non-differentiable hence, we update parameters for the generator model with policy gradients as described in Yu BIBREF15. We utilize AWD-LSTM BIBREF20 and TransformerXL BIBREF21 based language models. For model hyperparameters please to refer to Supplementary Section Table TABREF5. We use Adam optimizer BIBREF22 with $\beta 1= 0.7$ and $\beta 2= 0.8$ similar to BIBREF19 and use a batch size of 50. Other practices for LM training were the same as BIBREF21 and BIBREF20 for Transformer-XL and AWD-LSTM respectively. We refer to our proposed GAN as Creative-GAN and compare it to a baseline (a language model equivalent to our pre-trained generator) and a GumbelGAN model BIBREF14 across all proposed datasets. We use three creative English datasets with distinct linguistic characteristics: (1) A corpus of 740 classical and contemporary English poems, (2) a corpus of 14950 metaphor sentences retrieved from a metaphor database website and (3) a corpus of 1500 song lyrics ranging across genres. The mix of linguistic styles within this corpus offers the potential for interesting variation during the generation phase. We use the same pre-processing as in earlier work BIBREF19, BIBREF23. We reserve 10% of our data for test set and another 10% for our validation set. We first pre-train our generator on the Gutenberg dataset BIBREF24 for 20 epochs and then fine-tune BIBREF19 them to our target datasets with a language modeling objective. The discriminator's encoder is initialized to the same weights as our fine-tuned language model. Once we have our fine-tuned encoders for each target dataset, we train in an adversarial manner. The discriminator objective here is to score the quality of the creative text. The discriminator is trained for 3 iterations for every iteration of the generator, a practice seen in previous work BIBREF25. Creative-GAN relies on using the reward from the discriminator BIBREF12, BIBREF15 for backpropagation. We follow a similar training procedure for GumbelGAN. Outputs are generated through sampling over a multinomial distribution for all methods, instead of $argmax$ on the log-likelihood probabilities, as sampling has shown to produce better output quality BIBREF4. Please refer to Supplementary Section Table TABREF6 for training parameters of each dataset and Table TABREF5 for hyperparameters of each encoder. We pick these values after experimentation with our validation set. Training and output generation code can be found online. Evaluation and conclusion Evaluating creative generation tasks is both critical and complex BIBREF26. Along the lines of previous research on evaluating text generation tasks BIBREF26, we report the perplexity scores of our test set on the evaluated models in the Supplementary Section, Table TABREF4 Our model shows improvements over baseline and GumbelGAN. Common computational methods like BLEU BIBREF27 and perplexity are at best a heuristic and not strong indicators of good performance in text generation models BIBREF28. Particularly, since these scores use target sequences as a reference, it has the same pitfalls as relying on MLE. The advantages in this approach lie in the discriminator's ability to influence the generator to explore other possibilities. Sample outputs for our model can be found on our website . Supplementary Material In this section, we report our results on computational metrics, hyperparameters and training configurations for our models. Table TABREF4 shows the results of the perplexity score evaluation of the evaluated models, Table TABREF5 shows hyperparameters for each encoding method and Table TABREF6 shows our training parameters. In Table TABREF6, the values for Gutenberg dataset in columns, GumbelGAN and Creative-GAN are empty as we only pretrain our LMs with the Gutenberg dataset
Do they report results only on English data?
f20a389ace2267aa61eddcc235535452ccdae0e6
Do Ġthey Ġreport Ġresults Ġonly Ġon ĠEnglish Ġdata ?
Yes
<s> Do Ġthey Ġreport Ġresults Ġonly Ġon ĠEnglish Ġdata ? </s> Introduction Ġand Ġrelated Ġwork </s> Language Ġmodels Ġcan Ġbe Ġoptimized Ġto Ġrecognize Ġsyntax Ġand Ġsemantics Ġwith Ġgreat Ġaccuracy ĠB IB REF 0 . ĠHowever , Ġthe Ġoutput Ġgenerated Ġcan Ġbe Ġrepetitive Ġand Ġgeneric Ġleading Ġto Ġmon oton ous Ġor Ġun interesting Ġresponses Ġ( e . g ĠâĢ ľ I Ġdon 't Ġknow âĢ Ŀ ) Ġregardless Ġof Ġthe Ġinput ĠB IB REF 1 . ĠWhile Ġapplication Ġof Ġattention ĠB IB REF 2 , ĠB IB REF 3 Ġand Ġadvanced Ġdecoding Ġmechanisms Ġlike Ġbeam Ġsearch Ġand Ġvariation Ġsampling ĠB IB REF 4 Ġhave Ġshown Ġimprovements , Ġit Ġdoes Ġnot Ġsolve Ġthe Ġunderlying Ġproblem . ĠIn Ġcreative Ġtext Ġgeneration , Ġthe Ġobjective Ġis Ġnot Ġstrongly Ġbound Ġto Ġthe Ġground Ġtruth âĢĶ instead Ġthe Ġobjective Ġis Ġto Ġgenerate Ġdiverse , Ġunique Ġor Ġoriginal Ġsamples . ĠWe Ġattempt Ġto Ġdo Ġthis Ġthrough Ġa Ġdiscrim inator Ġwhich Ġcan Ġgive Ġfeedback Ġto Ġthe Ġgener ative Ġmodel Ġthrough Ġa Ġcost Ġfunction Ġthat Ġencourages Ġsampling Ġof Ġcreative Ġtokens . ĠThe Ġcontributions Ġof Ġthis Ġpaper Ġare Ġin Ġthe Ġusage Ġof Ġa ĠG AN Ġframework Ġto Ġgenerate Ġcreative Ġpieces Ġof Ġwriting . ĠOur Ġexperiments Ġsuggest Ġthat Ġgener ative Ġtext Ġmodels , Ġwhile Ġvery Ġgood Ġat Ġencaps ulating Ġsemantic , Ġsynt actic Ġand Ġdomain Ġinformation , Ġperform Ġbetter Ġwith Ġexternal Ġfeedback Ġfrom Ġa Ġdiscrim inator Ġfor Ġfine - tun ing Ġobjective less Ġdecoding Ġtasks Ġlike Ġthat Ġof Ġcreative Ġtext . ĠWe Ġshow Ġthis Ġby Ġevaluating Ġour Ġmodel Ġon Ġthree Ġvery Ġdifferent Ġcreative Ġdatasets Ġcontaining Ġpoetry , Ġmetaphors Ġand Ġlyrics . </s> Previous Ġwork Ġon Ġhandling Ġthe Ġshortcomings Ġof ĠM LE Ġinclude Ġlength - normal izing Ġsentence Ġprobability ĠB IB REF 5 , Ġfuture Ġcost Ġestimation ĠB IB REF 6 , Ġdiversity - boost ing Ġobjective Ġfunction ĠB IB REF 7 , ĠB IB REF 1 Ġor Ġpenal izing Ġrepeating Ġtokens ĠB IB REF 8 . ĠWhen Ġit Ġcomes Ġto Ġpoetry Ġgeneration Ġusing Ġgener ative Ġtext Ġmodels , ĠZhang Ġand ĠLap ata ĠB IB REF 9 , ĠYi Ġet Ġal . ĠB IB REF 10 Ġand ĠWang Ġet Ġal . ĠB IB REF 11 Ġuse Ġlanguage Ġmodeling Ġto Ġgenerate ĠChinese Ġpoems . ĠHowever , Ġnone Ġof Ġthese Ġmethods Ġprovide Ġfeedback Ġon Ġthe Ġquality Ġof Ġthe Ġgenerated Ġsample Ġand Ġhence , Ġdo Ġnot Ġaddress Ġthe Ġqualitative Ġobjective Ġrequired Ġfor Ġcreative Ġdecoding . ĠFor Ġthe Ġtask Ġof Ġtext Ġgeneration , ĠMask GAN ĠB IB REF 12 Ġuses Ġa ĠRein forcement ĠLearning Ġsignal Ġfrom Ġthe Ġdiscrim inator , ĠF MD - GAN ĠB IB REF 13 Ġuses Ġan Ġoptimal Ġtransport Ġmechanism Ġas Ġan Ġobjective Ġfunction . ĠG umb el GAN ĠB IB REF 14 Ġuses ĠG umb el - Soft max Ġdistribution Ġthat Ġreplaces Ġthe Ġnon - different iable Ġsample Ġfrom Ġa Ġcateg orical Ġdistribution Ġwith Ġa Ġdifferent iable Ġsample Ġto Ġpropagate Ġstronger Ġgrad ients . ĠLi Ġet Ġal . ĠB IB REF 1 Ġuse Ġa Ġdiscrim inator Ġfor Ġa Ġdiversity Ġpromoting Ġobjective . ĠYu Ġet Ġal . ĠB IB REF 15 Ġuse ĠSe q GAN Ġto Ġgenerate Ġpoetry Ġand Ġcomment Ġon Ġthe Ġperformance Ġof ĠSe q GAN Ġover ĠM LE Ġin Ġhuman Ġevaluations , Ġencouraging Ġour Ġstudy Ġof ĠG AN s Ġfor Ġcreative Ġtext Ġgeneration . ĠHowever , Ġthese Ġstudies Ġdo Ġnot Ġfocus Ġsolely Ġon Ġcreative Ġtext . </s> GAN s Ġfor Ġcreative Ġtext Ġgeneration </s> Using ĠG AN s , Ġwe Ġcan Ġtrain Ġgener ative Ġmodels Ġin Ġa Ġtwo - player Ġgame Ġsetting Ġbetween Ġa Ġdiscrim inator Ġand Ġa Ġgenerator , Ġwhere Ġthe Ġdiscrim inator Ġ( a Ġbinary Ġclass ifier ) Ġlearns Ġto Ġdistinguish Ġbetween Ġreal Ġand Ġfake Ġdata Ġsamples Ġand Ġthe Ġgenerator Ġtries Ġto Ġfool Ġthe Ġdiscrim inator Ġby Ġgenerating Ġauthentic Ġand Ġhigh Ġquality Ġoutput ĠB IB REF 16 . ĠG AN s Ġhave Ġshown Ġto Ġbe Ġsuccessful Ġin Ġimage Ġgeneration Ġtasks ĠB IB REF 17 Ġand Ġrecently , Ġsome Ġprogress Ġhas Ġbeen Ġobserved Ġin Ġtext Ġgeneration ĠB IB REF 13 , ĠB IB REF 12 , ĠB IB REF 15 . ĠOur Ġgenerator Ġis Ġa Ġlanguage Ġmodel Ġtrained Ġusing Ġback prop ag ation Ġthrough Ġtime ĠB IB REF 18 . ĠDuring Ġthe Ġpre - training Ġphase Ġwe Ġoptimize Ġfor ĠM LE Ġand Ġduring Ġthe ĠG AN Ġtraining Ġphase , Ġwe Ġoptimize Ġon Ġthe Ġcreativity Ġreward Ġfrom Ġthe Ġdiscrim inator . ĠThe Ġdiscrim inator 's Ġenc oder Ġhas Ġthe Ġsame Ġarchitecture Ġas Ġthe Ġgenerator Ġenc oder Ġmodule Ġwith Ġthe Ġaddition Ġof Ġa Ġpooled Ġdec oder Ġlayer . ĠThe Ġdec oder Ġcontains Ġ3 Ġ$ [ D ense ĠB atch ĠNormal ization , Re LU ] $ Ġblocks Ġand Ġan Ġadd t ional Ġ$ S igm oid $ Ġlayer . ĠThe Ġdiscrim inator Ġdec oder Ġtakes Ġthe Ġhidden Ġstate Ġat Ġthe Ġlast Ġtime Ġstep Ġof Ġa Ġsequence Ġconc aten ated Ġwith Ġboth Ġthe Ġmax - pool ed Ġand Ġmean - pool ed Ġrepresentation Ġof Ġthe Ġhidden Ġstates ĠB IB REF 19 Ġand Ġoutputs Ġa Ġnumber Ġin Ġthe Ġrange Ġ$ [ 0 , 1 ] $. ĠThe Ġdifficulty Ġof Ġusing ĠG AN s Ġin Ġtext Ġgeneration Ġcomes Ġfrom Ġthe Ġdiscrete Ġnature Ġof Ġtext , Ġmaking Ġthe Ġmodel Ġnon - different iable Ġhence , Ġwe Ġupdate Ġparameters Ġfor Ġthe Ġgenerator Ġmodel Ġwith Ġpolicy Ġgrad ients Ġas Ġdescribed Ġin ĠYu ĠB IB REF 15 . </s> We Ġutilize ĠAW D - L ST M ĠB IB REF 20 Ġand ĠTrans former XL ĠB IB REF 21 Ġbased Ġlanguage Ġmodels . ĠFor Ġmodel Ġhyper param eters Ġplease Ġto Ġrefer Ġto ĠSupplementary ĠSection ĠTable ĠT AB REF 5 . ĠWe Ġuse ĠAdam Ġoptim izer ĠB IB REF 22 Ġwith Ġ$\ beta Ġ1 = Ġ0 . 7 $ Ġand Ġ$\ beta Ġ2 = Ġ0 . 8 $ Ġsimilar Ġto ĠB IB REF 19 Ġand Ġuse Ġa Ġbatch Ġsize Ġof Ġ50 . ĠOther Ġpractices Ġfor ĠLM Ġtraining Ġwere Ġthe Ġsame Ġas ĠB IB REF 21 Ġand ĠB IB REF 20 Ġfor ĠTrans former - XL Ġand ĠAW D - L ST M Ġrespectively . ĠWe Ġrefer Ġto Ġour Ġproposed ĠG AN Ġas ĠCreative - GAN Ġand Ġcompare Ġit Ġto Ġa Ġbaseline Ġ( a Ġlanguage Ġmodel Ġequivalent Ġto Ġour Ġpre - trained Ġgenerator ) Ġand Ġa ĠG umb el GAN Ġmodel ĠB IB REF 14 Ġacross Ġall Ġproposed Ġdatasets . ĠWe Ġuse Ġthree Ġcreative ĠEnglish Ġdatasets Ġwith Ġdistinct Ġlinguistic Ġcharacteristics : Ġ( 1 ) ĠA Ġcorpus Ġof Ġ7 40 Ġclassical Ġand Ġcontemporary ĠEnglish Ġpoems , Ġ( 2 ) Ġa Ġcorpus Ġof Ġ149 50 Ġmetaphor Ġsentences Ġretrieved Ġfrom Ġa Ġmetaphor Ġdatabase Ġwebsite Ġand Ġ( 3 ) Ġa Ġcorpus Ġof Ġ1500 Ġsong Ġlyrics Ġranging Ġacross Ġgenres . ĠThe Ġmix Ġof Ġlinguistic Ġstyles Ġwithin Ġthis Ġcorpus Ġoffers Ġthe Ġpotential Ġfor Ġinteresting Ġvariation Ġduring Ġthe Ġgeneration Ġphase . ĠWe Ġuse Ġthe Ġsame Ġpre - processing Ġas Ġin Ġearlier Ġwork ĠB IB REF 19 , ĠB IB REF 23 . ĠWe Ġreserve Ġ10 % Ġof Ġour Ġdata Ġfor Ġtest Ġset Ġand Ġanother Ġ10 % Ġfor Ġour Ġvalidation Ġset . </s> We Ġfirst Ġpre - train Ġour Ġgenerator Ġon Ġthe ĠGutenberg Ġdataset ĠB IB REF 24 Ġfor Ġ20 Ġepoch s Ġand Ġthen Ġfine - t une ĠB IB REF 19 Ġthem Ġto Ġour Ġtarget Ġdatasets Ġwith Ġa Ġlanguage Ġmodeling Ġobjective . ĠThe Ġdiscrim inator 's Ġenc oder Ġis Ġinitialized Ġto Ġthe Ġsame Ġweights Ġas Ġour Ġfine - tun ed Ġlanguage Ġmodel . ĠOnce Ġwe Ġhave Ġour Ġfine - tun ed Ġenc od ers Ġfor Ġeach Ġtarget Ġdataset , Ġwe Ġtrain Ġin Ġan Ġadvers arial Ġmanner . ĠThe Ġdiscrim inator Ġobjective Ġhere Ġis Ġto Ġscore Ġthe Ġquality Ġof Ġthe Ġcreative Ġtext . ĠThe Ġdiscrim inator Ġis Ġtrained Ġfor Ġ3 Ġiterations Ġfor Ġevery Ġiteration Ġof Ġthe Ġgenerator , Ġa Ġpractice Ġseen Ġin Ġprevious Ġwork ĠB IB REF 25 . ĠCreative - GAN Ġrelies Ġon Ġusing Ġthe Ġreward Ġfrom Ġthe Ġdiscrim inator ĠB IB REF 12 , ĠB IB REF 15 Ġfor Ġback prop ag ation . ĠWe Ġfollow Ġa Ġsimilar Ġtraining Ġprocedure Ġfor ĠG umb el GAN . ĠOutput s Ġare Ġgenerated Ġthrough Ġsampling Ġover Ġa Ġmult in omial Ġdistribution Ġfor Ġall Ġmethods , Ġinstead Ġof Ġ$ arg max $ Ġon Ġthe Ġlog - like lihood Ġprobabilities , Ġas Ġsampling Ġhas Ġshown Ġto Ġproduce Ġbetter Ġoutput Ġquality ĠB IB REF 4 . ĠPlease Ġrefer Ġto ĠSupplementary ĠSection ĠTable ĠT AB REF 6 Ġfor Ġtraining Ġparameters Ġof Ġeach Ġdataset Ġand ĠTable ĠT AB REF 5 Ġfor Ġhyper param eters Ġof Ġeach Ġenc oder . ĠWe Ġpick Ġthese Ġvalues Ġafter Ġexperimentation Ġwith Ġour Ġvalidation Ġset . ĠTraining Ġand Ġoutput Ġgeneration Ġcode Ġcan Ġbe Ġfound Ġonline . </s> E val uation Ġand Ġconclusion </s> E val uating Ġcreative Ġgeneration Ġtasks Ġis Ġboth Ġcritical Ġand Ġcomplex ĠB IB REF 26 . ĠAlong Ġthe Ġlines Ġof Ġprevious Ġresearch Ġon Ġevaluating Ġtext Ġgeneration Ġtasks ĠB IB REF 26 , Ġwe Ġreport Ġthe Ġperplex ity Ġscores Ġof Ġour Ġtest Ġset Ġon Ġthe Ġevaluated Ġmodels Ġin Ġthe ĠSupplementary ĠSection , ĠTable ĠT AB REF 4 ĠOur Ġmodel Ġshows Ġimprovements Ġover Ġbaseline Ġand ĠG umb el GAN . ĠCommon Ġcomputational Ġmethods Ġlike ĠB LE U ĠB IB REF 27 Ġand Ġperplex ity Ġare Ġat Ġbest Ġa Ġhe uristic Ġand Ġnot Ġstrong Ġindicators Ġof Ġgood Ġperformance Ġin Ġtext Ġgeneration Ġmodels ĠB IB REF 28 . ĠParticularly , Ġsince Ġthese Ġscores Ġuse Ġtarget Ġsequences Ġas Ġa Ġreference , Ġit Ġhas Ġthe Ġsame Ġpitfalls Ġas Ġrelying Ġon ĠM LE . ĠThe Ġadvantages Ġin Ġthis Ġapproach Ġlie Ġin Ġthe Ġdiscrim inator 's Ġability Ġto Ġinfluence Ġthe Ġgenerator Ġto Ġexplore Ġother Ġpossibilities . ĠSample Ġoutputs Ġfor Ġour Ġmodel Ġcan Ġbe Ġfound Ġon Ġour Ġwebsite Ġ. </s> Supplement ary ĠMaterial </s> In Ġthis Ġsection , Ġwe Ġreport Ġour Ġresults Ġon Ġcomputational Ġmetrics , Ġhyper param eters Ġand Ġtraining Ġconfigurations Ġfor Ġour Ġmodels . ĠTable ĠT AB REF 4 Ġshows Ġthe Ġresults Ġof Ġthe Ġperplex ity Ġscore Ġevaluation Ġof Ġthe Ġevaluated Ġmodels , ĠTable ĠT AB REF 5 Ġshows Ġhyper param eters Ġfor Ġeach Ġencoding Ġmethod Ġand ĠTable ĠT AB REF 6 Ġshows Ġour Ġtraining Ġparameters . ĠIn ĠTable ĠT AB REF 6 , Ġthe Ġvalues Ġfor ĠGutenberg Ġdataset Ġin Ġcolumns , ĠG umb el GAN Ġand ĠCreative - GAN Ġare Ġempty Ġas Ġwe Ġonly Ġpret rain Ġour ĠL Ms Ġwith Ġthe ĠGutenberg Ġdataset