Language style transfer is the problem of migrating the content of a source sentence to a target style. In many applications, parallel training data are not available and source sentences to be transferred may have arbitrary and unknown styles. In this paper, we present an encoder-decoder framework under this problem setting. Each sentence is encoded into its content and style latent representations. By recombining the content with the target style, we can decode a sentence aligned in the target domain. To adequately constrain the encoding and decoding functions, we couple them with two loss functions. The first is a style discrepancy loss, enforcing that the style representation accurately encodes the style information guided by the discrepancy between the sentence style and the target style. The second is a cycle consistency loss, which ensures that the transferred sentence should preserve the content of the original sentence disentangled from its style. We validate the effectiveness of our proposed model on two tasks: sentiment modification of restaurant reviews, and dialog response revision with a romantic style. Style transfer is a long-standing research problem that aims at migrating the content of a sample from a source style to a target style. Recently, great progress has been achieved by applying deep neural networks to redraw an image in a particular style BID7 BID10 BID2 BID20 BID12 . However, until now very few approaches have been proposed for style transfer of natural language sentences, i.e., changing the style or genre of a sentence while preserving its semantic content. For example, we would like a system that can convert a given text piece in the language of Shakespeare BID14 ; or rewrite product reviews with a favored sentiment BID17 .One important issue on language style transfer is that parallel data are unavailable. For instance, considering the task of rewriting a negative review of a product to its counterpart with a positive sentiment, we can hardly find paired data that describe the same content. Yet , many text generation frameworks require parallel data, such as the popular sequence-to-sequence model in machine translation and document summarization BID16 , and thus are not applicable under this scenario. A few recent approaches have been proposed for style transfer with non-parallel data BID4 BID17 . Their key idea is to learn a latent representation of the content disentangled from the source style, and then recombine it with the target style to generate the corresponding sentence.All the above approaches assume that data have only two styles, and their task is to transfer sentences from one style to the other. However , in many practical settings, we may deal with sentences in more than two styles. Taking the review sentiment modification as an example again, some reviews may be neither positive nor negative, but in a neutral style. Moreover , even reviews considered negative can be categorized into more fine-grained sentiments, such as anger, sadness, boredom and other negative styles. It may be beneficial if such styles are treated differently. As another example, consider a chatbot with a coherent persona, which has a consistent language behavior and interaction style BID9 . A simple framework for this task is to first use human dialog data to train a chatbot system, such as a retrieval-based dialog model BID11 , and then transfer the output responses with a language style transfer model so that multi-round responses always have a consistent style. Note that the human dialog sentences are collected from different users, and users' expressions of the content and tones may be in different personalized characteristics. Thus the output responses retrieved from the dialog model may have the language style of any user. Simply treating the responses with a single style and employing the existing style transfer models would lead to unsatisfactory results. Hence, in this paper, we study the setting of language style transfer in which the source data to be transferred can have various (and possibly unknown) styles.Another challenging problem in language style transfer is that the transferred sentence should preserve the content of the original sentence disentangled from its style. To tackle this problem, BID17 assumed the source domain and the target domain share the same latent content space, and trained their model by aligning these two latent spaces. BID4 constrained that the latent content representation of the original sentence could be inferred from the transferred sentence. However, these attempts considered content modification in the latent content space but not the sentence space.In this work, we develop an encoder-decoder framework that can transfer a sentence from a source domain to its counterpart in a target domain. The training data in the two domains are non-parallel, and sentences in the source domain can have arbitrary language styles but those in the target domain are with a consensus style. We encode each sentence into two latent representations, one for the content disentangled from the style, and the other for the style. Intuitively, if a source sentence is considered having the target style with a high probability, its style representation should be close to the target style representation. To make use of this idea, we enforce that the discrepancy between an arbitrary style representation and the target style representation should be consistent with the closeness of its sentence style to the target style. A cycle consistency loss is further introduced to avoid content change by directly considering the transferred sentence. Its idea is that the generated sentence, when put back into the encoder and recombined with its original style representation, can recover the original sentence. We evaluate the performance of our proposed model on two tasks. The first is the sentiment modification task with its source domain containing more than one sentiments, and the second is to transfer general dialog responses to a romantic style. In this paper, we present an encoder-decoder framework for language style transfer, which allows for the use of non-parallel data and source data with various unknown language styles. Each sentence is encoded into two latent representations, one corresponding to its content disentangled from the style and and the other representing the style only. By recombining the content with the target style, we can decode a sentence aligned in the target domain. Specifically, we propose two loss functions, i.e., the style discrepancy loss and the cycle consistency loss, to adequately constrain the encoding and decoding functions. The style discrepancy loss is to enforce a properly encoded style representation while the cycle consistency loss is used to ensure that the style-transferred sentences can be transferred back to their original sentences. Experimental results on two tasks demonstrate that our proposed method outperforms the state-of-the-art style transfer method BID17 We randomly select 200 test samples from Yelp and perform human evaluations on four aspects of the results: (1) content: estimates if the content of an input sentence is preserved in a transferred sentence; content rating has 0 (changed), 1 (synonym substitution or partially changed), and 2 (unchanged); (2) sentiment: estimates if the sentiment of a transferred sentence is consistent with the target sentiment; sentiment rating has 0 (unchanged and wrong), 1 (changed but wrong), 2 (correct); (3) fluency: estimates the fluency of transferred sentences; fluency is rated from 1 (unreadable) to 4 (perfect); (4) overall: estimates the overall quality of transferred sentences; overall rating ranges from 1 (failed) to 4 (perfect).We hired five annotators and averaged their evaluations. TAB0 shows results on Yelp when the source domain contains not only negative sentences but also 150k positive sentences (row 3 in TAB0 ), and TAB0 shows results on Yelp when the target domain contains only 100k positive sentences ( row 1 in TAB3 ). As can be seen, our model is better in terms of sentiment accuracy and overall quality, which is consistent with the automatic evaluation results.
We present an encoder-decoder framework for language style transfer, which allows for the use of non-parallel data and source data with various unknown language styles.
{"src": "scitldr-AIC_train", "paper_id": "B1NKuC6SG"}