Controllable Natural Language Generation with Contrastive Prefixes

To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for controllable GPT2 generation, which utilizes a set of small attribute-specific vectors, called prefixes (Li and Liang, 2021), to steer natural language generation. Different from Li and Liang (2021), where each prefix is trained independently, we take the relationship among prefixes into consideration and train multiple prefixes simultaneously. We propose a novel supervised method and also an unsupervised method to train the prefixes for single-aspect control while the combination of these two methods can achieve multi-aspect control. Experimental results on both single-aspect and multi-aspect control show that our methods can guide generation towards the desired attributes while keeping high linguistic quality.


Introduction
The goal of controllable Natural Language Generation (NLG) is to guide generation towards the desired attributes in the concerned aspects of the text. For example, the aspect can be topic or sentiment, and sentiment may have two attributes: positive and negative. Previous work has focused on directly fine-tuning the existing models (Keskar et al., 2019;Hu et al., 2017;Ficler and Goldberg, 2017) or using a discriminator to guide generation (Dathathri et al., 2020;Krause et al., 2020;Holtzman et al., 2018). CTRL (Keskar et al., 2019) achieves controllability at the expense of training a large conditional LM. GeDi (Krause et al., 2020) also trains conditional LMs but uses them as discriminators to guide generation, introducing additional 345M parameters. Besides, GeDi focuses on single-aspect control, ignoring the need for multi-aspect control. PPLM (Dathathri et al., 2020) guides generation by iteratively updating the LM's hidden activations. However, this decoding strategy is extremely computationally intensive, resulting in a slow generation speed (Gehman et al., 2020). Prefix-tuning (Li and Liang, 2021) proposes to optimize a prefix, which is a small continuous taskspecific vector, as a lightweight alternative to finetuning an NLG task, such as table-to-text generation or summarization. Inspired by Li and Liang (2021), we propose to use prefixes, a set of small continuous attribute-specific vectors, to steer NLG. Compared with using an attribute model or a generative discriminator (Dathathri et al., 2020;Krause et al., 2020), using learned prefixes to achieve controllability has the following benefits. First, it introduces fewer additional parameters (~0.2%-2% of GPT2 parameters in our experiments). Second, using prefixes keeps the inference speed comparable to that of the original GPT2 model.
In a general sense, prefix-tuning (Li and Liang, 2021) can be considered as controlling the genera-tion of language models. Prefix-tuning views each prefix as an independent control task thus trains each prefix separately (top in Figure 1). However, one aspect of controllability in NLG involves multiple attributes, which might have a relationship with each other. For example, the sentiment aspect usually has two attributes: positive and negative, which are in opposition to each other. We think that this opposite relationship can be helpful to improve the controllability of a prefix. Therefore, we propose a novel supervised method and a novel unsupervised one in our framework, which takes the relationship among prefixes into consideration and trains multiple prefixes simultaneously with novel training objectives, as illustrated in Figure 1.
Experimental results on the single-aspect control tasks (sentiment control, detoxification, and topic control) show that our proposed methods can guide generation towards the target attribute while keeping high linguistic quality, even when only several dozen labeled examples are available. In addition to single-aspect control, multi-aspect control can be achieved by combining the proposed supervised method with the unsupervised method in our framework. Experimental results on the sentiment and topic control show that the prefixes trained with our method can successfully control these two aspects simultaneously.
Our main contributions are as follows: • We propose a novel framework that utilizes prefixes with frozen LMs as a lightweight alternative for controllable GPT2 generation.
• We propose a supervised method and an unsupervised method with novel objectives for prefix training, where the relationship among prefixes are considered and multiple prefixes are trained simultaneously.
• This work provides a unified perspective for single-aspect control and multi-aspect control. Experimental results show that our methods can effectively guide generation in both single-aspect control and multi-aspect control.
2 Related Work Ficler and Goldberg (2017) control the stylistic aspects of the generated text with a conditioned RNN (Recurrent Neural Network) LM. Holtzman et al. (2018) compose a committee of discriminators to guide an RNN generator towards the generations with the desired linguistic quality. Hu et al. (2017) aim at controlling the sentiment and tense of the generated text by combining variational autoencoders (VAE) and attribute discriminators. More recently, with the advent of Transformers and large pretrained language models, such as GPT2, an extensive body of work has focused on controlling the generation of these Transformerbased models. Keskar et al. (2019) train a 1.63 billion-parameter conditional transformer LM from scratch with 55 attribute control codes to guide generation. However, this method is expensive and lacks flexibility since the control codes are fixed. Dathathri et al. (2020) address these limitations by developing a plug-and-play model which leverages an attribute discriminator to perturb the LM's hidden activations. However, updating gradients at the token level results in slow inference. Instead of updating the hidden activations, Krause et al. (2020); Yang and Klein (2021); Lin and Riedl (2021) introduce generative discriminators to re-weight the next token distributions on the fly during inference, thus improving the inference speed.
Our work is mostly related to Yu et al. (2021); Li and Liang (2021). Yu et al. (2021) use a pretrained LM followed by an attribute alignment function to encode the tokens of the target attributes and the resulting hidden states are used to control generation. Different from their work, we do not take the tokens of the target attributes as input. Instead, we directly train a set of parameters, which acts as the prepended hidden states of GPT2, to control generation. Avoiding using attribute tokens can circumvent the problems when it is difficult to describe the desired attribute with only one word. Besides, Yu et al. (2021) focus on attributes disentanglement, which is not a focus in our work, so our training methods are different. Prefix-tuning (Li and Liang, 2021) can, in a general sense, be viewed as controlling the generation of LMs, where the LM is controlled to depict a specific NLG task, while in this work, the LM is controlled to carry specific attributes in a generation. Besides, our proposed methods for prefix training are different from Li and Liang (2021), as stated in Section 1.

Method
Our method uses prefixes to guide GPT2 generation, where a prefix is a continuous attributespecific vector prepended to the activations of GPT2. Prefixes are free parameters denoted as H θ . Different from Li and Liang (2021), where each prefix is trained independently, we consider the relationship among attributes and train multiple prefixes simultaneously, so H θ is of dimension N × M × D, where N is the number of prefixes. In single-aspect control, N equals the number of attributes in the concerned aspect. M is the length of a prefix. D = 2 × L × E is the dimension of the activation in GPT2, where L is the number of transformer layers, E is the hidden size, and 2 indicates one key vector and one value vector. Following Li and Liang (2021), we reparametrize H θ [i, j, :] = W i H θ [i, j, :] by a smaller parameter (H θ ) composed with a large matrix (W i ). After the training finishes, only H θ needs to be saved for generation while W and H θ can be discarded. Since the GPT2 parameters are kept frozen during training, they do not need to be saved either. Figure 2 shows an example of the generation process under the control of a trained prefix. The prefixes can be trained in a supervised, semi-supervised, or unsupervised way. Since the semi-supervised method is a combination of the supervised and the unsupervised method, we introduce the supervised and the unsupervised method in this section. For clarity, we introduce these methods under the single-aspect control setting.

Supervised Method
Suppose the concerned aspect has the attribute set Y , each training example is a pair of (x, y) where x is the input text and y ∈ Y is the attribute label of x. Note that the attribute label also indicates the ground truth index of the prefix in H θ , so y also refers to the prefix index in the following description. As mentioned in Section 1, we introduce an additional discriminative loss to train multiple prefixes simultaneously. Therefore, the training loss L sup is a weighted sum of the language model loss L LM and the discriminative loss L d : The computation of log p(x t |x <t , y) is parameterized as log p θ,γ (x t |x <t , H θ [y, :, :]), where γ is the set of fixed GPT2 parameters, and θ represents learnable prefix parameters. log p(x|y) = t log p(x t |x <t , y), so the parameterization of . "The book" is the given prompt. "is good" is the generated completion.
Note that each prefix can be trained independently using L LM alone, which would be the same as prefix-tuning (Li and Liang, 2021). Intuitively, prefixes trained by L LM are infused with the information of what is encouraged to generate. However, we observe that in controllable NLG, it is helpful to also infuse a prefix with the information of what is discouraged to generate. Given a training example (x, y), the prefix H θ [y, :, :] should be optimized towards generating x, while the other prefixes should be discouraged to generate x. To achieve this goal, all the prefixes in H θ should be trained simultaneously. Therefore, the discriminative loss L d is introduced. As in equation 3, optimizing L d improves the attribute alignment p(y|x) by increasing p(x|y) and lowering p(x|ȳ),ȳ ∈ Y \{y} at the same time. We assume uniform prior, so p(y) and p(y ) can be canceled out in Equation 3. Figure 3 illustrates the training process with two prefixes.

Unsupervised Method
In the unsupervised setting, we assume the attribute set Y of the concerned aspect is known. The training example consists of input text x only. The attribute label y is no longer available and thus the index of the prefix associated with x is unknown. In other words, the index of the prefix corresponding to x is a latent variable z, whose posterior distribution follows a categorical distribution. Inspired by VQ-VAE (van den Oord et al., 2017), we consider the prefixes as discrete latent representations. We take the backbone model in the above supervised method as the decoder and introduce an encoder to parameterize the categorical distribution q(z|x). According to q(z|x), a prefix index z is selected and the prefix H θ [z, :, :] is then fed into the decoder  to reconstruct the input text x. Since the selection process of the prefixes is non-differentiable, we use Gumbel-Softmax (GS) relaxation (Jang et al., 2017;Maddison et al., 2017) following Sønderby et al. (2017); Ramesh et al. (2021). Formally, q(z|x) is computed as follows: where τ is the temperature of Gumbel-Softmax, and Enc is the encoder function. We use a pretrained GPT-2 model followed by a linear layer as the encoder. To train the prefixes, the loss function is a weighted sum of the three loss terms: where L LM is the language model loss. Similar as that in the supervised method, the computation of log p(x t |x <t , z) is parameterized as log p θ,γ (x t |x <t , H θ [z, :, :]). L KL is the Kullback-Leibler divergence, where we assume the prior p(z) to be uniform. Note that these two terms constitute the loss function of VAE. Optimizing these two loss terms improves the evidence lower bound of log p(x). Similar to the intuition behind L d in the supervised method, if the ground truth prefix for x is H θ [y, :, :], then the other prefixes should be discouraged to generate x. However, L d requires the ground truth label y for computation. Instead, we introduce an unsupervised contrastive loss L c .
where m is a pre-set margin andz is another latent variable indicating the index of the opposite prefix of x. q(z|x) is computed as follows: L c is aimed at increasing the attribute alignment by pushing p(z|x) away from p(z|x) by a margin. The computation of p(z|x) is as follows: We assume uniform prior, so p(z) and p(z ) can be canceled out. Similar as the parameterization of log p(x|y) in the supervised method, the parameterization of log p(x|z) is the sum of log p θ,γ (x t |x <t , H θ [z, :, :]) over t. The training process is illustrated in Figure 4.

Experiments
We experiment with three tasks: sentiment control, detoxification, and topic control. We compare our method to GPT2, PPLM, and GeDi. We focus on English text in all the experiments and we experiment with GPT2-medium (345M parameters) for all the methods. We use the original implementation of PPLM and GeDi released by Dathathri et al. (2020) and Krause et al. (2020), and the hyperparameters are set to the reported value in the original paper. The detailed hyperparameters in each task are listed in appendix A. For the GPT2 model, we do experiments under two settings. First, the GPT2 model generates completions of each prompt in the evaluation dataset, which is denoted as GPT2medium. Second, GPT2-medium + prompt engineering prepends a guiding sentence to each testing prompt and then generates completions of each augmented prompt. We evaluate the linguistic quality and attribute alignment of the generation. The linguistic quality is evaluated using the perplexity calculated by GPT2-large (774M parameters).
To evaluate the robustness of our supervised method with the size of the training dataset, we experiment with the following three different settings: 1) using the complete training dataset; 2) using 1,000 examples per attribute for training; 3) using 24 examples per attribute for training. We evaluate our unsupervised method on the sentiment control task and the detoxification task, which are binary tasks. Note that different from the supervised method, our unsupervised method does not use any attribute labels, so the order of the attributes in the trained prefixes is undetermined. After the prefixes finish training using the unsupervised method, we manually check the order of the attributes.

Tasks
Sentiment Control Same as GeDi, we use IMDb movie reviews (Maas et al., 2011) to train our model. The number of prefixes is 2. Note that GeDi only uses 11.25k examples from the dataset for training. To be a fair comparison, we randomly sample 11.25k examples from the dataset to train our model. To evaluate the sentiment alignment of the generated text, we finetune a RoBERTa (Liu et al., 2019) classifier using the Yelp Review dataset (Zhang et al., 2015). The prompts used for evaluation are the same as those in the PPLM experiment (Dathathri et al., 2020). For each of the 15 prompts, 45 completions are generated. In the GPT2-medium + prompt engineering setting, we prepend each prompt with the guiding sentence "This is a negative review:" for negative sentiment control, and similarly, we prepend each prompt with "This is a positive review:" for positive sentiment control.
Detoxification We use Jigsaw Toxic Comment Classification Challenge Dataset 1 to train our model. The number of prefixes is 2. Google Perspective API 2 is used for toxicity evaluation. The testing prompts are collected from RealTox-icityPrompts (Gehman et al., 2020). We use the prompts categorized as "challenging" in the dataset. We further filter out the prompts with toxicity larger than 0.5, scored by Perspective. The resulted evaluation dataset consists of 203 prompts. For each of these prompts, 20 completions are generated. In the GPT2-medium + prompt engineering setting, we prepend each prompt with the guiding sentence "This is a non-toxic comment:".
Topic Control We experiment with the AGNews dataset and DBPedia dataset (Zhang et al., 2015). The number of prefixes is 4 and 14, respectively. The prompts used for evaluation are the same as those in the PPLM experiment (Dathathri et al., 2020). For each of the 20 prompts, 45 completions are generated. Same as that in GeDi, we split each of the original training datasets in half. One half is used to train prefixes, while the other half is used to train a RoBERTa topic classifier for topic relevance evaluation. In the GPT2-medium + prompt engineering setting, the guiding sentence follows the template "The following is about [TOPIC]". We do not compare with PPLM in the topic control task since PPLM uses a bag-of-words attribute model to do topic control, where the 7 predefined topics are different from the topics in the AGNews dataset or the DBPedia dataset.
All the experiments are conducted on NVIDIA Tesla V100 GPUs. The detailed hyper-parameters for each experiment are listed in appendix A.

Results
In the unsupervised setting, GPT2-medium + prompt engineering shows controllability on sentiment control (Table 1) and topic control (Table 3). However, this method does not work on the detoxification task (Table 2). Our unsupervised method significantly lowers the toxicity on the detoxification task and the ablation study shows that the contrastive loss L c is crucial. On the sentiment control task, our unsupervised method does not achieve good attribute alignment when the target sentiment is negative, but it performs well when the target sentiment is positive. One possible reason is that compared with the differences between toxic and normal sentences, the difference between positive sentiment and negative sentiment is more subtle, so it is more challenging for the GPT2 encoder in our unsupervised model to accurately separate the unlabeled data into two sentiments. As a result, the encoder's implicit criterion to categorize the input text may not be exactly the sentiment, which is also the reason that after removing the contrastive loss L c in the unsupervised loss function, the attribute relevance on the negative sentiment is higher while that on the positive sentiment is lower.
In the supervised setting with full data, our supervised method consistently achieves better controllability than PPLM while maintaining the linguistic quality of the generations (Table 1, 2). Although GeDi achieves a high attribute alignment score on the three tasks, it severely sacrifices the linguistic quality, as indicated by the high perplexity. In the few-shot setting, where the number of labeled training examples is reduced to 1000 or 24 examples per attribute, our supervised method can still maintain good controllability on the three tasks, showing the robustness of our method to the size of the training data.
Ablation study shows the importance of the discriminative loss L d in our supervised method. As mentioned in section 3, training without L d is equivalent to prefix-tuning. Comparing the results of Ours−L d and GPT2-medium show that directly using prefix-tuning can achieve controllability on the sentiment or the topic. However, it is less effective on detoxification. The reason is that different from topic control or sentiment control, detoxification requires the model to avoid generating some words or phrases according to the context, which can not be achieved by prefix-tuning. L d fills this gap by increasing p(x|y) and lowering p(x|ȳ) at the same time. Therefore, incorporating L d is of critical importance to the detoxification task. In the DBPedia topic control task, adding L d also achieves a large improvement on attribute alignment. The number of attributes in this task is much larger than that in the other tasks, so incorporating L d can effectively push the prefixes to capture the unique features of each topic.
We compare the average inference speed of our methods with the baselines ( Table 5). The inference speed of PPLM is several dozen times slower than that of the original GPT2 model. GeDi's inference speed is much faster than that of PPLM. The inference speed of our method is the closest to that of the original GPT2.

Human Evaluation
Besides automatic evaluation, we also conduct human evaluations on Amazon Mechanical Turk to compare the performance of the baselines and our methods. In each task, workers are presented with a prompt along with the completions generated by different methods. Workers are instructed to answer two questions:"Which one has the best linguistic quality?" and "The target attribute is [ATT]. Which one aligns best with the target attribute?". [ATT] is the control attribute used when generating the completions. In order to evaluate the linguistic quality and the attribute alignment separately, the workers are instructed not to consider the control aspect or the factual errors when answering the first question and not to consider the linguistic quality when answering the second question. The user interface provided to the workers is shown in the appendix ( Figure 5). We conduct human evaluations on the results of the sentiment control experiment and those of the AGNews topic control experiment separately. 100 tasks are randomly sampled from the results of each control experiment. Each task is assigned to 3 different Mechanical Turk workers and the annotations are aggregated by majority voting. To ensure data quality, we restrict the workers to be in Canada or United States with     a HIT approval rate higher than 95%. In total, 81 workers participated in the human evaluation. For the sentiment control task, we compare the results of GPT2-medium + prompt engineering, PPLM, GeDi, and our supervised method (with full training dataset). For the AGNews topic control task, PPLM is not evaluated as explained above. The results are shown in Table 4. The inter-annotator agreement on the sentiment task and the AGNews task is 0.39 and 0.30 in Fleiss' κ, respectively. Appendix B lists other details of the human evaluation.
In the sentiment control task, the result of human evaluation on linguistic quality is generally consistent with the result of automatic evaluation. However, different from the result of the automatic evaluation, annotators are more inclined to select Ours and GPT2 + prompt engineering when evaluating attribute alignment. Although the annotators are instructed not to consider linguistic quality when evaluating sentiment alignment, they tend to select the one with better linguistic quality when multiple completions exhibits equally good attribute alignment. In the AGNews topic control task, the result of human evaluation on attribute alignment is generally consistent with the result of automatic evaluation. However, in more than half of the linguistic quality questions, the annotators select Ours, although GPT2-medium + prompt engineering achieves lower perplexity than Ours. On inspection, we find that GPT2-medium + prompt  Table 6: Experimental results of the multi-aspect control task. "PPL.": perplexity scores. "Senti. Rel.": sentiment relevance. "Topic Rel.": topic relevance. "−L d / −L enc ": ablating loss terms as described in Eq. 3 and Eq. 12.
engineering in this task exhibits a more severe repetition problem compared to that in the sentiment control task. This inconsistency shows the limitation of using automatic evaluations, as alluded to in Welbl et al. (2021). Both human evaluation and automatic evaluation show that the linguistic quality of GeDi is inferior to that of the other methods. One possible reason is the length of the prompt. In the original experiment in Krause et al. (2020), each prompt is at least 150 characters for sentiment control evaluation and at least 30 characters for topic control evaluation. However, we use the prompts as in Dathathri et al. (2020), where the average prompt length is 11.8 characters for sentiment control evaluation and 14.5 characters for topic control evaluation. The generated examples are shown in the appendix (Table 7).

Multi-Aspect Control
Our method can also be applied to multi-aspect control. Directly applying our supervised method to multi-aspect control requires training examples with multi-aspect labels. However, such datasets are usually not readily available since most of the datasets are labeled for a single task. Although multi-aspect labeled examples are limited, we have training examples with single-aspect labels from multiple aspects, which can be utilized to achieve multi-aspect control. One method is to train a set of prefixes for each aspect separately using our supervised method and then concatenate the prefixes from different aspects for generation. This method is denoted as Ours (concatenation) in the result table. Another method is to train the prefixes of multiple aspects simultaneously by considering each single-aspect labeled example as partially labeled. We use a semi-supervised method for training, which is a combination of our supervised method and unsupervised method in Section 3. The model structure is the same as in the unsupervised method ( Figure 4). The loss function is as follows: where the latent variable z is the concatenation of the latent variable of each aspect, including both the supervised aspects and the unsupervised ones z = [z sup ; z uns ]. L enc is used to train the encoder. It is introduced because the partially labeled examples imply the ground truth indexes of the prefixes in the labeled aspect, providing supervision for both the prefix and the encoder. σ is the softmax function. We experiment with controlling the following two aspects simultaneously: sentiment and topic. We use the binary sentiment dataset from Amazon review (Zhang et al., 2015) and the DBPedia topic dataset. The prompts used for evaluation are the same as those in the topic control experiment. For each of the 20 prompts, 45 completions are generated. In the GPT2-medium + prompt engineering setting, the guiding sentence follows the template "This is a [SENTIMENT] review on [TOPIC]:". In Ours (concatenation), the sentiment prefixes and the topic prefixes are trained separately using our supervised method and then concatenated as multiaspect prefixes. In Ours (semi-supervised), we reuse the prefixes trained in the single-aspect control tasks to initialize H θ . All the experiments are conducted on NVIDIA Tesla V100 GPUs. The hyper-parameters are listed in appendix A.
Experimental results on multi-aspect control (Table 6) show that simply concatenating the prefixes trained for single-aspect control can effectively control the sentiment and topic simultaneously, and our experiments show that the order of the prefixes does not impact the result. On the other hand, training using the combination of our supervised and unsupervised methods can further improve the attribute alignment without sacrificing too much linguistic quality. Same as the observations stated in Section 4.1.2, removing the discriminative loss L d will significantly degrade the attribute relevance, especially the topic relevance. Removing the encoder loss L enc may achieve higher overall attribute relevance at the cost of linguistic quality, indicated by a higher perplexity. We present the generated examples in the appendix (Table 7).

Conclusion
We propose a novel framework for controllable GPT2 generation with frozen LMs, which utilizes contrastive prefixes to guide generation. Experimental results show that our framework can not only successfully guide generation from a single aspect but also achieve promising results on multiaspect control tasks. Besides the control tasks we experimented with, our proposed framework can be freely applied to other desired attributes.

Ethical Considerations
With our controlling methods, it is not one hundred percent guaranteed that the generations will have the desired attributes, but the probability for the generations to exhibit the desired attributes will increase. When applied to detoxification, although the probability of toxicity degeneration will decrease, the controlled language model may still produce unsafe text. We would like to clarify that the offensive language generated by the language model controlled with our methods does not represent any opinion of the authors.
Besides, our proposed methods control the highlevel attributes of the generation, such as toxicity, topic, or sentiment, but there is no guarantee of factual accuracy for the generation, which is a wellknown problem in NLG models. Our controlling methods may not be used for factual accuracy controlling. While reducing hallucination is not the focus of this work, knowledge-grounded generation techniques can be used to alleviate this problem.

Appendix A Hyperparameters
For PPLM and GeDi, we use the hyperparameters reported in their original work (Dathathri et al., 2020;Krause et al., 2020). Note that GeDi has multiple versions of submission available online and we refer to the latest one on OpenReivew.
Our methods are implemented using the Hugging face Transformers package. In all the experiments with our methods, the random seed is fixed to 42, and the optimizer is AdamW with a learning rate of 2e-5. D = 24 × 2 × 1024, where 24 is the number of hidden layers in GPT2-medium, 1024 is the size of hidden states in GPT2-medium, and 2 represent one key and one value. In the sentiment control task and the topic control tasks, the maximum generation length is set to 50 during evaluation while in the detoxification task the maximum generation length is set to 20. Unless stated otherwise, the prefix length M = 10.

Sentiment Control
In the Ours (unsupervised) setting, the training batch size is 8. ω 1 = 0.8, ω 3 = 2.0. The weight of the KL loss term ω 2 anneals from 0.001 to 0.1 during training while the temperature τ reduces from 1.0 to 0.5. The number of training epochs is 60. During training, we randomly mask the input tokens when computing the next token probabilities so as to force the prefix to preserve the key information of the input text. The mask rate is 0.5.
Detoxification In the Ours (unsupervised) setting, the training batch size is 8. ω 1 = 0.8, ω 3 = 2.0. The weight of the KL loss term ω 2 anneals from 0.001 to 0.1 during training while the temperature τ reduces from 1.0 to 0.5. The number of training epochs is 4. Same as in the sentiment control task, the mask rate is 0.5.
Multi-Aspect Control In the Ours (concatenation) setting, the sentiment prefix with length M = 10 and the topic prefix with length M = 10 are concatenated, so the resultant multi-aspect prefix has a length M = 20.

B Human Evaluation
The payment for each approved annotation is set to $0.6. The average completion time is 3 minutes 45 seconds per HIT (prorated to an hourly wage of $9.6). In fact, Bynum allegedly responded to Eley on Facebook: "A. Someone should tell me who my NBA team ____. Nice kid (10 pounds . . .

Ours
In fact, Bynum allegedly responded to Eley on Facebook: "Thanks for your suggestion. I'm going to start by using the phrase 'if you have something to . . .

GPT2+
The book contains some serious errors in spelling and grammar and there is little in the way of a coherent story. This review is therefore the negative review, for lack of a better term. It is a story about "The Dark Knight." It follows one. . .

PPLM
The book, The Big Empty, is the first of a series of three, to be released by the American Public Health Association in the fall. The report's goal is to "The new report ( ). A: It is . . .

Ours
The book I had read before I heard about the film and it was a huge disappointment. I am a huge fan of The Hobbit, so I was looking forward to watching this and this is not going to hold me back from enjoying the film. I am very . . .

Positive GPT2+
The potato chip is fantastic! I love it! My friends really liked it and the food is good as well. Went here for a quick lunch. We sat in the outdoor patio area next to a few of our friends. My sister and I . . .

PPLM
The potato, which is a staple of many people's diet, has been linked to the development of type 1 diabetes and heart problems. A group of British scientists has found the first evidence of an association between high-fiber eating and type 1 diabetes . . .

GeDi
The potato grower will adjust perfectly to your farm and farm animals. We offer high-quality fresh, all-natural produce that is grown outdoors. About Us Bib Flowermachine provides composting, hydroponics, water, fertile . . .

Ours
The potato chip is the classic American family meal. And while it's been around for decades, it still is the perfect dinner option for any family, whether you're a member of the household or not. But it is also an incredibly versatile meal. For example . . .

Sports GPT2+
Views on football. Football is the subject of a number of sports-related articles by the public domain, so this will not be repeated here. This article may not contain legal advice or should be considered legal advice in relation to your own legal . . .

GeDi
Views on Beckham MVP derby got into the mix Sunday weekend, as ESPN's Adam Schefter produced a great (& entire list we'll get to below) breakdown of all things Beckham. Basically, we popped the top of the pitcher (who may win to clear . . .

Ours
Views on this season are split. Some, like former Miami Dolphins quarterback Peyton Manning, believe the Patriots are a Super Bowl contender. Others, like former New England Patriots head coach Bill Belichick, say the Pats are a perennial loser.

World GPT2+
The central theme of the novel is the search for purpose and for meaning. However, the novel isn't just about these goals and meanings. It is also about life and death, personal relationships, and the way that life and death are often intertwined in the lives of . . .

GeDi
The central theme campaigner Najim Hasina uses is Kashmir peace, and with the Privy Council review being conducted towards the beginning of January, critical comments were placed on Delhi's artificiality andness in defence of watchdog. As has been stated, Rajesh G. . .

Ours
The central theme of the next few weeks will be the battle against terrorism, with Iraq at the top of the list.

{Negative, Company} Ours
The issue focused on accessories and software was one of the main reasons why Apple Inc. dropped the product line. The company did not realize that its product line would be the downfall of the company.

{Positive, Athlete} Ours
The issue focused on his game as a center back. He is an excellent athlete who has a strong work ethic. He is a good defensive midfielder who can make plays and get his team points. He plays a natural position as a right midfielder. Table 7: Examples of the generation. In the first column are control codes. "Negative": Negative Sentiment. "Positive": Positive Sentiment. The second column lists the methods. "GPT2+": GPT2-medium + prompt engineering. The given prompts are in bold. The guiding sentences of GPT2+ are omitted for brevity.