We rely on arguments in our daily lives to deliver our opinions and base them on evidence , making them more convincing in turn . However , finding and formulating arguments can be challenging . In this work , we present the Arg-CTRL-a language model for argument generation that can be controlled to generate sentence-level arguments for a given topic , stance , and aspect . We define argument aspect detection as a necessary method to allow this fine-granular control and crowdsource a dataset with 5,032 arguments annotated with aspects . Our evaluation shows that the Arg-CTRL is able to generate high-quality , aspectspecific arguments , applicable to automatic counter-argument generation . We publish the model weights and all datasets and code to train the Arg-CTRL . 1 Nuclear reactors produce radioactive waste ... Language models ( Bengio et al . , 2003 ) allow to generate text through learned distributions of a language and have been applied to a variety of areas like machine translation ( Bahdanau et al . , 2015 ) , summarization ( Paulus et al . , 2018 ) , or dialogue systems ( Wen et al . , 2017 ) . A rather new field for these models is the task of producing text with argumentative content ( Wang and Ling , 2016 ) . We believe this technology can support humans in the challenging task of finding and formulating arguments . A politician might use this to prepare for a debate with a political opponent or for a press conference . It may be used to support students in writing argumentative essays or to enrich one-sided discussions with counter-arguments . In contrast to retrieval methods , generation allows to combine and stylistically adapt text ( e.g. arguments ) based on a given input ( usually the beginning of a sentence ) . Current argument generation models , however , produce lengthy texts and allow the user little control over the aspect the argument should address ( Hua et al . , 2019 ; Hua and Wang , 2018 ) . We show that argument generation can be enhanced by allowing for a fine-grained control and limiting the argument to a single but concise sentence . Controllable language models like the CTRL ( Keskar et al . , 2019 ) allow to condition the model at training time to certain control codes . At inference , these can be used to direct the model 's output with regard to content or style . We build upon this architecture to control argument generation based solely on a given topic , stance , and argument aspect . For instance , to enforce focus on the aspect of cancer for the topic of nuclear energy , we input a control code " Nuclear Energy CON cancer " that creates a contra argument discussing this aspect , for instance : " Studies show that people living next to nuclear power plants have a higher risk of developing cancer . " . To obtain control codes from training data , we pre-define a set of topics to retrieve documents for and rely on an existing stance detection model to classify whether a sentence argues in favor ( pro ) or against ( con ) the given topic ( Stab et al . , 2018a ) . Regarding argument aspect detection , however , past work has two drawbacks : it either uses simple rule-based extraction of verb-and noun-phrases ( Fujii and Ishikawa , 2006 ) or the definition of aspects is based on target-concepts located within the same sentence ( Gemechu and Reed , 2019 ) . Aspects as we require and define them are not bound to any part-of-speech tag and ( 1 ) hold the core reason upon which the conclusion / evidence is built and ( 2 ) encode the stance towards a general but not necessarily explicitly mentioned topic the argument discusses . For instance : Topic : Nuclear Energy Argument : Running nuclear reactors is costly as it involves long-time disposal of radioactive waste . The evidence of this argument is based upon the two underlined aspects . While these aspects encode a negative stance towards the topic of " Nuclear Energy " , the topic itself is not mentioned explicitly in the argument . Our final controlled argument generation pipeline ( see Figure 1 ) works as follows : ( 1 ) We gather several million documents for eight different topics from two large data sources . All sentences are classified into pro- , con- , and non-arguments . We detect aspects of all arguments with a model trained on a novel dataset and concatenate arguments with the same topic , stance , and aspect into training documents . ( 2 ) We use the collected classified data to condition the Arg-CTRL on the topics , stances , and aspects of all gathered arguments . ( 3 ) At inference , passing the control code [ Topic ] [ Stance ] [ Aspect ] to the model will generate an argument that follows these commands . Our evaluation shows that the Arg-CTRL is able to produce aspect-specific , high-quality arguments , applicable to automatic counter-argument generation . The contributions are as follows : ( i ) We adapt and fine-tune the CTRL for aspect-controlled neural argument generation . ( ii ) We show that detecting argument aspects and conditioning the generation model on them are necessary steps to control the model 's training process and its perspective while generating . ( iii ) We propose several methods to analyze and evaluate the quality of ( controllable ) argument generation models . ( iv ) We develop a new scheme to annotate argument aspects and release a dataset with 5,032 samples . We apply the concept of controlled neural text generation to the domain of argument generation . Our Arg-CTRL is conditioned on topics , stances , and aspects and can reliably create arguments using these control codes . We show that arguments generated with our approach are genuine and of high argumentative and grammatical quality in general . Moreover , we show that our approach can be used to generate counter-arguments in a transparent and interpretable way . We fine-tune the Arg-CTRL on two different data sources and find that using mixed data from Common-Crawl results in a higher quality of generated arguments than using user discussions from Reddit-Comments . Further , we define argument aspect detection for controlled argument generation and introduce a novel annotation scheme to crowdsource argument aspect annotations , resulting in a high-quality dataset . We publish the model weights , data , and all code necessary to train the Arg-CTRL .
Argumentative content generation can support humans but current models produce lengthy texts and offer a little controllability on aspects of the argument for users.
They train a controllable language model on a corpus annotated with control codes provided by a stance detection model and introduce a dataset for evaluation.
The proposed model can generate arguments that are genuine and argumentative and grammatically correct and also counter-arguments in a transparent and interpretable way.