Recently, neural models have been proposed for headline generation by learning to map documents to headlines with recurrent neural networks. Nevertheless, as traditional neural network utilizes maximum likelihood estimation for parameter optimization, it essentially constrains the expected training objective within word level rather than sentence level. Moreover, the performance of model prediction significantly relies on training data distribution. To overcome these drawbacks, we employ minimum risk training strategy in this paper, which directly optimizes model parameters in sentence level with respect to evaluation metrics and leads to significant improvements for headline generation. Experiment results show that our models outperforms state-of-the-art systems on both English and Chinese headline generation tasks.
Headline generation is a well-defined task standardized in DUC-2003 and DUC-2004. Various approaches have been proposed for headline generation: rule-based, statistical-based and neural-based. The rule-based models create a headline for a news article using handcrafted and linguistically motivated rules to guide the choice of a potential headline. Hedge Trimmer @cite_6 is a representative example of this approach which creates a headline by removing constituents from the parse tree of the first sentence until it reaches a specific length limit. Statistical-based methods make use of large scale training data to learn correlations between words in headlines and articles @cite_12 . The best system on DUC-2004, TOPIARY @cite_8 combines both linguistic and statistical information to generate headlines. There is also method make use of knowledge bases to generate better headlines. With the advances of deep neural networks, there are growing works that design neural networks for headline generation. @cite_17 proposes an attention-based model to generate headlines. @cite_5 proposes a recurrent neural network with long short term memory (LSTM) @cite_11 for headline generation. @cite_9 introduces copying mechanism into encoder-deconder architecture inspired by the Pointer Networks @cite_4 .
@cite_11 @cite_4 @cite_8 @cite_9 @cite_6 @cite_5 @cite_12 @cite_17
2787752238 2341718173 2293941196 2735492478
Automatic headline generation is related to automatic text summarization and it is useful to solve information flood problems. This paper aims at generating a headline using a recurrent neural network which is based on a machine translation approach. Our headline generator consists of an encoder and a decoder and they are constructed with Long Short Term Memory, which is one of recurrent neural networks. The encoder constructs distributed representation from the first sentence in an article and the decoder generated headlines from the distributed representation. In our experiments, we confirmed that our proposed method could generate appropriate headlines but in some articles this method generates meaningless headlines. The results show that our proposed method is superior to another approach, statistical machine translation from the viewpoint of ROUGE, which is an evaluation score of automatic text summarization. Furthermore, we could find that using an input sentence in reverse order improves the quality of headline generation. Automatic headline generation is an important research area within text summarization and sentence compression. Recently, neural headline generation models have been proposed to take advantage of well-trained neural networks in learning sentence representations and mapping sequence to sequence. Nevertheless, traditional neural network encoder utilizes maximum likelihood estimation for parameter optimization, which essentially constraints the expected training objective within word level instead of sentence level. Moreover, the performance of model prediction significantly relies on training data distribution. To overcome these drawbacks, we employ minimum risk training strategy in this paper, which directly optimizes model parameters with respect to evaluation metrics and statistically leads to significant improvements for headline generation. Experiment results show that our approach outperforms state-of-the-art systems on both English and Chinese headline generation tasks. Automatic headline generation is a sub-task of document summarization with many reported applications. In this study we present a sequence-prediction technique for learning how editors title their news stories. The introduced technique models the problem as a discrete optimization task in a feature-rich space. In this space the global optimum can be found in polynomial time by means of dynamic programming. We train and test our model on an extensive corpus of financial news, and compare it against a number of baselines by using standard metrics from the document summarization domain, as well as some new ones proposed in this work. We also assess the readability and informativeness of the generated titles through human evaluation. The obtained results are very appealing and substantiate the soundness of the approach. Recently, neural models have been proposed for headline generation by learning to map documents to headlines with recurrent neural network. In this work, we give a detailed introduction and comparison of existing work and recent improvements in neural headline generation, with particular attention on how encoders, decoders and neural model training strategies alter the overall performance of the headline generation system. Furthermore, we perform quantitative analysis of most existing neural headline generation systems and summarize several key factors that impact the performance of headline generation systems. Meanwhile, we carry on detailed error analysis to typical neural headline generation systems in order to gain more comprehension. Our results and conclusions are hoped to benefit future research studies.