The encoder-decoder model is widely used in natural language generation tasks. However, the model sometimes suffers from repeated redundant generation, misses important phrases, and includes irrelevant entities. Toward solving these problems we propose a novel source-side token prediction module. Our method jointly estimates the probability distributions over source and target vocabularies to capture a correspondence between source and target tokens. The experiments show that the proposed model outperforms the current state-of-the-art method in the headline generation task. Additionally, we show that our method has an ability to learn a reasonable token-wise correspondence without knowing any true alignments.
Apart from the , some studies proposed methods to improve the performance of the headline generation task. incorporated AMR @cite_7 into the encoder to use the syntactic and semantic information of the source. also encoded additional information of the source such as TF-IDF, part-of-speech tags and named entities. modeled the typical structure of a headline, such as Who Action What'' with a variational auto-encoder. These approach improved the performance of the headline generation but it is unclear whether they can reduce the .
@cite_7
2787752238 2341718173 2735492478 2963585309
Automatic headline generation is related to automatic text summarization and it is useful to solve information flood problems. This paper aims at generating a headline using a recurrent neural network which is based on a machine translation approach. Our headline generator consists of an encoder and a decoder and they are constructed with Long Short Term Memory, which is one of recurrent neural networks. The encoder constructs distributed representation from the first sentence in an article and the decoder generated headlines from the distributed representation. In our experiments, we confirmed that our proposed method could generate appropriate headlines but in some articles this method generates meaningless headlines. The results show that our proposed method is superior to another approach, statistical machine translation from the viewpoint of ROUGE, which is an evaluation score of automatic text summarization. Furthermore, we could find that using an input sentence in reverse order improves the quality of headline generation. Automatic headline generation is an important research area within text summarization and sentence compression. Recently, neural headline generation models have been proposed to take advantage of well-trained neural networks in learning sentence representations and mapping sequence to sequence. Nevertheless, traditional neural network encoder utilizes maximum likelihood estimation for parameter optimization, which essentially constraints the expected training objective within word level instead of sentence level. Moreover, the performance of model prediction significantly relies on training data distribution. To overcome these drawbacks, we employ minimum risk training strategy in this paper, which directly optimizes model parameters with respect to evaluation metrics and statistically leads to significant improvements for headline generation. Experiment results show that our approach outperforms state-of-the-art systems on both English and Chinese headline generation tasks. Recently, neural models have been proposed for headline generation by learning to map documents to headlines with recurrent neural network. In this work, we give a detailed introduction and comparison of existing work and recent improvements in neural headline generation, with particular attention on how encoders, decoders and neural model training strategies alter the overall performance of the headline generation system. Furthermore, we perform quantitative analysis of most existing neural headline generation systems and summarize several key factors that impact the performance of headline generation systems. Meanwhile, we carry on detailed error analysis to typical neural headline generation systems in order to gain more comprehension. Our results and conclusions are hoped to benefit future research studies. Headline generation for spoken content is important since spoken content is difficult to be shown on the screen and browsed by the user. It is a special type of abstractive summarization, for which the summaries are generated word by word from scratch without using any part of the original content. Many deep learning approaches for headline generation from text document have been proposed recently, all requiring huge quantities of training data, which is difficult for spoken document summarization. In this paper, we propose an ASR error modeling approach to learn the underlying structure of ASR error patterns and incorporate this model in an Attentive Recurrent Neural Network (ARNN) architecture. In this way, the model for abstractive headline generation for spoken content can be learned from abundant text data and the ASR data for some recognizers. Experiments showed very encouraging results and verified that the proposed ASR error model works well even when the input spoken content is recognized by a recognizer very different from the one the model learned from.