Automatic summarisation is a popular approach to reduce a document to its main arguments. Recent research in the area has focused on neural approaches to summarisation, which can be very data-hungry. However, few large datasets exist and none for the traditionally popular domain of scientific publications, which opens up challenging research avenues centered on encoding large, complex documents. In this paper, we introduce a new dataset for summarisation of computer science publications by exploiting a large resource of author provided summaries and show straightforward ways of extending it further. We develop models on the dataset making use of both neural sentence encoding and traditionally used summarisation features and show that models which encode sentences as well as their local and global context perform best, significantly outperforming well-established baseline methods.
Datasets for extractive summarisation often emerged as part of evaluation campaigns for summarisation of news, organised by the Document Understanding Conference (DUC), and the Text Analysis Conference (TAC). DUC proposed single-document summarisation @cite_15 , whereas TAC datasets are for multi-document summarisation @cite_16 @cite_27 . All of the datasets contain roughly 500 documents.
@cite_27 @cite_15 @cite_16
2571932860 1598682430 2182572585
There has been a long history of research in text summarization by both the text retrieval and the natural language processing communities, but evaluation of this research has always presented problems. In 2001 NIST launched a new text summarization evaluation effort, guided by a roadmap from the research community and sponsored by the DARPA TIDES project. This paper is a report of the first formal evaluation in a new conference called the Document Understanding Conference (DUC). The summarization track at the Text Analysis Conference (TAC) is a direct continuation of the Document Understanding Conference (DUC) series of workshops, focused on providing common data and evaluation framework for research in automatic summarization. In the TAC 2008 summarization track, the main task was to produce two 100-word summaries from two related sets of 10 documents, where the second summary was an update summary. While all of the 71 submitted runs were automatically scored with the ROUGE and BE metrics, NIST assessors manually evaluated only 57 of the submitted runs for readability, content, and overall responsiveness.