Text Summarization Techniques: A Brief Survey <s> Evaluation of Automatically Produced Summaries <s> The TIPSTER Text Summarization Evaluation (SUMMAC) has developed several new extrinsic and intrinsic methods for evaluating summaries. It has established definitively that automatic text summarization is very effective in relevance assessment tasks on news articles. Summaries as short as 17% of full text length sped up decision-making by almost a factor of 2 with no statistically significant degradation in accuracy. Analysis of feedback forms filled in after each decision indicated that the intelligibility of present-day machine-generated summaries is high. Systems that performed most accurately in the production of indicative and informative topic-related summaries used term frequency and co-occurrence statistics, and vocabulary overlap comparisons between text passages. However, in the absence of a topic, these statistical methods do not appear to provide any additional leverage: in the case of generic summaries, the systems were indistinguishable in accuracy. The paper discusses some of the tradeoffs and challenges faced by the evaluation, and also lists some of the lessons learned, impacts, and possible future directions. The evaluation methods used in the SUMMAC evaluation are of interest to both summarization evaluation as well as evaluation of other ‘output-related’ NLP technologies, where there may be many potentially acceptable outputs, with no automatic way to compare them. <s> BIB001 </s> Text Summarization Techniques: A Brief Survey <s> Evaluation of Automatically Produced Summaries <s> In citation-based summarization, text written by several researchers is leveraged to identify the important aspects of a target paper. Previous work on this problem focused almost exclusively on its extraction aspect (i.e. selecting a representative set of citation sentences that highlight the contribution of the target paper). Meanwhile, the fluency of the produced summaries has been mostly ignored. For example, diversity, readability, cohesion, and ordering of the sentences included in the summary have not been thoroughly considered. This resulted in noisy and confusing summaries. In this work, we present an approach for producing readable and cohesive citation-based summaries. Our experiments show that the proposed approach outperforms several baselines in terms of both extraction quality and fluency. <s> BIB002
There have been several evaluation campaigns since the late 1990s in the US  . They include SUMMAC (1996-1998) BIB001 , DUC (the Document Understanding Conference, 2000-2007) [49] , and more recently TAC (the Text Analysis Conference, 2008-present) BIB002 . These conferences have primary role in design of evaluation standards and evaluate the summaries based on human as well as automatic scoring of the summaries. In order to be able to do automatic summary evaluation, we need to conquer three major difficulties: i) It is fundamental to decide and specify the most important parts of the original text to preserve. ii) Evaluators have to automatically identify these pieces of important information in the candidate summary, since this information can be represented using disparate expressions. iii) the readability of the summary in terms of grammaticality and coherence has to be evaluated.