A Survey of Unstructured Text Summarization Techniques <s> I. INTRODUCTION <s> In this paper, we propose two generic text summarization methods that create text summaries by ranking and extracting sentences from the original documents. The first method uses standard IR methods to rank sentence relevances, while the second method uses the latent semantic analysis technique to identify semantically important sentences, for summary creations. Both methods strive to select sentences that are highly ranked and different from each other. This is an attempt to create a summary with a wider coverage of the document's main content and less redundancy. Performance evaluations on the two summarization methods are conducted by comparing their summarization outputs with the manual summaries generated by three independent human evaluators. The evaluations also study the influence of different VSM weighting schemes on the text summarization performances. Finally, the causes of the large disparities in the evaluators' manual summarization results are investigated, and discussions on human text summarization patterns are presented. <s> BIB001 </s> A Survey of Unstructured Text Summarization Techniques <s> I. INTRODUCTION <s> This study evaluates the effectiveness of simple and expert searches in Google Scholar (GS), EconLit, GEOBASE, PAIS, POPLINE, PubMed, Social Sciences Citation Index, Social Sciences Full Text, and Sociological Abstracts. It assesses the recall and precision of 32 searches in the field of later-life migration: nine simple keyword searches and 23 expert searches constructed by demography librarians at three top universities. For simple searches, Google Scholar’s recall and precision are well above average. For expert searches, the relative effectiveness of GS depends on the number of results users are willing to examine. Although Google Scholar’s expert-search performance is just average within the first fifty search results, GS is one of the few databases that retrieves relevant results with reasonably high precision after the fiftieth hit. The results also show that simple searches in GS, GEOBASE, PubMed, and Sociological Abstracts have consistently higher recall and precision than expert searches. This can be attributed not to differences in expert-search effectiveness, but to the unusually strong performance of simple searches in those four databases. <s> BIB002
The rapid growth of online information services, social media and other digital format documents means that huge amounts of information are becoming immediately available and readily accessible to a large number of end-users. However, human ability to organize and understand a large number of documents is limited. This well-known information overload problem is most acute when we need to make a decision or understand something deeply, which typically involves reviewing several documents, but have limited time. Reading through long documents consumes precious time in understanding the gist of the document. Web search engines look for documents from the Internet based upon user supplied queries. They not only overwhelm users with too many results, they also provide documents that may not be very relevant to the topic being studied by the user. For example, if the user is searching using some keyword and the search engine finds it somewhere inside a document, that document will be a "search hit" even if the document is not really relevant to the keyword. The most common search method is based on maintaining an inverted list (text index) of documents' text. Not only precision is hurt by indexing every word in the document, excluding stop words, but also efficiency is adversely impacted. If summaries are indexed and searched instead, index size will be considerably smaller and search hits will be of better quality (fewer false positives) BIB002 . This can be explained using the definition of Precision and Recall measures used in information retrieval. Precision is defined as the percentage of the relevant items in the returned set and Recall is the percentage of the relevant items in the returned set compared to those in the collection. If the whole collection is retrieved, then the Recall is 100%, but Precision is low. Most search engines suffer from this problem (high Recall and low Precision). If search engines search only a document's primary ideas, instead of every word, then Recall will likely not be decreased but Precision will likely improve. Hence, an automated facility for summarizing documents to improve productivity is desirable. A good summarization system should include only sentences that are most important to a document's theme; it must also cover all documents' topics BIB001 . Using a summary instead of the whole document as a representative of what the document is about would mean processing a fraction (20% or less) of the document's text, yet yield better Precision and lesser processing time for search queries. In order to determine the requirements of a good summarization system, many text summarization approaches were reviewed. An in-depth review of text summarization literature was conducted and results from this study along with a description of each algorithm, its strengths and weaknesses are presented in this article. Section II presents an overview of the major types of text summarization techniques. Section III provides detailed information on unsupervised text summarization techniques. The evaluation techniques used for assessing the quality of text summarization systems are also discussed in section IV. It was found that due to the shortcomings of the text summarization approaches currently available, there is a lack of a universal approach for document summarization that provides high Precision and Recall with various types of text corpora.