Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> The end users of medical digital libraries need quick access to information that is specific to the patients under their care. We present a summarization system that finds and extracts results from multiple medical journal articles returned by a search, filters results that match the patient and merges and orders the remaining facts for the summary. Our approach features an integration of text categorization, information extraction, information fusion and text reformulation for the summarization task. <s> BIB001 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> Objective:: We present the summarization system in the PErsonalized Retrieval and Summarization of Images, Video and Language (PERSIVAL) medical digital library. Although we discuss the context of our summarization research within the PERSIVAL platform, the primary focus of this article is on strategies to define and generate customized summaries. Methods and material:: Our summarizer employs a unified user model to create a tailored summary of relevant documents for either a physician or lay person. The approach takes advantage of regularities in medical literature text structure and content to fulfill identified user needs. Results:: The resulting summaries combine both machine-generated text and extracted text that comes from multiple input documents. Customization includes both group-based modeling for two classes of users, physician and lay person, and individually driven models based on a patient record. Conclusions:: Our research shows that customization is feasible in a medical digital library. <s> BIB002 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> In this thesis, we present a user-sensitive approach to text summarization. One domain which would highly benefit front tailoring summaries to both individual and class-based user characteristics is the medical domain, where physicians and patients access similar information, each with their own needs and abilities. Our framework is a medical digital library for physicians and patients. We describe a summarizer, which generates summaries of findings in an input set of clinical studies. When a physician is treating a specific patient, he's looking for information relevant to the patient's history and problems. The summarizer takes the user's interests into account and presents only the findings pertaining to a user model, as approximated by an existing patient record. The same synthesis of information can also be of interest to the patient. The summarizer predicts which medical terms used in a text will be too technical for patients, and augments it, with appropriate definitions when necessary. ::: We adopt a generation-like architecture for our summarizer. However, because our input is textual and not semantic, new challenges arise. We operate over a content representation hybrid between full-semantic and extracted phrases. Our content organization strategy is dynamic and data-driven. This is in contrast to most summarizers which use no explicit strategies to order information extracted from several input documents. The result is more readable, coherent output. To generate the actual summary, the summarizer makes use of aggregation and phrasal generation. The result is a concise and fluent summary. ::: One key challenge when it comes to adapting a text for a different audience is identifying the bottleneck for reader comprehension. We analyzed corpora of technical and lay medical texts and qualified differences. We identified the presence of difficult vocabulary as the major obstacle to comprehension for lay readers. We designed an unsupervised method to predict which terms are incomprehensible for lay readers and provide the user with appropriate definitions. ::: Our methods are grounded on corpus analyses and feasibility studies conducted with physicians and consumers of health information. To assess the value of our work, we evaluated our summarizer both intrinsically and extrinsically. Our task-based evaluation conducted with physicians at the ICU demonstrates that personalized summaries help physicians access relevant information better than generic summaries. Evaluation with lay readers shows that our method to augment technical medical texts improves readers' comprehension significantly. <s> BIB003 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> Automatically extracting information needs from ad hoc clinical questions is an important step towards medical question answering. In this work, we first explored supervised machine-learning approaches to automatically classify an ad hoc clinical question into general topics. We then explored both unsupervised and supervised methods for automatically extracting keywords from an ad hoc clinical question. Our methods were evaluated on the 4,654 clinical questions maintained by the National Library of Medicine. Our best systems or methods showed F-score of 76% for the task of question-general topic classification and of 58% for extracting keywords from ad hoc clinical questions. <s> BIB004 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> This paper analyzes the topic identification stage of single-document automatic text summarization across four different domains, consisting of newswire, literary, scientific and legal documents. We present a study that explores the summary space of each domain via an exhaustive search strategy, and finds the probability density function (pdf) of the ROUGE score distributions for each domain. We then use this pdf to calculate the percentile rank of extractive summarization systems. Our results introduce a new way to judge the success of automatic summarization systems and bring quantified explanations to questions such as why it was so hard for the systems to date to have a statistically significant improvement over the lead baseline in the news domain. <s> BIB005 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> Objective ::: Clinical questions are often long and complex and take many forms. We have built a clinical question answering system named AskHERMES to perform robust semantic analysis on complex clinical questions and output question-focused extractive summaries as answers. <s> BIB006 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> We present an approach for extractive, query-focused, single-document summarisation of medical text. Our approach utilises a combination of target-sentence-specific and target-sentence-independent statistics derived from a corpus specialised for summarisation in the medical domain. We incorporate domain knowledge via the application of multiple domain-specific features, and we customise the answer extraction process for different question types. The use of carefully selected domain-specific features enables our summariser to generate content-rich extractive summaries, and an automatic evaluation of our system reveals that it outperforms other baseline and benchmark summarisation systems with a percentile rank of 96.8%. <s> BIB007 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> We propose a supervised classification approach for automatically determining the polarities of medical sentences. Our polarity classification approach is contextsensitive, meaning that the same sentence may have differing polarities depending on the context. Using a set of carefully selected features, we achieve 84.7% accuracy, which is significantly better than current state-of-the-art for the polarity classification task. Our analyses and experiments on a specialised corpus indicate that automatic polarity classification of key sentences can be utilised to generate evidence-based recommendations. <s> BIB008 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> 0:20 <s> Display Omitted We present an approach for query-focused extractive summarisation of medical text.Our approach is data-centric and utilises a specialised annotated corpus.We introduce target-sentence-specific summarisation to improve summarisation.We incorporate query information and domain-specific knowledge in various ways.Our approach performs significantly better than existing benchmark approaches. BackgroundEvidence-based medicine practice requires medical practitioners to rely on the best available evidence, in addition to their expertise, when making clinical decisions. The medical domain boasts a large amount of published medical research data, indexed in various medical databases such as MEDLINE. As the size of this data grows, practitioners increasingly face the problem of information overload, and past research has established the time-associated obstacles faced by evidence-based medicine practitioners. In this paper, we focus on the problem of automatic text summarisation to help practitioners quickly find query-focused information from relevant documents. MethodsWe utilise an annotated corpus that is specialised for the task of evidence-based summarisation of text. In contrast to past summarisation approaches, which mostly rely on surface level features to identify salient pieces of texts that form the summaries, our approach focuses on the use of corpus-based statistics, and domain-specific lexical knowledge for the identification of summary contents. We also apply a target-sentence-specific summarisation technique that reduces the problem of underfitting that persists in generic summarisation models. ResultsIn automatic evaluations run over a large number of annotated summaries, our extractive summarisation technique statistically outperforms various baseline and benchmark summarisation models with a percentile rank of 96.8%. A manual evaluation shows that our extractive summarisation approach is capable of selecting content with high recall and precision, and may thus be used to generate bottom-line answers to practitioners' queries. ConclusionsOur research shows that the incorporation of specialised data and domain-specific knowledge can significantly improve text summarisation performance in the medical domain. Due to the vast amounts of medical text available, and the high growth of this form of data, we suspect that such summarisation techniques will address the time-related obstacles associated with evidence-based medicine. <s> BIB009
• A. Sarker, D. Mollá, C. Paris automated identification of interventions and problems. The authors note that identification of outcomes is a much more difficult task, and they identify cue words (nouns, verbs and adjectives) that indicate the presence of outcomes in sentences. The outcome detection task is further subdivided into two sub-tasks -outcome identification and lexical boundary determination. In addition to sentence level outcome detection, the authors suggest that the polarity of outcomes play an important role in determining which sentences to choose as answers. Four categories of polarities are defined (positive, negative, no outcome and neutral). The authors use Support Vector Machines (SVMs) to classify sentences into the four categories, and show that best results are obtained by combining linguistic features with domain features. The outcomes of the polarity classification task are used in a multi-document summarisation approach to automatically find information from MEDLINE abstracts to answer a clinical question. Presence and polarity of outcomes, position of sentence in abstracts, length of sentences and Maximal Marginal Relevance (MMR) are used as features in a machine learning algorithm (SVM) used to solve the summarisation problem. Sentences in MEDLINE abstracts cited by Clinical Evidence (CE) articles are manually annotated for each clinical query, and sentences obtained from the automated approach are compared with these for evaluation. A total of 197 abstracts from 24 subsections in CE are annotated to give a total of 2,298 sentences. The outcome detection task is shown to have an accuracy of 83%. The polarity assessment task is shown to have an accuracy of 79.42%. For the summarisation task, it is observed that the identification of outcomes and polarity improves performance significantly. However, F-scores are shown to be very low (<0.50) in all cases. ROUGE is also used for evaluation but shows little variation in performance for distinct combinations of features. The outcomes of the EpoCare project show that abstractive summarisation approaches, which utilise automatic polarity classification of sentences, have the potential to be applied for the generation of evidence-based summaries. 4.4.5 PERSIVAL. PERSIVAL (PErsonalized Retrieval and summarisation of Image, Video and Language) BIB001 is a medical digital library designed to provide customised access to a distributed library of multimedia medical literature. It is not possible to discuss the whole project in this review, and hence we focus on the text summarisation component of the system that produces customised, abstractive summaries for persons from technical and non-technical backgrounds BIB002 BIB003 . The summarisation system takes as input three different sources: patient records, medical articles (about cardiology) that are appropriate for the patient, and a query by the user from which key words are extracted. The input articles are classified as prognosis, treatment or diagnosis. Relevant sentences from the 'Results' sections of the articles are extracted and stored in in a pre-defined template, which contains three sets of information: parameter(s), findings, and relation. The relation describes the relation between a parameter and a finding. The information extraction phase also identifies the extent to which the parameters are dependent, along with other meta-data, such as the position of the sentence from which a parameter has been extracted BIB002 . Following the extraction, the information is filtered to keep only the portions that are relevant to the patient's medical records, and hand-crafted templates are filled with the extracted elements. In the next step, the filled templates are converted into semantic representations via mapping them into a graph. Repetitions (when two nodes are connected via multiple vertices of the same type) and contradictions (when two nodes are connected by multiple vertices of different types) are identified from the graphs, and this information is used to generate a coherent summary. The representation is then ordered for summary generation: relations that are deemed relevant to the user question are given the highest preference followed by recitations and contradictions, which in turn are followed by preference based on the relation type (e.g., relations representing risks are given higher priority than association relations). Finally, dependent relations identified from one template are output together, and the final summary is generated using natural language generation techniques, along with hyperlinks to medical concepts. The summaries generated by the system are not evidence-based, and the intent of the system is to provide personalised information for users from both technical and non-technical backgrounds. The approaches applied by this system have the potential of being applied for evidence-based summarisation. 4.4.6 AskHERMES. AskHERMES 38 BIB006 ] is a clinical QA system that is capable of taking queries in natural language, performs thorough query analysis, and generates single-document, query-focused extracts from a group of texts that are relevant to the query. The input to the system, being in natural language, requires minimal query formulation by the user, and the interface allows quick navigation of the summaries. The system is demonstrated to outperform Google and UpToDate when answering complex clinical questions. AskHermes operates in five phases: a query analysis module automatically identifies the information needs posed by questions by generating a list of informative query terms; a related questions extraction module identifies a list of questions that are similar, based on the terms; an information retrieval module returns the relevant documents; an information extraction module identifies relevant passages from the source documents; and a summarisation and answer presentation module that analyses the identified relevant passages, identifies and removes redundant elements, and finally outputs summaries with structure. In the query analysis phase, a query is first classified into one of 12 general topics BIB004 , and then keywords are automatically extracted from the question using Conditional Random Fields. Following the retrieval of relevant documents, a two-layer hierarchical clustering is applied to group passages into different topics. Query terms and their UMLS mappings are used to assign the cluster labels. The first layer of clustering generates the topic labels for a tree structures, and a second layer of clustering is applied to provide more refined categories. The final set of clusters are ranked based on the key query terms that are found in in them, and redundancies are detected and removed using longest common substrings. AskHERMES is shown to perform comparably to state-of-the-art systems, and its potential application in the field of EBM is very promising. One issue is that the lengthy summaries do not satisfy the requirement of bottom-line recommendations required by practitioners. Due to the extractive nature of the summarisation, it is difficult to synthesise information from multiple documents and generate brief summaries. However, the system's performance suggests that customising the summary generation process to the type of question may be beneficial. Furthermore, the authors show that key query terms can be used to determine topics. 4.4.7 QSpec. QSpec 39   BIB007 BIB009 ] is a summarisation system that attempts to generate evidence-based summaries to complex medical queries posed by practitioners.The system utilises a publicly available data set 40 , which consists of real-life clinical queries posed by practitioners, evidence-based justifications in response to the queries (single-document summaries), and bottom-line recommendations (multi-document summaries). The proposed system consists of two primary components: (i) query-focused summariser BIB009 , and (ii) automatic evidence grader . The summariser module performs single-document, extractive summarisation based on multiple domain-specific and domain-independent 38 Available at: http://www.askhermes.org/index2.html. Accessed on 15th Jan, 2016. 39 http://web.science.mq.edu.au/ ∼ diego/medicalnlp/. Accessed on 2nd February, 2015. 40 http://sourceforge.net/projects/ebmsumcorpus/. Accessed on 2nd February, 2015. features, such as sentence positions, lengths, sentence classes, UMLS semantic types, and others. The system takes as input questions in natural language. Following the retrieval of the relevant documents, 41 single-document summaries are generated by selecting the three most appropriate sentences from each document. The crucial part of this component is the sentence selection process, which utilises statistics derived from the annotated data in the corpus. Using statistics associated with a range of topics (e.g., question-specific semantic types, associations between semantic types, maximal marginal relevance and others), the system attempts to apply weights to each sentence score, and finally ranks the sentences based on the scores. Multi-document summaries are generated by performing automatic, contextual polarity classification of sentences BIB008 . In addition to the multi-document summaries for each evidence, a supervised learning approach that utilises a sequence of classifiers  ] is applied to grade the quality of the evidence. The system is shown to perform significantly better than several baseline and benchmark systems on a scale that employs ROUGE F-scores for relative comparison of systems BIB005 . However, the generation of multi-document summaries is described as a much more difficult task, and the contextual polarity classifier is only applied to a subset of the corpus that addresses therapeutic questions. This system illustrates the usefulness of annotated corpora for this task, and it is likely that future research will utilise the publicly available data set that this system has been trained on.