Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> We describe our work in progress on natural language analysis in medical question-answering in the context of a broader medical text-retrieval project. We analyze the limitations in the medical domain of the technologies that have been developed for general question-answering systems, and describe an alternative approach whose organizing principle is the identification of semantic roles in both question and answer texts that correspond to the fields of PICO format. <s> BIB001 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> We explore a semantic abstraction approach to automatic summarization in the biomedical domain. The approach relies on a semantic processor that functions as the source interpreter and produces a list of predications. A transformation stage then generalizes and condenses this list, ultimately generating a conceptual condensate for a disorder input topic. The final condensate is displayed in graphical form. We provide a set of principles for the transformation stage and describe the application of this approach to multidocument input. Finally, we examine the characteristics and quality of the condensates produced. <s> BIB002 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> BackgroundTopic detection is a task that automatically identifies topics (e.g., "biochemistry" and "protein structure") in scientific articles based on information content. Topic detection will benefit many other natural language processing tasks including information retrieval, text summarization and question answering; and is a necessary step towards the building of an information system that provides an efficient way for biologists to seek information from an ocean of literature.ResultsWe have explored the methods of Topic Spotting, a task of text categorization that applies the supervised machine-learning technique naïve Bayes to assign automatically a document into one or more predefined topics; and Topic Clustering, which apply unsupervised hierarchical clustering algorithms to aggregate documents into clusters such that each cluster represents a topic. We have applied our methods to detect topics of more than fifteen thousand of articles that represent over sixteen thousand entries in the Online Mendelian Inheritance in Man (OMIM) database. We have explored bag of words as the features. Additionally, we have explored semantic features; namely, the Medical Subject Headings (MeSH) that are assigned to the MEDLINE records, and the Unified Medical Language System (UMLS) semantic types that correspond to the MeSH terms, in addition to bag of words, to facilitate the tasks of topic detection. Our results indicate that incorporating the MeSH terms and the UMLS semantic types as additional features enhances the performance of topic detection and the naïve Bayes has the highest accuracy, 66.4%, for predicting the topic of an OMIM article as one of the total twenty-five topics.ConclusionOur results indicate that the supervised topic spotting methods outperformed the unsupervised topic clustering; on the other hand, the unsupervised topic clustering methods have the advantages of being robust and applicable in real world settings. <s> BIB003 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> Knowing the polarity of clinical outcomes is important in answering questions posed by clinicians in patient treatment. We treat analysis of this information as a classification problem. Natural language processing and machine learning techniques are applied to detect four possibilities in medical text: no outcome, positive outcome, negative outcome, and neutral outcome. A supervised learning method is used to perform the classification at the sentence level. Five feature sets are constructed: unigrams, bigrams, change phrases, negations, and categories. The performance of different combinations of feature sets is compared. The results show that generalization using the category information in the domain knowledge base Unified Medical Language System is effective in the task. The effect of context information is significant. Combining linguistic features and domain knowledge leads to the highest accuracy. <s> BIB004 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> Physicians have many questions when caring for patients, and frequently need to seek answers for their questions. Information retrieval systems (e.g., PubMed) typically return a list of documents in response to a user’s query. Frequently the number of returned documents is large and makes physicians’ information seeking “practical only ‘after hours’ and not in the clinical settings”. Question answering techniques are based on automatically analyzing thousands of electronic documents to generate short-text answers in response to clinical questions that are posed by physicians. The authors address physicians’ information needs and described the design, implementation, and evaluation of the medical question answering system (MedQA). Although our long term goal is to enable MedQA to answer all types of medical questions, currently, we currently implement MedQA to integrate information retrieval, extraction, and summarization techniques to automatically generate paragraph-level text for definitional questions (i.e., “What is X?”). MedQA can be accessed at http://www.dbmi.columbia.edu/~yuh9001/research/MedQA.html. <s> BIB005 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> Multiple pieces of text describing various pieces of evidence in clinical trials are often needed in answering a clinical question. We explore a multi-document summarization approach to automatically find this information for questions about effects of using a medication to treat a disease. Sentences in relevant documents are ranked according to various features by a machine-learning approach. Those with higher scores are more important and will be included in the summary. The presence of clinical outcomes and their polarity are incorporated into the approach as features for determining importance of sentences, and the effectiveness of this is investigated, along with that of other textual features. The results show that information on clinical outcomes improves the performance of summarization. <s> BIB006 </s> Automated text summarisation and evidence-based medicine: A survey of two domains <s> Detailed Review of Systems: The EBM Perspective <s> As the number of electronic biomedical textual resources increases, it becomes harder for physicians to find useful answers at the point of care. Information retrieval applications provide access to databases; however, little research has been done on using automatic summarization to help navigate the documents returned by these systems. After presenting a semantic abstraction automatic summarization system for MEDLINE citations, we concentrate on evaluating its ability to identify useful drug interventions for 53 diseases. The evaluation methodology uses existing sources of evidence-based medicine as surrogates for a physician-annotated reference standard. Mean average precision (MAP) and a clinical usefulness score developed for this study were computed as performance metrics. The automatic summarization system significantly outperformed the baseline in both metrics. The MAP gain was 0.17 (p<0.01) and the increase in the overall score of clinical usefulness was 0.39 (p<0.05). <s> BIB007
While discussing approaches in the previous subsection, we attempted to only provide the reader with a flavour of the various directions in which medical text summarisation has progressed. We intentionally skipped some important systems and filtered out many. In this subsection, we discuss the characteristics of a number of systems that perform summarisation of medical text. For full QA systems, we primarily focus on their summarisation components. This review not only discusses the capabilities of the systems but also presents their strengths and weaknesses from the perspective of EBM. Table I BIB005 answers definitional questions by producing paragraph-level answers from MEDLINE and the web. Syntactic parsing, query formulation, and query classification techniques are used to prepare queries, and an IR engine retrieves relevant documents . The answer extraction component employs document zone detection and sentence categorisation via the identification of cue phrases. Definitional sentences are identified in this fashion. Hierarchical clustering BIB003 ] and centroid-based summarisation techniques ] are used for text summarisation. The system's ability to answer definitional questions is evaluated against three search engines-Google, OneLook and PubMed ; Evaluations show that Google is very effective in obtaining definitions, outperforming MedQA. MedQA has opened new directions in medical text summarisation. It has shown how supervised classification can be used for intermediate steps in medical text summarisation, such as query analysis. MedQA, however, is not capable of incorporating semantic information, and the intent of the system is very different from the requirements of the EBM practitioner. The key limitation of the system is that it can only answer definitional questions, and not real life-like complex ones. 36 that uses a statistical and knowledge-based approach and is particularly targeted towards the practice of EBM. The proposed system uses PICO representations of questions as queries which are sent to PubMed to retrieve an initial set of abstracts. From the abstracts, each of the PICO elements (Problem/Population, Intervention, Comparison and Outcome) are extracted using various techniques. MetaMap is extensively used to identify UMLS terms and their categories. The population extractor uses a series of hand-crafted rules to identify occurrences of population terms in the abstracts, with preference given to terms occurring earlier in the documents. The problem extractor identifies elements that represent the UMLS semantic group 'DISORDER'. The intervention and comparison elements are identified in a similar way-by recognition of multiple UMLS semantic types (e.g., clinical drug, diagnostic procedure, etc.). In structured abstracts, more weight is given if the semantic types occur in 'title', 'aims' or 'methods' sections, while, in unstructured abstracts, more weight is given if they appear towards the beginning of the document. Using the extracted knowledge, the authors re-rank the retrieved documents using a document ranking algorithm that takes into account the knowledge elements, strength of evidence and other task specific considerations . An outcome extractor extracts outcome sentences from the retrieved documents using a "strategy based on an ensemble of classifiers (a rule-based classifier, a bag-of-words classifier, an n-gram classifier, a position classifier, an abstract length classifier and a semantic classifier)" . Each sentence is given a probability based on the classifier scores, and the top-ranked sentences are chosen. As the final output, the system simply produces the top ranked sentences from the top re-ranked documents along with the question and the strength of recommendation. Only basic clinical questions such as 'What is the best drug treatment for X?' are addressed. The authors evaluate the performance of the knowledge extractors against different baselines and also manually evaluate the final output against a baseline that only presents top sentences from unranked documents. The techniques applied by the CQA system show the importance of statistics and domain-specific knowledge for medical text summarisation. There are several limitations of the system. It relies on PICO frames for queries (instead of natural language questions), and an information synthesis technique is absent at the end to produce a single answer from related documents. Furthermore, the algorithm applied to predict the qualities of evidences does not follow an evidence-based guideline. 4.4.3 Summarisation using SemRep. BIB002 ,  and BIB007 propose an approach to abstractive summarisation that primarily relies on utilising semantic information. The summariser uses SemRep as a semantic processor to perform source interpretation and predication listing, and relies on user-specified topics. A transformation stage generalises and condenses the list of predicates generating a conceptual condensate for the input topic. The transformation is carried out in four phases: Relevance, Connectivity, Novelty, and Saliency. First, the relevance phase is responsible for identifying predications for a particular topic (e.g., treatments). The UMLS semantic network is utilised, along with a controlled schema, to identify predications. The predications, which must conform to the schema, are called the 'core predications. The Connectivity phase is a generalisation process and retrieves all predications that share arguments with core predications. The Novelty phase condenses the summaries further by removing general argument predications (e.g., Pharmaceutical Preparations), which are predications that appear higher in the UMLS metathesaurus hierarchy. Finally, the Saliency phase calculates the occurrence of uses simple frequency measures to keep the representative predicates, predications, and arguments. The final summary is produced in the form of a graph, and the approach can be applied to both single documents and multiple documents without requiring any modifications. One of the drawbacks of the system is that it does not incorporate query information. The system is evaluated BIB007 ] on its capability to discover drug interventions for disorders/syndromes only (other forms of interventions are not taken into account). Therefore, the application domain of the system is very limited. Furthermore, only clinical trials are used as source documents. The performance of the system is compared to a baseline that selects drug names based on the frequencies of their occurrence in source texts. A scoring mechanism called the clinical usefulness score is used. It rewards the systems for identifying beneficial drug interventions and penalises them for identifying harmful or not useful ones. To assess the usefulness of drug interventions, drug listings in Clinical Evidence (CE) articles are used as gold standards, in place of manually annotated references. The system is shown to outperform the baseline in terms of both the clinical usefulness score and mean average precision (MAP). The summarisation approach applied by the SemRep system is simple, innovative, and effective. Its performance illustrates the importance of domain-specific semantic information, and the usefulness of distributional semantics in automated summarisation for this domain. Incorporating query-focus into the summarisation procedure could technique more applicable for EBM practice. Summary information requirements must be identified from clinical questions instead of manually identified topics, and the summarisation component should be able to identify information other than drug interventions. 4.4.4 EPoCare. The EpoCare (Evidence at Point of Care) 37 project BIB001  BIB004 BIB006 is an initiative by the University of Toronto to develop a clinical QA system. The current implementation relies heavily on automatic identification of PICO elements from both clinical questions and their corresponding answers. PICO keywords are first identified from the question and used as keywords for retrieval. The problem of identifying answers to a clinical question is divided into four sub-problems -(i) identifying roles (PICO elements) in the text, (ii) identifying the lexical boundary of each element, (iii) analysing the relationships between distinct elements and (iv) determining which combinations of roles are most likely to contain correct answers. The initial work presented by Niu and colleagues addresses simple treatment-type questions. MetaMap is used for 37 http://www.cs.toronto.edu/km/epocare/index.html. Accessed on 15th Jan, 2016.