Patent retrieval: a literature review <s> Interactive methods <s> Professional Search is usually a recall-oriented problem. For helping the user to get efficiently a concise overview, to quickly restrict the search space and to make sense of the results, in this article we present an exploratory strategy for professional search that is based on semantic post-analysis of the classical search results (of keyword based queries). The described strategy can exploit the metadata that are already available, as well as the results of textual clustering and entity mining that can be performed at query time. The outcome of this process (i.e. metadata, clusters and entities grouped in categories) complement the ranked list of results produced from the core search engine with useful information for the user. This extra information is useful not only for providing a concise overview of the search results, but also for supporting a faceted and session-based interaction scheme that allows the users to restrict their focus gradually and to explore other related information. To tackle the corresponding configuration requirements of this process, we show how one can exploit the (constantly evolving) Linked Data for specifying the entities of interest and for providing further information about the identified entities. In this article, apart from detailing the steps of this process, we present applications of this approach in the marine domain and in the domain of patent search. <s> BIB001 </s> Patent retrieval: a literature review <s> Interactive methods <s> Abstract We present PerFedPat, an interactive patent search system based on the federated search approach and the ezDL framework. PerFedPat provides core services to search, using a federated method, multiple online patent resources (currently Espacenet, Google patents, Patentscope and the MAREC collection), thus providing parallel access to multiple patent sources. PerFedPat hides complexity from the end user who uses a common single query tool for querying all patent datasets at the same time. PerFedPat provides cores services such as Boolean and fielded search, merging, grouping and filtering of results, and offers support for query history and search sessions. The second innovative feature of PerFedPat is that it has a pluggable and extensible architecture and it enables the use of multiple search tools which are integrated in PerFedPat. Currently tools are integrated for IPC classification search, faceted navigation, clustering of search results and machine translation of queries. As a result the system is able to provide a rich, personalized information seeking experience for different types of patent searches, potentially exploiting techniques from diverse areas such as distributed information retrieval, semantic search, machine learning and human–computer interaction. The system is available for download from the internet. <s> BIB002 </s> Patent retrieval: a literature review <s> Interactive methods <s> Patent search tasks are challenging and often require many hours or even days to be completed. Patent search systems that integrate multiple search tools could assist patent examiners to complete the demanding patent search tasks by using the set of search tools most suitable for the task at hand. PerFedPat is an interactive patent search system designed on this principle and based on the federated search approach and the ezDL framework. PerFedPat provides core services to search multiple online patent resources, while hiding complexity from the end user. The extensible architecture of the system enables the integration and parallel use of multiple search tools. In this paper, we present part of a user study of the PerFedPat system and we also discuss the results, mostly focused on the research question: could the patent examiners efficiently use a patent search system that integratesmultiple resources, search tools and UIs? <s> BIB003 </s> Patent retrieval: a literature review <s> Interactive methods <s> Abstract Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P <s> BIB004 </s> Patent retrieval: a literature review <s> Interactive methods <s> Mined Semantic Analysis (MSA) is a novel distributional semantics approach which employs data mining techniques. MSA embraces knowledge-driven analysis of natural languages. It uncovers implicit relations between concepts by mining for their associations in target encyclopedic corpora. MSA exploits not only target corpus content but also its knowledge graph (e.g., "See also" link graph of Wikipedia). Empirical results show competitive performance of MSA compared to prior state-of-the-art methods for measuring semantic relatedness on benchmark data sets. Additionally, we introduce the first analytical study to examine statistical significance of results reported by different semantic relatedness methods. Our study shows that, top performing results could be statistically equivalent though mathematically different. The study positions MSA as one of state-of-the-art methods for measuring semantic relatedness. <s> BIB005 </s> Patent retrieval: a literature review <s> Interactive methods <s> In this paper, we investigate the influence of term selection on retrieval performance on the CLEF-IP prior art test collection, using the Description section of the patent query with Language Model (LM) and BM25 scoring functions. We find that an oracular relevance feedback system that extracts terms from the judged relevant documents far outperforms the baseline and performs twice as well on MAP as the best competitor in CLEF-IP 2010. We find a very clear term selection value threshold for use when choosing terms. We also noticed that most of the useful feedback terms are actually present in the original query and hypothesized that the baseline system could be substantially improved by removing negative query terms. We tried four simple automated approaches to identify negative terms for query reduction but we were unable to notably improve on the baseline performance with any of them. However, we show that a simple, minimal interactive relevance feedback approach where terms are selected from only the first retrieved relevant document outperforms the best result from CLEF-IP 2010 suggesting the promise of interactive methods for term selection in patent prior art search. <s> BIB006 </s> Patent retrieval: a literature review <s> Interactive methods <s> In this demo we present a semantic framework for innovation and patent analytics powered by Mined Semantic Analysis (MSA). Our framework provides cognitive assistance to its users through a Web-based visual and interactive interface. First, we describe building a conceptual knowledge graph by mining user-generated encyclopedic textual corpus for semantic associations. Then, we demonstrate applying the acquired knowledge to support many cognition and knowledge based use cases for innovation analysis including technology exploration and landscaping, competitive analysis, literature and prior art search and others. <s> BIB007
Interactive patent retrieval is inevitable. As we can notice from the above review, effective fully automated retrieval of patent prior art is very challenging. Best methods perform around average in terms of PRES and much less in terms of MAP. Additionally, these methods require tuning a large number of parameters and thresholds whose optimal values differ according to the given query and the specific information need, for example, deciding which patent section to use, which PRF results, and which expansion terms and their respective weights. The answers of these questions are not deterministic and probably require multiple interaction cycles with the user in order to satisfy his/her information need. Current interactive methods in patent retrieval are more focused on better organization, integration, and utilization of structured and textual patent data than on better retrieval performance. In other words, patent retrieval is addressed as a professional search problem rather than prior art search problem. Fafalios and Tzitzikas BIB001 presented a keyword-based interactive search framework to support patent search. The interaction elements are presented through post-analysis of search results in the form of facet-based features such as static metadata (e.g., IPC codes), textual clustering, named entity extraction, semantic enrichments, and others. The framework was applied on patent search BIB002 and evaluated using user study of twelve patent examiners BIB003 . Evaluation responses indicated overall acceptance of the framework in terms of usability, ease of use, efficiency, learnability. However, the authors did not report on the effectiveness or success of the system helping patent examiners to find prior art. Shalaby and Zadrozny  proposed a visual interactive semantic framework for patent analysis which features semantic-based query expansion of search queries using Mined Semantic Analysis (MSA) BIB005 . In a nutshell, MSA builds an association knowledge graph using rule mining of concept rich textual corpora (e.g., Wikipedia). After mining the "See Also" link graph of Wikipedia, MSA could represent a topic query as a Bag of Concepts (BOC) derived from the association knowledge graph. This BOC could then be used to expand the original query terms. Figure 2 shows an example of the query expansion map of Cognitive Analytics. Another example is presented in Fig. 3 showing concept map of 10 patents of Bank of America using the abstract section. Users can interact with the concept map by removing nodes and updating the search results. Shalaby et al. BIB007 demonstrated the applicability of their framework to support tasks such as prior art search, competitive intelligence, technology landscape analysis and exploration. However, they did not provide a controlled study evaluating the performance of their method on benchmark collections. Developing interactive methods for patent retrieval is also motivated by recent analysis which showed significant performance improvement if only one relevant document was manually provided by the user BIB006 . Performance gains using technology-assisted review (TAR) BIB004  in domains such as electronic discovery motivates investigating the applicability of machine learning TAR protocols in patent retrieval. Technology-assisted review, such as patent retrieval, is a total-recall task where it is required to find all relevant documents to the search request with reasonable effort (time and cost). It is thus a human-in-the-loop process where a human expert manually annotates a subset of the documents as relevant or irrelevant. The underlying algorithm subsequently builds a ranking model by training on such annotations and uses this model to promote more relevant results and demote irrelevant ones as more documents are searched and annotated. This process stops when enough results are obtained. Typically, these algorithms utilize techniques such as continuous active learning combined with Boolean search in order to develop and adapt the ranking model  . Several questions still need to be addressed when it comes to investigating technologyassisted review protocols applicability to patent retrieval, as these protocols were only evaluated in ad hoc search scenarios. The complexity of patents terminology and availability of multiple sources of metadata would, likely, demonstrate many opportunities for adaptation and modifications to the current technology-assisted review protocols.