Patent retrieval: a literature review <s> Semantic-based methods <s> We develop a method for predicting query performance by computing the relative entropy between a query language model and the corresponding collection language model. The resulting clarity score measures the coherence of the language usage in documents whose models are likely to generate the query. We suggest that clarity scores measure the ambiguity of a query with respect to a collection of documents and show that they correlate positively with average precision in a variety of TREC test sets. Thus, the clarity score may be used to identify ineffective queries, on average, without relevance information. We develop an algorithm for automatically setting the clarity score threshold between predicted poorly-performing queries and acceptable queries and validate it using TREC data. In particular, we compare the automatic thresholds to optimum thresholds and also check how frequently results as good are achieved in sampling experiments that randomly assign queries to the two classes. <s> BIB001 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Noun phrases in queries are identified and classified into four types: proper names, dictionary phrases, simple phrases and complex phrases. A document has a phrase if all content words in the phrase are within a window of a certain size. The window sizes for different types of phrases are different and are determined using a decision tree. Phrases are more important than individual terms. Consequently, documents in response to a query are ranked with matching phrases given a higher priority. We utilize WordNet to disambiguate word senses of query terms. Whenever the sense of a query term is determined, its synonyms, hyponyms, words from its definition and its compound words are considered for possible additions to the query. Experimental results show that our approach yields between 23% and 31% improvements over the best-known results on the TREC 9, 10 and 12 collections for short (title only) queries, without using Web data. <s> BIB002 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Pseudo-relevance feedback assumes that most frequent terms in the pseudo-feedback documents are useful for the retrieval. In this study, we re-examine this assumption and show that it does not hold in reality - many expansion terms identified in traditional approaches are indeed unrelated to the query and harmful to the retrieval. We also show that good expansion terms cannot be distinguished from bad ones merely on their distributions in the feedback documents and in the whole collection. We then propose to integrate a term classification process to predict the usefulness of expansion terms. Multiple additional features can be integrated in this process. Our experiments on three TREC collections show that retrieval effectiveness can be much improved when term classification is used. In addition, we also demonstrate that good terms should be identified directly according to their possible impact on the retrieval effectiveness, i.e. using supervised learning, instead of unsupervised learning. <s> BIB003 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> This paper presents the system called PATATRAS (PATent and Article Tracking, Retrieval and AnalysiS) realized at the Humboldt University for the IP track of CLEF 2009. Our approach presents three main characteristics: 1. The usage of multiple retrieval models (KL, Okapi) and term index definitions (lemma, phrase, concept) for the three languages considered in the present track (English, French, German) producing ten different sets of ranked results. 2. The merging of the different results based on multiple regression models using an additional validation set created from the patent collection. 3. The exploitation of patent metadata and of the citation structures for creating restricted initial working sets of patents and for producing a final re-ranking regression model. As we exploit specific metadata of the patent documents and the citation relations only at the creation of initial working sets and during the final post ranking step, our architecture remains generic and easy to extend. <s> BIB004 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Relevance Feedback has proven very effective for improving retrieval accuracy. A difficult yet important problem in all relevance feedback methods is how to optimally balance the original query and feedback information. In the current feedback methods, the balance parameter is usually set to a fixed value across all the queries and collections. However, due to the difference in queries and feedback documents, this balance parameter should be optimized for each query and each set of feedback documents. In this paper, we present a learning approach to adaptively predict the optimal balance coefficient for each query and each collection. We propose three heuristics to characterize the balance between query and feedback information. Taking these three heuristics as a road map, we explore a number of features and combine them using a regression approach to predict the balance coefficient. Our experiments show that the proposed adaptive relevance feedback is more robust and effective than the regular fixed-coefficient feedback. <s> BIB005 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> We present our experiments and results for the DCU CNGL participation in the CLEF-IP 2010 Candidate Patent Search Task. Our work applied standard information retrieval (IR) techniques to patent search. In addition, a very simple citation extraction method was applied to improve the results. This was our second consecutive participation in the CLEF-IP tasks. Our experiments in 2009 showed that many sophisticated approach to IR do not improve the retrieval effectiveness for this task. For this reason of we decided to apply only simple methods in 2010. These were demonstrated to be highly competitive with other participants. DCU submitted three runs for the Prior Art Candidate Search Task, two of these runs achieved the second and third ranks among the 25 runs submitted by nine different participants. Our best run achieved MAP of 0.203, recall of 0.618, and PRES of 0.523. <s> BIB006 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Patent retrieval is a recall-oriented search task where the objective is to find all possible relevant documents. Queries in patent retrieval are typically very long since they take the form of a patent claim or even a full patent application in the case of prior-art patent search. Nevertheless, there is generally a significant mismatch between the query and the relevant documents, often leading to low retrieval effectiveness. Some previous work has tried to address this mismatch through the application of query expansion (QE) techniques which have generally showed effectiveness for many other retrieval tasks. However, results of QE on patent search have been found to be very disappointing. We present a review of previous investigations of QE in patent retrieval, and explore some of these techniques on a prior-art patent search task. In addition, a novel method for QE using automatically generated synonyms set is presented. While previous QE techniques fail to improve over baseline retrieval, our new approach show statistically better retrieval precision over the baseline, although not for recall. In addition, it proves to be significantly more efficient than existing techniques. An extensive analysis to the results is presented which seeks to better understand situations where these QE techniques succeed or fail. <s> BIB007 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> In an attempt to improve retrieval systems for the patent domain, significant efforts are invested to assist researchers in formulating better queries, preferably via automated query generation. Current work on query generation in patent retrieval is mostly based on statistical measures without considering whether these terms are the best choice. To learn from actual queries being posed by experts, we analyze query logs from USPTO patent examiners. Results show that US examiners pick the majority of query terms from the claim section, a large fraction of which, in turn, coincide with the subject feature terms which determine the extent of the protection of the patent right. Considering the lessons learned from evaluating existing search logs will help in improving (semi-) automated query generation. <s> BIB008 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Query expansion is a crucial step in recall-oriented domains such as Patent Searching. Currently, automatic query expansion in patent search is mostly based on statistical measures. Additional query terms are extracted from the query documents based on entropy measures. To automate query expansion in patent searching, we acquire lexical knowledge from Query Logs of USPTO Patent Examiners. Results show good performance in query expansion and patent searching using the lexical database. This will help improving (semi-) automated query expansion in patent searching. <s> BIB009 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> In this paper we analyze a highly professional search setting of patent examiners of the United Patent and Trademark Office USPTO. We gain insight into the search behavior of USPTO patent examiners to explore ways for enhancing query generation in patent searching. We show that query generation is highly patent domain specific and patent examiners follow a strict scheme for generating text queries. Means to enhance query generation in patent search are to suggest synonyms and equivalents, co-occurring terms and keyword phrases to the searchable features of the invention. Further, we show that term networks including synonyms and equivalents can be learned from the query logs for automatic query expansion in patent searching. <s> BIB010 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Relevance feedback methods generally suffer from topic drift caused by word ambiguities and synonymous uses of words. Topic drift is an important issue in patent information retrieval as people tend to use different expressions describing similar concepts causing low precision and recall at the same time. Furthermore, failing to retrieve relevant patents to an application during the examination process may cause legal problems caused by granting an existing invention. A possible cause of topic drift is utilizing a relevance feedback-based search method. As a way to alleviate the inherent problem, we propose a novel query phrase expansion approach utilizing semantic annotations in Wikipedia pages, trying to enrich queries with phrases disambiguating the original query words. The idea was implemented for patent search where patents are classified into a hierarchy of categories, and the analyses of the experimental results showed not only the positive roles of phrases and words in retrieving additional relevant documents through query expansion but also their contributions to alleviating the query drift problem. More specifically, our query expansion method was compared against relevance-based language model, a state-of-the-art query expansion method, to show its superiority in terms of MAP on all levels of the classification hierarchy. <s> BIB011 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> The availability of large volumes of granted patents and applications, all publicly available on the Web, enables the use of sophisticated text mining and information retrieval methods to facilitate access and analysis of patents. In this paper we investigate techniques to automatically recommend patents given a query patent. This task is critical for a variety of patent-related analysis problems such as finding relevant citations, research of relevant prior art, and infringement analysis. We investigate the use of latent Dirichlet allocation and Dirichlet multinomial regression to represent patent documents and to compute similarity scores. We compare our methods with state-of-the-art document representations and retrieval techniques and demonstrate the effectiveness of our approach on a collection of US patent publications. <s> BIB012 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> Patent prior art search is a task in patent retrieval with the goal of finding documents which describe prior art work related to a query patent. A query patent is a full patent application composed of hundreds of terms which does not represent a single focused information need. Fortunately, other relevance evidence sources (i.e., classification tags and bibliographical data) provide additional details about the underlying information need. In this article, we propose a unified framework that integrates multiple relevance evidence components for query formulation. We first build a query model from the textual fields of a query patent. To overcome the term mismatch, we expand this initial query model with the term distribution of documents in the citation graph, modeling old and recent domain terminology. We build an IPC lexicon and perform query expansion using this lexicon incorporating proximity information. We performed an empirical evaluation on two patent datasets. Our results show that employing the temporal features of documents has a precision enhancing effect, while query expansion using IPC lexicon improves the recall of the final rank list. <s> BIB013 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> In the patent domain significant efforts are invested to assist researchers in formulating better queries, preferably via automated query expansion. Currently, automatic query expansion in patent search is mostly limited to computing co-occurring terms for the searchable features of the invention. Additional query terms are extracted automatically from patent documents based on entropy measures. Learning synonyms in the patent domain for automatic query expansion has been a difficult task. No dedicated sources providing synonyms for the patent domain, such as patent domain specific lexica or thesauri, are available. In this paper we focus on the highly professional search setting of patent examiners. In particular, we use query logs to learn synonyms for the patent domain. For automatic query expansion, we create term networks based on the query logs specifically for several USPTO patent classes. Experiments show good performance in automatic query expansion using these automatically generated term networks. Specifically, with a larger number of query logs for a specific patent US class available the performance of the learned term networks increases. <s> BIB014 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> In the patent domain Boolean retrieval is particularly common. But despite the importance of Boolean retrieval, there is not much work in current research assisting patent experts in formulating such queries. Currently, these approaches are mostly limited to the usage of standard dictionaries, such as WordNet, to provide synonymous expansion terms. In this paper we present a new approach to support patent searchers in the query generation process. We extract a lexical database, which we call PatNet, from real query sessions of patent examiners of the United Patent and Trademark Office (USPTO). PatNet provides several types of synonym relations. Further, we apply several query term expansion strategies to improve the precision measures of PatNet in suggesting expansion terms. Experiments based on real query sessions of patent examiners show a drastic increase in precision, when considering support of the synonym relations, US patent classes, and word senses. <s> BIB015 </s> Patent retrieval: a literature review <s> Semantic-based methods <s> In this paper we study the impact of query term expansion QTE using synonyms on patent document retrieval. We use an automatically generated lexical database from USPTO query logs, called PatNet, which provides synonyms and equivalents for a query term. Our experiments on the CLEF-IP 2010 benchmark dataset show that automatic query expansion using PatNet tends to decrease or only slightly improve the retrieval effectiveness, with no significant improvement. An analysis of the retrieval results shows that PatNet does not have generally a negative effect on the retrieval effectiveness. Recall is drastically improved for query topics, where the baseline queries achieve, on average, only low recall values. But we have not detected any commonality that allows us to characterize these queries. So we recommend using PatNet for semi-automatic QTE in Boolean retrieval, where expanding query terms with synonyms and equivalents with the aim of expanding the query scope is a common practice. <s> BIB016
As we mentioned before, in PR queries can vary from few terms (e.g., survey memo) to thousands of terms (e.g., full patent application). Straightforward keyword-based PR proved to be ineffective simply because of the vocabulary mismatch between query terms and relevant patents content. Magdy et al.  showed that, in the CLEF-IP 2009 collection, 12% of the relevant documents have no common words with the search topics. This motivates the need for novel approaches to bridge this vocabulary mismatch gap. Several semantic-based methods have been proposed in attempt to match queries with relevant documents based on their meanings rather than relying on keyword matches only. Table 5 shows some semantic-based methods along with their performance results on benchmark datasets. Dictionary-based: semantic-based methods perform QRE by expanding the query to include other terms that have similar meanings to the original query terms. The first category of these methods are the dictionary-based techniques which use either generic BIB007 , technical BIB004 , or patent-specific dictionaries BIB013 BIB008 BIB010 BIB015 BIB016  for QRE. Generic dictionaries could be existing lexical databases such as WordNet  , while patent-specific dictionaries are lexical databases generated from patent-related data such as examiner's query logs. In either case, similar or related terms to the original query terms are retrieved from such dictionaries and used for query expansion. Magdy and Jones BIB007 explored the use of WordNet for query expansion in patent retrieval on the CLEF-IP 2010 collection. Overall, adding synonyms and hyponyms for nouns and verbs in the original query increased the MAP score slightly while decreased the PRES score significantly. Moreover, query execution time was increased considerably. The authors Indicates baseline performance considered this a "negative" result. As the use of WordNet was proven to be effective in other retrieval tasks BIB002  , more experiments are needed to affirm the authors' conclusion, for example, investigating the impact of using synonyms only or hyponyms only and expanding terms belonging to specific sections or ambiguous terms only. Recently, more research was focused on utilizing domain-specific and technical dictionaries rather than WordNet. Examiners' query logs have been an important resource for building such technical thesauri. Tannebaum and Rauber BIB009 BIB008 BIB010 BIB014 BIB015 and Tannebaum et al. BIB016 introduced an analysis of the USPTO examiners' search query logs. Their analysis, though on a subset of query logs, revealed interesting insights into patent examiners' search behavior which could be very useful for designing effective patent retrieval systems. For example, the authors noted about examiners' behavior while searching for prior art that: (1) the average query length is four terms, (2) search terms are mostly from the patent application under investigation, (3) expansion terms represent small percentage of query terms and mostly appear in the specific patent domain terminology, (4) the majority of query terms represents subject technical features that appears in the claims section, while very little percentage of them appears in the description section, (5) the majority of terms are nouns, followed by verbs, then adjectives, and (6) about half of the query operators used are "OR," followed by "AND," then proximity operators. Tannebaum et al. built upon these insights and introduced methods to automatically identify synonyms/equivalents, co-occurring terms, and proximity relations for expanding query terms by mining examiners' search logs. As we can notice, learning expansion terms from query logs might be misleading because not all query sessions succeed to identify prior art. Additionally, deeper analysis of the query logs considering other metadata such as relevant hits count might be useful in this regard. On the other hand, it would be more useful if we can model the features of these terms, for example, based on their location, frequency, part of speech, etc. From effectiveness perspective, evaluating the generated lexical knowledge on the CLEF-IP 2010 collection did not record significant improvement BIB016 . Therefore, the authors recommended using it in an interactive mode rather than automatic mode to semi-automate query generation. Corpus-based: the second category of semantic-based QRE is the corpus-based methods. In these methods, textual corpora are analyzed to extract semantically related concepts to query terms which can be used for query expansion. Al-Shboul and Myaeng BIB011 proposed a Wikipedia-based query expansion method which works by first creating a summary of each Wikipedia article containing the main category, all titles under the main category, and other categories with in/out links to the main category. At query time, query terms and phrases are matched with page summaries; then, phrases from matching pages are scored and selected for query expansion under the assumption that they are semantically related. Experiments on the subset of USPTO patents in the NTCIR-6 collection showed an increase in MAP over other query expansion techniques. However, the authors used IPC codes rather than citations as relevance judgments to topic queries which do not reflect the typical search practices, where it is needed to retrieve related patent documents not related to classification codes. Another corpus-based method was proposed by Magdy and Jones BIB007 , where synonym sets were automatically generated from the CLEF-IP patent corpus. The authors utilized parallel translations of patent sections in different languages to build a word-to-word translation model and infer synonymy relation when a word in one language is translated to multiple words in another language. These multiple words under some probabilistic threshold could be considered synonyms. Overall results using this method were better than PRF and Wordnetbased query expansion, but worse than the keyword-based baseline in Magdy and Jones BIB006 . The authors also showed that the performance of this method on some topics was better than the baseline which indicates its potential. The issue they raised is how to more effectively apply query expansion by selecting "good" terms BIB003 , or predicting query expansion performance beforehand BIB001 BIB005 . Such challenges can also be alleviated semi-automatically by developing intelligent and usable interactive query expansion frameworks which engage users in such decision. Finally, Krestel and Smyth BIB012 applied topic modeling of search hits in order to better rank retrieved patents. The results on a small collection of the USPTO patents showed improved MAP.