Patent retrieval: a literature review <s> CLEF-IP collections <s> This paper proposes a methodology for the construction of a patent test collection for the task of prior art search. Key to the justification of the methodology is an analysis of the nature and structure of patent documents and the patenting process. These factors enable a corpus of patent documents to be reverse engineered in order to arrive at high quality, realistic, relevance assessments. The paper first outlines the case for such a prior art search test collection along with the characteristics of patent documents, before describing the proposed method. Further research and development will be directed towards the application of this methodology to create a suite of prior art search topics for the evaluation of patent retrieval systems. We also include a preliminary analysis of its applica <s> BIB001 </s> Patent retrieval: a literature review <s> CLEF-IP collections <s> The CLEF-IP track ran for the first time within CLEF 2009. The purpose of the track was twofold: to encourage and facilitate research in the area of patent retrieval by providing a large clean data set for experimentation; to create a large test collection of patents in the three main European languages for the evaluation of cross-lingual information access. The track focused on the task of prior art search. The 15 European teams who participated in the track deployed a rich range of Information Retrieval techniques adapting them to this new specific domain and task. A large-scale test collection for evaluation purposes was created by exploiting patent citations. <s> BIB002 </s> Patent retrieval: a literature review <s> CLEF-IP collections <s> The patent system is designed to encourage disclosure of new technologies and novel ideas by granting exclusive rights on the use of inventions to their inventors, for a limited period of time. Before a patent can be granted, patent o ces around the world perform thorough searches to ensure that no previous similar disclosures were made. In the intellectual property terminology, such kind of searches are called prior art searches. In some industries, the number of granted patents a company owns has a high impact on the market value of the company. This underlines the importance of well-performed prior art searches. Together with the Trec Chem track [5], also organized by our institution, the Clef Ip e ort comes to complete the work that is being done in the series of Ntcir workshops (see for example [4]). The rst Clef Ip track ran within Clef 2009. The purpose of the track was twofold: to encourage and facilitate research in the area of patent retrieval by providing a large clean data set for experimentation; to create a large test collection of patents in the three main European languages for the evaluation of cross lingual information access. The Clef Ip data set includes documents published by the European Patent O ce (Epo) which contain a mixture of English, German and French content. The track focused on the task of prior art search. In 2010 and 2011, the Clef Ip track was organized as a benchmarking activity (lab) in the Clef conference. In these years, the main goal of the Clef Ip e ort remained the same to foster research in the patent retrieval area, and provide a large clean data set. To this end, the number of tasks in the track was increased and the data set was enlarged. Recognizing the importance of patent classi cations in the daily activity of an intellectual property professional, in 2010 the Clef Ip benchmarking activity included a patent classi cation task. The participants were asked to classify <s> BIB003 </s> Patent retrieval: a literature review <s> CLEF-IP collections <s> Relevance feedback methods generally suffer from topic drift caused by word ambiguities and synonymous uses of words. Topic drift is an important issue in patent information retrieval as people tend to use different expressions describing similar concepts causing low precision and recall at the same time. Furthermore, failing to retrieve relevant patents to an application during the examination process may cause legal problems caused by granting an existing invention. A possible cause of topic drift is utilizing a relevance feedback-based search method. As a way to alleviate the inherent problem, we propose a novel query phrase expansion approach utilizing semantic annotations in Wikipedia pages, trying to enrich queries with phrases disambiguating the original query words. The idea was implemented for patent search where patents are classified into a hierarchy of categories, and the analyses of the experimental results showed not only the positive roles of phrases and words in retrieving additional relevant documents through query expansion but also their contributions to alleviating the query drift problem. More specifically, our query expansion method was compared against relevance-based language model, a state-of-the-art query expansion method, to show its superiority in terms of MAP on all levels of the classification hierarchy. <s> BIB004 </s> Patent retrieval: a literature review <s> CLEF-IP collections <s> Patents are used by legal entities to legally protect their inventions and represent a multi-billion dollar industry of licensing and litigation. In 2014, 326,033 patent applications were approved in the US alone -- a number that has doubled in the past 15 years and which makes prior art search a daunting, but necessary task in the patent application process. In this work, we seek to investigate the efficacy of prior art search strategies from the perspective of the inventor who wishes to assess the patentability of their ideas prior to writing a full application. While much of the literature inspired by the evaluation framework of the CLEF-IP competition has aimed to assist patent examiners in assessing prior art for complete patent applications, less of this work has focused on patent search with queries representing partial applications. In the (partial) patent search setting, a query is often much longer than in other standard IR tasks, e.g., the description section may contain hundreds or even thousands of words. While the length of such queries may suggest query reduction strategies to remove irrelevant terms, intentional obfuscation and general language used in patents suggests that it may help to expand queries with additionally relevant terms. To assess the trade-offs among all of these pre-application prior art search strategies, we comparatively evaluate a variety of partial application search and query reformulation methods. Among numerous findings, querying with a full description, perhaps in conjunction with generic (non-patent specific) query reduction methods, is recommended for best performance. However, we also find that querying with an abstract represents the best trade-off in terms of writing effort vs. retrieval efficacy (i.e., querying with the description sections only lead to marginal improvements) and that for such relatively short queries, generic query expansion methods help. <s> BIB005
The Conference and Labs of the Evaluation Forum 4 (CLEF) is a European series of workshops which started in 2001 to foster research in Cross Language Information Retrieval (CLIR). The Intellectual Property (IP) track (CLEF-IP) which ran between (2009 and 2013) was organized to: BIB004 foster research in patent data analysis and (2) provide large and clean test collections of multi-language patent documents, specifically in the three main European languages (English, French, and German). Research labs have the opportunity to test their methods on multiple shared tasks such as PR, patent classification, image-based PR, image classification, flowchart recognition, and structure recognition BIB002 BIB003    . The CLEF-IP data collection are patent documents extracted from USPTO, EPO and WIPO data. It is provided through the Information Research Facility 5 (IRF) and hosted by Marec. BIB005 Patent documents are provided in XML format and have common Document Type Definition (DTD) schema. The collection was constructed according to the proposed methodology by Graf and Azzopardi BIB001 and is divided into two pools: 1. The corpus pool: Documents selected from this pool are provided for participating labs as training or lookup instances. 2. The topics pool: Documents selected from this pool are called topics, and they represent testing or evaluation instances. For example, in prior art search, the topic might be a patent application document for which it is required to retrieve prior art. In this scenario, the patents that constitute prior art are called relevance assessments and obtained from the corpus pool. The XML documents consist of the main textual sections such as bibliographic data, abstract, description, and claims. Each section is written in one or more languages (English, French, and/or German) and is denoted by a language code. At least the claims of granted patents (B1 documents) are written in the three languages because it is EPO requirement once a patent application is granted. CLEF-IP 2009 collection: this dataset was designed for the prior art search task  . The corpus pool contains documents published between (1985 and 2000) (∼ 2 m documents pertaining to ∼1 m unique patents). The topics pool contains documents published between (2001 and 2006) (∼ 0.7 m documents pertaining to ∼0.5 m individual patents). Top-ics are sets of documents from the topics pool with sizes ranging from 500 to 10,000 topics. Topics were assembled from granted patent documents including abstract, description, and claims sections. Citation information from the bibliographic data section was excluded. A major pitfall in this dataset is its topics, which were chosen from granted patent documents (B1 documents). Initially, the creators of the dataset were motivated by having topics from granted patent documents which have claims in three languages. This was thought to provide a kind of parallel corpus suitable for CLIR. The problem of using such documents is simple, it contradicts the practice of IP search professionals who start with the patent application document not the granted one. CLEF-IP 2010 collection: this dataset was created for the prior art search and patent classification tasks BIB002 . The corpus pool of this dataset contains documents with publication date before 2002 (∼ 2.6 m documents pertaining to ∼1.9 m unique patents). The topics pool contains documents published between (2002 and 2009) (∼ 0.8 m documents pertaining to ∼ 0.6 m unique patents). Topics for the prior art task are two sets of documents from the topics pool; a small set of 500 topics and a larger set of 2000 topics. Unlike the CLEF-IP 2009 dataset, topics are assembled from patent application documents rather than granted patent documents. CLEF-IP 2011 collection: this dataset was created as a test collection for four tasks: prior art search, patent classification, image-based prior art search, and image classification BIB003 . The topics and corpus pools were the same as in CLEF-IP 2010 dataset. For the prior art task, 3973 topics were provided as a separate archive of patent application documents. CLEF-IP 2012 collection: this dataset was created as a test collection for three tasks: passage retrieval starting from claims, chemical structure recognition, and flowchart recognition  . The topics and corpus pools were the same as in CLEF-IP 2010 dataset. The passage retrieval task is designed differently from previous CLEF-IP prior art search collections. The purpose for these tasks is to retrieve both documents and passages relevant to a set of claims. Topics for the passage retrieval task were extracted from patent applications published after 2001. Relevance judgments were the highly relevant citations only (i.e., marked X or Y) in the examiners' search reports (A4 documents) of chosen topic patents. CLEF-IP 2013 collection: this dataset was created as a test collection for two tasks: (1) passage retrieval from claims and (2) structure recognition from patent images  . The topics and corpus pools were the same as in CLEF-IP 2010 dataset. Similar to CLEF-IP 2012, the CLM task is designed to retrieve both documents and passages relevant to a set of claims. Topics for the passage retrieval task were extracted from patent applications published after 2002. Overall, the topics set contained 148 topics extracted from 69 patent applications.