Hanna Suominen, Liadh Kelly, Lorraine Goeuriot, Aurélie Névéol, Lionel Ramadier, Aude Robert, Evangelos Kanoulas, Rene Spijker, Leif Azzopardi, Dan Li, others &ndash; University of Turku, Turku, Finland; The Australian National University (ANU), Australia; Commonwealth Scientific and Industrial Research Organisation (CSIRO), University of Canberra, Canberra, Australia; Maynooth University, Maynooth, Ireland; Univ. Grenoble Alpes, CNRS, Grenoble, France; Université Paris-Saclay, Orsay, France; INSERM, France; University of Amsterdam, Amsterdam, Netherlands; Cochrane Netherlands and UMC Utrecht; Julius Center for Health Sciences and Primary Care, Utrecht, Netherlands; University of Strathclyde, Glasgow, UK; Queensland University of Technology, Brisbane, Australia; Vienna University of Technology, Vienna, Austria; Qatar Computing Research Institute, Doha, Qatar
Overview of the CLEF ehealth evaluation lab 2018
University of Turku, Turku, Finland; The Australian National University (ANU), Australia; Commonwealth Scientific and Industrial Research Organisation (CSIRO), University of Canberra, Canberra, Australia; Maynooth University, Maynooth, Ireland; Univ. Grenoble Alpes, CNRS, Grenoble, France; Université Paris-Saclay, Orsay, France; INSERM, France; University of Amsterdam, Amsterdam, Netherlands; Cochrane Netherlands and UMC Utrecht; Julius Center for Health Sciences and Primary Care, Utrecht, Netherlands; University of Strathclyde, Glasgow, UK; Queensland University of Technology, Brisbane, Australia; Vienna University of Technology, Vienna, Austria; Qatar Computing Research Institute, Doha, Qatar
ir/search-engine-evaluation, nlp/corpus-construction
This year we introduced clefehealth2018 corpus. This was crated by compiling Web pages of selected domains acquired from the CommonCrawl¹¹. An initial list of Websites was identified for acquisition. The list was built by submitting the CLEF 2018 base queries to the Microsoft Bing APIs (through the Azure Cognitive Services) repeatedly over a period of few weeks¹², and acquiring the URLs of the retrieved results. The domains of the URLs were then included in the list, except some domains that were excluded for decency reasons (e.g. pornhub.com). The list was further augmented by including a number of known reliable health Websites and other known unreliable health Websites, from lists previously compiled by health institutions and agencies. The corpus was divided into folders, by domain name. Each folder contained a file for each Webpage from the domain available in the CommonCrawl dump. In total, 2,021 domains were requested from the CommonCrawl dump of 2018-09¹³. Of the 2,021 domains in total, 1,903 were successfully acquired. The remaining domains were discarded due to errors, corrupted or incomplete data returned by the CommonCrawl API (a total of ten retries were attempted for each domain before giving up on a domain). Of the 1,903 crawled domains, 84 were not available in the CommonCrawl dump, and for these, a folder in the corpus exists and represents the domain that was requested; however, the folder is empty, meaning that it was not available in the dump. Note that .pdf documents were excluded from the data acquired from CommonCrawl. A complete list of domains and size of the crawl data for each domain is available at https://github.com/CLEFeHealth/CLEFeHealth2018IRtask/ blob/master/clef2018collection_listofdomains.txt. The full collection, clefehealth2018¹⁴, it contains 5,535,120 Web pages and its uncompressed size is about 480GB. In addition to the full collection, an alternative corpus named clefehealth2018_B¹⁵ was created by manually removing a number of domains that were not strictly health-related (e.g., news Websites). This subset contains 1,653 domains and its size is about 294GB, uncompressed.