FAQIR A Frequently Asked Questions Retrieval Test Collection
Frequently asked question (FAQ) collections are commonly used across the web to provide information about a specific domain (e.g. services of a company) With respect to traditional information retrieval, FAQ retrieval introduces additional challenges, the main ones being (1) the brevity of FAQ texts and (2) the need for topic specific knowledge. The primary contribution of our work is a new domain specific FAQ collection, providing a large number of queries with manually annotated relevance judgments. On this collection, we test several unsupervised baseline models, including both count based and semantic embedding based models, as well as a combined model. We evaluate the performance across different setups and identify potential venues for improvement. The collection constitutes a solid basis for research in supervised machine learning based FAQ retrieval.
TSD