Evaluating Task Dependent Taxonomies for Navigation
Taxonomies of concepts are important across many application domains, for instance, online shopping portals use catalogs to help users navigate and search for products. Task dependent taxonomies, e.g. adapting the taxonomy to a specific cohort of users, can greatly improve the effectiveness of navigation and search. However, taxonomies are usually created by domain experts and hence designing task dependent taxonomies can be an expensive process: this often limits the applications to deploy generic taxonomies. Crowdsourcing based techniques have the potential to provide a cost efficient solution to building task dependent taxonomies. In this paper, we present the first quantitative study to evaluate the effectiveness of these crowdsourcing based techniques. Our experimental study compares different task dependent taxonomies built via crowdsourcing and generic taxonomies built by experts. We design randomized behavioral experiments on the Amazon Mechanical Turk platform for navigation tasks using these taxonomies resembling real world applications such as product search. We record various metrics such as the time of navigation, the number of clicks performed, and the search path taken by a participant to navigate the taxonomy to locate a desired object. Our findings show that task dependent taxonomies built by crowdsourcing techniques can reduce the navigation time up to $20\ Our results, in turn,demonstrate the power of crowdsourcing for learning complex structures such as semantic taxonomies.
HCOMP