To study intents more systematically, the original study [1]} used surveys and also carried out manual annotation of a sample of queries. The finding was that navigational, informational and transactional queries are roughly 20%, 50% and 30% of queries, respectively. Based on this it was possible to set up system-oriented evaluation that also included these types of intents, for example capturing that the desired output is Navigational in the TREC Web Track [2]}. All of this suggests that log analysis of CIR systems, to understand usage patterns and gain insights into possible user intents, will be a crucial step to the future of the field. Bringing to light the kinds of CIR research systems and datasets that correctly reflect a real-world CIR system.
