@inproceedings{
ye2024query,
title={Query Augmentation with Brain Signals},
author={Ziyi Ye and Jingtao Zhan and Qingyao Ai and Yiqun LIU and Maarten de Rijke and Christina Lioma and Tuukka Ruotsalo},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=fmf8605lKj}
}
In the information retrieval scenario, query augmentation is an essential technique to refine semantically imprecise queries to align closely with users' actual information needs. 
Traditional methods typically rely on extracting signals from user interactions such as browsing or clicking behaviors to augment the queries, which may not accurately reflect the actual user intent due to inherent noise and the dependency on initial user interactions. 
To overcome these limitations, we introduce Brain-Aug, a novel approach that decodes semantic information directly from brain signals of users to augment query representation. 
Brain-Aug explores three-fold techniques: 
(1) Structurally, an adapter network is utilized to project brain signals into the embedding space of a language model, allowing query augmentation conditioned on both the users' initial query and their brain signals. 
(2) During training, we use a next token prediction task for query augmentation and adopt prompt tuning to efficiently train the brain adapter.
(3) At the inference stage, a ranking-oriented decoding strategy is implemented, enabling Brain-Aug to generate augmentations that improve ranking performance. 
We evaluate our approach on multiple functional magnetic resonance imaging (fMRI) datasets, demonstrating that Brain-Aug not only produces semantically richer queries but also significantly improves document ranking accuracy, particularly for ambiguous queries. 
These results validate the effectiveness of our proposed Brain-Aug approach, and reveal the great potential of leveraging internal cognitive states to understand and augment text-based queries.
Query Augmentation with Brain Signals
Ziyi Ye Jingtao Zhan Qingyao Ai Yiqun LIU Maarten de Rijke Christina Lioma Tuukka Ruotsalo