Query Augmentation with Brain Signals

摘要

In the information retrieval scenario, query augmentation is an essential technique to refine semantically imprecise queries to align more closely with users' actual information needs. Traditional methods typically rely on extracting signals from user interactions such as browsing or clicking behaviors to augment the queries, which may not accurately reflect the actual user intent due to inherent noise and the dependency on initial user interactions. To overcome these limitations, we introduce Brain-Aug, a novel approach that decodes semantic information directly from brain signals of users to augment query representation. Brain-Aug builds on three techniques: (i) Structurally, an adapter network is utilized to project brain signals into the embedding space of a language model, allowing query augmentation conditioned on both the users' initial query and their brain signals. (ii) During training, we use a next token prediction task for query augmentation and adopt prompt tuning to efficiently train the brain adapter. (iii)At the inference stage, a ranking-oriented decoding strategy is implemented, enabling Brain-Aug to generate augmentations that improve ranking performance. We evaluate our approach on multiple functional magnetic resonance imaging (fMRI) datasets, demonstrating that Brain-Aug not only produces semantically richer queries but also significantly improves document ranking accuracy, particularly for ambiguous queries. These results validate the effectiveness of Brain-Aug, and reveal the potential of using internal cognitive states to understand and augment text-based queries. Supplementary materials and code are available at https://github.com/YeZiyi1998/Brain-Query-Augmentation.