Linguistic Priors for Visual Decoupling: Towards Symmetric Vision-Brain Alignment

摘要

Brain visual decoding aims to recognize and reconstruct perceptual visual content from brain activity, providing a promising potential for the development of brain-computer interfaces and brain-inspired intelligence. However, this task faces a fundamental challenge of information asymmetry: while natural images contain complex visual scenes with objects and backgrounds, the corresponding brain signals reflect focused attention on central objects while being contaminated by various neural noise. Previous methods that directly align visual and brain representations often overlook this inherent asymmetry, resulting in suboptimal decoding performance. To address this, we propose linguistic-prior-guided visual decoupling method, which introduces object-oriented textual descriptions as semantic guidance to explicitly decouple foreground objects from complex backgrounds in natural images. This design enables the model to automatically focus on task-relevant visual concepts while effectively filtering out irrelevant neural noise in brain signals, achieving a transition from asymmetric vision-brain alignment to semantic symmetric alignment. Extensive experiments on the THINGS-EEG and THINGS-MEG datasets demonstrate that our method achieves new state-of-the-art performance in the zero-shot brain-to-image retrieval task. The source code is available at https://github.com/TKQXX/BVSA.