论文检索

Applications · Neuroscience, Cognitive Science

Connor Lane, Ratna Grandhi, Leema Krishna Murali, Mihir Tripathy, Shamus Zi Yang Sim, Will Beddow, Gianfranco Cortes, Suin Cho, Debojyoti Das, Sam Gijsen 等

We propose a simple strategy for training a foundation model on functional MRI (fMRI) data: we adapt the standard Vision Transformer to fMRI by first converting each 3D fMRI volume to a 2D map using a standard cortical flat map projection. We train spatiotemporal masked autoencoders (MAE) on 2.3K hours of fMRI flat map videos. Our model (CortexMAE) outperforms identical MAE models trained on parcel-averaged or native volume data. We perform the first quantitative scaling analyses for fMRI and observe strict power law scaling. Finally, we develop the first open evaluation suite for fMRI foundation models and use it to perform a comprehensive comparison. On cognitive state decoding, our model outperforms all models by a wide margin. On clinical trait prediction, however, we report an important mixed result: all models show inconsistent performance (including our own). We hope that by introducing reproducible benchmarks and a strong, simple baseline, we can help establish a clear frontier for fMRI foundation models. Code is available at \url{https://anonymous.4open.science/r/cortex_mae}.

Applications · Neuroscience, Cognitive Science

Yamin Li, Shiyu Wang, Chang Li, Ange Lou, Haatef Pourmotabbed, Sarah Goodale, Dario Englot, Daniel Moyer, Roza G Bayrak, Catie Chang

Functional magnetic resonance imaging (fMRI) provides dynamic measurements of human brain activity at high spatial resolution and depth, but its use is constrained by high cost, limited accessibility, and strict acquisition requirements. Synthesizing fMRI data from more accessible, non-invasive modalities such as electroencephalography (EEG) offers a promising alternative, enabling inference of deep brain dynamics from low-cost scalp recordings in naturalistic settings. Despite recent progress, existing EEG-to-fMRI translation methods typically rely on region-specific models and offer limited support for subject-level and dataset-level heterogeneity, restricting their generalizability. We propose UniEFS, a unified EEG-to-fMRI synthesis model that enables full-brain fMRI reconstruction while accommodating varying demographic and physiological contexts within a single model. Our approach leverages a pretrained fMRI decoder to embed rich spatial priors and introduces condition-aware prompt tokens that encode subject-level and experimental metadata, enabling effective handling of heterogeneous datasets. We extensively evaluate our model performance on eyes-closed resting-state data and demonstrate that it can reliably reconstruct temporally-resolved whole-brain fMRI activity, with strong potential to generalize to task-based fMRI and clinical populations in a zero-shot manner.

Applications · Neuroscience, Cognitive Science

Sijin Yu, Zijiao Chen, Zhenyu Yang, Zihao Tan, Jiakun Xu, Zhongliang Liu, shengxian chen, WENXUAN WU, Xiangmin Xu, Xin Zhang

Current fMRI decoders face a performance-fidelity trade-off where efficient ID encoders outperform geometrically-aligned surface-based models. We argue this is an artifact of inefficient surface tokenization and the failure to use anatomy as a predictive signal. We present **NeurIPS**, a framework that improves surface-based decoding by reframing anatomical variation from a nuisance to a powerful inductive prior. NeurIPS unites two innovations: a **Selective ROI Spherical Tokenizer (SRST)** for efficient geometric encoding, and a **Guided Mixture of Experts (SG-MoE)** that explicitly models individual anatomy using cortical features. On the Natural Scenes Dataset, NeurIPS establishes a new state-of-the-art for surface decoders and achieves performance comparable to strong 1D baselines. This is achieved with unprecedented efficiency, as the model converges dramatically faster (**10 vs. 600 epochs**). This efficiency enables rapid adaptation to new subjects using only **20\%** of data and remains stable when scaling the training cohort (4 to 8 subjects). Ablations provide evidence that these gains are driven by the model's use of cortical features, not by memorizing subject IDs. By leveraging anatomical priors, NeurIPS provides a principled and scalable path toward robust, generalizable brain decoding.

Applications · Neuroscience, Cognitive Science

Luca M. Schulze Buschoff, Konstantinos Voudouris, Can Demircan, Eric Schulz

Pre-trained vision language models do not have good intuitions about the physical world. Recent work has shown that supervised fine-tuning can improve model performance on simple physical tasks. However, fine-tuned models do not appear to learn robust physical rules that can generalize to new contexts. Based on research in cognitive science, we hypothesize that models need to interact with an environment to properly learn its physical dynamics. We train models that learn through interaction with the environment using reinforcement learning. While learning from interaction allows models to improve their within-task performance, it fails to produce models with generalizable physical intuitions. We find that models trained on one task do not reliably generalize to related tasks, even if the tasks share visual statistics and physical principles, and regardless of whether the models are trained through interaction.

Applications · Neuroscience, Cognitive Science

Pradeep Singh, Balasubramanian Raman

Efficient prediction and planning in structured environments often relies on spectral decompositions of transition operators, yet existing grid-cell and successor-representation theories implicitly assume “flat” action geometry where translations commute and a single Fourier eigenbasis suffices. We show that this assumption breaks in the presence of path-dependent effects—e.g., circulation, rotational drift, or topological loops—whose defining signature is nontrivial holonomy. We introduce a theory of directed-action prediction on discrete tori based on twisted translation operators forming a projective representation of the underlying motion group, and prove that the resulting controlled Markov operators admit an exact block-diagonalisation under a twisted Fourier transform: actions share a universal harmonic basis while their effects appear as small matrix-valued spectra rather than scalar eigenvalues. This yields closed-form resolvent expressions for the successor representation, a gauge-invariant transfer principle characterising when two environments admit identical predictive structure, and a curvature-induced lower bound showing that nonzero holonomy provably necessitates internal representational dimension. Together, these results generalise Fourier/grid-based prediction from commutative to curved action geometries, providing a principled foundation for generalisation under directed actions without learning environment-specific eigenvectors.

Applications · Neuroscience, Cognitive Science

Jiayu Lu, Yujin Wang, Xiaofeng Liu, Dandan Li, Bin Wang

Functional brain network analysis plays an important role in understanding and diagnosing psychiatric disorders. However, current methods struggle with subject variations, impairing the model’s generalization ability to the test set. To address this issue, we propose the Subject Invariance-aware Inverse Graph Contrastive Learning (SI-IGCL) model, which adopts a two-stage paradigm with self-supervised subject-invariant pre-training followed by supervised fine-tuning for identification. During the pre-training phase, we construct an inverse contrastive objective that reshapes the embedding space by repelling intra-subject and attracting inter-subject embeddings to learn subject-invariant representations, with an auxiliary correction term to avoid early optimization plateaus. Meanwhile, we incorporate a structure-preserving reconstruction constraint to preserve discriminative information. Moreover, a Hierarchical Topology Enhanced Transformer (HTET) module is designed to enable multi-level modeling of subject-invariant functional patterns. During the fine-tuning phase, a supervised classifier is integrated to perform psychiatric disorder classification. Extensive experiments demonstrate that our method outperforms all state-of-the-art methods. The code is available at https://anonymous.4open.science/r/SI-IGCL.

Applications · Neuroscience, Cognitive Science

Xiran Chen, Xiaoke Yang, Cunhang Fan, Jian Zhou, Zhao Lv

Auditory attention decoding (AAD) based on Electroencephalography (EEG) aims to identify the attended speaker in multi-speaker environments. However, existing methods typically overlook the crucial phase information of EEG signals, which limits their ability to distinguish structured neural patterns from random noise in the frequency domain and hinders robust decoding. To address these issues, this paper proposes a Phase-aware Complex Refinement Network (PCRNet) for AAD, which consists of a Temporal Context Calibration (TCC) module and a Dual-Domain Integration (DDI) module. Specifically, the TCC module captures long-range temporal dependencies through multi-scale temporal attention mechanism, while the DDI module employs a phase-guided spectral filtering strategy to dynamically suppress noise-dominated frequencies and refine the real and imaginary components separately. This design enables effective phase recalibration and enhances the discriminability of target features in the complex domain. Experimental results on three public datasets demonstrate that PCRNet outperforms state-of-the-art (SOTA) methods, particularly under challenging ultra-short 0.1-second windows.

Applications · Neuroscience, Cognitive Science

JinGyo Lim, Seunggyu Jeong, Seong-Eun Kim

Recent Spiking Transformer models have explored a variety of attention mechanisms beyond standard dot-product formulations. However, many existing similarity-based spiking attention formulations remain inherently sensitive to firing density, causing neurons with high spike rates to dominate attention scores regardless of semantic relevance. This density bias is particularly problematic in event-driven spiking representations, where sparse spike patterns often carry critical information. To address this limitation, we rethink spiking attention from a set-theoretic perspective. We propose DiceFormer, a novel Spiking Transformer architecture driven by Spike Dice Attention (SDA). Unlike traditional approaches, SDA replaces density-sensitive measures with a set similarity function derived from the Dice coefficient. By explicitly normalizing for firing density, SDA focuses on spike co-occurrence rather than high firing rates. We primarily evaluate DiceFormer on the challenging audio domain, where spike sparsity varies substantially across inputs. On AudioSet-20k, DiceFormer achieves a SOTA mAP of 0.161 with 54.3M parameters, outperforming prior SNN-based approaches and substantially narrowing the performance gap with ANN-based models. We also introduce Lin-SDA, a linearized version for computation efficiency, while achieving performance comparable to SDA. Beyond audio, we evaluate the effectiveness of SDA on CIFAR-100 to verify its applicability to the vision domain.

Applications · Neuroscience, Cognitive Science

Xinhong Xu, Yimeng Zhang, Qichen Qian, Yuanlong Zhang

Recent work suggests that large-scale, multi-animal modeling can significantly improve neural recording analysis. However, for functional calcium traces, existing approaches remain task-specific, limiting transfer across common neuroscience objectives. To address this challenge, we propose \textbf{CalM}, a self-supervised neural foundation model trained solely on neuronal calcium traces and adaptable to multiple downstream tasks, including forecasting and decoding. Our key contribution is a pretraining framework, composed of a high-performance tokenizer mapping single-neuron traces into a shared discrete vocabulary, and a dual-axis autoregressive transformer modeling dependencies along both the neural and the temporal axis. We evaluate CalM on a large-scale, multi-animal, multi-session dataset. On the neural population dynamics forecasting task, CalM outperforms strong specialized baselines after pretraining. With a task-specific head, CalM further adapts to the behavior decoding task and achieves superior results compared with supervised decoding models. Moreover, linear analyses of CalM representations reveal interpretable functional structures beyond predictive accuracy. Taken together, we propose a novel and effective self-supervised pretraining paradigm for foundation models based on calcium traces, paving the way for scalable pretraining and broad applications in functional neural analysis.

Applications · Neuroscience, Cognitive Science

Louis Schiekiera, Max Zimmer, Christophe Roux, Sebastian Pokutta, Fritz Günther

We investigate the extent to which an LLM’s hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer models, we run two experimental paradigms---similarity-based forced choice and free association---over a shared 5,000-word vocabulary, collecting 17.5M+ trials to build behavior-based similarity matrices. Using representational similarity analysis, we compare behavioral geometries to layerwise hidden-state similarity and benchmark against FastText, BERT, and cross-model consensus. We find that forced-choice behavior aligns substantially more with hidden-state geometry than free association. In a held-out-words regression, behavioral similarity (especially forced choice) predicts unseen hidden-state similarities beyond lexical baselines and cross-model consensus, indicating that behavior-only measurements retain recoverable information about internal semantic geometry. Finally, we discuss implications for the ability of behavioral tasks to uncover hidden cognitive states.

Applications · Neuroscience, Cognitive Science

KIEREN YU, Ziyang Liu, Chang Huang, Kaishun WU

EEG foundation models aim to learn transferable representations, yet EEG recordings are dominated by high-frequency noise and large cross-subject variability. Existing pretraining strategies such as masked autoencoding or autoregressive modeling often treat waveform reconstruction as the learning signal, making the objective sensitive to stochastic fluctuations rather than consistent neurophysiological structure. To address this overlap, we propose \textbf{PATCHCODE}, a region-aware discrete predictive learning framework that keeps the encoder input continuous while introducing region-aware discrete codes as stable supervision targets. We pretrain a masked predictive encoder on continuous EEG patches with dual-granularity learning: it predicts missing patch-level representations to preserve fine spatiotemporal structure, while aligning them to discretized code targets from a frozen tokenizer to anchor robust semantics. Extensive Experiments across ten downstream datasets spanning emotion recognition, motor imagery, sleep staging, and seizure detection demonstrate that PATCHCODE achieves competitive performance compared to state-of-the-art baselines, with notable gains in data efficiency under limited labels. Our code is available at https://anonymous.4open.science/r/PATCHCODE-323D/.

Applications · Neuroscience, Cognitive Science

Byungwoo Kang, Maceo Richards, Bernardo Sabatini

Credit assignment, the process of determining how changes in individual neurons and synapses influence a network’s output, is central to learning in brains and machines. Noise correlation-based methods, which estimate gradients by correlating perturbations of activity with changes in output, provide a biologically plausible solution to credit assignment but scales poorly as accurately estimating the Jacobian requires that the number of perturbations scale with network size. Moreover, isotropic noise conflicts with neurobiological observations that neural activity lies on a low-dimensional manifold. To address these drawbacks, we propose *neural manifold noise correlation* (NMNC), which performs credit assignment using perturbations restricted to the neural manifold. We show theoretically and empirically that the Jacobian row space aligns with the neural manifold in trained networks, and that manifold dimensionality scales slowly with network size. NMNC substantially improves performance and sample efficiency over vanilla noise correlation in convolutional networks trained on CIFAR-10, ImageNet-scale models, and recurrent networks. NMNC also yields representations more similar to the primate visual system than vanilla noise correlation. These findings offer a mechanistic hypothesis for how biological circuits could support credit assignment, and suggest that biologically inspired constraints may enable, rather than limit, effective learning at scale.

Applications · Neuroscience, Cognitive Science

Wei Xiong, Jiangtong Li, Jie Li, Kun Zhu, Changjun Jiang

Electroencephalography foundation models (EEG-FMs) have advanced brain signal analysis, but the lack of standardized evaluation benchmarks impedes model comparison and scientific progress. Current evaluations rely on inconsistent protocols that render cross-model comparisons unreliable, while a lack of diagnostic analyses obscures the internal mechanisms driving transfer efficiency and scaling behaviors. To address this, we introduce **EEG-FM-Bench**, a unified system for the standardized evaluation of EEG-FMs. The benchmark integrates 14 datasets across 10 paradigms and incorporates diverse experimental settings, including multiple fine-tuning strategies, task organizations, and classifier configurations, supported by tools for gradient and representation analysis. Our experiments and analysis reveal several critical insights: (1) multi-task learning acts as a critical regularizer to mitigate overfitting in data-scarce EEG contexts; (2) pre-training efficiency is currently limited by gradient conflicts between reconstruction objectives and downstream tasks; (3) model scaling deviates from typical laws, as compact architectures with domain-specific inductive biases consistently outperform significantly larger models. This benchmark enables fair comparison and reproducible analysis, shifting the field from fragmented results to interpretable advances.

Applications · Neuroscience, Cognitive Science

Sunghwan LEE, jihun kim, Chaelynn Kim, Jiyun Park, Jong-Hwan Lee

Decoding fMRI into natural language is challenging because strong, pre-trained language priors can dominate autoregressive generation, obscuring whether a model truly utilizes neural evidence. We introduce BIT-LLM, which exposes fMRI-derived tokens as a persistent key–value memory through interleaved cross-attention adapters, enabling repeated neural access throughout decoding. BIT-LLM is trained with a three-stage pipeline: (i) multimodal contrastive learning to obtain semantically aligned fMRI representations, (ii) supervised fine-tuning to learn the brain-LLM interface while freezing the encoder and backbone LLM, and (iii) reward-based finetuning to optimize sequence-level caption quality directly. On the NSD subject-heldout benchmark (S1-7 train, S8 test), BIT-LLM yields substantially improved captioning quality over prior baselines under greedy decoding. In addition to standard captioning metrics, we perform several complementary evaluations to assess the robustness of brain–language grounding. Specifically, we conduct perturbation-based sanity checks by zeroing fMRI inputs or shuffling voxel values, and examine whether internal representations and generated outputs change accordingly. BIT-LLM exhibits clear sensitivity to these perturbations, indicating effective utilization of voxel values and their spatial correspondence.

Applications · Neuroscience, Cognitive Science

Zijian Zhou, Honglin Cao, Ammar Belatreche, Wenjie Wei, Yimeng Shan, Yu Liang, Yu Yang, Shuai Wang, Yalan Ye, Malu Zhang 等

Transformer-based Spiking Neural Networks (SNNs) combine Transformer performance with SNN energy efficiency through an event-driven self-attention mechanism. However, Spiking Transformers still lag behind their Artificial Neural Network (ANN) counterparts. Most existing studies address this issue through new architectural designs, yet none has considered optimization algorithms specific to Spiking Transformers. Here, we first analyze the gradient characteristics of Spiking Transformers and identify excessive noise from surrogate gradient learning as a major challenge to stable training. We then provide a quantitative definition of noise in the gradient update direction and propose an adaptive gradient descent method for spiking transforms, named AdaS. Since moderate update direction noise can enhance generalization, whereas excessive noise degrades training, AdaS adaptively adjusts the update direction noise to an optimal level, thereby improving the performance of Spiking Transformers. We conduct extensive experiments on various Spiking Transformer architectures and datasets from both computer vision and natural language processing. The results demonstrate that the proposed AdaS consistently enhances performance across different Spiking Transformers, validating its effectiveness and generalizability. This work presents the first systematic investigation of optimization algorithms specifically tailored for SNNs, offering a practical tool to narrow the accuracy gap with ANNs while preserving the energy advantages of spike-based computation.

Applications · Neuroscience, Cognitive Science

Zijian Zhou, Wenjie Wei, Yu Liang, Jialin Li, Ammar Belatreche, Honglin Cao, Shuai Wang, Malu Zhang, Yang Yang, Haizhou Li

Spiking Neural Networks (SNNs) that leverage sparse binary spikes and temporal dynamics have emerged as energy-efficient alternatives to Artificial Neural Networks (ANNs). However, SNNs suffer from limited representational capacity due to the discrete nature of spikes. Existing solutions extending spike levels often overlook the constraints of the simulation time window, leading to a critical issue we identify as spike saturation-induced information homogenization. In this phenomenon, distinct high-amplitude inputs result in identical maximized spike counts, truncating the dynamic range and hindering the model’s ability to capture fine-grained semantic differences. To address this, we propose SmoothSpike, a novel method designed to enhance representational capacity by suppressing spike saturation. We first introduce a randomized Hadamard transformation to smooth neuronal inputs, theoretically proving its efficacy in constraining extreme values and reducing both saturation probability and input variability among saturated neurons. To further improve adaptability, we evolve this into a learnable orthogonal transformation. Initialized with Hadamard matrices and maintained orthogonal via Newton-Schulz iteration, this module dynamically adapts to varying input distributions during training. Extensive experiments on language modeling tasks show that SmoothSpike effectively mitigates the information homogenization problem and improves task performance. This positions SmoothSpike as a robust solution to bridge the performance gap between SNNs and ANNs.

Applications · Neuroscience, Cognitive Science

Yu Liang, Zijian Zhou, Wenjie Wei, Shuai Wang, Honglin Cao, Ammar Belatreche, Yu Yang, Malu Zhang, Yang Yang, Haizhou Li

Spiking Neural Networks (SNNs) offer a promising avenue toward energy-efficient language modeling by replacing multiply-accumulate operations with sparse, event-driven computation. However, constructing fully spiking language models reveals two fundamental challenges: (1) gradient degradation from dead neurons caused by diminishing input magnitudes in deep networks, and (2) reduced token selectivity due to the absence of softmax's competitive winner-takes-all mechanism. These limitations create a substantial performance gap that has hindered the practical deployment of spiking language models. To address these challenges, we introduce SpikingLM, a framework that bridges the efficiency of SNNs with the capabilities of modern language models through two key innovations. First, we propose Distribution-aware Scaling, which rescales linear outputs to an optimal range that prevents gradient vanishing. These parameters are fused into preceding linear layers at inference, incurring zero additional overhead. Second, we introduce Spike2Max, a hardware-efficient attention mechanism that restores winner-takes-all dynamics through base-2 exponentiation and max-subtraction. By exploiting the integer-valued nature of spike coincidence counts, Spike2Max replaces floating-point exponentials with bit-shift operations, reducing attention energy consumption by over 95\% compared to softmax. Extensive experiments demonstrate that SpikingLM achieves a 57.9\% reduction in energy consumption while delivering state-of-the-art performance on GLUE among spiking language models.

Applications · Neuroscience, Cognitive Science

Haowei Xu, Yixin Chen, Wanyi Fu, Hongbin Han, Zhaoheng Xie

Neuromodulation can be viewed as closed-loop control of high-dimensional spatiotemporal fields on irregular 3D morphologies, coupling membrane electrophysiology with ionic reaction–diffusion. This view supports high-rate feedback and systematic in-silico evaluation, yet is difficult in practice. Unlike classical PDE control with known equations on regular domains, neuronal microenvironments exhibit complex, often unknown biophysics on irregular shapes. High-fidelity simulators are too costly for real-time control with repeated planning. The discretized field is sparsely observed and must satisfy hard full-field safety constraints. We introduce **NeuronCtrl**, a modular operator-level framework for safe, closed-loop generative control of neuronal microenvironment dynamics. Given measurements, actions, and morphology, a history-conditioned observer infers the latent field, a morphology-aware neural operator predicts one-step dynamics, and a flow-matching conditional flow proposes actions conditioned on user preferences. Safety is enforced via complementary barrier-based mechanisms at both the action and field levels, ensuring constraint satisfaction with minimal intervention. When latency is critical, the multi-step generator is distilled into a single-step policy while retaining the same safety filter. Experiments across three high-fidelity 3D neuromodulation benchmarks spanning deep brain stimulation, extracellular reaction--diffusion control, and astrocytic potassium regulation, demonstrate superior trade-offs among cost, safety, and latency. Code is available at https://anonymous.4open.science/r/NeuronControl-D900.

Applications · Neuroscience, Cognitive Science

Jingjing Hu, Dan Guo, Haofan Cheng, Zeng ying, Zhan Si, Jinxing Zhou, Meng Wang

Despite the high accuracy of EEG-based emotion recognition, existing models remain opaque "black boxes", lacking semantic grounding between abstract neural features and human-interpretable states. In this paper, we reframe EEG explainability as a cross-modal generation task, shifting the paradigm from feature attribution to behavioral visualization. We introduce Facial Emoji Proxy Modeling, a novel framework that translates high-dimensional EEG signals into identity-agnostic facial emojis. Guided by the neuroscientific prior of neural-facial consistency, this approach grounds neural representations in the manifold of observable facial dynamics. Technically, our framework integrates FMENet, a specialized backbone modeling expression-relevant spatial synergies, and the Facial Emoji Learning Branch (FELB), which treats emoji reconstruction as a structured semantic regularizer. Extensive experiments on EAV and MMER benchmarks demonstrate that our method achieves state-of-the-art accuracy among EEG-only models. Crucially, it generates semantically faithful facial animations that provide a transparent, privacy-preserving window into the brain's emotional evolution, effectively allowing users to "see the emotion" directly from neural signals.

Applications · Neuroscience, Cognitive Science

Subba Reddy Oota, Satya Sai Srinath Namburi GNVV, Vijay Rowtula, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Raju Bapi

Recent work has shown that scaling large language models (LLMs) improves their alignment with human brain activity, yet it remains unclear what drives these gains or which representational properties are responsible. Although larger models often yield better task performance and brain alignment, they are increasingly difficult to analyze mechanistically. This raises a fundamental question: \emph{what is the minimal model capacity required to capture brain-relevant representations?} To address this question, we systematically investigate how constraining model scale and numerical precision affects brain alignment. We compare full-precision LLMs, small language models (SLMs), and compressed variants (quantized and pruned) by predicting fMRI responses during naturalistic language comprehension. Across model families up to 14B parameters, we find that 3B SLMs achieve brain predictivity indistinguishable from larger LLMs, whereas 1B models degrade substantially, particularly in semantic language regions. Brain alignment is remarkably robust to compression: most quantization and pruning methods preserve neural predictivity, with GPTQ as a consistent exception. Linguistic probing reveals a dissociation between task performance and brain predictivity: compression degrades discourse, syntax, and morphology, yet brain predictivity remains largely unchanged. Overall, brain alignment saturates at modest model scales and is resilient to compression, challenging common assumptions about neural scaling and motivating compact models for brain-aligned language modeling.