论文检索

Applications · Neuroscience, Cognitive Science

Michela Proietti, Roberto Capobianco, Mariya Toneva

Understanding the alignment between large language models (LLMs) and human brain activity can reveal computational principles underlying language processing. This work describes a pipeline to apply attribution methods to the brain-LLM alignment setting to identify the specific words most important for this alignment. As a case study, we leverage it to study a contentious research question about brain-LLM alignment: the relationship between brain alignment (BA) and next-word prediction (NWP). Across two naturalistic fMRI datasets, we find that BA and NWP rely on largely distinct word subsets: NWP exhibits recency and primacy biases with a focus on syntax, while BA prioritizes semantic and discourse-level information with a more targeted recency effect. This work advances our understanding of how LLMs relate to human language processing and highlights differences in feature reliance between BA and NWP. Beyond this study, our attribution method can be broadly applied to explore the cognitive relevance of model predictions in diverse language processing tasks.

Applications · Neuroscience, Cognitive Science

Subati Abulikemu, Tiago Azevedo, Michail Mamalakis, John Suckling

In network neuroscience, functional brain systems are often characterized using separate yet related graph-theoretic or spectral descriptors, overlooking how these properties covary and partially overlap across individuals and conditions. We anticipate that dense, weighted functional connectivity graphs lie on a low-dimensional latent geometry along which both topological and spectral structures vary smoothly at the population level. Although graph-based deep learning offers a powerful framework for modeling these brain connectomes, supervised approaches are constrained by the limited availability of labeled data. Existing unsupervised graph representation methods also typically focus on node-level embeddings, which are limited in capturing compact graph-level representations that preserve information from dense functional connectomes. To address these gaps, we learn compact brain graph representations using a graph transformer autoencoder, where domain-specific, aligned functional gradient geometry provides an inductive bias to guide learning. Despite being trained in a fully unsupervised manner, our approach meaningfully separates cognitive states and enables decoding of visual stimuli, with performance further improved by incorporating neural dynamics. In parallel, to enable generation of synthetic brain graphs, we fit a diffusion model to the learned latent representation and decode samples back to dense connectomes.

Applications · Neuroscience, Cognitive Science

Emily Cheng, Aditya Vaidya, Richard Antonello

Research has repeatedly demonstrated that intermediate hidden states extracted from large language models and speech audio models predict measured brain response to natural language stimuli. Yet, very little is known about the representation properties that enable this high prediction performance. Why is it the intermediate layers, and not the output layers, that are most effective for this unique and highly general transfer task? We give evidence that the correspondence between speech and language models and the brain derives from shared meaning abstraction and not their next-word prediction properties. In particular, models construct higher-order linguistic features in their middle layers, cued by a peak in the layerwise *intrinsic dimension*, a measure of feature complexity. We show that a layer's intrinsic dimension strongly predicts how well it explains fMRI and ECoG signals; that the relation between intrinsic dimension and brain predictivity arises over model pre-training; and finetuning models to better predict the brain causally increases both representations' intrinsic dimension and their semantic content. Results suggest that semantic richness, high intrinsic dimension, and brain predictivity mirror each other, and that the key driver of model-brain similarity is *rich meaning abstraction* of the inputs, where language modeling is a task sufficiently complex (but perhaps not the only) to require it.

Applications · Neuroscience, Cognitive Science

Zhenyu Liao, Di Yu, Changze Lv, Wentao Tong, Linshan Jiang, Sijie Ji, Xin Du, Hailiang Zhao, Xiaoqing Zheng, Shuiguang Deng

Millimeter-wave (mmWave) sensing enables privacy-preserving, always-on edge perception, but its measurements are often sparse, temporally irregular, and corrupted by high-frequency noise. Existing mmWave pipelines predominantly rely on artificial neural networks (ANNs), which achieve robustness through extensive preprocessing or deep architectures, thereby limiting their efficiency on edge devices. In this work, we study spiking neural networks (SNNs) for mmWave sensing from a mechanism–data alignment perspective. By leveraging the low-pass filtering behavior of leaky integrate-and-fire (LIF) dynamics, we analyze how their implicit temporal filtering interacts with the frequency structure of mmWave signals. Our analysis shows that when discriminative information resides in low-to-mid frequencies, LIF dynamics can inherently suppress high-frequency noise, clarifying when and why SNNs outperform ANNs. Based on this insight, we derive a principled criterion for configuring the membrane decay factor by matching the effective bandwidth of LIF dynamics to the data’s discriminative spectral content. Experimental results across four widely used mmWave datasets validate the proposed frequency-matching hypothesis, yielding an average test-accuracy improvement of 6.22% and a 3.64× reduction in theoretical energy consumption relative to ANN baselines, under a unified evaluation protocol.

Applications · Neuroscience, Cognitive Science

Jaeyoon Sim, Soojin Hwang, Seunghun Baek, Guorong Wu, Won Hwa Kim

Understanding complex interactions between brain regions is critical for early neurodegenerative disease classification such as Alzheimer’s Disease (AD) and Parkinson’s Disease (PD). While graph-based models are widely used to analyze brain networks, most existing approaches primarily focus on pairwise interactions between directly connected nodes, limiting their ability to capture higher-order dependencies across multiple regions. Although hypergraph-based methods have been proposed to model higher-order relations, many rely on predefined hyperedges or restrict learning to hyperedge weights, reducing flexibility and limiting their capacity to capture multi-resolution structural patterns. In this regard, we introduce an adaptive multi-scale hyperedge learning framework, i.e., MuHL, which constructs hierarchical node features and dynamically learns high-order interaction through continuous hyper-edge construction over multi-resolution graph signals. Extensive experiments on multiple brain network benchmarks demonstrate that MuHL consistently improves disease classification performance across different stages, and further identifies key regions of interest (ROIs) and their group-wise interactions from the learned hyperedges that are associated with disease progression, highlighting its potential as a powerful tool for brain network analysis with neurodegenerative disorders.

Applications · Neuroscience, Cognitive Science

Ganxi Xu, Zhao-Rong Lai, Yuting Tang, Yonghao Song, Guoxu Zhou, Boyu Wang, Jian Zhu, Jinyi Long

Visual prostheses hold great promise for restoring vision in blind individuals. While researchers have successfully utilized M/EEG signals to evoke visual perceptions during the brain decoding stage of visual prostheses, the complementary process of converting images into M/EEG signals in the brain encoding stage remains largely unexplored, hindering the formation of a complete functional pipeline. In this work, we present a novel image-to-brain signal framework that generates M/EEG from images by leveraging the diffusion transformer architecture enhanced with cross-attention mechanisms. Specifically, we employ a diffusion transformer (DiT) architecture based on denoising diffusion implicit models (DDIM) to achieve brain signal generation. To realize the goal of image-to-brain signal conversion, we use cross-attention mechanisms to align brain signal embeddings with CLIP image embeddings. Moreover, we leverage large language models (LLMs) to generate image captions, and concatenate the resulting CLIP text embeddings with CLIP image embeddings to form unified embeddings for cross-attention alignment, enabling our model to capture core semantic information. Furthermore, we introduce a learnable spatio-temporal position encoding that combines brain region embeddings with temporal embeddings to capture both spatial and temporal characteristics of brain signals. We evaluate the framework on two multimodal benchmark datasets (THINGS-EEG2 and THINGS-MEG) and demonstrate that it generates biologically plausible brain signals.

Applications · Neuroscience, Cognitive Science

Guobin Shen, Dongcheng Zhao, Yiting Dong, Qian Zhang, Yi Zeng

Artificial and biological systems may evolve similar computational solutions despite fundamental differences in architecture and learning mechanisms—a form of convergent evolution. We provide large-scale evidence for this phenomenon through comprehensive analysis of alignment between human brain activity and internal representations across over 600 AI models spanning language and vision domains (1.33M to 72B parameters). Analysis of 60 million alignment measurements reveals that higher-performing models spontaneously develop stronger brain correspondence without explicit neural constraints, with language models demonstrating markedly stronger correlations ($r=0.89, p<7.5 \times 10^{-13}$) than vision models ($r=0.53, p<2.0 \times 10^{-44}$). Crucially, longitudinal training analysis shows that brain alignment consistently emerges prior to performance improvements, suggesting that developing brain-like representations constitutes a fundamental stepping stone toward enhanced capabilities. We identify systematic organizational patterns reflecting human cognitive architecture: language models exhibit strongest alignment with limbic and integrative regions, while vision models show progressive correspondence with visual cortical hierarchies. These findings establish that optimization for task performance naturally drives AI systems toward human-like computational strategies.

Applications · Neuroscience, Cognitive Science

Bozhou Li, Chubo Liu, Yan Ding, Yufeng Zhang, Zhuo Tang, Kenli Li

ANN-to-SNN conversion offers energy-efficient inference but faces a fidelity-latency trade-off due to open-loop error accumulation. While conversion-aware training mitigates this, it sacrifices the generality of using off-the-shelf ANNs. We propose a closed-loop fine-tuning framework that calibrates these errors without altering the source model. Our approach employs a Dual Alignment Mechanism, utilizing global Kullback-Leibler divergence for output distillation and introducing an adaptive local Centered Kernel Alignment constraint, weighted by initial conversion loss, for feature alignment. We uncover a critical time-dependent dynamic: local constraints are essential for stabilizing representations in low-latency regimes (e.g., $T=8$) where global gradients are unstable, whereas global alignment drives fidelity at higher time steps. Experiments on CIFAR-10 demonstrate that our method achieves over 99\% of source ANN accuracy at $T=32$ (e.g., ResNet-18: 96.38\% vs.\ 96.39\%). Furthermore, this fine-tuning acts as a regularizer, yielding SNNs with input noise robustness that matches or exceeds the source ANN.

Applications · Neuroscience, Cognitive Science

Haoyu Wang, Wei Dai, Jialun Ma, Jiawei Zhang, Jinqi Liu, Mingchen Jiang, Mingqing Xiao, Yansen Wang, Dongqi Han, Dongsheng Li 等

Lateral connections (LCs) are ubiquitous in the cortical circuits. While modern deep learning architectures have rich intralayer interactions (e.g., convolutional mixing, normalization, or attention) to support feature selectivity and contextual modulation, explicit excitatory and inhibitory (E-I) LCs remain underexplored and unjustified additions for encoding models in both deep learning and neuroscience. In this work, we analyze and train stabilized supralinear networks (SSNs) with sufficiently strong recurrent excitation and feedback inhibition, using local unsupervised plasticity rules under natural image stimulation. We demonstrate that these LCs support a transition between dynamical regimes under different input conditions. During the transition, the network shifts from employing population coding to sparse coding balancing cost and performance: population coding extracts robust features from low-contrast or noisy inputs by recruiting more neurons while sparse coding encodes high-contrast, proper inputs efficiently with minimal cost. These results are then compared against sparse coding and ICA-based models. Our findings frame explicit E-I recurrent neural networks through the lens of dynamic coding strategies and provide insights into designing more adaptive and robust systems with a concrete example in vision.

Applications · Neuroscience, Cognitive Science

Mo Wang, Wenhao Ye, Junfeng Xia, Junxiang Zhang, Xuanye Pan, Minghao Xu, Haotian Deng, Hongkai Wen, Quanying Liu

Self-supervised fMRI foundation models have shown promising transfer performance, yet most rely on predefined region-level parcellations that discard fine-grained voxel information and introduce atlas-dependent biases. We propose Omni-fMRI, an atlas-free foundation model that operates directly on voxel-level signals. To enable scalable pretraining on 49,497 fMRI sessions across nine datasets, Omni-fMRI introduces a dynamic patching mechanism that substantially reduces computational cost while preserving informative spatial structure. To support reproducibility and fair comparison, we establish a comprehensive benchmark suite spanning 11 datasets and a diverse set of resting-state and task-based fMRI tasks. Experimental results demonstrate that Omni-fMRI consistently outperforms existing foundation models, providing a scalable and reproducible framework for atlas-free brain representation learning. Code is available.

Applications · Neuroscience, Cognitive Science

Zhe Jiao, Xiaodong He, Shanglin Zhou

While heavy-tailed synaptic weight distributions are pervasive in biological neural networks, their computational role---particularly in relation to generalization---remains poorly understood. To address this, we develop a novel optimal-transport-based optimization algorithm that incorporates key biological constraints, including Dale’s principle and heavy-tailed synaptic statistics, to train recurrent neural networks (RNNs) on a wide range of cognitive tasks. We show that these biologically constrained, heavy-tailed RNNs exhibit substantially improved generalization, which we further characterize within a PAC-Bayes framework. Our theoretical analysis and numerical experiments reveal two complementary mechanisms underlying this generalization enhancement. Topologically, heavy-tailed connectivity induces an effectively low-rank structure, which in turn yields low-dimensional neural dynamics. Geometrically, heavy-tailed connectivity intrinsically shapes task variable representations to lie near a linear manifold, thereby improving generalization for a linear readout strategy. Together, these results identify heavy-tailed connectivity as a biologically grounded intrinsic mechanism that promotes low-rank structure and favorable representational geometry, leading to improved generalization in flexible cognitive tasks.

Applications · Neuroscience, Cognitive Science

Timothy Kim, Ulises Obilinovic, Yiliu Wang, Eric SheaBrown, Uygar Sümbül

Connectivity structure shapes neural computation, but inferring this structure from population recordings is degenerate: multiple connectivity structures can generate identical dynamics. Recent work uses low-rank recurrent neural networks (lrRNNs) to infer low-dimensional latent dynamics and connectivity structure from observed activity, enabling a mechanistic interpretation of the dynamics. However, standard approaches for training lrRNNs can recover spurious structures irrelevant to the underlying dynamics. We first characterize the identifiability of connectivity structures in lrRNNs and determine conditions under which a unique solution exists. Then, to find such solutions, we develop an inference framework based on maximum entropy and continuous normalizing flows (CNFs), trained via flow matching. Instead of estimating a single connectivity matrix, our method learns the maximally unbiased distribution over connection weights consistent with observed dynamics. This approach captures complex yet necessary distributions such as heavy-tailed connectivity found in empirical data. We validate our method on synthetic datasets with connectivity structures that generate multistable attractors, limit cycles, and ring attractors, and demonstrate its applicability in recordings from rat frontal cortex during decision-making. Our framework shifts circuit inference from recovering connectivity to identifying which connectivity structures are computationally required, and which are artifacts of underconstrained inference.

Applications · Neuroscience, Cognitive Science

Noga Mudrik, Yuxi Chen, Gal Mishne, Adam Charles

Many fields collect large-scale temporal data through repeated measurements (`trials’), where each trial is labeled with a set of metadata variables spanning several categories. For example, a trial in a neuroscience study may be linked to a value from category (a): task difficulty, and category (b): animal choice. A critical challenge in time-series analysis is to understand how these labels are encoded within the multi-trial observations, and disentangle the distinct effect of each label entry across categories. Here, we present MILCCI, a novel data-driven method that i) identifies the interpretable components underlying the data, ii) captures cross-trial variability, and iii) integrates label information to understand each category's representation within the data. MILCCI extends a sparse per-trial decomposition that leverages label similarities within each category to enable subtle, label-driven cross-trial adjustments in component compositions and to distinguish the contribution of each category. MILCCI also learns each component’s corresponding temporal trace, which evolves over time within each trial and varies flexibly across trials. We demonstrate MILCCI’s performance through both synthetic and real-world examples, including voting patterns, online page view trends, and neuronal recordings.

Applications · Neuroscience, Cognitive Science

Lianghuan Huang, Yihao Li, Saeed Salehi, Yingshan Chang, Ansh Soni, Konrad Kording

Representations of the world, arguably, contain information about features (e.g. something is blue, something is a circle) but also information about which features are part of the same object (e.g. the circle is blue), which we call binding information. Any system with the ability to understand scenes with multiple objects must be able to solve the binding problem: it needs to know which features belong together. However, despite work showing that Vision Transformers (ViTs) know which patches belong together, it is not known whether current deep learning models learn binding information, i.e., for features. We may believe that there is not much binding information, after all misattributing features to wrong objects is a common failure of ViT-based architectures, especially in scenes with distracting objects. Here we formalize the binding problem with an information-theoretic approach, and introduce a probing method to measure binding information in model representations. We perform experiments measuring binding information in different datasets with different number of features, different occlusion levels of objects, synthetic (e.g., red, circle) versus natural features (e.g., bikes, running), as well as out-of-distribution feature combinations (e.g. blue penguins), while performing these experiments on several pre-trained ViTs. Our research demonstrates binding as a key ingredient to strong visual recognition and reasoning.

Applications · Neuroscience, Cognitive Science

Francesco Innocenti, El Mehdi Achour, Rafal Bogacz

Predictive coding (PC) is a biologically plausible alternative to standard backpropagation (BP) that minimises an energy function with respect to network activities before updating weights. Recent work has improved the training stability of deep PC networks (PCNs) by leveraging some BP-inspired reparameterisations. However, the full scalability and theoretical basis of these approaches remains unclear. To address this, we study the infinite width and depth limits of PCNs. For linear residual networks, we show that the set of width- and depth-stable feature-learning parameterisations for PC is exactly the same as for BP. Moreover, under any of these parameterisations, the PC energy with equilibrated activities converges to the BP loss in a regime where the model width is much larger than the depth, resulting in PC computing the same gradients as BP. Experiments show that these results hold in practice for deep nonlinear networks as long as an activity equilibrium is reached. Overall, this work unifies various previous theoretical and empirical results and has important practical implications for the scaling of PCNs.

Applications · Neuroscience, Cognitive Science

Todd Morrill, Christian-Gernot Pehle, Anthony Zador

Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events, treating spike timing and ordering as the representation rather than an artifact of time discretization. This viewpoint aligns with biological computation and with the native resolution of event sensors and neuromorphic processors, while enabling compute and memory that scale with the number of events. However, two challenges hinder practical, end-to-end trainable event-based SNN systems: 1) exact charge--fire--reset dynamics impose inherently sequential processing of input spikes, and 2) precise spike times must be solved without time bins. We address both. First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 43x speedups over sequential simulation while retaining exact hard-reset dynamics. Second, we implement differentiable spike-time solvers that compute spike times to machine precision without discrete-time approximations or restrictive analytic assumptions. We demonstrate the viability of training SNNs using our solutions on four event-based datasets on GPUs.

Applications · Neuroscience, Cognitive Science

Chenggang Chen, Zhiyu Yang, Xiaoqin Wang

Deep neural networks currently provide the leading quantitative models of neural responses in sensory systems. However, these networks remain implausible as models of sensory development, largely because they rely on supervised training with label efficiency far exceeding that of biological learning. Furthermore, these models are typically trained on manually curated datasets that lack the statistical properties of the natural environments to which the brain is exposed. Here, we demonstrate that models trained with unsupervised objectives on real-world data significantly outperform supervised models in predicting brain responses across both human auditory and visual cortex. We show that this performance advantage is not driven by network architecture or dataset size, but rather by the data distribution. Crucially, we find that unsupervised models trained on real-world data exhibit remarkable out-of-distribution generalization: a model trained exclusively on Mandarin speech accurately predicts English-driven brain responses, and a model trained on infant head-cam footage predicts adult visual responses to curated object images. Together, our results illustrate how deep neural networks can be used to reveal the real-world statistics that shape neural representations in the brain.

Applications · Neuroscience, Cognitive Science

Dongyang Ma, Zhengyu Ma, Yifan Huang, Chenlin Zhou, Wei Zhang, Yonghong Tian

Retina-like event cameras and brain-inspired Spiking Neural Networks (SNNs) demonstrate exceptional energy efficiency through bio-inspired sensing and computation. While SNNs are naturally well-suited to the asynchronous nature of event data, their practical applications face the following challenges: sensitivity to noise, dense representations that disrupt spike pathways, and insufficient multi-scale feature perception. To address the aforementioned challenges, we propose a bio-vision-inspired object detection method motivated by biological (bio) vision systems. First, at the micro level, this paper proposes a noise-filtering STATNF-Neuron architecture to address the current sensitivity of basic neurons to noise. Based on STATNF-Neurons, the paper introduces two bio-vision-inspired macro-structures: Events-to-Spikes Representation (E2S), which preserves spiking characteristics while mimicking the memory and noise-filtering abilities of retinal neurons; Bidirectional Multi-Scale Spiking Network (BiSNet), which simulates cortical information flow pathways to integrate multi-scale features in both directions, enhancing the network's ability to perceive information at multiple scales. Extensive experiments show that the proposed bio-vision-inspired method achieving state-of-the-art performance. Notably, it reaches 96.1\% accuracy on NCAR, 63.5\% mAP\textsubscript{50} on N-Caltech101, and 69.1\% mAP\textsubscript{50} on Gen1.

Applications · Neuroscience, Cognitive Science

Dennis Wu, Yi-Chun Hung, Braden Yuille, James Fitzgerald, Han Liu

Neural population geometry shapes downstream inference. Recent findings in neurobiology suggest that a hyperbolic structure underlies population activity. However, a theoretical framework for this phenomenon is still lacking. Here, we propose a plausible construction of hippocampal tuning curves that statistically induce hyperbolic geometry. Next, we establish a connection between neural decoding and associative memory by demonstrating that the Modern Hopfield Network update rule computes the optimal squared loss estimator under hyperbolic geometry. Furthermore, we introduce a novel associative memory model defined in hyperbolic space that yields significantly larger capacity than existing models. Our results suggest that animals encode spatial information as a latent hyperbolic cognitive map, which enhances both memory capacity and decoding accuracy.

Applications · Neuroscience, Cognitive Science

Yizhuo Lu, Changde Du, Qingyu Shi, Hang Chen, Jie Peng, Liuyun Jiang, Shuangchen Zhao, Huiguang He

Modeling the interplay between external stimuli and internal neural representations is a pivotal research area for Brain-Computer Interfaces (BCIs). A major limitation of prior work is the prevailing paradigm of specialized, single-task models, which curtails versatility and neglects inter-task synergies. To address this, we propose Mind-Omni, the first versatile framework that unifies seven distinct encoding and decoding tasks through a discrete diffusion paradigm. At its core is a novel Brain Tokenizer that transforms heterogeneous, continuous brain signals into standardized, discrete tokens. This enables direct, token-level interactions for mutual understanding and generation between any two or more modalities within a shared semantic space. To unlock advanced reasoning capabilities, we further curate a specialized Brain Question Answering (BQA) instruction-tuning dataset. Our model not only establishes a new state-of-the-art among multi-task unified frameworks but also provides strong evidence for multi-task synergy. By demonstrating performance competitive with, and at times superior to, larger specialized models, our work offers a powerful new paradigm for neural modeling and paves the way for foundation models of neural activity.