论文检索

Applications · Neuroscience, Cognitive Science

Haorui Sun, Ardyn Olszko, Niharika Singh, David Jangraw

We present Reading Observed At Mindless Moments (ROAMM), a large-scale multimodal dataset comprising 50 hours of simultaneous EEG and eye-tracking recorded during naturalistic multi-page reading from 44 participants, with annotations including eye events, page-level comprehension scores, and word-level mind-wandering (MW) labels obtained via a retrospective self-report paradigm. We introduce a standardized evaluation protocol for MW detection under leave-one-subject-out evaluation, achieving up to 0.609 AUROC using supervised models. We also report results for EEG-to-text decoding trained on non-MW segments and show that decoding performance decreases when MW-labeled segments are included. Overall, ROAMM provides a benchmark dataset for MW detection and EEG-to-text decoding tasks, and enables the study of attention-related degradation in language decoding from brain activity in naturalistic reading.

Applications · Neuroscience, Cognitive Science

Kartikay Agrawal, Vaishnavi N, Abhijeet Vikram, Vedant Sharma, Ayon Borthakur

Spiking neural networks have attracted increasing attention for their energy efficiency, multiplication-free computation, and sparse event-based processing. In parallel, state space models have emerged as a scalable alternative to transformers for long-range sequence modelling by avoiding quadratic dependence on sequence length. We propose here a spiking heterogeneous harmonic resonate-and-fire state space model (S$H^2$RFSSM), a second-order spiking SSM for classification and regression on ultra-long sequences. S$H^2$RFSSM outperforms transformers and first-order SSMs on average while eliminating matrix multiplications, making it highly suitable for resource-constrained applications. Furthermore, we introduce a kernel-based spiking regressor that enables accurate modelling of dependencies in sequences of up to 50k steps. We also observe a reduction in spiking operations and improved performance with heterogeneity and discretisation in harmonic resonate-and-fire neuronal layers. Overall, we evaluate Harmonic Resonate-and Fire layers across 17 diverse datasets, spanning sensors, time series, and classification to long-term forecasting. Our results demonstrate that S$H^2$RFSSM achieves superior long-range modelling capability with energy efficiency, positioning it as a strong candidate for signal processing on resource-constrained devices for human activity recognition, time series classification, and regression.

Applications · Neuroscience, Cognitive Science

Jinhan Liu, Mahsa Shoaran

Affective and cognitive disorders manifest as distributed, time-varying brain network dynamics across regions, channels, and time, challenging robust representation learning from EEG/sEEG for clinical diagnosis. We propose **RECTOR** (Masked **Re**gion–**C**hannel–**T**emp**or**al Modeling), an end-to-end self-supervised framework that unifies joint region-channel-temporal representation learning beyond fixed anatomical priors. At its core, **RECTOR-SA** is a hierarchical, block-sparse self-attention induced by Adaptive Functional Partitioning that evolves region structures from static anatomical definitions to adaptive functional regions. The self-supervision is driven by **Masked Topology and Representation Learning**, which jointly optimizes three complementary objectives: Masked Predictive Modeling, Topological Structure Modeling, and Cross-View Consistency. Across diverse benchmarks, RECTOR sets a new state-of-the-art in EEG emotion recognition and sEEG task-engagement classification. Crucially, its strong robustness to missing channels and cross-montage generalization underscores its potential for large-scale pre-training on heterogeneous EEG/sEEG, providing interpretable insights at both region and channel levels.

Applications · Neuroscience, Cognitive Science

Gustavo Grivol, Alexander Tuzhilin

Modeling decision-making outside of controlled environments requires accounting for asynchronous, exogenous signals, such as notifications or algorithmic feeds, that dynamically alter user response times. Standard Drift-Diffusion Models (DDM) become analytically intractable when drift rates vary continuously with time. In this paper, we derive a closed-form analytical approximation for the first-passage time distribution of a single-boundary DDM with time-dependent drift, valid in the high-threshold regime. The main result allows us to analytically study the optimal timing of external signals to maximize the probability of a user response within our approximation framework. To evaluate our response time model, we conduct an extensive empirical comparison with state-of-the-art methods for user watch-time prediction and evaluation in simulated environments.

Applications · Neuroscience, Cognitive Science

Xinyu Shi, Tong Bu, Zhaofei Yu

Spiking Neural Networks (SNNs) offer a notable energy-saving advantage compared to Artificial Neural Networks (ANNs) when deployed on neuromorphic hardware. While recent SNNs achieve superior performance using larger and deeper backbones, this comes at a cost of diminishing their energy-saving benefits. In this paper, we propose UniSparse, a unified sparsification framework for enhancing the energy efficiency of SNNs. We demonstrate that the affine parameters in batch normalization also serve as the learnable threshold of its subsequent spiking neurons. Based on this, we propose a novel spike sparsification method that reduces firing rate by constraining the affine parameters. As a complement to spike sparsification, we propose a weight pruning method based on the same energy constraint, which can be naturally integrated with spike sparsification. Experimental results demonstrate that UniSparse achieves a state-of-the-art trade-off between accuracy and energy efficiency across models and datasets. The sparsified ResNet-18 model requires only 7.04M SOPs for inference to achieve 92.38\% accuracy on the CIFAR-10 dataset. Our work highlights the great potential of deep SNNs in improving energy efficiency.

Applications · Neuroscience, Cognitive Science

Xinyu Shi, Zhaofei Yu

The training of deep Spiking Neural Networks (SNNs) has traditionally relied on Batch Normalization (BN), which stabilizes input currents and gradients during training. However, BN is not a universal solution. It is unsuitable for variable-length tasks and scenarios with reduced batch size, constraining the development of deep SNNs, where removing BN typically causes the training to fail to converge. This dependence stems not from a fundamental necessity of BN but from the current lack of reasonable initialization methods for SNNs. This paper addresses this core limitation by proposing SpikeInit, a novel initialization framework for SNNs. By modeling the response curve and gradient of spiking layers, SpikeInit initializes the weights and shape parameters of surrogate gradients to maintain stable firing rates during forward propagation and stable gradient magnitudes during backpropagation. Extensive experiments demonstrate that deep SNNs with SpikeInit can be trained stably without normalization and achieve superior performance compared to their normalized counterparts under identical settings. Furthermore, we demonstrate the scalability of SpikeInit by successfully training an ultra-deep, 1000-layer SNN without normalization. Our work provides a foundational step toward large-scale normalization-free SNN, liberating SNN design from the constraints of normalization.

Applications · Neuroscience, Cognitive Science

Yiwen Gu, Junchuan Gu, Haibin Shen, Kejie Huang

Spiking Neural Networks (SNNs) emulate the spiking behavior of biological neurons and are promising for energy-efficient neuromorphic computing. A widely used strategy to train SNNs is to convert pretrained Artificial Neural Networks (ANNs), where the accuracy and efficiency are determined by the spike encoding scheme. Traditional methods based on spike count or timing severely underutilize the available encoding space, leading to large accuracy degradation under low-timestep constraints. More expressive alternatives involve complex dynamics, which hinder scalability and practical deployment. To address these challenges, we propose Temporal Weighted Encoding (TWE). Through a simple recursive integration, spikes are implicitly assigned exponentially decaying weights, drawing an analogy to a temporal bit sequence. We systematically analyze the temporal mismatch caused by this weight pattern and propose temporal relaxation and threshold relaxation to resolve this issue, enabling fast and accurate activation encoding. Extensive experiments demonstrate that TWE achieves negligible conversion loss with significantly fewer timesteps, offering a scalable and efficient solution for SNN deployment.

Applications · Neuroscience, Cognitive Science

Binghao Ye, Wenjuan Li, Dengfeng Xue, Bing Li, Weiming Hu, Dong Liang, Kun Shang

Spiking Neural Networks (SNNs) have garnered increasing attention for their biological plausibility, energy efficiency, and temporal modeling capability. Due to the non-differentiability of spike generation, a widely used supervised training method for SNNs is backpropagation through time with surrogate gradients, which achieves competitive performance with a small number of timesteps. Intuitively, scaling timesteps should further improve performance by enriching temporal dynamics. However, we observe timestep scaling paradox (TSP), a counter-intuitive degradation in accuracy when scaling timesteps. We investigate TSP and link it to long-term temporal gradient vanishing and weakened cross-timestep dependencies. To address this, we propose the Timestep-Scalable (TS) neuron model. It introduces long-term memory reconsolidation to enhance cross-timestep information flow and enable effective learning with more timesteps. In parallel, a temporal forgetting mechanism periodically truncates the accumulation path, suppressing excessive temporal information buildup and improving training stability. Supported by theoretical analysis and extensive experiments, TS consistently improves performance when scaling timesteps. Beyond gains from timestep scaling, it attains state-of-the-art results on EEG signals, event-based recognition, and time-series forecasting, while remaining strong on conventional image classification and object detection datasets.

Applications · Neuroscience, Cognitive Science

Minxu Liu, Donghai Guan, Chuhang Zheng, Chunwei Tian, Jie Wen, Qi Zhu

Understanding and decoding brain activity into visual representations is a fundamental challenge at the intersection of neuroscience and artificial intelligence. While electroencephalogram (EEG) visual decoding has shown promise due to its non-invasive and low-cost nature, existing methods suffer from Hierarchical Neural Encoding Neglect (HNEN), a critical limitation in which flat neural representations fail to model the brain’s hierarchical visual processing. Inspired by the hierarchical organization of visual cortex, we propose ViEEG, a neuro-inspired framework that addresses HNEN. ViEEG decomposes each visual stimulus into three biologically aligned components, namely contour, foreground object, and contextual scene, which serve as anchors for a three-stream EEG encoder. These EEG features are progressively integrated via cross-attention routing, simulating cortical information flow from low-level to high-level vision. We further adopt hierarchical contrastive learning for EEG-CLIP representation alignment, enabling zero-shot object recognition. Extensive experiments on THINGS-EEG dataset demonstrate that ViEEG significantly outperforms previous methods by a large margin in both subject-dependent and subject-independent settings. Results on THINGS-MEG dataset further confirm ViEEG's generalization to different neural modalities. Our framework not only advances the performance frontier but also sets a new paradigm for EEG brain decoding. Code and pretrained models will be available.

Applications · Neuroscience, Cognitive Science

Kaiwei Che, Wei Fang, Zhengyu Ma, Yifan Huang, Peng Xue, Li Yuan, Yonghong Tian

Spiking Neural Networks (SNNs), with their event-driven and biologically inspired mechanisms, are well-suited for energy-efficient neuromorphic hardware. Neural coding, which is critical to SNNs, determines how information is represented via spikes. While Time-to-First-Spike (TTFS) coding uses a single spike per neuron to offer extreme sparsity and energy efficiency, it often suffers from unstable training and low accuracy due to its sparse firing. To address these challenges, we propose a training framework that incorporates parameter initialization, training normalization, a temporal output decoder, and a re-evaluation of the pooling layer. The proposed parameter initialization and training normalization mitigate signal diminishing and gradient vanishing, which helps stabilize training. Our output decoder aggregates temporal spikes to encourage earlier firing, thereby reducing latency. The re-evaluation of the pooling layer demonstrates that max-pooling violates single-spike constraints, which should be avoided, whereas average-pooling preserves them. Experiments show that our framework stabilizes and accelerates training, reduces latency, and achieves state-of-the-art accuracy for step-by-step TTFS SNNs on MNIST ($99.48\%$), Fashion-MNIST ($92.90\%$), CIFAR10 ($90.56\%$), CIFAR100 ($70.27\%$) and DVS Gesture ($95.83\%$).

Applications · Neuroscience, Cognitive Science

Ezekiel Williams, Alexandre Payeur, Guillaume Lajoie

Biological and neuromorphic recurrent neural networks (RNNs) are subject to spatial and temporal locality constraints on the information that can plausibly be used during learning. A common strategy to satisfy these constraints is to modify gradient descent by neglecting non-local terms to varying degrees, as in random feedback local online (RFLO) learning and truncated backpropagation through time (tBPTT). However, the learning dynamics of these algorithms, and how they compare with BPTT, remain poorly understood. Here, we apply dynamical systems theory to data-aligned linear RNNs to compare stationary solutions, stability properties, and convergence rates, finding a close relationship between BPTT and one-step tBPTT—in contrast with qualitatively distinct behavior for RFLO. We further observe that the solutions learned by RFLO are restricted to low-rank perturbations of initial parameters, a result which holds beyond the data-aligned setting. Our work provides fundamental insights into how locality constraints shape RNN learning dynamics, with implications for neuroscientific models of learning and alternative optimization approaches for state-space models.

Applications · Neuroscience, Cognitive Science

Abdülkadir Gökce, Yingtian Tang, Martin Schrimpf

Task-optimized neural networks are the leading in-silico models of sensory cortex, yet the field lacks a unified understanding of which modeling choices drive improved brain alignment. Prior NeuroAI work is fragmented across datasets and modalities, making it difficult to determine robust scaling trends. Here, we systematically investigate the scaling laws of model-to-brain alignment across 8 neural datasets (spanning electrophysiology, fMRI, EEG, and MEG) and over 600 models with diverse architectures and pretraining configurations. We report three key scaling trends: (1) *Pretraining saturation*: Alignment improves with pretraining compute and data scale but saturates across all recording modalities. (2) *Complementary fine-tuning*: Hybrid task & neural data optimization yields consistent improvements in alignment that generalize across datasets and modalities. (3) *Mapping scaling*: Increasing the number of neural samples to fit model-to-brain mappings yields log-linear gains with the largest impact on alignment. Finally, we propose a novel subject-shared cross-attention mapping which drastically reduces parameter count and improves alignment. Taken together, these results establish multimodal scaling laws that guide resource allocation for next-generation brain models.

Applications · Neuroscience, Cognitive Science

Junfeng Zuo, Yuhang He, Wenhao Zhang, Fang Fang, Si Wu

Navigation in complex environments relies on internal spatial representations that guide action. While the brain employs a diverse repertoire of spatial tuning cells—including grid, place, and head-direction cells—a normative theory linking these static neural codes to the dynamic process of navigation remains elusive. In this work, we propose a Unified Laplacian Framework derived from first principles of representational smoothness and efficiency. We first demonstrate that diverse spatial codes emerge naturally as spectral decompositions of the Laplacian operator. Crucially, bridging the gap from representation to action, we derive a biologically plausible navigation policy based on the Green's function potential. We show that this potential encodes the environment's intrinsic geometry to enable simple, trap-free gradient ascent, achieving significantly improved sample efficiency and generalization in goal-reaching tasks. Furthermore, we demonstrate that these spectral representations can be learned directly from high-dimensional visual inputs, confirming its plausibility in realistic environments. Our results suggest that the "cognitive map" can be viewed as a spectral embedding of the Laplacian, providing a rigorous foundation for spatial cognition in both biological and artificial agents.

Applications · Neuroscience, Cognitive Science

Amrith Lotlikar, Ian Tanoh, Praful Vasireddy, Andrew Lanpouthakoun, Michael A Sommeling, Ramandeep Vilkhu, A.J. Phillips, Alexander Sher, Alan Litke, Scott Linderman 等

Multi-compartment Hodgkin–Huxley (HH) models provide a principled framework for predicting neural dynamics and responses to electrical stimulation. However, fitting HH biophysical parameters typically requires intracellular recordings, which are invasive and low-throughput, limiting the ability to capture the geometry and cell-specific properties of many neurons in a given neural circuit. Multi-electrode arrays (MEAs) offer a scalable alternative—high-density extracellular measurements from full neural populations—but HH model complexity has so far precluded reliable biophysical inference from extracellular data alone. Here, we introduce a framework that rapidly infers HH parameters from designed features of extracellular MEA measurements by leveraging differentiable biophysical simulation and simulation-based inference, unlocking a wide range of downstream applications. In this work, we focus on a central goal of translational neuroengineering: predicting neural spiking responses to candidate neurostimulation patterns that would take hours to measure clinically. To validate our approach, we collected hundreds of hours of stimulation and recording data from isolated macaque retina with a 30 µm-pitch 512-electrode array. Our framework predicted previously unseen multi-electrode stimulation responses with 90.4\% accuracy using HH models fit from only a few minutes of recording, replacing hours of stimulus testing.

Applications · Neuroscience, Cognitive Science

Jiahe Li, Junru Chen, Fanqi Shen, Jialan Yang, Jada Li, Zhizhang Yuan, Baowen Cheng, Li Meng, Yang Yang

Enabling natural communication through brain–computer interfaces (BCIs) remains one of the most profound challenges in neuroscience and neurotechnology. While existing frameworks offer partial solutions, they are constrained by oversimplified semantic representations and a lack of interpretability. To overcome these limitations, we introduce **Semantic Intent Decoding(SID)**, a novel framework that translates neural activity into natural language by modeling meaning as a flexible set of compositional semantic units. SID is built on three core principles: semantic compositionality, continuity and expandability of semantic space, and fidelity in reconstruction. We present **BrainMosaic**, a deep learning architecture implementing SID. BrainMosaic decodes multiple semantic units from EEG/SEEG signals using set matching and then reconstructs coherent sentences through semantic-guided reconstruction. This approach moves beyond traditional pipelines that rely on fixed-class classification or unconstrained generation, enabling a more interpretable and expressive communication paradigm. Extensive experiments on multilingual EEG and clinical SEEG datasets demonstrate that SID and BrainMosaic offer substantial advantages over existing frameworks, paving the way for natural and effective BCI-mediated communication.

Applications · Neuroscience, Cognitive Science

Kai Sun, Peibo Duan, Yongsheng Huang, Nanxu Gong, Levin Kuhlmann

Spiking Neural Networks (SNNs), inspired by biological neurons, are attractive for their event-driven energy efficiency but still fall short of Artificial Neural Networks (ANNs) in accuracy. Knowledge distillation (KD) has emerged as a promising approach to narrow this gap by transferring ANN knowledge into SNNs. Temporal-wise distillation (TWD) leverages the temporal dynamics of SNNs by providing supervision across timesteps, but it applies a constant teacher output to all timesteps, mismatching the inherently evolving temporal process of SNNs. Moreover, while TWD improves per-timestep accuracy, truncated inference still suffers from full-length temporal information loss due to the progressive accumulation process. We propose **MEOM** (**M**any **E**yes, **O**ne **M**ind), a unified KD framework that enriches supervision with diverse temporal perspectives through mask-weighted teacher features and progressively aligns truncated predictions with the full-length prediction, thereby enabling more reliable inference across all timesteps. Extensive experiments and theoretical analyses demonstrate that MEOM achieves state-of-the-art performance on multiple benchmarks. Code is available at https://github.com/KaiSUN1/MEOM.

Applications · Neuroscience, Cognitive Science

Francesco Carzaniga, Michael Hersche, Abu Sebastian, Kaspar Schindler, Abbas Rahimi

Learning from multi-variate time-series with heterogeneous channel configurations remains a fundamental challenge for deep neural networks, particularly in clinical domains such as intracranial electroencephalography (iEEG), where channel setups vary widely across subjects. In this work, we introduce multi-variate parallel attention (MVPA), a novel self-attention mechanism that disentangles content, temporal, and spatial attention, enabling flexible, generalizable, and efficient modeling of time-series data with varying channel counts and configurations. We use MVPA to build MVPFormer, a generative foundation model for human electrophysiology, trained to predict the evolution of iEEG signals across diverse subjects. To support this and future efforts by the community, we release the SWEC iEEG dataset, the largest publicly available iEEG dataset to date, comprising nearly 10,000 hours of recordings from heterogeneous clinical sources. MVPFormer leverages MVPA to achieve strong generalization across subjects, demonstrating expert-level performance in several iEEG tasks. MVPFormer surpasses state-of-the-art (SOTA) Transformer baselines in seizure detection across the SWEC, the MAYO, and the FNUSA datasets, while also achieving SOTA performance on four Brain TreeBank iEEG decoding tasks (volume, pitch, onset, and speech). We further validate MVPA on standard time-series forecasting and classification tasks, where it matches or exceeds the performance of existing attention-based models. Together, our contributions establish MVPA as a general-purpose attention mechanism for heterogeneous time-series and MVPFormer as the first open-source, open-weights, and open-data iEEG foundation model with SOTA clinical performance. The code and weights are available at https://github.com/IBM/multi-variate-parallel-transformer. The SWEC iEEG dataset is available at https://huggingface.co/datasets/NeuroTec/SWEC_iEEG_Dataset.

Applications · Neuroscience, Cognitive Science

Yijie Lu, Zhiyi Pan, Renrui Zhang, Yanhao Jia, Ronggang Wang, Zhaokun Zhou

Spiking Transformer has sparked growing interest, with the Spiking Self-Attention merging spikes with self-attention to deliver both energy efficiency and competitive performance. However, existing work primarily focuses on 2D visual tasks, and in the domain of 3D point clouds, the disorder and complexity of spatial information, along with the scale of the point clouds, present significant challenges. For point clouds, we introduce spiking discrepancy, measuring differences in spike features to highlight key information, and then construct the Spiking Discrepancy Attention Mechanism (SDAM). SDAM contains two variants: the Spiking Element Discrepancy Attention captures local geometric correlations between central points and neighboring points, while the Spiking Intensity Discrepancy Attention characterizes structural patterns of point clouds based on macroscopic spike statistics. Moreover, we propose a Spatially-Aware Spiking Neuron. Based on these, we construct a hierarchical Spiking Discrepancy Transformer. Experimental results demonstrate that our method achieves state-of-the-art performance within the Spiking Neural Networks and exhibits impressive performance compared to Artificial Neural Networks along with a few parameters and significantly lower theoretical energy consumption.

Applications · Neuroscience, Cognitive Science

William Qian, Cengiz Pehlevan

Training recurrent neural networks (RNNs) to perform neuroscience-style tasks has become a popular way to generate hypotheses for how neural circuits in the brain might perform computations. Recent work has demonstrated that task-trained RNNs possess a strong simplicity bias. In particular, this inductive bias often causes RNNs trained on the same task to collapse on effectively the same solution, typically comprised of fixed-point attractors or other low-dimensional dynamical motifs. While such solutions are readily interpretable, this collapse proves counterproductive for the sake of generating a set of genuinely unique hypotheses for how neural computations might be performed. Here we propose Iterative Neural Similarity Deflation (INSD), a simple method to break this inductive bias. By penalizing linear predictivity of neural activity produced by standard task-trained RNNs, we find an alternative class of solutions to classic neuroscience-style RNN tasks. These solutions appear distinct across a battery of analysis techniques, including representational similarity metrics, dynamical systems analysis, and the linear decodability of task-relevant variables. Moreover, these alternative solutions can sometimes achieve superior performance in difficult or out-of-distribution task regimes. Our findings underscore the importance of moving beyond the simplicity bias to uncover richer and more varied models of neural computation.

Applications · Neuroscience, Cognitive Science

Roman Beliy, Amit Zalcher, Jonathan Kogman, navve wasserman, michal Irani

Reconstructing images seen by people from their fMRI brain recordings provides a non-invasive window into the human brain. Despite recent progress enabled by diffusion models, current methods often lack faithfulness to the actual seen images. We present ``Brain-IT'', a brain-inspired approach that addresses this challenge through a Brain Interaction Transformer (BIT), allowing effective interactions between clusters of functionally-similar brain-voxels. These functional-clusters are shared by all subjects, serving as building blocks for integrating information both within and across brains. All model components are shared by all clusters \& subjects, allowing efficient training with limited amount of data. To guide the image reconstruction, BIT predicts two complementary localized patch-level image features: (i) high-level semantic features which steer the diffusion model toward the correct semantic content of the image; and (ii) low-level structural features which help to initialize the diffusion process with the correct coarse layout of the image. BIT's design enables direct flow of information from brain-voxel clusters to localized image features. Through these principles, our method achieves image reconstructions from fMRI that faithfully reconstruct the seen images, and surpass current SotA approaches both visually and by standard objective metrics. Moreover, with only 1-hour of fMRI data from a new subject, we achieve results comparable to current methods trained on full 40-hour recordings.