论文检索

Applications · Neuroscience, Cognitive Science

Junfeng Lu, Zijie Yu, SHAOYANG CUI, Gan He, Ruiqin Xiong, Kai Du, Tiejun Huang

Biophysically detailed neural networks represent a promising frontier for brain-inspired AI, offering intrinsic spatio-temporal dynamics to enhance the expressivity and computational density of deep learning systems. However, general-purpose deep learning frameworks suffer from a fundamental mismatch between their dense parallel optimizations and the irregular, tree-structured complexity of biological mechanisms. In this work, we propose **HelioX**, a **GPU-native** framework designed to unify high-performance simulation with scalable training. Unlike approaches that adapt biology to existing deep learning tools, HelioX adopts a "GPU-to-Biophysics" paradigm. We tailor the underlying GPU parallelism to biological structures by implementing custom-fused CUDA kernels for both the Dendritic Hierarchical Scheduling (DHS) algorithm and its gradient propagation. This design eliminates the runtime overhead of generic automatic differentiation and enables multi-stream concurrency for spike generation and equation assembly. Experimental results demonstrate that HelioX outperforms standard simulators (NEURON) by orders of magnitude and surpasses prior GPU-based solvers in both speed and scalability. We successfully train deep biophysical MLPs and organism-scale biophysical neural networks (e.g., the BAAIWorm C. elegans model) on a single consumer-grade GPU. HelioX establishes a new standard for computational efficiency, enabling the training of biophysically detailed models at scales previously unattainable.

Applications · Chemistry, Physics, and Earth Sciences

Kexin Zhang, Weichen Qin, Yue Teng, Jiale Yu, Yuanyuan Ma, jinyu lin, Liping Sun, Jie Zheng, Jingyi Yu

The emergence of Vibe Researching is transforming scientific research into an interactive workflow, where agents orchestrate complex tasks via the Model Context Protocol (MCP). In this ecosystem, scientific tools must evolve from offline simulators into responsive Agent Skills. However, diffusion-based protein docking models—a core component of the current deep learning infrastructure for structural biology—suffer from excessively high latency, rendering them incompatible with real-time agentic interaction. To bridge this gap, we present a compute-efficient vertical foundation model that synergizes architectural optimization with generative consistency. First, we leverage Progressive Consistency Regularization (PCR) to compress complex generative dynamics into a few-step predictor, achieving sub-second latency. Second, we propose Residual Quantization, using mixed-precision on residual streams to alleviate memory bottlenecks while preserving numerical precision. Our approach achieves state-of-the-art (SOTA) docking accuracy while attaining a two-order-of-magnitude speedup ($>300\times$) over AlphaFold3, establishing a new efficiency standard for high-throughput virtual screening. By transforming molecular docking into an interactive, real-time tool, this work establishes a scalable, deep-learning infrastructure for the next generation of AI-driven drug discovery.

Kexin Zhang, Weichen Qin, Yue Teng, Jiale Yu, Yuanyuan Ma, jinyu lin, Liping Sun, Jie Zheng, Jingyi Yu

The emergence of Vibe Researching is transforming scientific research into an interactive workflow, where agents orchestrate complex tasks via the Model Context Protocol (MCP). In this ecosystem, scientific tools must evolve from offline simulators into responsive Agent Skills. However, diffusion-based protein docking models—a core component of the current deep learning infrastructure for structural biology—suffer from excessively high latency, rendering them incompatible with real-time agentic interaction. To bridge this gap, we present a compute-efficient vertical foundation model that synergizes architectural optimization with generative consistency. First, we leverage Progressive Consistency Regularization (PCR) to compress complex generative dynamics into a few-step predictor, achieving sub-second latency. Second, we propose Residual Quantization, using mixed-precision on residual streams to alleviate memory bottlenecks while preserving numerical precision. Our approach achieves state-of-the-art (SOTA) docking accuracy while attaining a two-order-of-magnitude speedup ($>300\times$) over AlphaFold3, establishing a new efficiency standard for high-throughput virtual screening. By transforming molecular docking into an interactive, real-time tool, this work establishes a scalable, deep-learning infrastructure for the next generation of AI-driven drug discovery.

Applications · Chemistry, Physics, and Earth Sciences

Haokai Hong, Wanyu LIN, KC Tan

Large-scale molecular dynamics simulations are essential in understanding chemical and biological processes, necessitating the accurate and efficient modeling of interatomic interactions. Existing learning-based methods generally are based on message passing mechanisms; they are either not scalable or too coarse to offer accurate modeling. We propose a new message passing framework that can effectively and efficiently model interatomic interactions for simulating large-scale molecular dynamics at full atomic resolution. Specifically, our framework is stacked with a sequence of message passing neural network layers, each realizing the message passing over a distinct and dilated star-structured path. These star-structured paths are constructed progressively along dilated regions to capture the distance-dependent interactions. The crux of our framework is that it resolves the problem of dense interatomic interactions of large-scale atomic systems with sparser and region-based message passing graphs. We evaluate the framework on four benchmarks: MD22 (molecules with 42–370 atoms), Chignolin (a 166-atom protein featuring diverse conformations), the AdK dataset (a protein trajectory with up to 3,000 atoms), and the MISATO dataset (over 10,000 heterogeneous protein-ligand complexes with systems up to 40,000 atoms). Comprehensive evaluations demonstrate that our approach delivers state-of-the-art performance overall across various benchmarks.

Deep Learning · Algorithms

Keigo Nishida, Eren Mehmet KIRAL, Kenichi Bannai, Mohammad Emtiyaz Khan, Thomas Moellenhoff

We propose a new algorithm enabling stable training under low-precision computations. We call this algorithm Log-normal Multiplicative Dynamics (LMD), and derive it by taking inspiration from the robustness of biological neural networks. Such networks use synapses whose size follow log-normal distribution and whose fluctuations follow noisy multiplicative dynamics. There has been no scalable algorithm to train modern deep networks that incorporates all such synaptic properties. We are able to include them in LMD by using a variational formulation where a log-normal posterior distribution is used. We show several results that involve low-precision matrix multiplications in the forward passes. This includes results for training Vision Transformer and GPT-2 from scratch. Our findings suggest that biologically inspired multiplicative dynamics offer a promising direction for future energy-efficient hardware.

Applications · Chemistry, Physics, and Earth Sciences

Pietro Sittoni, Emanuele Zangrando, Angelo Alberto Casulli, Nicola Guglielmi, Francesco Tudisco

Deep learning-based methods have shown remarkable effectiveness in solving PDEs, largely due to their ability to enable fast simulations once trained. However, despite the availability of high-performance computing infrastructure, many critical applications remain constrained by the substantial computational costs associated with generating large-scale, high-quality datasets and training models. In this work, inspired by studies on the structure of Green's functions for elliptic PDEs, we introduce Neural-HSS, a parameter-efficient architecture built upon the Hierarchical Semi-Separable (HSS) matrix structure that is provably data-efficient for a broad class of PDEs. We theoretically analyze the proposed architecture, proving that it satisfies exactness properties even in very low-data regimes. We also investigate its connections with other architectural primitives, such as the Fourier neural operator layer and convolutional layers. We experimentally validate the data efficiency of Neural-HSS on the three-dimensional Poisson equation over a grid of two million points, demonstrating its superior ability to learn from data generated by elliptic PDEs in the low-data regime while outperforming baseline methods. Finally, we demonstrate its capability to learn from data arising from a broad class of PDEs in diverse domains, including electromagnetism, fluid dynamics, and biology.

Applications · Health / Medicine

Zelin Zang, WenZhe Li, Yongjie Xu, Chang Yu, Changxi Chi, Jingbo Zhou, Zhen Lei, Stan Z Li

In single-cell research, tracing and analyzing high-throughput single-cell differentiation trajectories is crucial for understanding biological processes. Key to this is the robust modeling of hierarchical structures that govern cellular development. Traditional methods face limitations in computational cost, performance, and stability. VAE-based approaches have made strides but still require branch-specific network modules, limiting their scalability and stability, while often suffering from posterior collapse. To overcome these challenges, we introduce HDTree, a generative modeling framework designed for robust lineage inference. HDTree captures tree relationships within a hierarchical latent space using a unified hierarchical codebook and employs a quantized diffusion process to model continuous cell state transitions. By aligning the generative process with the Waddington landscape, this method not only improves stability and scalability but also enhances the biological plausibility of inferred lineages. HDTree's effectiveness is demonstrated through comparisons on both general-purpose and single-cell datasets, where it outperforms existing methods in lineage inference accuracy, reconstruction quality, and hierarchical consistency. These contributions enable accurate and efficient modeling of cellular differentiation paths, offering reliable insights for biological discovery.

Applications · Health / Medicine

Yinbo Liu, Keyang Ye, Wenshan Sun, Handi Gao, Tian Tian

Reconstructing unified continuous dynamics from sparse, non-contiguous, and unpaired point cloud snapshots remains a fundamental challenge in spatiotemporal analysis for computer vision and developmental biology. Existing methods, including scene flow and Optimal Transport-based approaches, are limited either by explicit reliance on point-wise correspondences or by cumulative errors from frame-to-frame propagation and temporal discontinuity, and limited ability to model multi-attribute dynamics such as gene expression and population changes. We propose FlowCloud, a variational Neural Ordinary Differential Equation (Neural ODE) generative framework. FlowCloud aggregates information from all observed time points into a joint latent representation that initializes a Neural ODE $z(t)$ enabling continuous spatiotemporal evolution modeling while mitigating propagation-induced errors and preserving temporal consistency. Training is performed without predefined correspondences using a multi-faceted objective with complementary roles: Sinkhorn distance for global distribution alignment, Chamfer distance for local geometric consistency, trajectory regularization to encourage smooth and physically plausible dynamics, and supervised losses for multi-attribute prediction. Experiments on human motion and developmental biology datasets demonstrate improved interpolation accuracy and promising short-term extrapolation performance. By unifying geometry, attributes, and population dynamics within a continuous latent framework, FlowCloud offers a novel and robust solution for continuous dynamic reconstruction from unstructured spatiotemporal observations.

Deep Learning · Generative Models and Autoencoders

Phil Sidney Ostheimer, Mayank Kumar Nagda, Andriy Balinskyy, Gabriel Rodrigues, Jean Radig, Carl Herrmann, Stephan Mandt, Marius Kloft, Sophie Fellenz

Diffusion models (DMs) excel on dense continuous data, but are not designed for sparse continuous data. They do not model exact zeros that represent the deliberate absence of a signal. As a result, they erase sparsity patterns and perform unnecessary computation on mostly zero entries. With Sparsity-Exploiting Diffusion (SED), we model only non-zero values, preserving sparsity. SED delivers computational savings while maintaining or improving generation quality by skipping zeros during training and inference. Across physics and biology benchmarks, SED matches or surpasses conventional DMs and domain-specific baselines, while vision experiments provide intuitive insights into the limitations of dense DMs and the benefits of SED.

Optimization · Zero-order and Black-box Optimization

Mingyue Qin, Shuyu Yin, Qinghai Guo, Xiaolin Huang, Peilin Liu, Fei Wen

The human brain is a biologically instantiated on-device neural system that integrates both learning and inference in a unified architecture, which enables rapid and flexible learning on-the-fly. This extraordinary capability is achieved through non-BP learning mechanisms, whereas BP is computationally and memory intensive that unsuitable for on-chip edge learning. Zeroth-order (ZO) optimization methods, which resemble biologically plausible perturbation-based learning, offer a promising alternative that enables learning with only forward passes and hence can significantly reduce the complexity of on-chip hardware implementation. However, in this work we show that applying ZO methods to spiking neural networks (SNNs) is non-trivial due to the step-function nature of spiking activation. We analyze the challenges posed by the spiking activation, and reveal a variance amplification effect of it. Based on this insight, we propose a subspace-based ZO (SZO) method that leverages the intrinsic low-dimensional structure of the SNN optimization trajectory. By learning in a low-dimensional subspace, SZO substantially enhances ZO learning efficacy, achieving accuracy comparable to first-order (FO) methods with faster learning speed than full-space BP. We evaluate SZO on model training from scratch, continual training, and unsupervised adaptation. Experimental results demonstrate that SZO closely approaches FO training performance for the first time while offering fast learning speed.

Applications · Health / Medicine

Zhenglun Kong, Mufan Qiu, John Boesen, xiang lin, Sukwon Yun, Tianlong Chen, Manolis Kellis, Marinka Zitnik

Understanding how cellular morphology, gene expression, and spatial context jointly shape tissue function is a central challenge in biology. Image-based spatial transcriptomics technologies now provide high-resolution measurements of cell images and gene expression profiles, but existing methods typically analyze these modalities in isolation or at limited resolution. We address the problem by introducing SPATIA, a multi-level generative and predictive model that learns unified, spatially aware representations by fusing morphology, gene expression, and spatial context from the cell to the tissue level. SPATIA also incorporates a novel spatially conditioned generative framework for predicting cell morphologies under perturbations. Specifically, we propose a confidence-aware flow matching objective that reweights weak optimal-transport pairs based on uncertainty. We further apply morphology-profile alignment to encourage biologically meaningful image generation, enabling the modeling of microenvironment-dependent phenotypic transitions. We assembled a multi-scale dataset consisting of 25.9 million cell-gene pairs across 17 tissues. We benchmark SPATIA against 18 models across 12 tasks, spanning categories such as phenotype generation, annotation, clustering, gene imputation, and cross-modal prediction. SPATIA achieves improved performance over state-of-the-art models, improving generative fidelity by 8\% and predictive accuracy by up to 3\%.

Deep Learning · Self-Supervised Learning

Chengwei Zhou, Gourav Datta

Spiking Neural Networks (SNNs) offer a promising alternative to traditional artificial neural networks by leveraging sparse, event-driven computation that closely mimics biological neurons. When deployed on neuromorphic hardware, SNNs enable substantial energy savings due to their temporal and asynchronous processing. However, training SNNs remains fundamentally difficult because the non-differentiable nature of spike generation breaks the bidirectional gradient flow required in modern self-supervised learning (SSL) frameworks. In this work, we introduce the first fully SSL framework for SNNs that scales to large-scale visual tasks without requiring labeled fine-tuning. Our method leverages intrinsic spike-time dynamics by aligning representations across time steps and augmented views. To address gradient mismatch during surrogate training, we propose the MixedLIF neuron model, which combines a spiking path with an antiderivative-based surrogate path during training to stabilize optimization, while retaining a fully spiking and energy-efficient architecture at inference. We also introduce two temporal objectives, Cross Temporal Loss and Boundary Temporal Loss, that align multi-time-step outputs to improve learning efficiency. Our approach achieves competitive results across ResNet and Vision Transformer-based SNNs on CIFAR-10, CIFAR10-DVS, and ImageNet-1K. Our approach further generalizes through transfer learning from ImageNet-1K to downstream tasks. Notably, our self-supervised SNNs match or exceed the performance of some non-spiking SSL models, demonstrating both representational strength and energy efficiency.

Deep Learning · Foundation Models

Ethan Roland, Murat Cubuktepe, Erick Martinez, Stijn Servaes, Keenan Pepper, Michael Vaiana, Diogo de Lucena, Judd Rosenblatt, Addie Foote, Cem Anil 等

AI developers face a dual-use dilemma. The same capability that helps one user cure a disease can help another synthesize one. This dilemma could be resolved by access control, granting different users access to different AI capabilities. A gold standard for access control would be to serve models with different capabilities to different users. However, training and deploying multiple models is prohibitively expensive. We address this challenge by developing gradient-routed mixture-of-experts (GR-MoE), a pretraining method that selectively updates experts to induce specialization. Ablating an expert at inference time removes its capability, approximating a model trained on filtered data. We evaluate GR-MoE on synthetic stories and realistic dual-use data spanning biology, cybersecurity, nuclear physics, and code. On realistic data, GR-MoE preserves performance on retained capabilities while achieving 30% compute efficiency on forget capabilities. GR-MoE limits recovery more effectively than post-hoc unlearning and preserves capabilities better than LoRA. GR-MoE's advantages improve when scaled from 48M to 2B parameters, approaching multiple data filtered models in a single training run.

Applications · Neuroscience, Cognitive Science

Guobin Shen, Dongcheng Zhao, Yiting Dong, Qian Zhang, Yi Zeng

Artificial and biological systems may evolve similar computational solutions despite fundamental differences in architecture and learning mechanisms—a form of convergent evolution. We provide large-scale evidence for this phenomenon through comprehensive analysis of alignment between human brain activity and internal representations across over 600 AI models spanning language and vision domains (1.33M to 72B parameters). Analysis of 60 million alignment measurements reveals that higher-performing models spontaneously develop stronger brain correspondence without explicit neural constraints, with language models demonstrating markedly stronger correlations ($r=0.89, p<7.5 \times 10^{-13}$) than vision models ($r=0.53, p<2.0 \times 10^{-44}$). Crucially, longitudinal training analysis shows that brain alignment consistently emerges prior to performance improvements, suggesting that developing brain-like representations constitutes a fundamental stepping stone toward enhanced capabilities. We identify systematic organizational patterns reflecting human cognitive architecture: language models exhibit strongest alignment with limbic and integrative regions, while vision models show progressive correspondence with visual cortical hierarchies. These findings establish that optimization for task performance naturally drives AI systems toward human-like computational strategies.

Applications · Neuroscience, Cognitive Science

Zhe Jiao, Xiaodong He, Shanglin Zhou

While heavy-tailed synaptic weight distributions are pervasive in biological neural networks, their computational role---particularly in relation to generalization---remains poorly understood. To address this, we develop a novel optimal-transport-based optimization algorithm that incorporates key biological constraints, including Dale’s principle and heavy-tailed synaptic statistics, to train recurrent neural networks (RNNs) on a wide range of cognitive tasks. We show that these biologically constrained, heavy-tailed RNNs exhibit substantially improved generalization, which we further characterize within a PAC-Bayes framework. Our theoretical analysis and numerical experiments reveal two complementary mechanisms underlying this generalization enhancement. Topologically, heavy-tailed connectivity induces an effectively low-rank structure, which in turn yields low-dimensional neural dynamics. Geometrically, heavy-tailed connectivity intrinsically shapes task variable representations to lie near a linear manifold, thereby improving generalization for a linear readout strategy. Together, these results identify heavy-tailed connectivity as a biologically grounded intrinsic mechanism that promotes low-rank structure and favorable representational geometry, leading to improved generalization in flexible cognitive tasks.

Xinyou Wang, Liang Hong, Jiasheng Ye, Zaixiang Zheng, Shujian Huang, Quanquan Gu

Proteins are shaped by gradual evolution under biophysical and functional constraints. Protein language models learn rich evolutionary constraints from large-scale sequence data, and discrete diffusion–based protein language models (e.g., DPLMs) have emerged as a promising framework for both understanding and generation. However, existing DPLMs typically rely on masking-based absorbing diffusion, which conflicts with a basic biological intuition: proteins evolve through accumulated edits rather than emerging from masked tokens. As a result, these frameworks lack explicit pretraining objectives for substitution and insertion/deletion (indel) operations, which in turn limits both optimization-style post-editing and flexible guided generation. To address these limitations, we present DPLM-Evo, an evolutionary discrete diffusion framework that explicitly predicts substitution, insertion, and deletion operations during denoising. \method decouples a fixed-length latent alignment space from the variable-length observed sequence space, making indel-aware generation tractable and enabling adaptive scaffold growth throughout the process with negligible computational overhead. To further align substitutions with real evolutionary dynamics, we introduce a contextual evolutionary noising kernel that induces biologically informed, context-dependent mutation patterns. Across tasks, \method improves sequence understanding and achieves state-of-the-art performance on ProteinGym in the single-sequence setting, while also enabling variable-length simulated evolution, guided generation, and post-editing or optimization of existing proteins via explicit edit trajectories.

Qibing Ren, Xinhao Song, Ke Fan, Lijun Li, Zhanpeng Zhou, Gongshen Liu, Junchi Yan, Lizhuang Ma, Jing Shao

The capabilities of large language models (LLMs), particularly large reasoning models (LRMs), are rapidly advancing. This raises concerns about whether LRMs can maintain their safety awareness throughout long-form reasoning. Frustratingly, we identify a prevalent safety issue across LLMs and LRMs, where LRMs can reveal dangerous thoughts, leading to harmful knowledge elicitation when confronting sensitive yet benign topics. For example, when explaining the chemical context of Lewisite, a biological weapon, LRMs analyze its synthesis in their reasoning without recognizing the associated risks. We refer to this issue as the unintended elicitation issue. Experiments on our benchmark show that it is a common issue across current LRMs due to their strong multi-step reasoning capabilities. To address this issue, we propose placing LLMs in our synthesized open-ended environments, allowing them to self-search for a safety reasoning pattern to respond responsibly and helpfully. We first design a scalable data synthesis pipeline to generate data that triggers the ``unintended elicitation'' issue. We further propose a safety-first reward model design, which prioritizes safety while also evaluating the helpfulness of responses and the faithfulness of reasoning. Experiments show that our method improves safety, reduces over-refusal, and maintains strong helpfulness, paving the way for safer deployment in high-stakes domains.

Deep Learning · Graph Neural Networks

Mridul Gupta, Samyak Jain, Vansh Ramani, HARIPRASAD KODAMANA, Sayan Ranu

Graph Neural Networks (GNNs) are powerful tools for learning from graph-structured data, but their scalability is increasingly strained by the size of real-world graphs in domains like recommender systems, fraud detection, and molecular biology. Graph condensation—the task of generating a smaller synthetic graph that retains the performance of models trained on the original—has emerged as a promising solution. However, the dominant approach of gradient matching introduces a fundamental contradiction: it requires training on the full dataset to create the compressed version, thereby undermining the goal of efficiency. Worse still, these methods suffer from high computational overhead, poor generalization across GNN architectures, and brittle reliance on specific model configurations. Equally concerning is the community's reliance on misleading evaluation protocols such as node compression ratios, which fail to reflect true resource savings, condensation overhead, and illusory application to neural architecture search. These shortcomings are not incidental—they are systemic, and they obstruct meaningful progress. In this position paper, we argue that graph condensation, in its current form, needs a reset. We call for moving beyond full-dataset training and model-dependent design, and instead advocate for methods that are lightweight, architecture-agnostic, and practically deployable. By identifying key methodological flaws and outlining concrete research directions, we aim to reorient the field toward approaches that deliver on the true promise of condensation: efficient, generalizable, and usable GNN training at scale.

Payam Dibaeinia, Sudarshan Babu, Mei Knudson, Ali ElSheikh, Yibo Wen, Han Liu, Jason Perera, Aly Khan

The intersection of AI and biology has entered a phase of explosive growth, driven by the ambition to build "Virtual Cells" or computational models capable of predicting cellular responses to any perturbation. Following the success of structural biology (e.g., AlphaFold) and large language models, the field has converged on training massive, high-capacity models on large-scale single-cell data. This position paper argues that scaling model capacity is insufficient to solve the Virtual Cell problem because the primary failure mode is a *lack of adequate coverage over diverse biological contexts*, not insufficient model expressivity. We support this claim by reviewing recent studies showing that simple baselines perform on par with sophisticated architectures within a given biological context, and current models fail to consistently generalize across contexts. We connect this finding to the causal inference literature on transportability and contrast it with domains where scaling has succeeded. We substantiate our argument through analysis of a state-of-the-art model on a 22-million-cell immunology dataset. We conclude that the community faces a *causal transport problem* that cannot be solved by accumulating more data from the same distributions. Instead, we argue that contextual diversity and causal representation learning deserve increased emphasis, complementing ongoing scaling of model capacity and data volume.

Applications · Computer Vision

Nedyalko Prisadnikov, Danda Pani Paudel, Yuqian Fu, Luc Van Gool

This position paper argues that the next generation of vision encoders should be image size agnostic and task driven. The source of our inspiration is biological. Not a structural aspect of biological vision, but a behavioral trait – efficiency. We focus on a couple of ways in which vision in nature is efficient, but modern vision encoders not. We – humans and animals – deal with vast quantities of visual data, and need to be smart where we focus our limited energy – it depends on the task. It is our belief that vision encoders should be dynamic and the computational complexity should depend on the task at hand rather than the size of the image. We, also, provide concrete first steps towards our vision – a proof-of-concept solution for image classification. Despite classification being not very representative for what we are trying to achieve, it shows that our approach is feasible and promising.