论文检索

Theory · Everything Else

Alireza Jafari, Heman Shakeri, Hadi Daneshmand

Single-cell perturbation analysis aims to predict how cellular states change after interventions such as drug treatments or genetic edits. A central difficulty is that pre- and post-perturbation measurements are typically observed as *unpaired* populations, so accurate prediction requires inferring a latent coupling and learning a transition map. In this position paper, we argue that the *measurement time gap* is the key experimental knob controlling both the computational tractability of coupling and the effective model complexity. We identify a critical time gap $\Delta$ that induces a phase transition, under biologically inspired conditions; for "measurement-time $< \Delta$", matching is polynomial-time tractable and the task reduces to supervised learning, whereas for "measurement-time $>\Delta$", recovering the matching is NP-hard in the worst case. The required conditions are restricted isometry of the initial states and temporal smoothness of the transition dynamics. We complement the theory with empirical evidence on synthetic and biological datasets showing a sharp regime change as the time gap increases. Furthermore, we demonstrate that a linear model can match or exceed the performance of higher-capacity neural approaches when our conditions hold.

Applications · Chemistry, Physics, and Earth Sciences

Yuxing Wang, Wenyi Zhang, Yilong Zou, Jing Huang

Computational identification of lipid-binding proteins is critical for both fundamental research and therapeutic development. Existing models are typically trained in a fully supervised manner, treating unlabeled samples as negatives. However, missing evidence does not imply non-binding, leading to systematic false negatives. Pocket-level lipid-binding prediction also remains underexplored compared to residue- or sequence-level approaches. To bridge these gaps, we present **LipoPU**, a pocket-centric predictor that formulates lipid-binding learning under a ranking-based positive-unlabeled objective, and supports both binary lipid-binding detection and multi-label lipid category prediction. LipoPU learns an attention-based pocket representation that is robust to ambiguous pocket definitions while providing residue-level interpretability. Experiments show consistent gains over supervised baselines and prior pocket-level work, and a structural case study recovers a literature-supported allosteric lipid-binding pocket while highlighting biologically informative residues.

Deep Learning · Large Language Models

Lukas Fesser, Yasha Ektefaie, Ada Fang, Sham Kakade, Marinka Zitnik

Relational reasoning is the ability to infer relations that jointly bind multiple entities, attributes, or variables. While this capability is essential for scientific reasoning, most existing evaluations of relational reasoning in large language models focus on structured inputs such as tables, graphs, or synthetic relational tasks, and do not isolate the sources of difficulty that arise from higher-arity relational binding. We study this problem through the lens of *Relational Complexity (RC)*, defined as the minimum number of independent entities or operands that must be simultaneously bound to apply a relation. RC provides a principled way to vary reasoning difficulty independently of confounders such as input size, vocabulary, and representational choices. Building on RC, we introduce REL, a generative benchmark framework spanning algebra, chemistry, and biology that varies RC within each domain. Evaluating frontier LLMs, we observe a consistent and monotonic degradation in performance as RC increases, even when the total number of entities is held fixed. This failure mode persists under increased test-time compute and with in-context learning, suggesting a limitation tied to the arity of the required relational binding rather than insufficient inference steps or exposure to examples. Our results identify a well-defined regime of higher-arity reasoning in which current models struggle and motivate revisiting reasoning benchmarks through the lens of relational complexity.

Deep Learning · Large Language Models

Haotian Xu, Jiannan Yang, Tian Gao, Lily Weng, Tengfei Ma

Activation sparsity offers a compelling route to accelerate large language model (LLM) inference by selectively suppressing hidden activations, yet existing approaches exhibit severe accuracy degradation at high sparsity. We show that this failure stems from representational instability: *activation sparsity disrupts input-dependent activation learned during pretraining, inducing distribution shifts in hidden states*. We address this issue by reframing activation sparsity as a representational alignment problem and introducing **Spontaneous Neurons (SPON)**, a lightweight mechanism inspired by spontaneous neural activity in biological systems. SPON injects a small set of learnable, input-independent activation vectors that act as persistent representational anchors for sparse computation. These vectors are trained via distribution matching to the dense model and can be absorbed into bias terms after training, incurring negligible inference overhead. Across multiple LLM backbones, SPON consistently restores performance, stabilizes latent representations, and preserves generalization. Our results establish SPON as an effective and principled solution for reliable activation-sparse inference, and offer new insights into knowledge retention in LLMs.

Applications · Neuroscience, Cognitive Science

Francesco Innocenti, El Mehdi Achour, Rafal Bogacz

Predictive coding (PC) is a biologically plausible alternative to standard backpropagation (BP) that minimises an energy function with respect to network activities before updating weights. Recent work has improved the training stability of deep PC networks (PCNs) by leveraging some BP-inspired reparameterisations. However, the full scalability and theoretical basis of these approaches remains unclear. To address this, we study the infinite width and depth limits of PCNs. For linear residual networks, we show that the set of width- and depth-stable feature-learning parameterisations for PC is exactly the same as for BP. Moreover, under any of these parameterisations, the PC energy with equilibrated activities converges to the BP loss in a regime where the model width is much larger than the depth, resulting in PC computing the same gradients as BP. Experiments show that these results hold in practice for deep nonlinear networks as long as an activity equilibrium is reached. Overall, this work unifies various previous theoretical and empirical results and has important practical implications for the scaling of PCNs.

Deep Learning · Generative Models and Autoencoders

Feifan Jiang, Yinan Bu, Shihao Wu, Gongjun Xu, Ji Zhu

Network data are ubiquitous across the social sciences, biology, and information systems. Generating realistic synthetic network data has broad applications from network simulation to scientific discovery. However, many existing black-box approaches for network generation tend to overfit observed data while overlooking characteristic network structure, and incur substantial computational overhead at scale. These practical challenges call for synthetic network generation methods that are both efficient and capable of capturing structural properties of networks. In this paper, we introduce Synthetic Network Generation via Latent Embedding Reconstruction (SyNGLER), a general and efficient framework for synthetic network generation that builds on latent space network models. Given an observed network, SyNGLER first learns low-dimensional latent node embeddings via a latent space network model and then reconstructs the latent space by building a distribution-free generator over these embeddings. For generation, SyNGLER first samples (or resamples) node embeddings from the generator in the latent space and then produces synthetic networks using the latent space network model. Through the latent space framework, SyNGLER preserves unique characteristics in networks such as sparsity and node degree heterogeneity, while allowing for efficient training with lower computational cost than many existing deep architectures. We provide theoretical guarantees by developing consistency results regarding the distance between the true and synthetic edge distributions. Empirical studies further demonstrate the effectiveness of SyNGLER, where SyNGLER efficiently produces networks that better preserve key network characteristics such as network moments and degree distributions compared with existing approaches.

Applications · Neuroscience, Cognitive Science

Todd Morrill, Christian-Gernot Pehle, Anthony Zador

Continuous-time, event-native spiking neural networks (SNNs) operate strictly on spike events, treating spike timing and ordering as the representation rather than an artifact of time discretization. This viewpoint aligns with biological computation and with the native resolution of event sensors and neuromorphic processors, while enabling compute and memory that scale with the number of events. However, two challenges hinder practical, end-to-end trainable event-based SNN systems: 1) exact charge--fire--reset dynamics impose inherently sequential processing of input spikes, and 2) precise spike times must be solved without time bins. We address both. First, we use parallel associative scans to consume multiple input spikes at once, yielding up to 43x speedups over sequential simulation while retaining exact hard-reset dynamics. Second, we implement differentiable spike-time solvers that compute spike times to machine precision without discrete-time approximations or restrictive analytic assumptions. We demonstrate the viability of training SNNs using our solutions on four event-based datasets on GPUs.

Applications · Neuroscience, Cognitive Science

Dongyang Ma, Zhengyu Ma, Yifan Huang, Chenlin Zhou, Wei Zhang, Yonghong Tian

Retina-like event cameras and brain-inspired Spiking Neural Networks (SNNs) demonstrate exceptional energy efficiency through bio-inspired sensing and computation. While SNNs are naturally well-suited to the asynchronous nature of event data, their practical applications face the following challenges: sensitivity to noise, dense representations that disrupt spike pathways, and insufficient multi-scale feature perception. To address the aforementioned challenges, we propose a bio-vision-inspired object detection method motivated by biological (bio) vision systems. First, at the micro level, this paper proposes a noise-filtering STATNF-Neuron architecture to address the current sensitivity of basic neurons to noise. Based on STATNF-Neurons, the paper introduces two bio-vision-inspired macro-structures: Events-to-Spikes Representation (E2S), which preserves spiking characteristics while mimicking the memory and noise-filtering abilities of retinal neurons; Bidirectional Multi-Scale Spiking Network (BiSNet), which simulates cortical information flow pathways to integrate multi-scale features in both directions, enhancing the network's ability to perceive information at multiple scales. Extensive experiments show that the proposed bio-vision-inspired method achieving state-of-the-art performance. Notably, it reaches 96.1\% accuracy on NCAR, 63.5\% mAP\textsubscript{50} on N-Caltech101, and 69.1\% mAP\textsubscript{50} on Gen1.

General Machine Learning · Transfer, Multitask and Meta-learning

Yuan Li, Heng Yang, Renzhi Chen, Ke Li

Foundation models (FMs) pretrained on large-scale sequence data have emerged as a promising paradigm for RNA biology, yet the mechanisms underlying their transferability remain unclear. In this work, we conduct a systematic investigation of transfer learning in RNA FMs across diverse structural and functional tasks. Our results demonstrate that frozen representations from pretrained RNA FMs are not universally transferable, and that the hierarchical feature reuse paradigm prevalent in computer vision does not generally extend to RNA FMs. Instead, pretraining primarily benefits downstream tasks by providing a favorable optimization initialization when pretraining and downstream objectives are well aligned, which accelerates convergence toward flatter minima associated with improved generalization. Overall, our findings characterize pretraining as an optimization prior whose effectiveness is governed by task alignment and model capacity, offering principled guidance for future RNA FMs.

Social Aspects · Accountability, Transparency, and Interpretability

Shasha Zhou, Mingyu Huang, Ke Li

Advances in machine learning and computational power have unlocked the predictive potential of the human genome, yet biologists increasingly demand that these models also elucidate the underlying biological mechanisms. While interpretable machine learning (IML) techniques have been increasingly applied to bridge this gap, there has been a pervasive reliance on anecdotal validation: the vast majority of research employs a single IML method and reports only isolated successful instances. Through a benchmarking study on transcription factor binding, we demonstrate the risks of current practices. We show that different IML methods can often (1) yield contradictory explanations for identical predictions, (2) fail to localize known regulatory motifs, and (3) do not faithfully reflect the model's internal decision process. In light of this, we argue for a validation framework analogous to clinical trials. Just as trials require rigorous design and the reporting of adverse events, genomic interpretability must move beyond cherry-picked plausibility toward systematic assessment of consistency, faithfulness, and biological validity. To facilitate this, we propose a tiered framework to guide the rigorous evaluation and reporting of genomic IML methods.

Applications · Chemistry, Physics, and Earth Sciences

Tianmeng Hu, Biao Luo, Ke Li

Designing RNA sequences that reliably fold into specific secondary structures is essential for understanding their biological functions but remains a challenging computational problem. We propose CocoRNA, a cooperative multi-agent reinforcement learning framework for RNA inverse design. CocoRNA simplifies the design task by decomposing it into smaller sub-problems, each solved collaboratively by multiple agents. This approach reduces the complexity of the problem and improves the exploration of design policies. During training, a centralized critic uses global structural information to guide the agents, enabling them to jointly optimize their design strategies. As a result, CocoRNA learns high-quality RNA design policies that generalize effectively to unseen structures without additional training. Experiments on the Rfam dataset demonstrate that CocoRNA substantially outperforms state-of-the-art methods in both success rate and design speed. Further experiments on other biological sequence design tasks highlight the effectiveness and broad potential of CocoRNA for complex design tasks.

Deep Learning · Foundation Models

Aleksandr Medvedev, Karthik Viswanathan, Praveenkumar Kanithi, Kirill Vishniakov, Prateek Munjal, Clement Christophe, Tiago Magalhaes, Marco Pimentel, Ronnie Rajan, SHADAB KHAN

Existing genomic foundation models (GFMs) typically treat DNA as raw nucleotide sequences, often overlooking the regulatory context required to interpret genetic variation accurately. We introduce BioToken, a tokenization framework that directly encodes variants and biological annotations into genomic representations, and BioFM, a parameter-efficient model built on this architecture. By leveraging biological inductive biases, BioFM outperforms state-of-the-art models and specialized baselines like Enformer on benchmarks including pathogenicity and expression prediction while requiring 100-fold less compute than current large-scale genomic models. These findings demonstrate that explicitly modeling biological structure yields more robust and efficient genomic representations than scaling alone.

Deep Learning · Theory

Jian Zhang, Yue Sun, Wangzi Yao, Tielin Zhang

The advancement of artificial neural networks (ANNs) has been driven by diverse and well-established architectural designs, especially in connectivity. Biological neural networks, which exhibit a rich variety of neurodynamic circuits, offer a valuable source of inspiration for developing novel ANN models. In this study, we analyze the meta-connectivity structure and introduce a network motif-based approach, in which 13 distinct motifs are modeled as functional building blocks. These motifs represent low-dimensional, fundamental components of larger network architectures. Through rigorous theoretical analysis, we classify these motifs into a three‑layer hierarchical classification of their dynamical regimes and demonstrate that their hierarchical proportions critically shape collective neural dynamics. Furthermore, by embedding motif distributions into recurrent neural networks (RNNs), we show that these motifs can selectively enhance either network robustness or flexibility. Collectively, our findings provide a theoretical framework—supported by extensive experiments—for understanding how specific network motifs influence the computational properties of artificial intelligence systems via their underlying dynamics. This motif-driven approach offers significant potential for analyzing and modulating neural dynamics in ANNs.

Deep Learning · Other Representation Learning

Ye Xiao, Ruikun Li, Zhenyu Yang, Andrey Vasnev, Junbin Gao

Existing parameter isolation-based methods in continual learning employ diverse designs to learn more tasks within a limited model capacity. However, most of their designs inevitably incur substantial computational overhead if their model capacity is enlarged to accommodate further tasks as the task stream continually grows, resulting in a significant efficiency bottleneck. In this paper, we propose a novel GNN framework with a biological neuron-inspired architecture, termed the capacity-agnostic GNN (CAGNN), to simultaneously overcome catastrophic forgetting and boost efficiency under capacity expansion. Unlike other methods that employ full network propagation, CAGNN leverages graph contextual information to support the construction of task-specific subnetworks and decouples subnetworks during both training and inference, while enabling effective knowledge transfer between tasks. Intensive experiments demonstrate CAGNN's superiority to the state of the art, in terms of effectiveness as well as computational efficiency.

Applications · Neuroscience, Cognitive Science

Mathis Pink, Vy Vo, Qinyuan Wu, Jianing Mu, Javier Turek, Uri Hasson, Kenneth Norman, Sebastian Michelmann, Alexander Huth, Mariya Toneva

Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the difficulty of mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and, if so, how LLMs capture core behavioral signatures of humans of a central aspect of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on a full-length novel, we show substantial similarities between the human and model performances on the temporal order memory task. We next perform long-context mechanistic interpretability analyses to reveal the underlying mechanisms in the model, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.

Theory · Reinforcement Learning and Planning

Abdullah Akgül, Gulcin Baykal, Manuel Haussmann, Mustafa Mert Çelikok, Melih Kandemir

Optimal control of complex environments with robotic systems faces two complementary and intertwined challenges: efficient organization of sensory state information and far-sighted action planning. Because the reinforcement learning framework addresses only the latter, it tends to deliver sample-inefficient solutions. Active inference is the state-of-the-art process theory that explains how biological brains handle this dual problem. However, its applications to artificial intelligence have thus far been limited to extensions of existing model-based approaches. We present a formal abstraction of reinforcement learning algorithms that spans model-based, distributional, and model-free approaches. This abstraction seamlessly integrates active inference into the distributional reinforcement learning framework, making its performance advantages accessible without transition dynamics modeling.

Applications · Computer Vision

Manuel Traub, Martin V Butz

Current state-of-the-art segmentation models encode entire images before focusing on specific objects. This wastes computational resources. We introduce FLIP (Fovea-Like Input Patching), a parameter-efficient vision model that realizes object segmentation through biologically-inspired top-down attention. FLIP selectively samples multi-resolution patches centered on objects of interest from the input. As a result, it allocates high-resolution processing to object centers while maintaining coarser peripheral context. This off-grid, scale-invariant design enables FLIP to outperform META's Segment Anything models (SAM, SAM2 and fast variants) by large margins: With more than 440$\times$ fewer parameters, FLIP-Tiny (0.51M parameters) reaches a mean IoU of 78.24\% while SAM2-L reaches 75.87\% IoU (224.45M parameters). FLIP-Large even achieves 80.33\% mean IoU (96.6M parameters), still running about $2.3\times$ faster than SAM2-L. We evaluate on six benchmarks in total. In five established benchmarks (Hypersim, KITTI-360, OpenImages, COCO, LVIS) FLIP consistently outperforms SAM and various variants of it. In our novel ObjaScale dataset, which stress-tests scale invariance with objects ranging from 0.0001\% up to 25\% of the image area, we show that FLIP segments even very small objects accurately, where existing models fail severely. FLIP opens new possibilities for real-time, object-centric vision applications and offers much higher energy efficiency. We believe that FLIP can act as a powerful foundation model, as it is very well-suited to track objects over time, for example, when being integrated into slot-based scene segmentation architectures.

Deep Learning · Self-Supervised Learning

Prithviraj Tarale, Kiet Chu, Abhishek Varghese, Kai-Chun Liu, Maxwell A. Xu, Mohit Iyyer, Sunghoon Lee

Wearable accelerometers have enabled large-scale health and wellness monitoring, yet learning robust human-activity representations has been constrained by the scarcity of labeled data. While self-supervised learning offers a potential remedy, existing approaches treat sensor streams as unstructured time series, overlooking the underlying biological structure of human movement, a factor we argue is critical for effective Human Activity Recognition (HAR). We introduce a novel tokenization strategy grounded in the *submovement theory* of motor control, which posits that continuous wrist motion is composed of superposed elementary basis functions called submovements. We define our token as the *movement segment*, a unit of motion composed of a finite sequence of submovements that is readily extractable from wrist accelerometer signals. By treating these segments as tokens, we pretrain a Transformer encoder via masked movement-segment reconstruction to model the temporal dependencies of movement segments, shifting the learning focus beyond local waveform morphology. Pretrained on the NHANES corpus ($\approx$ 28k hours; $\approx$ 11k participants; $\approx$ 10M windows), our representations outperform strong wearable SSL baselines across six subject-disjoint HAR benchmarks. Furthermore, they demonstrate stronger data efficiency in data-scarce settings. Code and pretrained weights will be made publicly available.

Social Aspects · Accountability, Transparency, and Interpretability

Jeremy Herbst, Jae Hee Lee, Stefan Wermter

Mixture-of-Experts (MoE) architectures have become the dominant choice for scaling Large Language Models (LLMs), activating only a subset of parameters per token. While primarily adopted for computational efficiency, it remains an open question whether their sparsity makes them inherently easier to interpret than dense feed-forward networks (FFNs). We compare MoE experts and dense FFNs using $k$-sparse probing and find that expert neurons are consistently less polysemantic, with the gap widening as routing becomes sparser. This suggests that sparsity pressures both individual neurons and entire experts toward monosemanticity. Leveraging this finding, we *"zoom out"* from the neuron to the expert level as a more effective unit of analysis. We validate this approach by automatically interpreting hundreds of experts. This analysis allows us to resolve the debate on specialization: experts are neither broad domain specialists (e.g., biology) nor simple token-level processors. Instead, they function as fine-grained task experts, specializing in linguistic operations or semantic tasks (e.g., retrieving specific bioinformatics software tools). Our findings suggest that MoEs are inherently interpretable at the expert level, providing a clearer path toward large-scale model interpretability.

Applications · Chemistry, Physics, and Earth Sciences

Jiahao Chen, Letian Gao, Yanhaozhu, Wenbiao Zhou, Bing Su, Zhi Lu, Bo Huang

Recent advances in generative modeling have enabled significant progress in structure-based drug design (SBDD). Existing methods typically condition molecule generation on empty binding pockets from holo complexes, overlooking informative components such as the filler (ligands and solvent). Here, we leverage low-resolution electron density (ED) derived from the filler as a physically grounded condition for \textit{de novo} drug design. We consider two types of ED—calculated and cryo-EM/X-ray—obtainable from computational or experimental sources, supporting unified pre-training and experimental integration. Compared with rigid pocket representations, experimental ED naturally captures conformational flexibility and provides a more faithful description of the binding environment. Based on this, we introduce EDMolGPT, a decoder-only autoregressive framework that generates molecules from low-resolution ED point clouds. By grounding generation in physically meaningful density signals, EDMolGPT mitigates structural bias and produces molecules with 3D conformations. Evaluations on 101 biological targets verify the effectiveness. Code will be released upon acceptance.