论文检索

Deep Learning · Generative Models and Autoencoders

Linze Li, Zong-Wei Hong, Shen Zhang, Bo Lin, Jinglun Li, Yao Tang, Jiajun Liang

While Flow Matching theoretically guarantees constant-velocity trajectories, we identify a critical breakdown in high-dimensional practice: the Velocity Deficit. We show that the MSE objective systematically underestimates velocity magnitude, causing generated samples to fail to reach the data manifold—a phenomenon we term Integration Lag. To rectify this, we propose Initial Energy Injection, instantiated via two complementary methods: the training-based Magnitude-Aware Flow Matching (MAFM) and the training-free Scale Schedule Corrector (SSC). Both are grounded in our discovery of a crucial asymmetry: velocity contraction causes harmful kinetic stagnation at the trajectory's start, yet acts as a beneficial denoising mechanism at its end. Empirically, SSC yields significant efficiency gains with zero retraining and just one line of code. On ImageNet-1k (256x256), it improves FID by 44.6% (from 13.68 to 7.58) and achieves a 5x speedup, enabling a 50-step generator (FID 7.58) to beat a 250-step baseline (FID 8.65). Furthermore, our methods generalize to Text-to-Image tasks and high-resolution generation, improving FID on MS-COCO by ~22%.

Applications · Chemistry, Physics, and Earth Sciences

Panagiotis Antoniadis, Beatrice Pavesi, Ole Winther, Simon Olsson

Molecular dynamics (MD) is a central computational tool in physics, chemistry, and biology, enabling quantitative prediction of experimental observables as expectations over high-dimensional molecular distributions such as Boltzmann distributions and transition densities. However, conventional MD is fundamentally limited by the high computational cost required to generate independent samples. Generative molecular dynamics (GenMD) has recently emerged as an alternative, learning surrogates of molecular distributions either from data or through interaction with energy models. While these methods enable efficient sampling, their transferability across molecular systems is often limited. In this work, we show that incorporating auxiliary sources of information can improve the data efficiency and generalization of transferable implicit transfer operators (TITO) for molecular dynamics. We find that coarse-grained TITO models are substantially more data-efficient than Boltzmann emulators, and that incorporating protein language model (PLM) embeddings further improves out-of-distribution generalization. Our approach, PLaTITO, achieves state-of-the-art performance on equilibrium sampling benchmarks for out-of-distribution protein systems, including fast-folding proteins. We further study the impact of additional conditioning signals---such as structural embeddings, temperature, and large-language-model-derived embeddings---on model performance.

Applications · Health / Medicine

Haoyang Luan, Gufeng Yu, Letian Chen, Zhenran Xiao, Yueshan Huang, Junkun Guo, Yang Yang

The *de novo* generation of high-affinity epitopes tailored to specific major histocompatibility complex (MHC) proteins is a pivotal challenge in computational immunotherapy. However, current methods struggle to effectively integrate the MHC context into the generation process, and often fail to guarantee high binding affinity due to the neglect of discriminative signals from non-binders. To bridge these gaps, we present **EpiCoCo**, a probabilistic framework for **Epi**tope generation via MHC-context **Co**-modeling and **Co**ntrastive affinity learning. EpiCoCo treats the pMHC complex as a dynamic, co-adaptive system by operating on the joint E(3) graph. In addition, we introduce Contrastive Affinity Guidance (CAG), an inference mechanism that leverages the gradient difference between learned high- and low-affinity distributions. CAG actively drives the generation trajectory towards high-affinity manifolds while utilizing repulsive signals to filter out candidates with poor binding potential. Extensive evaluations demonstrate that EpiCoCo achieves a mean binding free energy of -45.20 REU, a 23% improvement over the state-of-the-art, while maintaining high structural plausibility. The results validate that context co-modeling and negative-informed guidance are essential for generating valid, high-potency immunotherapeutics.

Social Aspects · Safety

Shuqiang Wang, Wei Cao, Jiaqi Weng, Jialing Tao, Licheng Pan, Hui Xue', Zhixuan Chu

Large Reasoning Models (LRMs) are increasingly integrated into systems requiring reliable multi-step inference, yet this growing dependence exposes new vulnerabilities related to computational availability. In particular, LRMs exhibit a tendency to “overthink’’—producing excessively long and redundant reasoning traces—when confronted with incomplete or logically inconsistent inputs. This behavior significantly increases inference latency and energy consumption, forming a potential vector for denial-of-service (DoS) style resource exhaustion. In this work, we investigate this attack surface and propose an automated black-box framework that induces overthinking in LRMs by systematically perturbing the logical structure of input problems. Our method employs a hierarchical genetic algorithm (HGA) operating on structured problem decompositions, and optimizes a composite fitness function designed to maximize both response length and reflective overthinking markers. Across four state-of-the-art reasoning models, the proposed method substantially amplifies output length, achieving up to a 26.1× increase on the MATH benchmark and consistently outperforming benign and manually crafted missing-premise baselines. We further demonstrate strong transferability, showing that adversarial inputs evolved using a small proxy model retain high effectiveness against large commercial LRMs. These findings highlight overthinking as a shared and exploitable vulnerability in modern reasoning systems, underscoring the need for more robust defenses.

Applications · Chemistry, Physics, and Earth Sciences

Yunyang Li, Lin Huang, Luojia Xia, Wenhe Zhang, Mark Gerstein

Generative models for 3D molecular conformations must respect Euclidean symmetries and concentrate probability mass on thermodynamically favorable, mechanically stable structures. However, E(3)-equivariant diffusion models often reproduce biases from semi-empirical training data rather than capturing the equilibrium distribution of a high-fidelity Hamiltonian. While physics-based guidance can correct this, it faces two computational bottlenecks: expensive quantum-chemical evaluations (e.g., DFT) and the need to repeat such queries at every sampling step. We present Elign, a post-training framework that amortizes both costs. First, we replace expensive DFT evaluations with a faster, pretrained foundational machine-learning force field (MLFF) to provide physical signals. Second, we eliminate repeated run-time queries by shifting physical steering to the training phase. To achieve the second amortization, we formulate reverse diffusion as a reinforcement learning problem and introduce Force--Energy Disentangled Group Relative Policy Optimization (FED-GRPO) to fine-tune the denoising policy. FED-GRPO includes a potential-based energy reward and a force-based stability reward, which are optimized and group-normalized independently. Experiments show that Elign generates conformations with lower gold-standard DFT energies and forces, while improving stability. Crucially, inference remains as fast as unguided sampling, since no energy evaluations are required during generation.

Deep Learning · Large Language Models

Siya Qi, Yudong Chen, Runcong Zhao, Qinglin Zhu, Zhanghao Hu, Wei Liu, Yulan He, Zheng Yuan, Lin Gui

Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available during generation, among which attention offers a direct view of grounding behavior. However, existing approaches typically rely on coarse summaries that fail to capture fine-grained instabilities in attention. Inspired by signal processing, we introduce a frequency-aware perspective on attention by analyzing its variation during generation. We model attention distributions as discrete signals and extract high-frequency components that reflect rapid local changes in attention. Our analysis reveals that hallucinated tokens are associated with high-frequency attention energy, reflecting fragmented and unstable grounding behavior. Based on this insight, we develop a lightweight hallucination detector using high-frequency attention features. Experiments on the RAGTruth and HalluRAG benchmarks show that our approach achieves performance gains over verification-based, internal-representation-based, and attention-based methods across models and tasks.

Deep Learning · Algorithms

Jose Marie Antonio Miñoza

Spiking Neural Networks (SNNs) offer energy-efficient, biologically plausible computation but suffer from non-differentiable spike generation, necessitating reliance on heuristic surrogate gradients. This paper introduces **UltraLIF**, a principled framework that replaces surrogate gradients with *ultradiscretization*, a mathematical formalism from tropical geometry providing continuous relaxations of discrete dynamics. The central insight is that the max-plus semiring underlying ultradiscretization naturally models neural threshold dynamics: the log-sum-exp function serves as a differentiable soft-maximum that converges to hard thresholding as a learnable temperature parameter $\eps \to 0$. Two neuron models are derived from distinct dynamical systems: UltraLIF from the LIF ordinary differential equation (temporal dynamics) and UltraDLIF from the diffusion equation modeling gap junction coupling across neuronal populations (spatial dynamics). Both yield fully differentiable SNNs trainable via standard backpropagation with no forward-backward mismatch. Theoretical analysis establishes pointwise convergence to classical LIF dynamics with quantitative error bounds and bounded non-vanishing gradients. Experiments on six benchmarks spanning static images, neuromorphic vision, and audio demonstrate improvements over surrogate gradient baselines, with gains most pronounced in single-timestep ($T{=}1$) settings on neuromorphic and temporal datasets. An optional sparsity penalty enables significant energy reduction while maintaining competitive accuracy.

Social Aspects · Everything Else

Guan-Ming Chiu

In this position paper, we argue that the machine learning community should adopt standardized carbon footprint reporting as part of routine scientific practice. Training large models can emit hundreds of tons of CO2, yet environmental costs remain largely invisible in publications. We contend that without energy and emissions metrics, claims of model efficiency are incomplete: a method cannot be deemed ''efficient'' without specifying efficient at what. This gap undermines scientific rigor and reproducibility, as identical experiments in different locations yield vastly different carbon footprints. We put forth reporting guidelines comprising five standardized metrics, practical measurement tools, and integration with community benchmarks, with a phased three-stage adoption process. We address alternative views, including concerns about measurement complexity and potential barriers for resource-limited researchers. To promote equity, we advocate for dual reporting of energy and carbon, reference-grid normalization, and acceptance of approximate estimates. This paper calls on venues, reviewers, authors, and institutions to establish carbon awareness as a foundational element of responsible ML research.

Probabilistic Methods · Monte Carlo and Sampling Methods

Arran Carter, Sanghyeok Choi, Kirill Tamogashev, Víctor Elvira, Esmeralda S. Whitammer

Sampling from a distribution $p(x) \propto e^{-\mathcal{E}(x)}$ known up to a normalising constant is an important and challenging problem in statistics. Recent years have seen the rise of a new family of amortised sampling algorithms, commonly referred to as diffusion samplers, that enable fast and efficient sampling from an unnormalised density. Such algorithms have been widely studied for continuous-space sampling tasks; however, their application to problems in discrete space remains largely unexplored. Although some progress has been made in this area, discrete diffusion samplers do not take full advantage of ideas commonly used for continuous-space sampling. In this paper, we propose to bridge this gap by introducing off-policy training techniques for discrete diffusion samplers. We show that these techniques improve the performance of discrete samplers on both established and new synthetic benchmarks. Next, we generalise discrete diffusion samplers to the task of bridging between two arbitrary distributions, introducing data-to-energy Schrödinger bridge training for the discrete domain for the first time. Lastly, we showcase the application of the proposed diffusion samplers to data-free posterior sampling in the discrete latent spaces of image generative models.

Probabilistic Methods · Monte Carlo and Sampling Methods

Jinwoo Kim, Sékou-Oumar Kaba, Jiyun Park, Seunghoon Hong, Siamak Ravanbakhsh

We study the problem of transformation inversion on general Lie groups: a datum is transformed by an unknown group element, and the goal is to recover an inverse transformation that maps it back to the original data distribution. We take a probabilistic view and model the posterior over transformations as a Boltzmann distribution defined by an energy function on data space. To sample from this posterior, we introduce a diffusion process on Lie groups that keeps all updates on-manifold and only requires computations in the associated Lie alge- bra. Our method, Transformation-Inverting Energy Diffusion (TIED), relies on a new trivialized target-score identity that enables efficient score-based sampling of the transformation posterior. As a key application, we focus on test-time equivariance, where the objective is to improve the robustness of pretrained neural networks to input transformations. Experiments on image homographies and PDE symmetries demonstrate that TIED can restore transformed inputs to the training distribution at test time, showing improved performance over strong canonicalization and sampling baselines.

Applications · Neuroscience, Cognitive Science

Xinyu Shi, Tong Bu, Zhaofei Yu

Spiking Neural Networks (SNNs) offer a notable energy-saving advantage compared to Artificial Neural Networks (ANNs) when deployed on neuromorphic hardware. While recent SNNs achieve superior performance using larger and deeper backbones, this comes at a cost of diminishing their energy-saving benefits. In this paper, we propose UniSparse, a unified sparsification framework for enhancing the energy efficiency of SNNs. We demonstrate that the affine parameters in batch normalization also serve as the learnable threshold of its subsequent spiking neurons. Based on this, we propose a novel spike sparsification method that reduces firing rate by constraining the affine parameters. As a complement to spike sparsification, we propose a weight pruning method based on the same energy constraint, which can be naturally integrated with spike sparsification. Experimental results demonstrate that UniSparse achieves a state-of-the-art trade-off between accuracy and energy efficiency across models and datasets. The sparsified ResNet-18 model requires only 7.04M SOPs for inference to achieve 92.38\% accuracy on the CIFAR-10 dataset. Our work highlights the great potential of deep SNNs in improving energy efficiency.

Deep Learning · Everything Else

Longhua Li, Lei Qi, Qi Tian, Xin Geng

While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks. We observe that output feature drift induced by parameter updates is inherently low-rank, and theoretically prove that preserving parameters along the principal directions of this drift minimizes the output reconstruction error. Motivated by this, we propose **E**nergy-Concentrated and **E**nergy-Ordered **Lo**w-**R**ank **A**daptation (E$^2$-LoRA). By explicitly ordering and concentrating knowledge into leading ranks, E$^2$-LoRA frees capacity for subsequent tasks. Furthermore, we design a dynamic rank allocation strategy to balance stability and plasticity by jointly optimizing energy retention and model plasticity. Extensive experiments across multiple benchmarks demonstrate that E$^2$-LoRA achieves state-of-the-art performance.

Applications · Neuroscience, Cognitive Science

Yiwen Gu, Junchuan Gu, Haibin Shen, Kejie Huang

Spiking Neural Networks (SNNs) emulate the spiking behavior of biological neurons and are promising for energy-efficient neuromorphic computing. A widely used strategy to train SNNs is to convert pretrained Artificial Neural Networks (ANNs), where the accuracy and efficiency are determined by the spike encoding scheme. Traditional methods based on spike count or timing severely underutilize the available encoding space, leading to large accuracy degradation under low-timestep constraints. More expressive alternatives involve complex dynamics, which hinder scalability and practical deployment. To address these challenges, we propose Temporal Weighted Encoding (TWE). Through a simple recursive integration, spikes are implicitly assigned exponentially decaying weights, drawing an analogy to a temporal bit sequence. We systematically analyze the temporal mismatch caused by this weight pattern and propose temporal relaxation and threshold relaxation to resolve this issue, enabling fast and accurate activation encoding. Extensive experiments demonstrate that TWE achieves negligible conversion loss with significantly fewer timesteps, offering a scalable and efficient solution for SNN deployment.

Applications · Chemistry, Physics, and Earth Sciences

ZHIRAN HOU, TINGHUAI MA, Huan Rong, Li Jia, Anouar Imel, Heng Zhang, Ming Li

Molecular property prediction from 3D structures is fundamentally constrained by the scarcity of labeled data. To address this challenge, researchers have adapted various self-supervised pre-training methods from computer vision and natural language processing; however, these approaches often neglect the fundamental physical principles unique to molecular systems. When grounded in physical principles, denoising pre-training can be formally shown to be equivalent to learning molecular force fields.However, existing methods uniformly apply a uniform noise scheme across all molecules, which introduces systematic bias in molecular distribution modeling. To overcome this limitation, we propose MOES-Pred, a novel denoising pre-training framework that employs an energy sentinel mechanism to dynamically adjust molecule-specific noise perturbations. By incorporating chemical prior knowledge, we design molecule-specific noising strategies that expand conformational sampling coverage and improve the fidelity of molecular distribution modeling. Extensive experiments demonstrate that MOES-Pred consistently surpasses existing methods, achieving state-of-the-art performance on both force prediction tasks and downstream quantum chemical property predictions.

Edoardo Cetin, Stefano Peluchetti, Emilio Castillo, Akira Naruse, Mana Murakami, Llion Jones

Scaling autoregressive large language models (LLMs) has had an unprecedented impact, but at vast computational costs. In this work, we tackle these costs by leveraging unstructured sparsity within an LLM's feedforward layers, which account for the majority of its parameters and execution FLOPs. To achieve this, we rework how computation is done on modern GPUs when sparsity is detected, introducing a set of new CUDA and Triton kernels that minimize computation and memory overheads during LLM inference and training. To substantiate our gains, we provide a quantitative study of LLM sparsity, demonstrating that simple L1 regularization can induce over 99% sparsity with negligible impact on downstream performance. When paired with our kernels, we show that these sparsity levels translate into substantial throughput, energy efficiency, and memory usage benefits that increase with model scale. The code and kernels shared with this submission will be released under an open-source license to promote adoption and future research to turn sparsity into a new practical axis for the efficiency and scalability of modern foundation models.

Theory · Probabilistic Methods

Binglin Li, Matthew Reed, Seong-Tae Kim

Compositional data analysis has gained increasing attention due to the widespread occurrence of simplex-valued data, including microbiome data. However, existing kernel or distance-based nonparametric two-sample tests are often designed for Euclidean data and rely on square-root or log-transformations, motivating the need for a unified framework for nonparametric two-sample testing applicable to both compositional and directional data. We propose a studentized spherical harmonic energy distance-based two-sample test over a fixed dimensional underlying space, incorporating U-statistics theory and recent developments of studentization in the context of compositional and directional data. We establish asymptotic normality of our studentized test statistics constructed via spherical harmonics theory, avoiding the need for permutation or bootstrap tests. Simulations demonstrate convergence to the limiting distribution, empirical size control, and improved power in certain scenarios. Our proposed framework paves a new direction for nonparametric testing in non-Euclidean data analysis.

Applications · Everything Else

Zhongju Yuan, Geraint Wiggins, Dick Botteldooren

Audio provides critical situational cues, yet current Audio Language Models (ALMs) face an attention bottleneck in long-form recordings where dominant background patterns can dilute rare, salient events. We introduce NAACA, a training-free NeuroAuditory Attentive Cognitive Architecture that reframes attention allocation as an auditory salience filtering problem. At its core is OWM, a neuro-inspired Oscillatory Working Memory that maintains stable attractor-like states and triggers higher-cognition ALM processing only when adaptive energy fluctuations signal perceptual salience, triggering higher-level reasoning. On XD-Violence, NAACA improves AudioQwen’s average precision (AP) from 53.50\% to 70.60\% while reducing unnecessary ALM invocations. Furthermore, qualitative case studies on the Urban Soundscapes of the World (USoW) dataset show that OWM captures novel events and subcategory shifts while remaining robust to transient pauses and ambient urban noise.

Applications · Neuroscience, Cognitive Science

Binghao Ye, Wenjuan Li, Dengfeng Xue, Bing Li, Weiming Hu, Dong Liang, Kun Shang

Spiking Neural Networks (SNNs) have garnered increasing attention for their biological plausibility, energy efficiency, and temporal modeling capability. Due to the non-differentiability of spike generation, a widely used supervised training method for SNNs is backpropagation through time with surrogate gradients, which achieves competitive performance with a small number of timesteps. Intuitively, scaling timesteps should further improve performance by enriching temporal dynamics. However, we observe timestep scaling paradox (TSP), a counter-intuitive degradation in accuracy when scaling timesteps. We investigate TSP and link it to long-term temporal gradient vanishing and weakened cross-timestep dependencies. To address this, we propose the Timestep-Scalable (TS) neuron model. It introduces long-term memory reconsolidation to enhance cross-timestep information flow and enable effective learning with more timesteps. In parallel, a temporal forgetting mechanism periodically truncates the accumulation path, suppressing excessive temporal information buildup and improving training stability. Supported by theoretical analysis and extensive experiments, TS consistently improves performance when scaling timesteps. Beyond gains from timestep scaling, it attains state-of-the-art results on EEG signals, event-based recognition, and time-series forecasting, while remaining strong on conventional image classification and object detection datasets.

Applications · Neuroscience, Cognitive Science

Kaiwei Che, Wei Fang, Zhengyu Ma, Yifan Huang, Peng Xue, Li Yuan, Yonghong Tian

Spiking Neural Networks (SNNs), with their event-driven and biologically inspired mechanisms, are well-suited for energy-efficient neuromorphic hardware. Neural coding, which is critical to SNNs, determines how information is represented via spikes. While Time-to-First-Spike (TTFS) coding uses a single spike per neuron to offer extreme sparsity and energy efficiency, it often suffers from unstable training and low accuracy due to its sparse firing. To address these challenges, we propose a training framework that incorporates parameter initialization, training normalization, a temporal output decoder, and a re-evaluation of the pooling layer. The proposed parameter initialization and training normalization mitigate signal diminishing and gradient vanishing, which helps stabilize training. Our output decoder aggregates temporal spikes to encourage earlier firing, thereby reducing latency. The re-evaluation of the pooling layer demonstrates that max-pooling violates single-spike constraints, which should be avoided, whereas average-pooling preserves them. Experiments show that our framework stabilizes and accelerates training, reduces latency, and achieves state-of-the-art accuracy for step-by-step TTFS SNNs on MNIST ($99.48\%$), Fashion-MNIST ($92.90\%$), CIFAR10 ($90.56\%$), CIFAR100 ($70.27\%$) and DVS Gesture ($95.83\%$).

Applications · Health / Medicine

Erjian Zhang, Yatong Hao, Liejun Wang, Zhiqing Guo

While multi-task learning based automatic radiology report generation (RRG) is widely adopted to ensure clinical consistency, most focus on architectural designs yet remain limited to coarse linear scalarization strategies. These strategies can not effectively balance the hard constraints of discriminative clinical supervision with the smoothness requirements of report generation. To address these problems, we analyze the failure mechanism of linear scalarization from the perspective of gradient dynamics for the first time, utilizing the stochastic differential equation (SDE) framework to formalize it as a "Double Dilemma'' of drift term deviation and diffusion term decay. Based on this, we propose a backbone-agnostic optimizer named Conflict-Averse Magnitude-Enhanced Gradient Descent (CAME-Grad). Through conflict-averse direction rectification and magnitude-enhanced energy injection, the algorithm not only ensures geometric validity, but also avoids the local optimal solution. Then, the adaptive gradient fusion technology is used to establish a dynamic balance between the theoretical optimal direction and the task-specific inductive bias. Experiments show that as a universal plug-and-play optimizer, CAME-Grad brings substantial and consistent improvements across eight diverse RRG methods, elevating overall clinical efficacy performance by an average of 2.3% on MIMIC-CXR and 1.9% on IU X-Ray. Our code is available at https://anonymous.4open.science/r/CAME-Grad-04A7.