论文检索

Applications · Chemistry, Physics, and Earth Sciences

Yotam Peled, David Zenati, Eliya Nachmani

Realizing the full potential of quantum computation requires Quantum Error Correction (QEC). QEC reduces error rates by encoding logical information across redundant physical qubits, enabling errors to be detected and corrected. A common decoder used for this task is Minimum Weight Perfect Matching (MWPM) a graph-based algorithm that relies on edge weights to identify the most likely error chains. In this work, we propose a data-driven decoder named Neural Minimum Weight Perfect Matching (NMWPM). Our decoder utilizes a hybrid architecture that integrates Graph Neural Networks (GNNs) to extract local syndrome features and Transformers to capture long-range global dependencies, which are then used to predict dynamic edge weights for the MWPM decoder. To facilitate training through the non-differentiable MWPM algorithm, we formulate a novel proxy loss function that enables end-to-end optimization. Our findings on the toric code under depolarizing noise demonstrate thresholds of 17.9\% and 10.95\%, nearing the 18.9\% and 11.0\% maximum likelihood bounds, highlighting the advantage of hybrid decoders that combine the predictive capabilities of neural networks with the algorithmic structure of classical matching.

Deep Learning · Algorithms

Naili Xing, Shaofeng Cai, Lingze Zeng, Jiaqi Zhu, Peng LU, Jian Pei, Beng Chin Ooi

Recent advances have shifted the paradigm of tabular learning toward tabular foundation models, yet their accuracy relies on a heavy inference cost that scales poorly with context size. Deep neural networks remain a highly competitive and more efficient modeling paradigm when equipped with well-designed architectures; however, identifying such architectures in a data-adaptive and budget-aware manner remains challenging. We propose pTNAS, the first progressive neural architecture search (NAS) approach tailored for tabular data, to enable fast identification of a viable architecture and continuously improve its search performance as more budget becomes available. pTNAS adopts a filter-and-refine optimization strategy that combines both efficient training-free and effective training-based architecture evaluation. At the filtering phase, we introduce pTProxy, a novel zero-cost proxy specifically designed for tabular networks that jointly captures architectural trainability and expressivity, to facilitate fast filtering of large architecture search spaces. At the refinement phase, pTNAS employs a fixed-budget scheduling algorithm to accurately identify the best-performing architecture from a small set of promising candidates. We further propose a budget-aware coordinator to optimize budget allocation holistically. Experiments show that pTNAS reduces the time to reach the globally best architecture by up to 82.75 X compared with other NAS approaches, and improves average predictive accuracy and end-to-end efficiency by up to 4.95 X compared with TabPFN.

Applications · Health / Medicine

Binghao Liu, Wenzheng Zhao, Zhijie Zheng, Fei Gu

Effective DNA modeling demands the integration of complex patterns such as local motifs, long-range dependencies, and periodic signals. Yet, architectures like CNNs, Transformers, and SSMs are hindered by static or time-domain-exclusive designs, which limit their representational flexibility. To address this, we introduce the **Synergistic Plasticity Unit (SPU)**, a scalable architecture that achieves multi-level plasticity through three synergistic layers. Specifically, SPU integrates a *Locus Plasticity Layer* (LPL) to capture fine-grained local motifs via token-specific convolution operations, while utilizing a *Domain Plasticity Layer* (DPL) to form multi-domain global features by concurrently modeling sequential (time) and spectral (frequency) patterns. Furthermore, it incorporates a *Saliency Plasticity Layer* (SPL) to optimize information flow through dual-axis saliency scoring. Supported by theoretical analysis, extensive empirical validation, and in-depth biological interpretation, this unified design enables SPU to achieve state-of-the-art performance with quasi-linear complexity, establishing a robust and principled paradigm for DNA modeling. Code will be available upon acceptance.

Theory · Optimization

Sumaya Abdul Rahman, Seckhen Cuellar, Ghani Raissov, Mohammad Raza

Natural language interfaces can greatly benefit the accessibility and usability of optimization modeling, and recent advances in large language models (LLMs) show promise in automatically translating textual problem descriptions into executable solver formulations. However, a key challenge for existing approaches is to ensure that the inferred formulation correctly implements the intended task, even if it may execute without errors. We introduce \textsc{VeriSimpl}, a solver–LLM framework for robust natural-language-to-optimization formalization. Our approach is based on the idea of \emph{simplification-based verification}, where the optimization solver is leveraged to generate simplified diagnostic queries about a candidate formulation to allow the LLM to tractably reason about the correctness of the formulation with respect to the task description. We present such simplification strategies along different dimensions with respect to problem constraints and decision variables, which allow the LLM to reason locally under fixed global contexts. Evaluations on a range of optimization benchmarks show how our approach provides consistent improvements in accuracy over existing methods, while also providing a novel high-precision self-verification signal.

Deep Learning · Theory

Sai Niranjan Ramachandran, Suvrit Sra

Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2× computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.

General Machine Learning · Transfer, Multitask and Meta-learning

HuiYu Yi, Xu Zhiming, Dunwei Tu, Zhicheng Wang, Baile Xu, Furao Shen

The Nearest Class Mean (NCM) classifier is widely favored in Class-Incremental Learning (CIL) for its superior resistance to catastrophic forgetting compared to Fully Connected layers. While Neural Collapse (NC) theory supports NCM's optimality by assuming features collapse into single points, non-linear feature drift and insufficient training in CIL often prevent this ideal state. Consequently, classes manifest as complex manifolds rather than collapsed points, rendering the single-point NCM suboptimal. To address this, we propose Hierarchical-Cluster SOINN (HC-SOINN), a novel classifier that captures the topological structure of these manifolds via a ``local-to-global'' representation. Furthermore, we introduce Structure-Topology Alignment via Residuals (STAR) method, which employs a fine-grained pointwise trajectory tracking mechanism to actively deform the learned topology, allowing it to adapt precisely to complex non-linear feature drift. Theoretical analysis and Procrustes distance experiments validate our framework's resilience to manifold deformations. We integrated HC-SOINN into seven state-of-the-art methods by replacing their original classifiers, achieving consistent improvements that highlight the effectiveness and robustness of our approach. Code is available at \url{https://anonymous.4open.science/r/icml2026-9B60}.

Social Aspects · Safety

Jason Lucas, Pureheart Ogheneogaga Irikefe, Adaku Uchendu, Umniya Najaer, Cornelius Adejoro, Patrice Sterling, Dongwon Lee

Large language models are deployed globally as universal systems, yet their safety mechanisms remain English-optimized. This creates a Dual Curse for speakers of low-resource languages: a Harmfulness Curse where harmful content generation rises from 1\% in English to 35\% in languages like Hausa, Igbo, and Javanese, and a Relevance Curse where instruction-following drops by 20 percentage points, making these systems simultaneously more dangerous and less useful. Drawing on a PRISMA-guided systematic review of 207 studies, we demonstrate that this disparity stems from a pre-training bottleneck: reward models achieve only 49--50\% accuracy in low-resource languages (equivalent to random chance), rendering post-hoc alignment structurally ineffective. These technical failures become governance hazards when at least 22 countries mandate automated content moderation, creating an infrastructure that is exploitable for censorship. Therefore, we propose a socio-technical framework addressing this inequity: (1) safety context distillation during pre-training (achieving 78--89\% harm reduction); (2) participatory harm specification by affected communities; and (3) evaluation metrics jointly tracking attack resistance and false refusal rates across languages.

Social Aspects · Security

Xian Gao, Bo Hui, MIN-TE SUN, Wei-Shinn Ku

Data attribution has become a core primitive for pricing, auditing, and governing machine learning pipelines, yet current attribution methods implicitly assume that attribution value faithfully reflects participants' contributions. We show that this assumption can fail: a single participant in a standard distributed training workflow can substantially inflate its measured attribution value while keeping global utility intact. Our attribution-first attack uses a latent optimization procedure that injects small, utility-preserving synthetic batches to exploit non-IID label coverage and evaluator sensitivities. Across datasets, models, and multiple marginal-utility evaluators, the attack consistently raises the adversary’s attribution value and reshapes the relative attribution structure among benign clients without degrading accuracy or triggering geometry-based defenses. These results demonstrate that attribution itself constitutes a new attack surface and motivate the development of attribution-robust and incentive-compatible scoring mechanisms.

Theory · Optimization

Sihan Zeng, Sujay Bhatt, Sumitra Ganesh

Neural networks are typically trained with a single learning rate across all layers. While recent empirical evidence suggests that assigning layer-specific learning rates can accelerate training, a principled understanding of the conditions and mechanisms under which non-uniform learning rates are beneficial remains limited. In this work, we investigate non-uniform learning rates through the lens of Stackelberg optimization. Specifically, we demonstrate that training neural networks with a smaller learning rate for the body layers and a larger learning rate for the final layer can be interpreted as a two-time-scale alternating gradient descent algorithm applied to a Stackelberg reformulation of the original objective. We establish finite-time convergence guarantees for the algorithm under broad conditions that accommodate constraint sets and non-smooth activation functions. Beyond convergence, we identify two mechanisms by which non-uniform learning rates can outperform uniform learning rates: (i) we show that certain problem instances induce a Stackelberg objective with stronger optimization structure than the original objective, yielding faster convergence to globally optimal solutions, (ii) our numerical analysis reveals that the Stackelberg objective can exhibit substantially sharper local curvature, especially in early training, which leads to more informative gradients and learning acceleration. Experiments in supervised learning and reinforcement learning support our findings.

Applications · Computer Vision

Kaiyang Ji, Bingsheng Qian, Binghuan Wu, Kangyi Chen, Ye Shi, Jingya Wang

We study real-time audio-responsive character control as a deployment-faithful problem: strictly causal, bounded-latency streaming that must generate coherent full-body motion at interactive frame rates while the audio condition can change abruptly (tempo shifts, drops, or user edits). Prior music-to-motion systems are largely optimized for offline generation with global context, and degrade in streaming rollouts where conditioning history becomes stale or unreliable. We introduce DiscoForcing, a streaming audio-driven diffusion framework that combines a causal music encoder that captures rhythmic structure and phase dynamics with a diffusion-forcing sequence model trained under heterogeneous noise levels across the temporal horizon. Building on this, we design a hybrid temporal schedule and a history-guided streaming sampler to explicitly trade off responsiveness against long-horizon consistency under non-stationary audio. Implemented in an end-to-end real-time interactive system with online avatar playback and humanoid deployment workflows, DiscoForcing delivers more stable long-horizon rollouts and sharper audio–motion alignment than prior baselines under matched causality and latency constraints while maintaining real-time throughput.

Deep Learning · Algorithms

Menghao Lv, Huiqiong Wang, Li Sun, Mingli Song

Post-training quantization (PTQ) suffers from severe accuracy degradation in ultra-low-bit regimes. To address this challenge, we propose SHARP-Q, a unified framework grounded in Information Geometry that aligns the quantization objective with the intrinsic Fisher geometry. Following a "Rectify-then-Approximate" strategy, SHARP-Q first preconditions the optimization landscape via Hessian-Aware Rectification (HAR) and subsequently approximates the rectified Fisher Information Matrix through Dynamic Fisher-Subspace Compensation (DFSC). Our findings reveal a pivotal insight: precise geometric alignment enables hardware-friendly uniform quantizers to consistently outperform specialized non-uniform designs. Extensive experiments across representative Vision Transformer and convolutional architectures confirm that SHARP-Q establishes new state-of-the-art results, achieving substantial accuracy gains in the challenging W2A2 and W3A3 settings. Code is available in the supplementary material.

Probabilistic Methods · Monte Carlo and Sampling Methods

Panagiotis Theodoropoulos, Juno Nam, Evangelos Theodorou, Jaemoo Choi

Transportation on graphs is a fundamental challenge across many domains, where decisions must respect topological and operational constraints. Despite the need for actionable policies, existing graph-transport methods lack this expressivity. They rely on restrictive assumptions, fail to generalize across sparse topologies, and scale poorly with graph size and time horizon. To address these issues, we introduce Generalized Schrödinger Bridge on Graphs (GSBoG), a novel scalable data-driven framework for learning executable controlled continuous-time Markov chain (CTMC) policies on arbitrary graphs under state cost augmented dynamics. Notably, GSBoG learns trajectory-level policies, avoiding dense global solvers and thereby enhancing scalability. This is achieved via a likelihood optimization approach, satisfying the endpoint marginals, while simultaneously optimizing intermediate behavior under state-dependent running costs. Extensive experimentation on challenging real-world graph topologies shows that GSBoG reliably learns accurate, topology-respecting policies while optimizing application-specific intermediate state costs, highlighting its broad applicability and paving new avenues for cost-aware dynamical transport on general graphs.

General Machine Learning · Methodology

Pankaj Bhagwat, Zhixian Yang, yihao wang, Bei Jiang, Linglong Kong

Conformal Prediction (CP) provides rigorous finite-sample coverage guarantees, yet its statistical efficiency hinges critically on the size of the calibration set. In data-scarce regimes, CP often suffers from volatile quantile estimation, leading to overly conservative and wide prediction intervals. To address this, we propose Random Score Alignment-Conformal Prediction (RSA-CP), a simple framework designed to improve sample efficiency in small-sample CP. Instead of requiring the computationally intensive generation of full synthetic datasets, RSA-CP enhances calibration by directly aligning real scores with a high-resolution reference score distribution. By employing an optimal transport mapping, our framework refines "step-like" quantile increments through a globally optimal use of reference information. We provide theoretical guarantees establishing that RSA-CP maintains robust coverage without any distributional assumptions on the reference scores. Empirical evaluations demonstrate that RSA-CP consistently produces shorter and more precise prediction intervals while maintaining finite-sample coverage guarantees. Overall, RSA-CP offers a computationally efficient and theoretically grounded solution for robust uncertainty quantification under limited data.

Deep Learning · Algorithms

Mingsheng Cao, Hongliang Chen, Ming Hu, Fei Gao, Qiaolong Ding, Wenke Huang, Xiaofei Xie, Junlong Zhou

Although Federated Learning (FL) offers advantages in privacy-preserving for cross-device collaborative learning, its practical deployment remains severely constrained by heterogeneous hardware resources and non-IID (non-independent and identically distributed) data across devices. Sub-model extraction has emerged as a widely adopted strategy for enabling collaborative training among devices with heterogeneous models. However, existing sub-model extraction methods in FL typically rely on coarse-grained stochastic selection or rigid rule-based neuron selection, which severely limits training performance. Specifically, stochastic strategies lead to severe parameter conflicts under non-IID data distributions, while rule-based approaches lack diversity in neuron selection per device, preventing comprehensive parameter optimization. To address this problem, this paper presents a novel sub-model extraction-based FL framework, named SpineFL, which adopts a backbone-sharing mechanism and an activation-guided pruning strategy for sub-model extraction. Specifically, SpineFL decomposes each global model layer into two portions: i) a mandatory backbone shared by all the sub-models to maintain model generalization, and ii) a dynamic portion for sub-model extraction. SpineFL adopts the activation-guided selection strategy to probabilistically select neurons according to their activation frequency from the dynamic portion to generate sub-model, where neurons exhibiting higher historical activation are more likely to be included, thereby simultaneously addressing parameter conflicts while preserving selection diversity. Experimental results demonstrate that compared with state-of-the-art heterogeneous FL methods, SpineFL can achieve up to 3.28% accuracy improvement.

Optimization · Non-Convex

Zuqi Zhang, Chenghe Sun, Xiangyi Chu, Wei-Han Yu, Ka-Fai Un, Rui Martins, Pui-In Mak, Jiawei Xu

Adaptive rounding techniques in Post-Training Quantization (PTQ) enable the efficient deployment of Large Language Models (LLMs) with low resource and data dependencies. While learning-based rounding methods are accurate yet costly, compensation-based approaches offer a highly efficient alternative. However, synergizing these two to realize their full potential is hindered by spatiotemporal misalignment in the decoupled paradigm. Key challenges include temporal parameter conflict, the invalidation of the initial Round-to-Nearest (RTN) assumption, and spatially-inconsistent optimization objectives. This paper introduces STLA, a novel rounding-optimized PTQ framework that achieves both fast and accurate LLM quantization. STLA resolves temporal inconsistency through cluster-wise integrated rounding optimization, which collocates the learning and compensation phases. STLA achieves spatial alignment through a unified global objective derived from the Schur Complement, enabling the solver to look ahead and align local rounding decisions with the optimal future compensation of remaining weights. Furthermore, we propose a Hessian-guided clustering strategy that exploits both diagonal and off-diagonal information to maximize intra-cluster error cancellation. Extensive experiments demonstrate that STLA establishes a new state-of-the-art for low-bit PTQ while maintaining high computational efficiency. The code is available at https://anonymous.4open.science/r/STLA.

Deep Learning · Large Language Models

Junyi Li, Xiaowei Qian, Yingyi Zhang, Wenlin Zhang, Guojing Li, Sheng Zhang, Xiao Han, Yichao Wang, Xiangyu Zhao

Recent advances in tool-integrated language agents have significantly improved their ability to solve complex reasoning tasks. However, existing alignment methods predominantly focus on maximizing task accuracy, while overlooking auxiliary objectives such as tool-use efficiency, which are essential for practical deployment. To address this gap, we introduce {ParetoPO}, a two-stage multi-objective optimization framework for aligning tool-using large language models (LLMs) under competing objectives. In the first stage, ParetoPO leverages hypervolume-guided dynamic scalarization to adapt reward weights based on global Pareto frontier progress. In the second stage, it replaces scalarized learning signals with Pareto-ranking-based advantage computation, promoting nondominated trajectories through dominance-aware credit assignment. This design enables fine-grained, action-level optimization across multiple conflicting objectives. Experimental results on mathematic reasoning and deep search tasks show that ParetoPO consistently discovers policies with superior accuracy-efficiency trade-offs compared to static and heuristic baselines.

Optimization · Non-Convex

Kangke Cheng, Jiawei Huang, Jingni Song, Wanlin Zhang, Bangxian Han, Hu Ding

The Gromov-Wasserstein (GW) distance enables comparison across different spaces but remains fragile to structural noise due to its global quadratic coupling. Existing robust extensions primarily rely on node-centric mass relaxation. However, we argue that this strategy is far from sufficient: it only addresses node-induced structural noise (outliers) while neglecting edge-induced distortions where spurious connections exist between valid nodes. To overcome this limitation, we propose the Structurally Robust Gromov-Wasserstein (SRGW) distance, a novel formulation that adaptively filters geometric distortions during optimization. By introducing a structure-aware dual-mask mechanism, our method effectively isolates these stubborn structural outliers while preserving strict marginal constraints for balanced transport. We solve this objective using a Mask-Guided GW Algorithm, which jointly optimizes the transport plan and the structural noise filters. We provide a rigorous theoretical analysis proving that our algorithm converges to a critical point under the Kurdyka-Łojasiewicz framework. Extensive experiments on synthetic geometric matching and real-world subgraph alignment benchmarks demonstrate that Mask-Guided GW achieves superior alignment quality, particularly under severe structural noise.

Theory · Deep Learning

Hongkang Li, Hancheng Min, Rene Vidal

Transformer-based diffusion models have demonstrated remarkable performance at generating high-quality samples. However, our theoretical understanding of the reasons for this success remains limited. For instance, existing models are typically trained by minimizing a denoising objective, which is equivalent to fitting the score function of the training data. However, we do not know why transformer-based models can match the score function for denoising, or why gradient-based methods converge to the optimal denoising model despite the non-convex loss landscape. To the best of our knowledge, this paper provides the first convergence analysis for training transformer-based diffusion models. More specifically, we consider the population Denoising Diffusion Probabilistic Model (DDPM) objective for denoising data that follow a \textit{multi-token Gaussian mixture} distribution. We theoretically quantify the required number of tokens per data point and training iterations for the global convergence towards the Bayes optimal risk of the denoising objective, thereby achieving a desired score matching error. A deeper investigation reveals that the self-attention module of the trained transformer implements a \emph{mean denoising} mechanism that enables the trained model to approximate the oracle Minimum Mean Squared Error (MMSE) estimator of the injected noise in the diffusion steps. Numerical experiments validate these findings.

Deep Learning · Large Language Models

Zheng Chen, Weifeng Yang, Jianxiao Tang, Buhui Yao

Sparse Mixture-of-Experts (MoE) language models enable conditional computation but face deployment challenges due to the "memory wall": while few experts are activated per token, the entire model must reside in memory. Existing expert pruning methods primarily rely on independent ranking, failing to account for the complex inter-dependencies and redundancies between experts. In this paper, we formulate post-training MoE pruning as a reconstruction-driven subset selection problem, aiming to minimize layer-output distortion under a cardinality constraint. We introduce SCHUR-A*, an algorithm that leverages A* search to achieve globally optimal expert selection within each layer. To maintain computational tractability, we derive a novel, admissible heuristic upper bound using a Schur-complement-based relaxation of the reconstruction objective. This tight bound allows for aggressive pruning of the search space while mathematically guaranteeing optimality. Furthermore, we propose an automated strategy to balance fidelity and memory reduction across heterogeneous layers via knee-point detection. Extensive experiments on Qwen3-30B-A3B demonstrate that SCHUR-A* significantly outperforms greedy and ranking-based baselines, maintaining comparable performance even under aggressive pruning ratios.

Deep Learning · Large Language Models

Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang, Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su 等

The reasoning patterns of large language models (LLMs) remain opaque, and Reinforcement learning (RL) typically assigns uniform credit across an entire generation, blurring the distinction between pivotal and routine steps. This work treats attention as a natural substrate for interpreting LLM reasoning and a window for aligning optimization with its internal dynamics. We first distinguish attention heads between locally and globally focused information processing and reveal that locally focused heads produce a sawtooth pattern near the diagonal indicating phrasal chunks, while globally focused heads expose tokens that exert broad downstream influence over future tokens. We quantify these with two metrics measuring the extent of backward attention within a clipped window and the average attention a token receives from subsequent tokens, respectively. Taken together, these signals reveal a recurring preplan-and-anchor mechanism, where the model first performs a long-range contextual reference to generate an introductory token, which is immediately followed by or coincides with a semantic anchor token that organizes subsequent reasoning. Leveraging these insights, we introduce three novel RL strategies that dynamically perform targeted credit assignment to critical nodes (preplan tokens, anchor tokens, and their temporal coupling) and show consistent performance gains across various reasoning tasks.