论文检索

Deep Learning · Large Language Models

Zichong Li, Chen Liang, Liliang Ren, Tuo Zhao, Yelong Shen, Weizhu Chen

Large language models (LLMs) increasingly operate in settings that require reliable long-context understanding, such as retrieval-augmented generation and multi-document reasoning. A common strategy is to fine-tune pretrained short-context models at the target sequence length. However, we find that standard long-context adaptation can remain brittle: model accuracy depends strongly on the absolute placement of relevant evidence, exhibiting high positional variance even when controlling for task format and difficulty. We propose *RoPE-Perturbed Self-Distillation*, a training regularizer that improves positional robustness. The core idea is to form alternative ``views'' of the same training sequence by perturbing its RoPE indices---effectively moving parts of the context to different positions---and to train the model to produce consistent predictions across views via self-distillation. This encourages reliance on semantic signals instead of brittle position dependencies. Experiments on long-context adaptation of Llama-3-8B and Qwen-3-4B demonstrate consistent gains on long-context benchmarks, including up to 12.04\% improvement on RULER-64K for Llama-3-8B and 2.71\% on RULER-256K for Qwen-3-4B after SFT, alongside improved length extrapolation beyond the training context window.

Deep Learning · Theory

Yujie Shen, Zihan Wang, Jian Qian, Qi Lei

Training data reconstruction from KKT conditions has shown striking empirical success, yet it remains unclear when the resulting KKT equations have unique solutions and, even in identifiable regimes, how to reliably recover solutions by optimization. This work hereby focuses on these two complementary questions: identifiability and optimization. On the identifiability side, we discuss the sufficient conditions for KKT system of two-layer networks with polynomial activations to uniquely determine the training data, providing a theoretical explanation of when and why reconstruction is possible. On the optimization side, we introduce sample splitting, a curvature-aware refinement step applicable to general reconstruction objectives (not limited to KKT-based formulations): it creates additional descent directions to escape poor stationary points and refine solutions. Experiments demonstrate that augmenting several existing reconstruction methods with sample splitting consistently improves reconstruction performance.

Deep Learning · Foundation Models

Yonggang Zhang, Zhiqin Yang, Wei Xue, Dong Fang, Bo Han, Yike Guo

Direct Preference Optimization (DPO) has emerged as a popular alternative to Reinforcement Learning from Human Feedback (RLHF), offering theoretical equivalence with simpler implementation. We prove this equivalence is _conditional_ rather than universal, depending on an implicit assumption frequently violated in practice: the RLHF-optimal policy must prefer human-preferred responses. When this assumption fails, DPO optimizes _relative advantage_ over the reference policy rather than _absolute alignment_ with human preferences, leading to pathological convergence where policies decrease DPO loss while preferring dispreferred responses. We characterize when this assumption is violated, show the existence of an undesirable solution space, and prove that DPO and RLHF optimize fundamentally different objectives in such cases. To address this, we introduce Constrained Preference Optimization (CPO), augmenting RLHF with constraints for provable alignment. We further provide a geometric interpretation through soft margin ranking, revealing DPO implements margin ranking with potentially negative targets. Our theoretical analysis establishes when DPOs' guarantees hold and provides solutions preserving simplicity with provable alignment. Comprehensive experiments on standard benchmarks demonstrate that CPO achieves state-of-the-art performance.

Probabilistic Methods · Bayesian Models and Methods

Luwei Wang, Dagmara Panas, Ke Wang, Sohan Seth, Bruce Guthrie

Constrained clustering incorporates prior knowledge in the form of pairwise constraints to guide data partitioning. While effective, existing Bayesian approaches are often limited in scalability to large datasets and provide weak interpretability due to the lack of explicit feature relevance modeling. We propose BASIL, a scalable Bayesian semi-supervised clustering framework that leverages stochastic variational inference to jointly infer cluster assignments and feature importance weights. This joint formulation enables the identification of discriminative features consistent with the imposed constraints. To robustly handle noisy or inconsistent supervision, BASIL introduces an adaptive constraint-weighting mechanism that down-weights unreliable constraints. Experiments on synthetic and real-world benchmarks demonstrate that our approach achieves competitive clustering performance while improving scalability and interpretability over existing baselines. We further demonstrate applicability to large-scale health data, including medical imaging and electronic health records.

Deep Learning · Large Language Models

Chenglin Li, Yisen Xu, Zehao Wang, Shin Hwei Tan, Tse-Hsun (Peter) Chen

Repository-level automated program repair (APR) requires long-horizon reasoning over interdependent decisions. However, most LLM-based approaches reconstruct repair reasoning independently for each issue, failing to reuse successful patterns from prior repairs, even though real-world repositories contain many related issues with shared structure or constraints. Existing methods typically rely on forward exploration, which operates under outcome uncertainty, incurs substantial inference-time overhead, and can drift from the final correct patch. We propose Conditional Reasoning Distillation (ConRAD), which leverages in-repository resolved issues by reconstructing repair reasoning backward from verified patches and distilling outcome-consistent, stage-wise repair reasoning plans. Injected at inference time, these plans guide fault localization and patch generation, replacing open-ended exploration with constrained inference without fine-tuning or search. On SWE-Bench Lite, ConRAD improves Pass@1 by 10.4\% (GPT-4o), 8.6\% (DeepSeek-V3), and 10.3\% (GPT-5), demonstrating a scalable inference-time alternative to forward exploration for long-horizon APR.

Deep Learning · Large Language Models

Ke Wang, ZEHAO Yu, Luwei Wang, Yongchao Huang

Recent work shows that chain based sampling for power shaped trajectory distributions can deliver large test time gains from a fixed base LLM and can approach RL trained reasoners such as GRPO. Deployment is the bottleneck. Autoregressive Metropolis Hastings is inherently serial, limits GPU utilization, and exhibits extreme tail latency at high budgets, reaching p95 $=1318$s on MATH500 at $128\times$. We propose Adaptive Sequential Monte Carlo (ASMC), a parallel particle inference method that targets power shaped trajectory distributions while adapting particle populations to problem hardness. To make resampling practical for Transformers, we introduce cache coherent resampling, which realizes ancestry updates by reordering KV caches and other particle bound tensors, avoiding prefix recomputation. On MATH500 at the same budget, ASMC attains $80.6\%$ exact match accuracy with p95 $=73.7$s, improving the accuracy to tail latency trade off over both sequential MCMC and best of $n$. We further analyze particle degeneracy and find that collapse severity, measured by low $\mathrm{ESS}_{\min}/N$, strongly predicts failures, while sensitivity to the resampling scheme is limited.

Renxiang Guan, Xiang Yang, Hao Yu, Siwei Wang, Suyuan Liu, Wenjing Yang, Jun-Jie Huang, Ao Li, Xinwang Liu, Yuhua Tang

The rapid expansion of remote sensing technology has generated massive amounts of unlabeled multi-view data distributed across different institutions. Analyzing this data presents significant challenges, as centralized processing incurs prohibitive communication costs and raises data privacy concerns. To address these issues, this paper proposes a novel deep federated multi-view clustering (MVC) framework tailored for remote sensing data. Unlike existing methods that transmit sensitive data features, our approach shares only privatized prototypes masked with adaptive noise, ensuring both communication efficiency and privacy preservation. First, we employ superpixel segmentation to reduce the spatial dimensionality of remote sensing data, lowering computational burdens. Furthermore, to resolve the inconsistency of cluster assignments across different clients, we design a co-occurrence structural alignment module that synchronizes local models. Finally, we incorporate a wasserstein prototype contrastive learning mechanism, which models clusters as distributions rather than points, to enhance global consistency and robustness against data heterogeneity. Extensive experiments on four public datasets demonstrate that our framework achieves superior clustering performance and efficiency compared to state-of-the-art methods.

Deep Learning · Attention Mechanisms

Francesco D'Angelo, Oğuz Yüksel, Swathi Narashiman, Nicolas Flammarion

Induction heads are attention circuits believed to underlie in-context learning in transformers, yet a precise characterization of the estimators they implement remains elusive. We study transformers trained on order-$k$ Markov chains and prove that a two-layer disentangled transformer implements a soft context-matching estimator that aggregates contributions from all partial context matches, weighted exponentially by their degree of overlap. This mechanism admits two complementary smoothing interpretations. First, prepending a beginning-of-sequence (BOS) token induces additive pseudo-counts, recovering Dirichlet-style smoothing. Second, a finite attention temperature enables interpolation across context orders, analogous to Jelinek–Mercer smoothing but with data-dependent weights that adapt to each sequence's local structure. Experiments on trained transformers confirm that learned attention patterns match our theoretical construction and approach Bayes-optimal performance, where hard counting fails. Our results bridge mechanistic interpretability of induction heads with classical statistical smoothing, revealing that transformers learn to regularize in-context estimation rather than simply count.

Deep Learning · Large Language Models

Yingru Li, Jiacai Liu, Jiawei Xu, Yuxuan Tong, Ziniu Li, Baoxiang Wang

Policy gradient methods for Large Language Models (LLMs) optimize a policy $\pi_\theta$ via a surrogate objective computed from samples of a rollout policy $\pi_{\text{roll}}$. However, modern LLM-RL pipelines suffer from unavoidable implementation divergences—such as backend discrepancies, Mixture-of-Experts routing discontinuities, and distributed training staleness. These factors cause an off-policy mismatch ($\pi_{\text{roll}} \neq \pi_\theta$), leading to approximation errors between the surrogate and true objectives, often precipitating training collapse. We demonstrate that classical trust region bounds on this error scale as $O(T^2)$ with sequence length $T$, rendering them vacuous for long-horizon tasks. To address this, we derive two tighter bounds: a *Pinsker-Marginal* bound scaling as $O(T^{3/2})$ and a *Mixed* bound scaling as $O(T)$. Crucially, both bounds depend on $\mathcal{D}_{\text{KL}}^{\max}$—the maximum token-level KL divergence across the sequence. As this is a *sequence-level* quantity, it cannot be controlled by token-independent methods like PPO clipping. We propose Trust Region Masking (TRM), which masks entire sequences that violate the trust region. TRM theoretically provides the first non-vacuous monotonic improvement guarantees and empirically improves training stability for long-horizon LLM-RL.

Deep Learning · Attention Mechanisms

Oğuz Yüksel

This paper studies simple transformers on a high-order Markov chain, where the model must incorporate knowledge from multiple past positions, each with different statistical importance. We show that transformers learn the task incrementally, with each stage induced by the acquisition or copying of information from a subset of positions via a sparse attention pattern. Notably, the learning dynamics transition from competitive, where all heads focus on the statistically most important attention pattern, to cooperative, where different heads specialize in different patterns. We explain these dynamics using a set of simplified differential equations, which characterize the stage-wise learning process and analyze the training trajectories. As transformers progress through these stages, they climb a complexity ladder defined via simpler misspecified hypothesis classes until reaching the full model class. Overall, our work provides theoretical explanations for how transformers learn in stages even without an explicit curriculum and provides insights into the emergence of complex behaviors and generalization, with relevance to applications such as natural language processing and algorithmic reasoning.

Reinforcement Learning · Everything Else

Yizhi Li, Qingshui Gu, Zhoufutu Wen, Ziniu Li, Ruibin Yuan, Tianshun Xing, Shuyue Guo, Tuney Zheng, 周欣, Xingwei Qu 等

Recent advancements in aligning large language models via reinforcement learning have achieved remarkable gains in solving complex reasoning problems, but at the cost of expensive on-policy rollouts and limited exploration of diverse reasoning paths. In this work, we introduce TreePO, involving a self-guided rollout algorithm that views sequence generation as a tree-structured searching process. Composed of dynamic tree sampling policy and fixed-length segment decoding, TreePO leverages local uncertainty to warrant additional branches. By amortizing computation across common prefixes and pruning low-value paths early, TreePO essentially reduces the per-update compute burden while preserving or enhancing exploration diversity. Key contributions include: (1) a segment-wise sampling algorithm that alleviates the KV cache burden through contiguous segments and spawns new branches along with an early-stop mechanism; (2) a tree-based segment-level advantage estimation that considers both global and local proximal policy optimization. and (3) analysis on the effectiveness of probability and quality-driven dynamic divergence and fallback strategy. We empirically validate the performance gain of \modelname on a set reasoning benchmarks and the efficiency saving of GPU hours from 22% up to 43% of the sampling design for the trained models, meanwhile showing up to 40% reduction at trajectory-level and 35% at token-level sampling compute for the existing models. While offering a free lunch of inference efficiency, TreePO reveals a practical path toward scaling RL-based post-training with fewer samples and less compute.

Deep Learning · Robustness

Yankai Chen, Hanrong Zhang, Bowei He, Philip Yu, Xue Liu

Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of Inference-time Element Corruption. This refers to scenarios where deployed models encounter element-level degradations, such as outliers or missing components, that may distort the set representation and degrade performance. To address this, we propose SW-DRSO, a distributionally robust optimization framework tailored for sets. Rather than minimizing loss solely on the observed training data, SW-DRSO optimizes the worst-case expected loss over a family of plausible inference-time variations. We further introduce a barycentric adversary that transforms the intractable search for worst-case corrupted sets into a differentiable and efficient optimization process. Extensive experiments across four tasks demonstrate that SW-DRSO effectively enhances robustness against corruption while maintaining high overall performance.

Deep Learning · Other Representation Learning

Xinyu Mao, Junsi Li, Haoji Zhang, Yu Liang, Ming Sun

Fine-grained cross-modal alignment is pivotal for multimodal reasoning yet remains limited by Semantic Sparsity Bias—a fundamental asymmetry where dense visual signals are under-represented by sparse textual captions. This disparity leads to the inadvertent suppression of contextually vital visual regions (patch redundancy) and hinders precise concept grounding (patch ambiguity). While Multimodal Large Language Models (MLLMs) offer rich descriptive capabilities, their naive integration often induces semantic drift due to inconsistencies with sparse ground-truth captions. To systematically resolve these challenges, we present the Semantic-Enhanced Patch Slimming (SEPS) framework. Central to SEPS is a novel Dual-Granularity Semantic Calibration mechanism, which synthesizes a Holistic Visual-Linguistic Anchor from MLLMs to synergize with original sparse queries. This mechanism transforms patch selection into a semantic consensus process, ensuring that retained patches satisfy both local discriminability and global contextual integrity. Furthermore, we propose a Salience-Guided Metric Aggregation strategy to mitigate the similarity dilution effect inherent in global mean pooling, thereby amplifying highly-relevant patch-word correspondences. Extensive experiments on Flickr30K and MS-COCO datasets demonstrate that SEPS surpasses existing state-of-the-art approaches across diverse backbones, delivering significant performance gains in text-to-image retrieval tasks. The complete implementation is available at https://anonymous.4open.science/r/SEPS/.

Applications · Computer Vision

Zhiwei Zheng, Dongyin Hu, Mingmin Zhao

Radio Frequency (RF) sensing has emerged as a powerful, privacy-preserving alternative to vision-based methods for various perception tasks. However, building high-quality RF datasets in dynamic and diverse environments remains a major challenge. To address this, we introduce WaveVerse, a prompt-based, scalable framework that simulates realistic RF signals from generated indoor scenes with human motions guided by spatial paths, enabling diverse and feasible behaviors without manual trajectory design. WaveVerse features a language-guided 4D world generator and a physics-based signal simulator that enables realistic simulation of RF signals in diverse environments. It employs a phase-coherent ray tracer that preserves both spatial and temporal phase consistency. The simulated signals show high fidelity on phase-sensitive benchmarks, and closely align with both real-world collected measurements and simulations from a proprietary electromagnetic solver. When used for data augmentation, WaveVerse consistently improves performance in downstream tasks like RF imaging and human activity recognition, with gains that grow with the amount of simulated data and surpass existing methods.

Reinforcement Learning · Inverse

Tian Xu, Chenyang Wang, Xiaochen Zhai, Ziniu Li, Yi-Chen Li, Yang Yu

Adversarial imitation learning (AIL) achieves high-quality imitation by mitigating compounding errors in behavioral cloning (BC), but often exhibits training instability due to adversarial optimization. To avoid this issue, a class of non-adversarial Q-based imitation learning (IL) methods, represented by IQ-Learn, has emerged and is widely believed to outperform BC by leveraging online environment interactions. However, this paper revisits IQ-Learn and demonstrates that it provably reduces to BC and suffers from an imitation gap lower bound with quadratic dependence on horizon, therefore still suffering from compounding errors. Theoretical analysis reveals that, despite using online interactions, IQ-Learn uniformly suppresses the Q-values for all actions on states uncovered by demonstrations, thereby failing to generalize. To address this limitation, we introduce a primal-dual framework for distribution matching, yielding a new Q-based IL method, Dual Q-DM. The key mechanism in Dual Q-DM is incorporating Bellman constraints to propagate high Q-values from visited states to unvisited ones, thereby achieving generalization beyond demonstrations. We prove that Dual Q-DM is equivalent to AIL and can recover expert actions beyond demonstrations, thereby mitigating compounding errors. To the best of our knowledge, Dual Q-DM is the first non-adversarial IL method that is theoretically guaranteed to eliminate compounding errors. Experimental results further corroborate our theoretical results.

Deep Learning · Large Language Models

Yingru Li, Jiawei Xu, Ziniu Li, Jiacai Liu, Wei Liu, Yuxuan Tong, Longtao Zheng, Zhenghai Xue, Yaxiang Zhang, Tianle Cai 等

Reinforcement Learning for Large Language Models (LLMs) often suffers from training collapse in long-horizon tasks due to exploding gradient variance. To mitigate this, baseline is commonly introduced for advantage computation; however, traditional value models remain difficult to optimize, and standard group-based baselines overlook sequence heterogeneity. Although classic optimal baseline theory can achieve global variance reduction, it neglects token heterogeneity and requires prohibitive gradient-based computation. In this work, we derive the Optimal Token Baseline (OTB) from first principles, proving that gradient updates should be weighted inversely to their cumulative gradient norm. To ensure efficiency, we propose the Logit-Gradient Proxy that approximates the gradient norm using only forward-pass probabilities. Our method achieves training stability and matches the performance of large group sizes ($N=32$) with only $N=4$, reducing token consumption by over 65\% across single-turn and tool-integrated reasoning tasks.

Deep Learning · Large Language Models

Junying Chen, Xinyuan Xie, Ziniu Li, Benyou Wang

Domain adaptation transforms general-purpose LLMs into specialized experts for specific domains or tasks. This process typically follows a two-stage recipe: first, Supervised Fine-Tuning (SFT) to inject domain knowledge or induce specific behaviors (e.g., reasoning patterns), followed by Reinforcement Learning (RL) for self-improvement. However, *does RL truly require a pre-SFT as cold-start phase?* We argue that pre-SFT is inherently problematic: (1) it indiscriminately reinforces knowledge and behaviors from references regardless of whether the LLM has already acquired them, leading to distribution contraction that constrains subsequent exploration; (2) it introduces substantial overhead in multi-stage training and data curation. While our pilot studies reveal that, without pre-SFT, RL struggles to acquire off-policy knowledge from scratch, we bridge this gap with **One-stage Policy Optimization (OnePO)**. OnePO is an SFT-free paradigm that enables LLMs to selectively internalize off-policy knowledge and behaviors directly during RL evolution. Crucially, we design an **Adaptive Objective Evolution** mechanism for rapid knowledge injection and a **Teacher Retirement** mechanism that prevents off-policy anchoring. Experiments demonstrate that OnePO successfully transforms the Qwen3-8B-Base model into a high-performance medical LLM in one RL stage, achieving competitive performance on HealthBench (67.2) and other benchmarks using only 20K samples. This highlights SFT-free RL can efficiently cultivate domain experts without the need for traditional multi-stage pipelines.

Applications · Everything Else

Qiaosheng Chen, Yang Liu, Lei Li, Kai Chen, Qipeng Guo, Gong Cheng, fei yuan

While Large Language Models (LLMs) hold promise for automating science and education, generating interactive scientific demonstrations demands a complex synthesis of deep domain knowledge and precise reactive coding. Current benchmarks fail to capture this synergy, largely bifurcating into static code generation or text-only reasoning. To address this, we introduce \textsc{InteractScience}, the first benchmark dedicated to evaluating the holistic creation of interactive scientific applications. We propose a novel hybrid framework that integrates programmatic functional testing for logic verification with visually-grounded qualitative assessment for rendering fidelity. Our evaluation of 30 leading models across five disciplines reveals critical gaps in grounding scientific reasoning within interactive interfaces. By standardizing this combined capability, \textsc{InteractScience} establishes a crucial foundation for reliable AI-driven tools in science and education.

General Machine Learning · Representation Learning

Shengju Yu, Suyuan Liu, Wenhao SHAO, Siwei Wang, Dayu Hu, Yiu-ming Cheung

In multi-view clustering (MVC), conventional anchor learning based models implicitly assume a uniform distribution of anchors across clusters, which could lead to inferior representation, especially when clusters vary significantly in size, as larger clusters require more anchors so as to adequately capture their intrinsic structural complexity. To alleviate this, we design a method termed FCFMVC that explicitly encourages proportional anchor allocation. To be specific, we transfer anchor allocation to discrete sample-cluster learning via bipartite graph bridge, and then backpropagate cluster state consisting of size and dispersion degree to guide anchor assignment. This allows the model to integrate cluster cardinality awareness and structural compactness directly into anchor distribution. On the other hand, we regard anchors as pseudo-samples, introduce an anchor-cluster indicator matrix on each view, and directly constrain the number of anchors assigned to each cluster within a tolerance margin. These two paths are further coupled through anchor-sample label alignment, and collaboratively facilitate anchor generation from fine-grained (anchor-level) to coarse-grained (cluster-level) structures. Besides, the entire optimization operation with linear time and space cost makes FCFMVC well-scalable to large-scale tasks. Experiments on datasets with diverse scales confirm the effectiveness of our FCFMVC.

Social Aspects · Accountability, Transparency, and Interpretability

Advaith Malladi, Shashank Srivastava

Natural-language explanations are widely used to interpret machine learning models, yet many prioritize human plausibility over accurately reflecting or predicting model behavior. Prior approaches often rely on human-written rationales, producing post-hoc explanations that neither align with the model’s decision function nor generalize. We introduce OPEX , a natural-language explanation model that directly optimizes for behavioral faithfulness: the ability of an explanation to reflect and predict a model’s observable input–output behavior. OPEX is trained using reinforcement learning with Group Relative Policy Optimization (GRPO), optimizing two complementary metrics: recoverability, which measures whether explanations recover model predictions on seen examples, and simulatability, which measures prediction of model behavior on unseen inputs. Across structured and text-based tasks, OPEX achieves high simulatability (∼0.85) and recoverability (∼0.99), outperforming GPT-4o, LLaMA-3.3-70B, and human-written explanations; despite having a 8B-parameter backbone. Human user studies show a 15% improvement in classification accuracy over competent baselines