论文检索

Applications · Computer Vision

Zhiwei Zheng, Dongyin Hu, Mingmin Zhao

Radio Frequency (RF) sensing has emerged as a powerful, privacy-preserving alternative to vision-based methods for various perception tasks. However, building high-quality RF datasets in dynamic and diverse environments remains a major challenge. To address this, we introduce WaveVerse, a prompt-based, scalable framework that simulates realistic RF signals from generated indoor scenes with human motions guided by spatial paths, enabling diverse and feasible behaviors without manual trajectory design. WaveVerse features a language-guided 4D world generator and a physics-based signal simulator that enables realistic simulation of RF signals in diverse environments. It employs a phase-coherent ray tracer that preserves both spatial and temporal phase consistency. The simulated signals show high fidelity on phase-sensitive benchmarks, and closely align with both real-world collected measurements and simulations from a proprietary electromagnetic solver. When used for data augmentation, WaveVerse consistently improves performance in downstream tasks like RF imaging and human activity recognition, with gains that grow with the amount of simulated data and surpass existing methods.

Reinforcement Learning · Inverse

Tian Xu, Chenyang Wang, Xiaochen Zhai, Ziniu Li, Yi-Chen Li, Yang Yu

Adversarial imitation learning (AIL) achieves high-quality imitation by mitigating compounding errors in behavioral cloning (BC), but often exhibits training instability due to adversarial optimization. To avoid this issue, a class of non-adversarial Q-based imitation learning (IL) methods, represented by IQ-Learn, has emerged and is widely believed to outperform BC by leveraging online environment interactions. However, this paper revisits IQ-Learn and demonstrates that it provably reduces to BC and suffers from an imitation gap lower bound with quadratic dependence on horizon, therefore still suffering from compounding errors. Theoretical analysis reveals that, despite using online interactions, IQ-Learn uniformly suppresses the Q-values for all actions on states uncovered by demonstrations, thereby failing to generalize. To address this limitation, we introduce a primal-dual framework for distribution matching, yielding a new Q-based IL method, Dual Q-DM. The key mechanism in Dual Q-DM is incorporating Bellman constraints to propagate high Q-values from visited states to unvisited ones, thereby achieving generalization beyond demonstrations. We prove that Dual Q-DM is equivalent to AIL and can recover expert actions beyond demonstrations, thereby mitigating compounding errors. To the best of our knowledge, Dual Q-DM is the first non-adversarial IL method that is theoretically guaranteed to eliminate compounding errors. Experimental results further corroborate our theoretical results.

Deep Learning · Large Language Models

Yingru Li, Jiawei Xu, Ziniu Li, Jiacai Liu, Wei Liu, Yuxuan Tong, Longtao Zheng, Zhenghai Xue, Yaxiang Zhang, Tianle Cai 等

Reinforcement Learning for Large Language Models (LLMs) often suffers from training collapse in long-horizon tasks due to exploding gradient variance. To mitigate this, baseline is commonly introduced for advantage computation; however, traditional value models remain difficult to optimize, and standard group-based baselines overlook sequence heterogeneity. Although classic optimal baseline theory can achieve global variance reduction, it neglects token heterogeneity and requires prohibitive gradient-based computation. In this work, we derive the Optimal Token Baseline (OTB) from first principles, proving that gradient updates should be weighted inversely to their cumulative gradient norm. To ensure efficiency, we propose the Logit-Gradient Proxy that approximates the gradient norm using only forward-pass probabilities. Our method achieves training stability and matches the performance of large group sizes ($N=32$) with only $N=4$, reducing token consumption by over 65\% across single-turn and tool-integrated reasoning tasks.

Deep Learning · Large Language Models

Junying Chen, Xinyuan Xie, Ziniu Li, Benyou Wang

Domain adaptation transforms general-purpose LLMs into specialized experts for specific domains or tasks. This process typically follows a two-stage recipe: first, Supervised Fine-Tuning (SFT) to inject domain knowledge or induce specific behaviors (e.g., reasoning patterns), followed by Reinforcement Learning (RL) for self-improvement. However, *does RL truly require a pre-SFT as cold-start phase?* We argue that pre-SFT is inherently problematic: (1) it indiscriminately reinforces knowledge and behaviors from references regardless of whether the LLM has already acquired them, leading to distribution contraction that constrains subsequent exploration; (2) it introduces substantial overhead in multi-stage training and data curation. While our pilot studies reveal that, without pre-SFT, RL struggles to acquire off-policy knowledge from scratch, we bridge this gap with **One-stage Policy Optimization (OnePO)**. OnePO is an SFT-free paradigm that enables LLMs to selectively internalize off-policy knowledge and behaviors directly during RL evolution. Crucially, we design an **Adaptive Objective Evolution** mechanism for rapid knowledge injection and a **Teacher Retirement** mechanism that prevents off-policy anchoring. Experiments demonstrate that OnePO successfully transforms the Qwen3-8B-Base model into a high-performance medical LLM in one RL stage, achieving competitive performance on HealthBench (67.2) and other benchmarks using only 20K samples. This highlights SFT-free RL can efficiently cultivate domain experts without the need for traditional multi-stage pipelines.

Applications · Everything Else

Qiaosheng Chen, Yang Liu, Lei Li, Kai Chen, Qipeng Guo, Gong Cheng, fei yuan

While Large Language Models (LLMs) hold promise for automating science and education, generating interactive scientific demonstrations demands a complex synthesis of deep domain knowledge and precise reactive coding. Current benchmarks fail to capture this synergy, largely bifurcating into static code generation or text-only reasoning. To address this, we introduce \textsc{InteractScience}, the first benchmark dedicated to evaluating the holistic creation of interactive scientific applications. We propose a novel hybrid framework that integrates programmatic functional testing for logic verification with visually-grounded qualitative assessment for rendering fidelity. Our evaluation of 30 leading models across five disciplines reveals critical gaps in grounding scientific reasoning within interactive interfaces. By standardizing this combined capability, \textsc{InteractScience} establishes a crucial foundation for reliable AI-driven tools in science and education.

General Machine Learning · Representation Learning

Shengju Yu, Suyuan Liu, Wenhao SHAO, Siwei Wang, Dayu Hu, Yiu-ming Cheung

In multi-view clustering (MVC), conventional anchor learning based models implicitly assume a uniform distribution of anchors across clusters, which could lead to inferior representation, especially when clusters vary significantly in size, as larger clusters require more anchors so as to adequately capture their intrinsic structural complexity. To alleviate this, we design a method termed FCFMVC that explicitly encourages proportional anchor allocation. To be specific, we transfer anchor allocation to discrete sample-cluster learning via bipartite graph bridge, and then backpropagate cluster state consisting of size and dispersion degree to guide anchor assignment. This allows the model to integrate cluster cardinality awareness and structural compactness directly into anchor distribution. On the other hand, we regard anchors as pseudo-samples, introduce an anchor-cluster indicator matrix on each view, and directly constrain the number of anchors assigned to each cluster within a tolerance margin. These two paths are further coupled through anchor-sample label alignment, and collaboratively facilitate anchor generation from fine-grained (anchor-level) to coarse-grained (cluster-level) structures. Besides, the entire optimization operation with linear time and space cost makes FCFMVC well-scalable to large-scale tasks. Experiments on datasets with diverse scales confirm the effectiveness of our FCFMVC.

Social Aspects · Accountability, Transparency, and Interpretability

Advaith Malladi, Shashank Srivastava

Natural-language explanations are widely used to interpret machine learning models, yet many prioritize human plausibility over accurately reflecting or predicting model behavior. Prior approaches often rely on human-written rationales, producing post-hoc explanations that neither align with the model’s decision function nor generalize. We introduce OPEX , a natural-language explanation model that directly optimizes for behavioral faithfulness: the ability of an explanation to reflect and predict a model’s observable input–output behavior. OPEX is trained using reinforcement learning with Group Relative Policy Optimization (GRPO), optimizing two complementary metrics: recoverability, which measures whether explanations recover model predictions on seen examples, and simulatability, which measures prediction of model behavior on unseen inputs. Across structured and text-based tasks, OPEX achieves high simulatability (∼0.85) and recoverability (∼0.99), outperforming GPT-4o, LLaMA-3.3-70B, and human-written explanations; despite having a 8B-parameter backbone. Human user studies show a 15% improvement in classification accuracy over competent baselines

Deep Learning · Large Language Models

Ziniu Li, Congliang Chen, Tianyun Yang, Tian Ding, Ruoyu Sun, Ge Zhang, Wenhao Huang, Zhiquan Luo

Large Language Models (LLMs) can improve via reinforcement learning by generating trajectories to discover better solutions. This exploration process represents an investment of finite GPU compute to obtain learning signals. However, current methods typically allocate a small, uniform budget to every task, which is inefficient and ineffective: easy tasks consistently succeed while difficult tasks consistently fail. For policy optimization algorithms such as Group Relative Policy Optimization (GRPO), both edge cases produce zero gradients, resulting in wasted computation. We address this by reframing exploration budget allocation as a resource optimization problem. Viewing each task's exploration as an `"item'' with a distinct "learning value'' and "computational cost'', we establish a connection to the classical knapsack problem and derive an optimal assignment rule. When applied to GRPO, our method increases the ratio of effective gradients by 40\%. As a computational "free lunch'', it enables substantially larger budgets (e.g., 93) for challenging tasks—allocations that would be expensive under a uniform allocation framework. These efficiency gains translate to meaningful improvements on mathematical reasoning benchmarks, with average gains of 2--4 points and peak gains of 9 points. Notably, achieving comparable performance with traditional homogeneous allocation would require approximately $2\times$ the computational resources.

Deep Learning · Other Representation Learning

Xiaolong Chen, Fuyuan Xiao, Xiaohong Zhang, Zehong Cao, Chin-teng Lin

Out-of-Distribution (OOD) detection requires accurately classifying In-Distribution (ID) samples while effectively distinguishing anomalous OOD data. However, existing methodologies predominantly rely on real-valued magnitude features, neglecting the semantic richness embedded in phase information, and often lack a systematic theoretical framework for quantitively modeling uncertainty. To address this dual limitation of incomplete feature representation and insufficient uncertainty modeling, the Trustworthy Quantum Evidence Neural Network (TrustworthyQENN) is proposed, a novel framework bridging complex-valued representation learning with Generalized Quantum Evidence Theory (GQET). Specifically, Supervised Complex-Valued Contrastive Learning (SCVCL) is proposed to synchronize amplitude distributions with phase correlations, thereby enforcing high intra-class compactness and inter-class separability for ID data. A quantum evidence generation mechanism based on GQET is subsequently devised, where the OOD state is formally grounded as the quantum empty set within a Hilbert space. Furthermore, the Generalized Quantum Evidential Combination Rule (GQECR) is leveraged to fuse multi-view evidence, thereby achieving trustworthy inference. Extensive experiments on the MSTAR, EuroSAT, and FUSAR-Ship benchmarks substantiate the superiority of TrustworthyQENN, which achieves a peak AUROC of 95.94% on the MSTAR dataset while consistently outperforming state-of-the-art methods across all evaluated scenarios.

Applications · Robotics

Guoxin Lian, Shuo Wang, Yucheng Wang, Yongcai Wang, Maiyue Chen, kaihui.wang, Bo Zhang, Zhizhong Su, Deying Li, Zhaoxin Fan

Vision-Language Navigation (VLN) requires agents to follow natural language instructions in partially observed 3D environments, motivating map representations that aggregate spatial context beyond local perception. However, most existing approaches rely on hand-crafted maps constructed independently of the navigation policy. We argue that maps should instead be learned representations shaped directly by navigation objectives rather than exhaustive reconstructions. Based on this insight, we propose MapDream, a map-in-the-loop framework that formulates map construction as autoregressive bird’s-eye-view (BEV) image synthesis. The framework jointly learns map generation and action prediction, distilling environmental context into a compact three-channel BEV map that preserves only navigation-critical affordances. Supervised pre-training bootstraps a reliable mapping-to-control interface, while the autoregressive design enables end-to-end joint optimization through reinforcement fine-tuning. Experiments on R2R-CE and RxR-CE achieve state-of-the-art monocular performance, validating task-driven generative map learning.

Deep Learning · Theory

Bolin Shen, Zhan Cheng, Neil Gong, Fan Yao, Yushun Dong

Machine Learning as a Service (MLaaS) has become a widely adopted method for delivering deep neural network (DNN) models, allowing users to conveniently access models via APIs. However, such services have been shown to be highly vulnerable to Model Extraction Attacks (MEAs). While numerous defense strategies have been proposed, verifying the ownership of a suspicious model with strict theoretical guarantees remains a challenging task. To address this gap, we introduce CREDIT a certified defense against MEAs. Specifically, we employ mutual information to quantify the similarity between DNN models, propose a practical verification threshold, and provide rigorous theoretical guarantees for ownership verification based on this threshold. We extensively evaluate our approach on several mainstream datasets and achieve state-of-the-art performance. Our implementation is publicly available at: \url{https://anonymous.4open.science/r/CREDIT}.

General Machine Learning · Evaluation

Xiangyu Jiang

Large language models increasingly rely on retrieval-augmented generation (RAG) to ground responses in external corpora. Yet, even with strong retrievers, generated statements can remain unsupported, and the resulting citations are often not reliable indicators of evidence. We introduce CiteGuard, a RAG decoding layer that treats sentence-level factuality as a multiple-testing problem and combines conformal calibration with false-discovery-rate control. CiteGuard converts claim–evidence scores into p-values for the null hypothesis "unsupported" and uses BH/BY procedures to decide which claims to keep (with citations) and which to abstain on. On FEVER and Natural Questions, CiteGuard reduces the false-discovery rate among accepted claims from 28–31% (vanilla RAG) to below 10% at α=0.10, while retaining 86–92% of supported claims. This yields a user-controlled risk budget: practitioners can trade off faithfulness and coverage via α, with finite-sample guarantees under standard exchangeability assumptions.

Deep Learning · Attention Mechanisms

Zhuoqing Song, Peng Sun, Huizhuo Yuan, Quanquan Gu

In standard causal attention, each token's query, key, and value (QKV) are static and encode only preceding context. We introduce CAuSal aTtention with Lookahead kEys (CASTLE), an attention mechanism that continually updates each token's keys as the context unfolds. We term these updated keys lookahead keys because they belong to earlier positions yet integrate information from tokens that appear later relative to those positions, while strictly preserving the autoregressive property. Although the mechanism appears sequential, we derive a mathematical equivalence that avoids explicitly materializing lookahead keys at each position and enables efficient parallel training. On language modeling benchmarks, CASTLE consistently outperforms standard causal attention across model scales, reducing validation perplexity and improving performance on a range of downstream tasks.

Deep Learning · Attention Mechanisms

Timon Klein, Jonas Kusch, Sebastian Sager, Stefan Schnake, Steffen Schotthöfer

The pursuit of reducing the memory footprint of the self-attention mechanism in multi-headed self attention (MHA) spawned a rich portfolio of methods, e.g., group-query attention (GQA) and multi-head latent attention (MLA). The methods leverage specialized low-rank factorizations across embedding dimensions or attention heads. From the point of view of classical low-rank approximation, these methods are unconventional and raise questions of which objects they really approximate and how to interpret the low-rank behavior of the resulting representations. To answer these questions, this work proposes a generalized view on the weight objects in the self-attention layer and a factorization strategy, which allows us to construct a parameter efficient scheme, called Tucker Attention. Tucker Attention requires an order of magnitude fewer parameters for comparable validation metrics, compared to GQA and MLA, as evaluated in LLM and ViT test cases. Additionally, Tucker Attention~encompasses GQA, MLA, MHA as special cases and is fully compatible with flash-attention and rotary position embeddings (RoPE). This generalization strategy yields insights of the actual ranks achieved by MHA, GQA, and MLA, and further enables simplifications for MLA.

Reinforcement Learning · Deep RL

Shaojun Xu, Xiaoling Zhou, Yihan Lin, Yapeng Meng, Xinglong Ji, Luping Shi, Rong Zhao

Model-Based Reinforcement Learning (MBRL) leverages latent imagination for sample efficiency, yet remains constrained by **Historical Tethering**: imagination is typically initialized from observed states. This creates a learning asymmetry, where the world model’s manifold discovery outpaces the policy's sparse-reward optimization. We propose **Mind Dreamer (MD)**, a framework that operationalizes **Active Counterfactual Reasoning (ACR)** to transcend Markovian continuity. MD reformulates discovery as the minimization of a global Relay Manifold Expected Free Energy (R-EFE); by invoking a latent-space $do$-operator, MD utilizes an adversarial generator to synthesize non-continuous **latent jumps** to epistemic blind spots that are physically plausible yet cognitively challenging. To resolve the credit assignment paradox across these spatial ruptures, we derive the **Relay Value Function (RVF)** and **Relay Uncertainty Function (RUF)**. These potentials treat synthesized anchors as latent bridges, propagating pragmatic and epistemic value through a principled Bellman-style formulation. Notably, we prove that the Uncertainty propagation across discontinuities necessitates a quadratic discount $\gamma^2$, establishing a formal epistemic horizon. Theoretically, MD acts as an optimal importance sampler that expands the manifold's spectral gap, reducing the hitting time to critical bottleneck states. Empirically, MD achieves a **1.67$\times$ average speedup** over DreamerV3 on DeepMind Control Suite, reaching **8.8$\times$** in sparse-reward tasks.

Probabilistic Methods · Monte Carlo and Sampling Methods

Benjamin Patrick Evans, Sumitra Ganesh, Leo Ardon

Large language models (LLMs) achieve remarkable generative performance, yet their output quality is dependent on the decoding strategy. While sampling-based methods (e.g., top-$k$, nucleus) and search-and-select based methods (e.g., beam search, best-of-$n$, majority voting) can improve upon greedy decoding, both approaches suffer from limitations: sampling commits to a single path, while search often expends excessive computation regardless of task complexity. We introduce **Entropy-informed DEcodiNg** (EDEN), a plug-and-play, model-agnostic decoding framework that adaptively allocates computation based on the model’s own uncertainty, approximating higher width beam search with *fewer generations required*. At each generation step, EDEN estimates the entropy of the output token distribution and adjusts the branching factor monotonically with the entropy, expanding more candidates in high-entropy regions and following a greedier path in low-entropy regions, improving sample efficiency. Experiments across complex tasks, including mathematical reasoning, code generation, and scientific questions, demonstrate that EDEN consistently improves output quality over existing decoding strategies, achieving better trade-offs between accuracy and token generations than fixed beam search approaches. By treating next token selection as a noisy maximisation problem, we prove that branching factors monotone in entropy are guaranteed to find better (i.e. more probable) continuations than any fixed branching factor within the same total computation budget, motivating the dynamic branching.

Social Aspects · Accountability, Transparency, and Interpretability

Charles Ye, Jasmine Cui, Dylan Hadfield-Menell

Language models remain vulnerable to prompt injection attacks despite extensive safety training. We trace this failure to role confusion: models assign roles based on how text sounds, not where it actually comes from. We design novel role probes to capture how models internally identify “who is speaking.” These reveal why prompt injection works: untrusted text that imitates a role inherits that role’s authority. We test this insight by injecting fabricated reasoning into user prompts and tool outputs, achieving average success rates of 60% on StrongREJECT and 61% on agent exfiltration, across multiple open- and closed-weight models with near-zero baselines. Strikingly, the degree of internal role confusion strongly predicts attack success before generation begins. Our findings reveal a fundamental gap: security is defined at the interface but authority is assigned in latent space. More broadly, we introduce a unifying, mechanistic framework for prompt injection, demonstrating that diverse prompt-injection attacks exploit the same underlying role-confusion mechanism. Code available at: https://anonymous.4open.science/r/role-science-B522.

Deep Learning · Foundation Models

Sujung Hong, Chanyong Yoon, Seong Jae Hwang

Large diffusion vision–language models (LDVLMs) have recently demonstrated competitive performance on multimodal tasks, emerging as a promising alternative to autoregressive models. They enable parallel decoding for efficient inference and leverage bidirectional attention to capture global context. Despite these advances, their behavior under long-form generation remains underexplored. In this work, we show that existing LDVLMs suffer from repetitive generation and lead to degraded visual grounding. Through analysis, we identify two underlying causes of these failures. First, repetitive generation originates from a mask token prior. Because generation tokens are initialized as mask tokens, their hidden representations progressively drift toward a shared prior direction over generation steps. Second, a fundamental misalignment exists between the positional attention bias and the iterative unmasking process. This discrepancy suppresses the model's attention toward informative visual tokens, leading to degradation in visual grounding. Based on these insights, we propose a training-free approach that mitigates both issues. Specifically, we introduce Mask Prior Suppression and Monotonic RoPE Scaling, which mitigate mask prior drift and positional attention collapse during decoding. Experiments on general multimodal benchmarks and visual grounding tasks demonstrate improvements over baseline LDVLMs, with robust gains on long-form description benchmarks. Overall, our results show that these failures can be effectively addressed with a lightweight, plug-and-play strategy that requires no additional training and generalizes across diverse LDVLM architectures.

Reinforcement Learning · Everything Else

Fang YUAN, Quanjun Yin, Siqi Shen, Yuxiang Xie, Junqiang Yang, Long Qin, Junjie Zeng, Qinglun Li

Unsupervised Environment Design (UED) offers a promising paradigm for improving reinforcement learning generalization by adaptively shaping training environments, but it requires reliable environment evaluation to remain effective. However, existing UED methods evaluate environments using indirect proxy signals such as regret, value-based errors, or Monte Carlo, which suffer from bias, high variance, or substantial computational overhead and fail to reflect agent realized learning progress. To address these limitations, we propose Parameter Change Environment Design (PACE), which evaluates an environment through the policy parameter change induced by training on that environment, directly grounding environment selection in realized learning progress. Specifically, PACE assigns environment value using a first-order approximation of the policy optimization objective, where the improvement induced by an environment is proportional to the squared $\ell_2$ norm of the corresponding parameter update, enabling low-variance and computation-efficient evaluation without additional rollouts. Experiments on MiniGrid and Craftax show that PACE consistently outperforms established UED baselines, achieving higher IQM and smaller Optimality Gap on OOD evaluations, including an IQM of 96.4% and an Optimality Gap of 17.2% on MiniGrid.

Social Aspects · Safety

Nivya Talokar, Ayush Kumar Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

LLM-based agents increasingly execute real-world workflows via tools and memory. Granting LLMs such powers enables ill-intended adversaries to likewise use these agents to carry out complex misuse scenarios. Existing agent-misuse benchmarks largely test single-prompt instructions, leaving a gap in measuring how agents end up helping with harmful or illegal tasks over multiple turns. We introduce **STING** (*Sequential Testing of Illicit N-step Goal execution*), an automated red-teaming framework that constructs a step-by-step illicit plan grounded in a benign persona and iteratively probes a target agent with adaptive follow-ups, using judge agents to track phase completion. We further introduce an analysis framework that models multi-turn red-teaming as a time-to-first-jailbreak random variable, enabling analysis tools like discovery curves, hazard-ratio attribution by attack language, and a new metric: Restricted Mean Jailbreak Discovery. Across AgentHarm scenarios, STING yields substantially higher illicit-task completion than single-turn prompting and chat-oriented multi-turn baselines adapted to tool-using agents. In multilingual evaluations across six non-English settings, we find that attack success and illicit-task completion do *not* consistently increase in lower-resource languages, diverging from common chatbot findings. Overall, STING provides a practical way to evaluate and stress-test agent misuse in realistic deployment settings, where interactions are inherently multi-turn and often multilingual.