论文检索

Applications · Computer Vision

Borui Kang, Jinrui Gu, Tao Feng, Qi Fan, Yinghuan Shi, Lei Wang, Wenbin Li, Yang Gao

Continual learning of Vision-Language Model (VLM) aspires to empower foundation models with new expertise without compromising their universal zero-shot capabilities. However, this pursuit faces a critical ''dual-forgetting'' challenge: the catastrophic forgetting of newly acquired classes (Incremental Knowledge Forgetting, IKF) and the insidious erosion of foundational zero-shot capabilities (Pre-trained Knowledge Forgetting, PKF). Existing evaluations often ignore PKF or assess it via confounded protocols where positive transfer on semantically similar domains creates an illusion of retention, masking severe foundational degradation. To address this, we propose the Dual-Forgetting-Aware Class-Incremental Learning (DFA-CIL) framework and the Similarity-Calibrated Retention (SCR) metric. Unlike standard averaging, SCR utilizes the frozen pre-trained feature space to inversely weight performance based on semantic similarity, effectively mitigating the confounding gains to stress-test foundational stability. Building on this, we propose DFA-MoE, a functionally heterogeneous Parameter-Efficient Fine-Tuning (PEFT) method. DFA-MoE architecturally decouples optimization objectives by assigning a momentum-enhanced contrastive expert for feature alignment, and separate plasticity experts that combine classification with auxiliary contrastive learning to adapt to new tasks while retaining historical knowledge. Extensive experiments demonstrate that our framework uncovers the hidden fragility of existing methods and achieves a state-of-the-art balance in preserving both incremental and pre-trained knowledge.

Social Aspects · Alignment

Yonghui Yang, Wenjian Tao, Jilong Liu, Xingyu Zhu, Junfeng Fang, Huang Weibiao, Le Wu, Richang Hong, Tat-Seng Chua

Safety alignment remains brittle under domain shift and noisy preference supervision. Existing robust alignment methods predominantly focus on data uncertainty in alignment data, while being less effective at addressing failures caused by optimization-induced fragility. In this work, we revisit robustness for safety alignment from an optimization geometry perspective, highlighting optimization-induced fragility as a complementary factor to data-space uncertainty. To address this, we propose ShaPO, a geometry-aware preference optimization framework that enforces worst-case alignment objectives via selective geometry control. We instantiate ShaPO at two levels: Token-level ShaPO stabilizes likelihood-based surrogate optimization, while Reward-level ShaPO enforces reward-consistent optimization and provides improved robustness under noisy supervision. Across diverse safety benchmarks and noisy preference settings, ShaPO consistently improves safety robustness over popular preference optimization methods. Moreover, ShaPO composes cleanly with data-robust objectives, yielding additional gains and empirically supporting the proposed optimization-geometry perspective. Our results highlight optimization geometry as a complementary and actionable axis for robust LLM safety alignment. The code is available at \url{https://anonymous.4open.science/r/ShaPO-D1B0}.

Deep Learning · Generative Models and Autoencoders

Sayedmoslem Shokrolahi, Jae-Mo Kang, Il-Min Kim

Text-to-image (T2I) diffusion models (DMs) have achieved remarkable generative quality but still exhibit the risk to produce biased and inappropriate images. A promising line of prior work aims to mitigate this issue by learning interpretable and linearly controllable concepts from semantic spaces, such as the U-Net bottleneck. However, these methods rely entirely on the bottleneck layer in U-Net and therefore cannot be directly applied to modern ViT-based DMs, including FLUX and PixArt. In this work, we present a model agnostic framework for discovering interpretable and linearly controllable semantic attributes across any T2I DMs backbone. We first show that multi-modal attention heads in ViT-based DMs encode interpretable and (near) linear semantic structures similar to those in the U-Net bottleneck. Based on this insight, we introduce a method that learns external concept vectors, which are added to the multi-modal attention heads for ViT-based DMs or to the bottleneck layer for U-Net-based DMs, while keeping the pretrained model frozen. Experiments across SDXL, SD3.5, PixArt, and FLUX demonstrate that these concept vectors provide interpretability, linearity, and highly improved fairness while preserving visual fidelity. The code is included in the supplementary material.

Applications · Everything Else

Ali Noshad, Zishan Zheng, Yinjun Wu

To reduce LLM costs and latency, semantic caching systems must accurately identify when a new prompt matches a cached one. Current methods often rely on simplistic similarity measures, which limit their effectiveness. We introduce MVR-cache, a novel semantic caching approach that significantly improves retrieval accuracy by integrating Multi-Vector Retrieval (MVR). MVR-cache is built upon a learnable segmentation model that intelligently splits prompts, enabling fine-grained similarity comparisons via MaxSim. We derive the model’s training objectives from a rigorous theoretical analysis. This can ensure that optimizing these objectives directly maximizes cache hits under strict correctness constraints. To solve the resulting non-differentiable combinatorial optimization problem, we leverage a reinforcement learning-based training strategy with the theoretically grounded objectives as the reward. Experimental results on established benchmarks across diverse tasks confirm that in comparison to the state-of-the-art, MVR-cache consistently increases the cache hit rates by up to 25% while maintaining the same correctness guarantees.

Applications · Neuroscience, Cognitive Science

Shunchi Zhang, Jin Lu, Chuanyang Jin, Yichao Zhou, Zhining Zhang, Tianmin Shu

Effective real-world assistance requires AI agents with robust Theory of Mind (ToM): inferring human mental states from their behavior. Despite recent advances, several key challenges remain, including (1) online inference with robust uncertainty updates over multiple hypotheses; (2) efficient reasoning suitable for real-time assistance; and (3) the lack of ground-truth mental state annotations in real-world domains. We address these challenges by introducing *MindZero*, a self-supervised reinforcement learning framework that trains language models to perform efficient and robust online mental reasoning. During training, the model is rewarded for generating mental state hypotheses that maximize the likelihood of observed actions estimated by a planner, similar to model-based ToM reasoning. This method thus eliminates the need for explicit mental state annotations. After training, *MindZero* internalizes model-based reasoning, and performs mental inference in a single forward pass at test time. We evaluate *MindZero* in four challenging mental reasoning and AI assistance domains. *MindZero* matches the robustness of explicit model-based methods while significantly accelerating inference, outperforming state-of-the-art methods by a large margin. These results demonstrate that mental reasoning can be learned as a self-supervised skill, bridging the gap between robustness and efficiency in ToM modeling.

Deep Learning · Large Language Models

Shuaidi Wang, Zhan Zhuang, HUANG Ruping, Yu Zhang

Diffusion Large Language Models (dLLMs) have emerged as a promising non-autoregressive generative paradigm. Given the prohibitive computational cost of full fine-tuning, Parameter-Efficient Fine-Tuning (PEFT) has become the standard approach. However, existing PEFT methods (e.g., LoRA), originally tailored for autoregressive models, rely on static parameters that are agnostic to the noise level. Consequently, they ignore the intrinsic dynamics of the diffusion process, where input distributions and generation difficulty shift significantly along the denoising trajectory, rendering them suboptimal for dLLMs. To address this, we propose **N**oise-**a**ware Low-**R**ank **A**daptation (NaRA), which introduces a low-rank core matrix generated by a lightweight, globally shared hypernetwork conditioned on the noise level. This design enables the update matrices to vary continuously along the diffusion process while keeping parameter and latency overhead negligible. We provide a theoretical justification for the proposed NaRA framework and empirically demonstrate consistent improvements over noise-agnostic baselines across commonsense reasoning, mathematical reasoning, and code generation benchmarks.

General Machine Learning · Evaluation

Yuanzhi Liu, Shousheng Zhao, Bo Zhou, Kongming Liang, Zhanyu Ma

Evaluation benchmarks play a central role in assessing vision–language models (VLMs). However, most existing multimodal benchmarks are static, making them increasingly vulnerable to data contamination, temporal staleness, and high construction costs. In this work, we introduce MMBench-Live, a multi-agent-driven dynamic multimodal benchmark that supports continuous updates without human in the loop. MMBench-Live is maintained through an end-to-end automated pipeline that integrates structured benchmark description, real-time data acquisition, and verifiable question–answer (QA) generation, enabling scalable, live, and low-cost benchmark updates. To ensure reliable evaluation across versions, we further propose a distribution-consistent updating strategy based on semantic task interpretation and feedback-driven data collection and filtering. Based on MMBench-Live, we conduct systematic evaluations of multiple open-source VLMs and analyze their performance, cross-version consistency, and data contamination, providing empirical evidence for the effectiveness of the proposed dynamic benchmark updating framework.

Deep Learning · Large Language Models

Kaustubh Vijaykumar Pethkar, Ziyang Xiong, Zuofeng Shang, Yingcong Li

Continual incorporation of new knowledge is essential for the long-term evolution of large language models (LLMs). Existing approaches typically rely on parameter-update algorithms to mitigate catastrophic forgetting, yet they suffer from fundamental limitations: 1) forgetting is unavoidable as the amount of newly injected knowledge grows; and 2) model updates are often irreversible. As modern LLMs become increasingly expressive, it is natural to question whether large-scale weight updates are necessary for acquiring a small amount of new knowledge. In this work, we propose a principled framework that models autoregressive language generation as a Markov process over tokens, where model memory is represented by a Markov transition matrix. Under this formulation, incorporating new knowledge/tokens corresponds to extending the state space, and preserving existing transitions guarantees retention of previously learned knowledge. We then prove a sample complexity bound for incorporating new tokens via a token-to-dictionary mapping strategy. In particular, for learning the transition behavior of each new token, the required number of samples scales linearly with the number of existing tokens it is mapped to. To realize this mapping, we propose an embedding-tuning algorithm that requires minimal parameter updates and induces zero forgetting. Experimental results further demonstrate the effectiveness of our method and validate our theoretical findings.

Social Aspects · Accountability, Transparency, and Interpretability

Liuyuan Wen, Xun Zhu, Lihao Huang, Wenbin Li, Yang Gao

Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual stream geometry during multi-operand addition, we identify the **Iso-Raw-Sum Trajectory (IRST)**, a topological manifold where representations are anchored by semantic digits and modulated by continuous carry fibers. We propose the **Noisy Quantization Model**, which frames arithmetic errors as *topological slippages* caused by internal neural noise pushing a continuous, latent *carry potential* across quantization thresholds. This geometric framework further elucidates *probe versatility*, explaining how lightweight probes can disentangle conflicting latent signals (such as ground truth versus hallucination) from a single activation vector. Finally, we validate these insights through a geometric consistency check method that effectively detects and corrects these quantization failures during inference.

Ali Rad, Khashayar Filom, Darioush Keivan, Peyman Mohajerin Esfahani, Ehsan Kamalinejad

Reinforcement learning with verifiable rewards (RLVR) trains a policy by verifying sampled completions and reinforcing higher-scoring outputs, but practical verifiers (e.g., incomplete unit tests or noisy judges) are prone to false positives and false negatives. We ask when such noise merely slows learning and when it reverses it. Modeling GRPO-style RLVR as a bandit over recurring \emph{reasoning modes}, we derive mean-field replicator-style (natural-selection) flow on the probability simplex. The dynamics decouples into within-correct-mode competition and a one-dimensional evolution for the mass on incorrect modes, whose drift is determined solely by Youden's index $J=\mathrm{TPR}-\mathrm{FPR}$. This yields a sharp phase transition: when $J>0$, the incorrect mass is driven toward extinction (learning); when $J=0$, the process is neutral; and when $J<0$, incorrect modes amplify until they dominate (anti-learning and collapse). In the learning regime $J>0$, noise primarily rescales convergence time (``rate, not fate''). Experiments on verifiable programming tasks under synthetic noise reproduce the predicted $J=0$ boundary. Beyond noise, the framework offers a general lens for analyzing RLVR stability, convergence, and algorithmic interventions.

Deep Learning · Large Language Models

Xiangyang Zhu, Yuan Tian, Qi Jia, Kaiwei Zhang, Zicheng Zhang, Chunyi Li, Zijian Chen, Lu Sun, Kaiyuan Ji, Renrui Zhang 等

The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address thess problems, we introduce \textbf{SafeSci}, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises \textbf{SafeSciBench}, a multi-disciplinary benchmark with 0.25M samples, and \textbf{SafeSciTrain}, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 21 advanced LLMs, revealing critical vulnerabilities in current models. Finally, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of models. Our work provides both a diagnostic tool and a practical resource for building safer scientific AI systems.

Houcheng Jiang, Junfeng Fang, Jiaxin Wu, Tianyu Zhang, Chen Gao, Xiang Wang, Xiangnan He, Yang Deng

Weak-to-strong generalization provides a promising paradigm for scaling large language models (LLMs) by training stronger models on samples from aligned weaker ones, without requiring human feedback or explicit reward modeling. However, its robustness and generalization are hindered by the noise and biases in weak-model outputs, which limit its applicability in practice. To address this challenge, we leverage implicit rewards, which approximate explicit rewards through log-likelihood ratios, and reveal their structural equivalence with Contrastive Decoding (CD), a decoding strategy shown to reduce noise in LLM generation. Building on this connection, we propose \textbf{Contrastive Weak-to-Strong Generalization (ConG)}, a framework that employs contrastive decoding between pre- and post-alignment weak models to generate higher-quality samples. This approach enables more reliable capability transfer, denoising, and improved robustness, substantially mitigating the limitations of traditional weak-to-strong methods. Empirical results across different model families confirm consistent improvements, demonstrating the generality and effectiveness of ConG. Taken together, our findings highlight the potential of ConG to advance weak-to-strong generalization and provide a promising pathway toward AGI. Our code is available at: \url{https://anonymous.4open.science/r/ConG/}

Probabilistic Methods · Graphical Models

Mustafa Seddiqi, Tiberiu Popa, Marta Kersten-Oertel

Learning effective value heuristics for combinatorial search is difficult, as prior methods rely on surrogate supervision or costly downstream search to assess progress. We introduce a trajectory-aware probabilistic framework that models uncertainty in cost-to-go labels instead of treating them as fixed targets. Heuristic learning is cast as inference over state trajectories using an HMM-style model (Rabiner, 1989), where estimated depth-change dynamics define transitions and forward–backward inference yields soft supervision. To evaluate heuristic quality without search, we propose a large-scale local ranking metric that measures a model’s ability to order neighboring states. On the Rubik’s Cube, our approach consistently improves local ranking accuracy and downstream search performance under matched computational budgets.

Applications · Time Series

Chen-Hui Song, Shuoling Liu, Liyuan Chen

While deep learning has revolutionized financial forecasting through sophisticated architectures, the design of the supervision signal itself is rarely scrutinized. We challenge the canonical assumption that training labels must strictly mirror inference targets, uncovering the Label Horizon Paradox: the optimal supervision signal often deviates from the prediction goal, shifting across intermediate horizons governed by market dynamics. We theoretically ground this phenomenon in a dynamic signal-noise trade-off, demonstrating that generalization hinges on the competition between marginal signal realization and noise accumulation. To operationalize this insight, we propose a bi-level optimization framework that autonomously identifies the optimal proxy label within a single training run. Extensive experiments on large-scale financial datasets demonstrate consistent improvements over conventional baselines, thereby opening new avenues for label-centric research in financial forecasting.

Reinforcement Learning · Deep RL

Chengxuan Lu, Zhenquan Zhang, Shukuan Wang, Qunzhi Lin, Baigui Sun, Yang Liu

Post-training with reinforcement learning (RL) has recently shown strong promise for advancing multimodal agents beyond supervised imitation. However, RL remains limited by poor data efficiency, particularly in settings where interaction data are scarce and quickly become outdated. To address this challenge, GIPO (Gaussian Importance sampling Policy Optimization) is proposed as a policy optimization objective based on truncated importance sampling, replacing hard clipping with a log-ratio–based Gaussian trust weight to softly damp extreme importance ratios while maintaining non-zero gradients. Theoretical analysis shows that GIPO introduces an implicit, tunable constraint on the update magnitude, while concentration bounds guarantee robustness and stability under finite-sample estimation. Experimental results show that GIPO achieves state-of-the-art performance among clipping-based baselines across a wide range of replay buffer sizes, from near on-policy to highly stale data, while exhibiting superior bias–variance trade-off, high training stability and improved sample efficiency. Code is provided in supplementary material.

Junfeng Fang, Nachuan Chen, Houcheng Jiang, Dan Zhang, Xiangnan He, Tat-Seng Chua, Xiang Wang

Large language models are increasingly deployed in streaming scenarios, rendering conventional post-hoc safeguards ineffective as they fail to interdict unsafe content in real-time. While streaming safeguards based on token-level supervised training could address this, they necessitate expensive annotations and suffer from severe overfitting. In this work, we challenge the paradigm that streaming safety must rely on token-level supervised training. Instead, it is an inherent capability of well-trained post-hoc safeguards, as they already encode token-level risk signals in hidden representations. Hence, we introduce NExT-Guard, a training-free framework that achieves streaming safeguards by monitoring interpretable latent features from Sparse Autoencoders (SAEs). It uses pretrained SAEs from publicly available base LLMs, enabling flexible, low-cost deployment without token-level supervision. Experimental results show that NExT-Guard outperforms both post-hoc and streaming safeguards based on supervised training, with superior robustness across models, SAE variants, and risk scenarios. These results make NExT-Guard a universal and scalable paradigm for real-time safety, accelerating the practical deployment of streaming safeguards.

Deep Learning · Other Representation Learning

Max Hartman, Vidhata Jayaraman, Moulik Choraria, Akhil Bhimaraju, Lav Varshney

Vision–language models achieve incredible performance across a wide range of tasks, but their large size makes inference costly. Recent work has shown that multimodal processing contains significant redundancies, making it possible to skip certain layers with minimal performance loss. Yet current pruning techniques remain ad-hoc, relying on heuristics or hyperparameter sweeps rather than principled criteria for determining when layer skipping is beneficial. In this paper, we propose a unified framework that characterizes the redundancy conditions under which pruning can enhance efficiency without sacrificing performance. Central to our approach are experimentally verifiable and interpretable notions of redundancy that can be evaluated without requiring downstream task performance as a metric. Applying this framework, we corroborate prior findings that both early and late vision tokens are redundant across models, and we validate our conditions by showing they align with actual performance degradation. Beyond these empirical results, our framework provides a theoretically grounded understanding of redundancy in VLMs and unifies many of the ideas behind modern layer-skipping techniques.

Deep Learning · Large Language Models

Zirui Zhu, Yong Liu, Yang Luo, Hailun Xu, Kanchan Sarkar, Kun Xu, Yang You

Reward models play a fundamental role in aligning large language models with human preferences. Existing methods predominantly follow two paradigms: scalar discriminative preference models, which are efficient but lack interpretability, and generative judging models, which offer richer reasoning at the cost of higher computational overhead. We observe that the log-probability margin between verdict tokens strongly correlates with prediction correctness, providing a reliable proxy for instance difficulty without additional inference cost. Building on this insight, we propose CAMEL, a confidence-gated reflection framework that performs a lightweight single-token preference decision first and selectively invokes reflection only for low-confidence instances. To induce effective self-correction, we train the model via reinforcement learning with counterfactual prefix augmentation, which exposes the model to diverse initial verdicts and encourages genuine revision. Empirically, CAMEL achieves state-of-the-art performance on three widely used reward-model benchmarks with 82.9\% average accuracy, surpassing the best prior model by 3.2\% and outperforming 70B-parameter models using only 14B parameters, while establishing a strictly better accuracy-efficiency Pareto frontier.

Deep Learning · Large Language Models

Wenwen Qiang, Ziyin Gu, Jiahuan Zhou, Jie Hu, Jingyao Wang, Changwen Zheng, Hui Xiong

Training stability remains a critical bottleneck for Group Relative Policy Optimization (GRPO), often manifesting as a trade-off between reasoning plasticity and general capability retention. We identify a root cause as the geometric conflict between plasticity and stability gradients, which leads to destructive interference. Crucially, we argue that deterministic projection methods are suboptimal for GRPO as they overlook the intrinsic stochasticity of group-based gradient estimates. To address this, we propose Probabilistic Conflict Resolution (PCR), a Bayesian framework that models gradients as random variables. PCR dynamically arbitrates conflicts via an uncertainty-aware ``soft projection'' mechanism, optimizing the signal-to-noise ratio. Extensive experiments demonstrate that PCR significantly smooths the training trajectory and achieves superior performance in various reasoning tasks.

Ali Rad, Khashayar Filom, Darioush Keivan, Peyman Mohajerin Esfahani, Ehsan Kamalinejad

Reinforcement learning with verifiable rewards (RLVR) is a cornerstone of post-training for large reasoning models, yet widely used algorithms such as Group Relative Policy Optimization (GRPO) often exhibit \textbf{diversity collapse}. We provide a geometric diagnosis by formalizing GRPO as a dynamical flow on the probability simplex. Under a mode-based coarse-graining of rollouts, we show that GRPO induces a \textbf{collision field} over correct modes, monotonically pushing towards simplex vertices and thus yielding a \textbf{winner-take-all} regime. To address this systematically, we introduce \textbf{G$^2$RPO (Geometric GRPO)}, which reshapes RLVR via principled \textbf{vector-field editing}. Concretely, we intervene at the advantage level by adding granularity bonuses inversely proportional to mode probabilities, encouraging underrepresented correct modes. The bonus has a natural geometric interpretation, and its potential performance side effects can be mitigated, thereby avoiding the usual accuracy--diversity trade-off. In experiments with 7B and 14B models trained on a math reasoning task and evaluated on \textbf{AIME 2024/2025}, GRPO loses up to \textbf{57\%} of active correct modes. In contrast, G$^{2}$RPO increases active correct-mode coverage by \textbf{172\%--205\%}, reduces concentration on any single correct mode, prevents the late-stage \emph{entropy crash}, and improves \texttt{pass@1} by \textbf{+1.4} to \textbf{+7.9} points relative to GRPO. Overall, diversity is not merely a regularizer but a \textbf{geometric property} to be controlled to improve the model without trapping it in a single dominant strategy.