论文检索

Social Aspects · Alignment

Aakash Sen Sharma, Debdeep Sanyal, Manodeep Ray, Vivek Srivastava, Shirish Karande, Murari Mandal

Post-training alignment of large language models (LLMs) relies on large-scale human annotations guided by policy specifications that change over time. Cultural shifts, value reinterpretations, and regulatory or industrial updates make static alignment increasingly brittle. As policies evolve, deployed models can diverge from current alignment objectives, creating an Alignment–Reality Gap that is difficult to audit or correct. Existing remediation typically requires re-annotation under revised guidelines, which introduces systematic challenges, including guideline ambiguity, annotator interpretation drift, and reduced consistency at scale. We introduce TRACE (Triage and Re-align by Alignment Conflict Evaluation), a framework that transforms re-alignment into a structured optimization problem over existing data without requiring fresh human annotation. Leveraging a stronger model as a proxy judge, TRACE operates via a three-stage pipeline: (1) triaging preference pairs into inversion, suppression, or retention categories based on alignment conflicts; (2) computing an alignment impact score via bi-level optimization to prioritize high-leverage samples; and (3) executing updates using a hybrid objective that combines relational losses (e.g., IPO) for preference inversion and punitive losses (e.g., NPO) for response suppression. Experiments on Qwen2.5-7B, Gemma-2-9B, and Llama-3.1-8B demonstrate robust re-alignment on synthetic benchmarks and the PKU-SafeRLHF dataset without degrading general utility. This work provides a scalable approach for LLM realignment under evolving data annotation policies and alignment guidelines.

Deep Learning · Large Language Models

Lingjing Kong, Xin Liu, Guangyi Chen, Martin Q. Ma, Xiangchen Song, Yuekai Sun, Mikhail Yurochkin, Taylor W. Killian, Russ Salakhutdinov, Kun Zhang 等

Reinforcement learning (RL) has emerged as a key mechanism for transforming LLMs into robust reasoners. While supervised fine-tuning (SFT) often limits models to the distribution of observed reasoning traces, RL post-training significantly improves performance on out-of-distribution (OOD) tasks that require unfamiliar recombinations of familiar steps. We argue that this improvement is driven by **compositional generalization**, which we formalize through a **Hierarchical Latent Selection Model**. In this framework, reasoning traces are generated by a cascade of discrete latent selection variables corresponding to reusable atomic modules, including both skills (local operations) and routing mechanisms (how intermediate information is selected, reused, and composed). We theoretically show that RL’s exploratory nature provides sufficient coverage to identify latent structure and enable compositional generalization. We design controlled experiments to validate this theory. Our results demonstrate that RL can extract atomic modules from compound traces and recombine them to solve new configurations. Moreover, we find that training on compound traces can yield stronger generalization than training on isolated atomic modules. Finally, we investigate relations between SFT and RL and identify an effective protocol in which SFT ensures coverage of all atomic modules, while RL focuses on novel compositions beyond the SFT support to encourage exploration.

Theory · Reinforcement Learning and Planning

Yike Zhao, Onno Eberhard, Malek khammassi, Ali Sayed, Michael Muehlebach

The family of linear recurrent neural networks has shown strong performance as recurrent memory units in partially observable reinforcement learning. We provide a theoretical justification for their empirical effectiveness by constructing and studying two linear filters: (i) the first exactly reproduces the pre–softmax logits of the belief vector in a hidden Markov model (HMM) under a deterministic transition matrix, thereby serving as a sufficient statistic for optimal policy learning, (ii) the second achieves vanishing state-decoding error under a nearly deterministic transition matrix, thus reducing state ambiguity to near zero. The results extend to action-controlled HMMs, where the corresponding linear filters become time-varying with action-dependent dynamics. We illustrate our main results through numerical experiments and further show that the constructed linear filter serves as a strong feature extractor in a small reinforcement learning game.

Applications · Computer Vision

Yinuo Zhang, Zian Cheng, Ziya Zhao, Zongyu Li, Bingshuo Liu, Qingbin Liu, Junxian Cai, chen, Zhiying Tu, Dianhui Chu 等

Photoshop is a professional image editing software whose complex multi-level menus, fine-grained operations, and layer-based non-destructive editing pose substantial challenges for automated agents. Existing GUI benchmarks and methods primarily target web interfaces and short-horizon, low-complexity tasks, falling short in modeling the multi-step decision-making and semantic understanding required by professional graphic software. We introduce PSBench, the first benchmark specifically designed for image editing in Adobe Photoshop, consisting of 600 human-annotated tasks across three difficulty levels, with tasks drawn from official tutorials and popular real-world workflows. PSBench covers core functionalities such as canvas adjustment, layer manipulation, and filter application, and provides fine-grained evaluation metrics tailored to each task category. Our experiments show that even the state-of-the-art system, Agent S3, achieves a success rate of only 18.09\% on difficult tasks, indicating that GUI agents still face considerable challenges in operating complex professional software. Furthermore, human-in-the-loop evaluations reveal that MLLMs, when serving as interactive assistants, can significantly improve novice users’ task completion rates and reduce operation time.

Applications · Neuroscience, Cognitive Science

Raymond Khazoum, Daniela Fernandes, Aleksandr Krylov, Qin Li, Stephane Deny

Mental rotation—the ability to compare objects seen from different viewpoints—is a fundamental example of mental simulation and spatial world modeling in humans. Here we propose a mechanistic model of human mental rotation, leveraging recent advances in deep, equivariant, and neuro-symbolic learning. Our model consists of three stacked components: (1) an equivariant neural encoder, producing 3D spatial representations of objects from images, (2) a neuro-symbolic object encoder, deriving symbolic objects descriptions from these spatial representations, and (3) a neural decision agent, comparing these symbolic descriptions to prescribe rotation simulations in 3D latent space via a recurrent pathway. Our model design is guided by the abundant experimental literature on mental rotation, which we complemented with experiments in VR where participants could at times manipulate the objects to compare. Our model captures well the performance, response times and behavior of participants in our and others' experiments, and through ablation studies we demonstrate the necessity of each component. Our work adds to a recent collection of deep neural models of human spatial reasoning, further demonstrating the potency of integrating deep, equivariant, and symbolic representations to model the human mind.

Deep Learning · Large Language Models

Shvat Messica, Jiawen Zhang, Kevin Li, Theodoros Tsiligkaridis, Marinka Zitnik

Time series reasoning tasks increasingly start from a natural language question and require targeted analysis of time series. Relevant evidence may be global or confined to a few short segments, so the model must decide what to inspect. Most existing methods compress the full series into a fixed representation before inference, preventing question-adaptive analysis. We introduce ARTIST, an approach that formulates time-series reasoning as a sequential decision problem and trains models to interleave reasoning with adaptive temporal segment selection. ARTIST uses a controller-reasoner architecture and reinforcement learning to optimize segment selection based on answer correctness, allowing the model to actively acquire task-relevant information during inference. We evaluate ARTIST on six time-series reasoning benchmarks against large language models, vision-language models, and prior time series reasoning systems. ARTIST improves average accuracy by 6.46 percentage points over the strongest model, with the largest gains on rare event localization and multi-segment evidence accumulation. Supervised fine-tuning improves performance, and reinforcement learning yields further gains by optimizing question-adaptive segment selection. Across datasets, ARTIST achieves higher accuracy while using a smaller fraction of the input time series, highlighting the importance of learned, selective data utilization for time series reasoning.

Theory · Learning Theory

Hubert Chan, Elaine Shi, Mengshi Zhao, Mingxun Zhou

Modern federated and streaming learning systems often release intermediate models, so privacy must hold for the full trajectory under adaptive interaction. Motivated by participation privacy, we study single-edit neighboring user streams, where one insertion/deletion shifts all subsequent updates and defeats standard Hamming-neighbor continual-release analyses. We give an auditable modular recipe. A randomized buffering wrapper emits bins of size $[U,2U]$, reducing single-edit streams to a Hamming-style per-bin update stream with explicit backlog/delay guarantees, where $U$ is calibrated by the privacy parameters $(\varepsilon,\delta)$. We then prove a certification theorem for independently decomposable (prefix-causal, fresh-noise) continual mechanisms: any non-adaptive Hamming-neighbor DP proof lifts to adaptive inputs. Together, these ingredients yield trajectory-level $(\varepsilon,\delta)$-DP for single-edit streams using standard primitives (e.g., tree prefix sums), with an explicit privacy--latency link via $U$. Streaming DP-SGD experiments validate the privacy-utility-latency tradeoffs and the induced delay distributions.

Applications · Computer Vision

YinYun Yan, Liping Zhang, Tingran Wang, Jiaxin Deng, Changshuo Wang, Limin Jiang, Shanwei Gao, Xin Ning

Current linear State-Space Models for 3D point clouds typically rely on 1D serialization (e.g., Hilbert curves) for global modeling. Such rigid ordering disrupts spatial continuity in dense scenes, introducing what we term Serialization Bias. We propose AnIsoNet, a framework that decouples anisotropic geometry from isotropic semantics via two dedicated modules: Local Anisotropy Geometric Modeling (LAGM) and Global Isotropy Semantic Aggregation (GISA). LAGM employs ellipsoidal encoding to capture local directionality without global order. GISA adapts to geometric characteristics via two modes: content-based accumulation (Identity Mode) for dense scenes and Morton serialization for sparse objects. This eliminates redundant multi-view scanning while maintaining O(N) complexity. Experiments show that avoiding artificial serialization in dense scenes achieves 82.62 % mIoU on S3DIS (surpassing PCM by 3.0 %), while Morton serialization for sparse objects achieves 94.21 % OA on ScanObjectNN (+1.6 %). On ScanNetV2, we reach 78.52 % mIoU, surpassing PTv3 (77.5 %) without pre-training. We achieve these results with only 12.2 M parameters (26.4 % of PTv3's）

Applications · Computer Vision

Haotian Wu, Gen Li, Di You, Pier Luigi Dragotti, Deniz Gunduz

We propose a frequency-aware perceptual optimization framework for low-complexity image compression, realized as a **Re**alism-enhanced **Re**gion-based **I**mplicit **C**odec (Re2IC). Re2IC models visual perception via saliency-guided region partitioning and local–global perceptual modulation. To enhance realism under complexity constraints, we introduce wavelet–Wasserstein distortion (WA-WD), a frequency-decomposed perceptual distortion that balances fidelity and realism through subband-aware modeling and provides a more reliable approximation than standard Wasserstein distortion. Together, these designs enable fine-grained spatial–spectral optimization, allowing Re2IC to achieve superior rate–perception trade-offs, outperforming generative codecs such as HiFiC while using less than $1\%$ of their decoding cost. Extensive experiments demonstrate state-of-the-art perceptual performance among overfitted codecs. Beyond compression, WA-WD serves as a standalone, tunable perceptual metric with strong alignment to human preference (Pearson 94.6\%, Spearman 92.3\%) and competitive performance across multiple IQA benchmarks.

Applications · Computer Vision

Kun Ouyang, Yuanxin Liu, Xinhao Li, Linli Yao, Xiangyu Zeng, Haoning Wu, Hao Zhou, Fandong Meng, Jie Zhou, Xu SUN

Verbal logic and visual mental simulation are two essential components of human intelligence. Modern Large Language Models (LLMs) have demonstrated strong verbal reasoning capabilities through textual Chain-of-Thought (CoT) reasoning. In contrast, current Video Generation Models (VGMs) struggle with visual reasoning tasks such as spatial planning. We attribute this limitation to two fundamental gaps: (i) VGMs are predominantly trained on general-purpose video corpora emphasizing perceptual fidelity over visual reasoning, leaving reasoning abilities underdeveloped; (ii) most VGMs generate videos in a single pass without mechanisms to explore alternative reasoning trajectories and to revise intermediate errors. Motivated by these limitations, we introduce **ReaForest**, a framework that fosters the reasoning capacity of VGMs in spatial planning through both training-time activation and inference-time scaling. ReaForest comprises three key components: **(1)** ReaGen-27k, a dataset covering diverse spatial planning tasks that require multi-step reasoning, which activates basic reasoning capabilities of VGMs for spatial planning; **(2)** Reflective Entropy-Aware Test-Time Scaling (ReaTTS), an inference framework that evolves multiple reasoning branches while enabling failure recovery; **(3)** Hierarchical constraint verification, which provides actionable feedback for ReaTTS based on decomposed constraints. Extensive experiments demonstrate that ReaForest substantially surpasses advanced textual reasoning models (e.g., Gemini-2.5-Pro) and video generation models (e.g., Sora-2). ReaForest exhibits emergent properties including self-correction, parallel thinking, and scalable reasoning, advancing VGMs toward human-like visual mental simulation.

Social Aspects · Accountability, Transparency, and Interpretability

Jeremy Herbst, Jae Hee Lee, Stefan Wermter

Mixture-of-Experts (MoE) architectures have become the dominant choice for scaling Large Language Models (LLMs), activating only a subset of parameters per token. While primarily adopted for computational efficiency, it remains an open question whether their sparsity makes them inherently easier to interpret than dense feed-forward networks (FFNs). We compare MoE experts and dense FFNs using $k$-sparse probing and find that expert neurons are consistently less polysemantic, with the gap widening as routing becomes sparser. This suggests that sparsity pressures both individual neurons and entire experts toward monosemanticity. Leveraging this finding, we *"zoom out"* from the neuron to the expert level as a more effective unit of analysis. We validate this approach by automatically interpreting hundreds of experts. This analysis allows us to resolve the debate on specialization: experts are neither broad domain specialists (e.g., biology) nor simple token-level processors. Instead, they function as fine-grained task experts, specializing in linguistic operations or semantic tasks (e.g., retrieving specific bioinformatics software tools). Our findings suggest that MoEs are inherently interpretable at the expert level, providing a clearer path toward large-scale model interpretability.

General Machine Learning · Evaluation

Yueqi Zhang, Jin Hu, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yiwei Li, Jiayi Shi, Chuyi Tan, Ji Zhang, Boyuan Pan 等

The prohibitive cost of evaluating Large Language Models (LLMs) necessitates efficient alternatives to full-scale benchmarking. Prevalent approaches address this by identifying a small coreset of items to approximate full-benchmark performance. However, existing methods must estimate a reliable item profile from response patterns across many source models, which becomes statistically unstable when the source pool is small. This dependency is particularly limiting for newly released benchmarks with minimal historical evaluation data. We argue that discrete correctness labels are a lossy view of the model's decision process and fail to capture information encoded in hidden states. To address this, we introduce \textsc{RepCore}, which aligns heterogeneous hidden states into a unified latent space to construct representative coresets. Using these subsets for performance extrapolation, \textsc{RepCore} achieves precise estimation accuracy with as few as ten source models. Experiments on five benchmarks and over 200 models show consistent gains over output-based baselines in ranking correlation and estimation accuracy. Spectral analysis further indicates that the aligned representations contain separable components reflecting broad response tendencies and task-specific reasoning patterns.

Applications · Computer Vision

Qingdong He, Chaoyi Wang, Peng TANG, Yifan Yang, Xiaobin Hu

Video subtitle removal is essential for content localization and media re-editing, yet existing mask-guided diffusion methods face critical limitations: training inefficiency requiring extensive annotations and full model fine-tuning, inference complexity demanding explicit mask sequences, and static prior utilization unable to adapt to quality variations. We present CLEAR (Context-aware Learning for End-to-end Adaptive subtitle Removal), a lightweight adapter-based framework addressing these challenges through three technical innovations. First, self-supervised prior learning (Stage I) extracts occlusion guidance from video pairs using pixel differences as weak supervision, eliminating annotation dependency while learning generalizable subtitle features across languages. Second, LoRA-based adaptive refinement (Stage II) enables parameter-efficient training that preserves pre-trained visual priors while achieving true mask-free end-to-end inference without external detection modules. Third, adaptive focal weighting dynamically adjusts prior influence based on local quality assessment, effectively handling diverse subtitle styles and noisy guidance signals. Extensive experiments demonstrate CLEAR's superior performance in multilingual subtitle removal while requiring only 0.77% trainable parameters, establishing a new paradigm for efficient video text removal without inference-time mask dependencies.

Applications · Health / Medicine

Shujun Xia, Haokun Lin, Yichen WU, Yinan Zhou, Zixuan Li, Zhongwei Wan, Xingrun Xing, Yefeng Zheng, Xiang Li, Caifeng Shan 等

LLMs hold great promise for healthcare applications, but fast-changing medical knowledge can quickly make their outputs outdated or inaccurate, limiting use in high-stakes settings. Model editing can update LLMs without full retraining, but parameter-based methods often break locality and are risky in medicine, making retrieval-based editing a better fit. However, applying model editing methods to the medical domain has two key challenges: (1) retrieval-based methods suffer from representation overlap within the medical knowledge space that causes inaccurate retrieval and reduces editing accuracy; (2) existing medical editing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world applications. To address these challenges, we construct MedVersa, an expanded benchmark that evaluates single and batch edits across broader medical coverage under strict locality constraints. We then propose MedREK, a retrieval-based editing framework that integrates a shared query–key module for precise matching with an attention-based prompt encoder for informative guidance. Experiments across various medical benchmarks show that our MedREK consistently improves key metrics and provides the first validated solution for batch editing in medical LLMs.

Deep Learning · Generative Models and Autoencoders

Yunqiao Yang, Haokun Lin, Guanzhong Wu, Ying Wei

Recent advances in image generation have been largely driven by image tokenization, which compresses raw pixels into compact latent representations. While existing tokenizers excel at preserving low-level visual details through reconstruction-based training, they often lack explicit semantic guidance, which limits their ability to capture semantically structured representations and thus hinders their performance on downstream tasks like image generation. To overcome this limitation, we propose a novel tokenization framework that incorporates high-level semantics through two key innovations: (1) a text-integrated encoder that jointly processes images and textual descriptions to produce semantically enriched latent representations, and (2) a concept-guided training objective that leverages sparse autoencoders to decompose pre-trained vision-language model features to a semantic concept space, employing sparse and disentangled concept indices for guidance. Our approach achieves strong alignment with semantic concepts, maintaining high reconstruction fidelity with an rFID of 1.39 on ImageNet, while achieving a gFID of 2.65 on the class-conditional image generation task and 10.73 on the text-to-image generation task. By infusing high-level semantic structures into low-level visual fidelity, our method bridges the reconstruction-generation divide and drives generative modeling as a powerful foundation.

Optimization · Discrete and Combinatorial Optimization

Xinyi Ke, Kai Li, Junliang Xing, Yifan Zhang, Jian Cheng

Large language models (LLMs) have enabled rapid progress in automatic heuristic discovery (AHD), yet most existing methods are predominantly limited by static evaluation against fixed instance distributions, leading to potential overfitting and poor generalization under distributional shifts. We propose Algorithm Space Response Oracles (ASRO), a game-theoretic framework that reframes heuristic discovery as a program level co-evolution between solver and instance generator. ASRO models their interaction as a two-player zero-sum game, maintains growing strategy pools on both sides, and iteratively expands them via LLM-based best-response oracles against mixed opponent meta-strategies, thereby replacing static evaluation with an adaptive, self-generated curriculum. Across multiple combinatorial optimization domains, ASRO consistently outperforms static-training AHD baselines built on the same program search mechanisms, achieving substantially improved generalization and robustness on diverse and out-of-distribution instances.

Qiuhong Shen, Xingyi Yang, Xinyin Ma, Gongfan Fang, Xinchao Wang

Diffusion-based large language models generate text by gradually filling in masked tokens, yet they remain slow because they usually decode only a few tokens per step. Parallel decoding, which unmasks multiple tokens simultaneously, promises acceleration but often degrades quality when too many tokens are predicted at once. We identify the root cause: when decoding is viewed as iterative token grouping, overly permissive grouping places interdependent tokens in the same step, violates the conditional independence assumption, and amplifies reliance on noisy context even when the top prediction is already correct. We introduce SPEED, a framework that enlarges safe parallel groups through complementary training and inference designs. At training time, a sharpened teacher distillation objective selectively aligns the student to teacher-correct positions using a temperature-scaled KL term together with a masked language modeling loss, producing a student that assigns more probability mass to correct token identities and elevates more positions above the decoding threshold. At inference time, Slow–Fast Decoding partitions tokens by sensitivity to revealed context using token-wise Jensen–Shannon Divergence computed with and without access to the preceding block, decoding high-sensitivity tokens jointly while deferring low-sensitivity tokens until sufficient context resolves them. Through extensive experiments, our framework attains up to 12.2× speedup on LLaDA-8B-Instruct and 6.7× on Dream-7B-Instruct with accuracy close to greedy decoding across standard reasoning and code benchmarks.

Deep Learning · Other Representation Learning

Jaeyung Kim, YoungJoon Yoo

Vector Quantized Variational Autoencoder (VQ-VAE) has become a fundamental framework for learning discrete representations in image modeling. However, VQ-VAE models must tokenize entire images using a finite set of codebook vectors, and this capacity limitation restricts their ability to capture rich and diverse representations. In this paper, we propose ArcCosine Additive Margin VQ-VAE (ArcVQ-VAE), a novel vector quantization framework that introduces a spherical angular-margin prior (SAMP) for the codebook of a conventional VQ-VAE. The proposed SAMP consists of Ball-Bounded Norm Regularization, which constrains all codebook vectors within a time-dependent Euclidean ball, and ArcCosine Additive Margin Loss, which encourages greater angular separability among latent vectors. This formulation promotes more discriminative and uniformly dispersed latent representations within the constrained space, thereby enabling codebook vectors to capture richer information and leading to improved codebook utilization. Experimental results on standard image reconstruction and generation tasks show that ArcVQ-VAE outperforms baseline models in terms of reconstruction accuracy, representation diversity, and sample quality.

Applications · Computer Vision

Kangpeng Hu, Quansen Sun, Tao Wang, Yinghui Sun, Weihao Zhang

Regional refinement for interactive segmentation is of great necessity to ensure the fidelity of segmented pixels nearby user-prompted locations, which specifies a local window (\ie, focus view) for the latest click after a global prediction, where local pixels are revisited and optimized using numerous refining structures. Previous methods either utilize a two-stage pipeline to estimate the focus view or manually preset a fixed scope for all clicks, while the former is time-consuming, the latter fails to capture the correlation among click position, object geometry, and focus intensity. In this paper, we inherit the core idea of FCFI \cite{wei2023focused} and dedicate a one-stage framework characterized with \textbf{E}laborate \textbf{F}ocus \textbf{P}rior (EFPNet). Concretely, EFPNet outputs an erroneous mask \wrt historical feedback and newly-placed click in an end-to-end manner, which deduces precise focus region according to its max-connected component, followed with feedback correction considering image, feature and mask affinity. We further design a clicked-with-focus mechanism for efficient feedback integration. Extensive studies on four benchmarks have revealed outstanding performance of EFPNet for both efficacy and computational overhead.

Theory · Deep Learning

Hongkang Li, Songtao Lu, Xiaodong Cui, Pin-Yu Chen, Meng Wang

The Mamba model has gained significant attention for its computational advantages over Transformer-based models, while achieving comparable performance across a wide range of language tasks. Like Transformers, Mamba exhibits in-context learning (ICL) capabilities, i.e., making predictions for new tasks based on a prompt containing input-label pairs and a query, without requiring fine-tuning. Despite its empirical success, the theoretical understanding of Mamba remains limited, largely due to the nonlinearity introduced by its gating mechanism. To the best of our knowledge, this paper presents the first theoretical analysis of the training dynamics of a one-layer Mamba model, which consists of a linear attention component followed by a nonlinear gating layer, and its ICL generalization on unseen binary classification tasks, even when the prompt includes additive outliers. Our analysis shows that Mamba leverages the linear attention layer to select informative context examples and uses the nonlinear gating layer to suppress the influence of outliers. By establishing and comparing to the analysis of linear Transformers under the same setting, we show that although Mamba may require more training iterations to converge, it maintains accurate predictions even when the proportion of outliers exceeds the threshold that a linear Transformer can tolerate. These theoretical findings are supported by empirical experiments.