论文检索

Social Aspects · Privacy

Zhiyu Sun, Minrui Luo, Yu Wang, Tianxing He, Zhili Chen

Large language models (LLMs) are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive information. Locate-then-edit methods, as a mainstream paradigm of model editing, offer a promising solution by modifying model parameters without retraining. However, in this work, we reveal a critical vulnerability of this paradigm: the parameter updates inadvertently serve as a side channel, enabling attackers to recover the edited data. We propose a two-stage reverse-engineering attack named KSTER (KeySpaceReconsTruction-then-EntropyReduction) that leverages the low-rank structure of these updates. First, we theoretically show that the row space of the update matrix encodes a "fingerprint" of the edited subjects, enabling accurate subject recovery via spectral analysis. Second, we introduce an entropy-based prompt recovery attack that reconstructs the semantic context of the edit. Extensive experiments on multiple LLMs demonstrate that our attacks can recover edited data with high success rates. Furthermore, we propose subspace camouflage, a defense strategy that obfuscates the update fingerprint with semantic decoys. This approach effectively mitigates reconstruction risks without compromising editing utility. Our code is available at https://anonymous.4open.science/r/EditATK-26B6.

Applications · Computer Vision

Kabilan Elangovan, Daniel Ting

This position paper argues that explanation stability claims are scientifically invalid without cross-method validation. Just as statistical significance requires specifying the test statistic, stability must be validated across multiple attribution paradigms or explicitly scoped to a single method’s computational objective. In controlled chest X-ray experiments, DenseNet201, ResNet50V2, and InceptionV3 achieve >99% AUC but exhibit reversed stability rankings across attribution methods. LayerCAM ranks InceptionV3 highest (IoU 0.777), while Grad-CAM++ favors DenseNet201, reducing InceptionV3’s score by 17.3%. These findings establish that explanation stability is an emergent property of the model–method pair, not an intrinsic model trait. We call for mandatory cross-method validation in XAI research and urge that regulatory submissions specify attribution methods to avoid illusionary safety assurances.

Minrui Luo, Zhiheng Zhang

Synthetic Nearest Neighbors (SNN) provides a principled solution to causal matrix completion under missing-not-at-random (MNAR) by exploiting local low-rank structure through fully observed anchor submatrices. However, its effectiveness critically relies on sufficient data availability within each treatment level, a condition that often fails in settings with multiple or complex treatments. In this work, we propose Mixed Synthetic Nearest Neighbors (MSNN), a new entry-wise causal identification estimator that integrates information across treatment levels. We show that MSNN retains the finite-sample error bounds and asymptotic normality guarantees of SNN, while enlarging the effective sample size available for estimation. Empirical results on synthetic and real-world datasets illustrate the efficacy of the proposed approach, especially under data-scarce treatment levels.

Deep Learning · Large Language Models

Kabilan Elangovan, Jasmine Ong, Daniel Ting

This position paper argues that as Large Language Models (LLMs) increasingly consume synthetic data, parametric representations can no longer serve as reliable witnesses of factual provenance. Current architectures, which treat fluent outputs as implicitly grounded, create a critical epistemic failure mode: systems emit accurate-looking claims with no recoverable lineage to verifiable sources. We advance the position that referenceability and explicit traceability of claims to accessible evidence must be enforced as a non-negotiable system invariant. Distinct from Retrieval-Augmented Generation (RAG), which enriches generation with external context, we propose a negative safety constraint: in factual settings, no atomic claim should be emitted unless it is evidence-gated by identifiers that entail it; otherwise, the system must abstain. To operationalize this, we introduce a “separation-of-powers” architecture that decouples parametric generation from factual authorization, along with a diagnostic metric—Parametric Leakage Ratio (PLR)—to quantify ungrounded factual emissions. We conclude that enforcing a strict provenance–parametric divide is essential to prevent safety certifications from legitimizing unverifiable outputs in high-stakes domains such as healthcare.

Deep Learning · Generative Models and Autoencoders

Wenbin Duan, Yan Shu, Zhuoyuan Fu, Fangmin Zhao, Yan Li, Yaru Zhao, Binyang Li

Rectified-flow-based diffusion transformers, particularly FLUX, have demonstrated outstanding performance in high-quality image generation. However, achieving fast and accurate inversion—transforming images back to latent noise for faithful reconstruction and editing—remains a challenging bottleneck due to the discretization errors of linear solvers. This paper introduces \textbf{SlerpFlow}, a straightforward yet highly effective zero-shot approach that unlocks the full potential of FLUX for high-fidelity inversion and editing. Unlike existing approaches (e.g., RF-Solver) that rely on complex numerical approximations such as high-order Taylor expansions to correct trajectory errors, we present a geometric view based on the Manifold Hypothesis: the empirically observed trajectory curvature is not a numerical artifact, but rather serves as a necessary ``centripetal force" that constrains the flow to remain on the data manifold. Guided by this insight, SlerpFlow integrates Spherical Linear Interpolation (Slerp) to rectify flow velocity directions on the hypersphere, strictly adhering to the intrinsic curvature of the latent space. Crucially, by caching the corrected velocity for subsequent steps, SlerpFlow achieves high-precision inversion while maintaining the computational efficiency of a first-order Euler solver. Extensive experiments on FLUX-based reconstruction and editing tasks demonstrate that our method delivers superior structural consistency and lower reconstruction errors compared to state-of-the-art baselines without requiring additional training.

Deep Learning · Large Language Models

Yang Wan, Zheng Cao, Zhenhao Zhang, Zhengwen Zeng, Shuheng Shen, Changhua Meng, Linchao Zhu

Large language models excel as few-shot learners when provided with appropriate demonstrations, yet this strength becomes problematic in multi-turn agent scenarios, where LLMs erroneously mimic their own previous responses as few-shot examples. Through attention analysis, we identify \textbf{conversational inertia}, a phenomenon where models exhibit strong diagonal attention to previous responses, which is associated with imitation bias that constrains exploration. This reveals a tension when transforming few-shot LLMs into agents: longer context enriches environmental feedback for exploitation, yet also amplifies conversational inertia that undermines exploration. Our key insight is that for identical states, actions generated with longer contexts exhibit stronger inertia than those with shorter contexts, enabling construction of preference pairs without environment rewards. Based on this, we propose Context Preference Learning to calibrate model preferences to favor low-inertia responses over high-inertia ones. We further provide context management strategies at inference time to balance exploration and exploitation. Experimental results across eight agentic environments and one deep research scenario validate that our framework reduces conversational inertia and achieves performance improvements.

Deep Learning · Graph Neural Networks

Yancheng Chen, Dun Ma, Shuai Zhang, Yang Liu, Xixun Lin, Xiangyu Zhao, Wenguo Yang, Wei Chen, Chuan Zhou

Graph Foundation Models (GFMs), built upon the *Pre-training and Adaptation* paradigm, have emerged as a research hotspot in graph learning. For GNN-based GFMs, graph prompt tuning has become the prevailing adaptation method for downstream tasks. Although recent methods explain why graph prompt tuning works, how to rigorously measure its adaptation capacity remains an open problem. Addressing this problem is critical for understanding the capability limits of graph prompt tuning and for developing more powerful adaptation methods. In this paper, we propose Prismatic Space Theory (PS-Theory), a novel mathematical framework to quantify the capacity of adaptation methods, while focusing on establishing the upper bound for the adaptation capacity of graph prompt tuning. Building upon the proposed PS-Theory, we further introduce **M**essage **T**uning for **G**FMs (MTG), a lightweight approach that injects a small set of learnable message prototypes into each layer of the GNN backbone to adaptively guide message fusion without updating pre-trained weights. Through our PS-Theory, we prove that the adaptation capacity of MTG can exceed the theoretical upper bound of graph prompt tuning. Extensive experiments demonstrate that MTG consistently outperforms graph prompt baselines across diverse benchmark datasets, providing strong empirical support for our theoretical findings. Our code is available at https://anonymous.4open.science/r/MTG.

Deep Learning · Large Language Models

Jing Bi, Luchuan Song, Dingxin Zhang, Pinxin Liu, Guangyu Sun, Lianggong Wen, Weidong Cai, Chen Chen, Chenliang Xu

Vision–language models (VLMs) now support both direct Instruct and explicit-reasoning Thinking modes, but practitioners lack principled ways to decide when reasoning helps or how much computation to allocate at test time. We investigate whether VLMs encode meta-cognitive signals for adaptive inference. We derive oracle labels for two properties: (1) reasoning helpfulness—whether explicit reasoning improves accuracy, and (2) optimal generation length—the minimal token budget for correctness. Probing final-layer representations in InternVL and Qwen3-VL models, we find Thinking models encode these signals more linearly than Instruct models, suggesting reasoning-oriented training enhances meta-cognitive structure. Head-wise attribution reveals two circuits: length-control heads in lower layers, and reasoning/difficulty heads in higher layers. Causal interventions confirm these roles: scaling length heads controls output length with little accuracy loss, while scaling reasoning heads enables a perception–reasoning trade-off, improving accuracy by up to 5.3\%. These effects generalize across benchmarks. Our results show reasoning-tuned VLMs develop localized, manipulable circuits for meta-cognitive control, enabling test-time steering of computation and reasoning without retraining.

Deep Learning · Generative Models and Autoencoders

Xinbao Qiao, Xianglong Du, Wei Liu, Jingqi Zhang, Peihua Mai, Meng Zhang, Yan (James) Pang

The proliferation of recursive synthetic data training promises to alleviate data scarcity but introduces the existential risk of model collapse, wherein recursive training on synthetic data erodes distributional tails and homogenizes outputs. Current literature identifies data selection as a pivotal solution, employing verifiers to prune datasets in pursuit of synthetic samples that approximate the true data manifold. However, this approach hinges on the fragile and often unrealistic assumption that a perfect verifier possesses global distributional knowledge. In real-world scenarios characterized by data silos, such as fragmented healthcare consortia or proprietary financial institutions, this assumption is invalidated by the inherent fragmentation of knowledge. We theoretically prove that such siloed selection accelerates model collapse, driving diversity decay governed by a power law. To bridge this gap, we propose an automated filtering criterion that synergizes the sensitivity theorem with Wasserstein geometry. Specifically, multiple parties collaboratively compute geodesic interpolations and the Wasserstein Barycenter as proxy measures, without exchanging raw data. These proxies serve as a collective reference, enabling multiple parties to score synthetic data rather than relying on a single biased perspective in a data silo. Empirical results show the failure of the baseline on skewed distributions, whereas our methods effectively prevent collapse. Code available at [Anonymous Github](https://anonymous.4open.science/r/When-Sample-Selection-Bias-Precipitates-Model-Collapse-D8D4).

Applications · Chemistry, Physics, and Earth Sciences

Stefan Pranger, Bernhard Ramsauer, Oliver Hofmann, Bettina Könighofer

Scanning tunneling microscopy (STM) enables precise manipulation of individual atoms and molecules, offering a pathway to constructing nanoscale assemblies with rich quantum mechanical behavior. Despite its potential, STM-based fabrication remains limited by the inherent complexity of manipulation procedures and the extensive manual effort required. In this work, we take a substantial step toward autonomous manufacturing with STMs by introducing a novel AI-based planning framework for molecular assembly and a high-fidelity simulation environment. Our framework computes collision-free assembly plans that minimize the total distance traveled by molecules. Given an assignment of molecules to target positions, satisfiability solving is used to compute execution schedules in which each molecule has an empty corridor available when it is scheduled to move. Reinforcement learning (RL) agents then execute sequences of STM actions to manipulate molecules to their targets. We further introduce NanoAssemblyGym, a high-fidelity simulation environment for molecular manipulation built on the Gymnasium API, allowing seamless integration with existing RL libraries and workflows. Using NanoAssemblyGym, we demonstrate autonomous assembly of structures containing up to 420 molecules.

Deep Learning · Attention Mechanisms

Zeru Shi, Zhenting Wang, Fan Yang, Qifan Wang, Ruixiang Tang

We investigate the origins of massive activations in large language models (LLMs) and identify a specific layer named the Massive Emergence Layer (ME Layer), that is consistently observed across model families, where massive activations first emerge and subsequently propagate to deeper layers through residual connections. We show that, within the ME Layer both the RMSNorm and the FFN parameters jointly contribute to the emergence of massive activations. Once formed, the massive activation token representation remains largely invariant across layers, reducing the diversity of hidden representations passed to the attention module. Motivated by this limitation, we propose a simple and effective method to reduce the rigidity of the massive activation token. Our approach consistently improves LLM performance across multiple tasks, including instruction following and math reasoning, in both training free and fine tuning settings. Moreover, we show that our method mitigates attention sinks by selectively weakening their influence, elucidating their origin at the hidden state level and shedding new light on principled mitigation strategies.

Social Aspects · Everything Else

Ziming Hong, Tianyu Huang, Runnan Chen, Shanshan Ye, Mingming Gong, Bo Han, Tongliang Liu

Recent studies have extended diffusion-based instruction-driven 2D image editing pipelines to 3D Gaussian Splatting (3DGS), enabling faithful manipulation of 3DGS assets and greatly advancing 3DGS content creation. However, it also exposes these assets to serious risks of unauthorized editing and malicious tampering. Although imperceptible adversarial perturbations against diffusion models have proven effective for protecting 2D images, applying them to 3DGS encounters two major challenges: view-generalizable protection and balancing invisibility with protection capability. In this work, we propose the first editing safeguard for 3DGS, termed AdLift, which prevents instruction-driven editing across arbitrary views and dimensions by lifting strictly bounded 2D adversarial perturbations into 3D Gaussian-represented safeguard. To ensure both adversarial perturbations effectiveness and invisibility, these safeguard Gaussians are progressively optimized across training views using a tailored Lifted PGD, which first conducts gradient truncation during back-propagation from the editing model at the rendered image and applies projected gradients to strictly constrain the image-level perturbation. Then, the resulting perturbation is backpropagated to the safeguard Gaussian parameters via an image-to-Gaussian fitting operation. We alternate between gradient truncation and image-to-Gaussian fitting, yielding consistent adversarial-based protection performance across different viewpoints and generalizes to novel views. Empirically, qualitative and quantitative results demonstrate that AdLift effectively protects against state-of-the-art instruction-driven 2D image and 3DGS editing.

Applications · Time Series

Heng Zhou, Xin Sun, Chao LI

Real-world time series inherently exhibit complex temporal patterns. Within chaotic systems, significant mixing and entanglement occur between different time-varying modes. Given that time series exhibit distinctly different patterns at various sampling scales, downsampling to extract multiscale features is a common approach. However, conventional downsampling causes high-frequency components in the original signal, those exceeding the new Nyquist frequency, to undergo spectral folding. This erroneously introduces spurious low-frequency patterns, perceived as low-frequency noise, thereby leading to the ***aliasing problem***. To address this problem, we propose a Decomposition-Prevention-Fusion architecture framework called **DMANet**, which introduces the **D**ynamic **M**ultiscale **A**nti-Aliasing **Net**work. Specifically, DMANet comprises two key components: Multiscale Convolutional Downsampling, designed to capture temporal dependencies and inter-channel interactions, and an Anti-Aliasing Operation, which includes Pre-Sampling Anti-Aliasing Filtering and Post-Sampling Interpolation. These designs guarantee the fidelity of multiscale features before and after downsampling. We show that by mitigating the risk of aliasing, our proposed simple convolutional downsampling architecture achieves performance competitive with common baselines. Our codes are available at https://anonymous.4open.science/r/DMANet.

Reinforcement Learning · Deep RL

Amir Moeini, Minjae Kwon, Alper Bozkurt, Yuichi Motai, Rohan Chandra, Lu Feng, Shangtong Zhang

In-context reinforcement learning (ICRL) is an emerging RL paradigm where an agent, after pretraining, can adapt to out-of-distribution test tasks without any parameter updates, instead relying on an expanding context of interaction history. While ICRL has shown impressive generalization, safety during this adaptation process remains unexplored, limiting its applicability in real-world deployments where test-time behavior is expected to be safe. In this work, we propose SCARED: Safe Contextual Adaptive Reinforcement via Exact-penalty Dual, the first method that promotes safe adaptation of ICRL under the constrained Markov decision process framework. During the parameter-update-free adaptation process, our agent not only maximizes the reward but also keeps the accumulated cost within a user-specified safety budget. We also demonstrate that the agent actively reacts to the safety budget; with a higher safety budget, the agent behaves more aggressively, and with a lower safety budget the agent behaves more conservatively. Across challenging benchmarks, SCARED consistently enables safe and robust in-context adaptation, outperforming existing ICRL and safe meta-RL baselines.

Applications · Health / Medicine

Chunlei Li, Zixuan Zheng, Yilei Shi, Guanglu Dong, Pengfei Li, Jingliang Hu, Xiao Zhu, Lichao Mou

Mislabeled samples in training datasets severely degrade the performance of deep networks, as overparameterized models tend to memorize erroneous labels. We address this challenge by proposing a novel approach for mislabeled data detection that leverages training dynamics. Our method is grounded in the key observation that correctly labeled samples exhibit consistent entropy decrease during training, while mislabeled samples maintain relatively high entropy throughout the training process. Building on this insight, we introduce a signed entropy integral (SEI) statistic that captures both the magnitude and temporal trend of prediction entropy across training epochs. SEI is broadly applicable to classification networks and demonstrates particular effectiveness when integrated with contrastive language-image pretraining (CLIP) architectures. Through extensive experiments on three medical imaging datasets---a domain particularly susceptible to labeling errors due to diagnostic complexity---spanning diverse modalities and pathologies, we demonstrate that SEI achieves state-of-the-art performance in mislabeled data identification, outperforming existing methods while maintaining computational efficiency and implementation simplicity.

Reinforcement Learning · Deep RL

Minjae Kwon, Josephine Lamp, Lu Feng

Safe Reinforcement Learning (RL) algorithms are typically evaluated under fixed training conditions. We investigate whether training-time safety guarantees transfer to deployment under distribution shift, using diabetes management as a safety-critical testbed. We benchmark safe RL algorithms on a unified clinical simulator and reveal a safety generalization gap: policies satisfying constraints during training frequently violate safety requirements on unseen patients. We demonstrate that test-time shielding, which filters unsafe actions using learned dynamics models, effectively restores safety across algorithms and patient populations. Across eight safe RL algorithms, three diabetes types, and three age groups, shielding achieves Time-in-Range gains of 13-14\% for strong baselines such as PPO-Lag and CPO while reducing clinical risk index and glucose variability. Our simulator and benchmark provide a platform for studying safety under distribution shift in safety-critical control domains.

Deep Learning · Large Language Models

Qingyue Zhang, Chang Chu, Tianren Peng, Qi Li, Xiangyang Luo, Zhihao Jiang, Shao-Lun Huang

LoRA has become a widely adopted method for PEFT, and its initialization methods have attracted increasing attention. However, existing methods have notable limitations: many methods do not incorporate target-domain data, while gradient-based methods exploit data only at a shallow level by relying on one-step gradient decomposition. In this paper, we establish a theoretical framework for data-aware LoRA initialization. Starting from minimizing the expectation of the parameter discrepancy between the fine-tuned and target models, we derive an optimization problem with two components: a bias term, which is related to the parameter distance between the fine-tuned and target models, and is approximated using a Fisher–gradient formulation to preserve anisotropy; and a variance term, which accounts for the uncertainty introduced by sampling stochasticity through the Fisher information. Solving this problem yields an optimal initialization strategy for LoRA, based on which we develop an efficient algorithm, LoRA-DA. Empirical results across multiple benchmarks demonstrate that LoRA-DA consistently improves final accuracy over existing initialization methods. Additional studies show faster, more stable convergence, robustness across ranks, and only a small initialization overhead for LoRA-DA. The source code will be released upon publication.

Deep Learning · Algorithms

Jiayu Zhang, Changbang Li, Canran Xiao

In-context learning (ICL) is a practical way to adapt large models, yet under strict context limits it remains unclear how to spend scarce tokens without being misled by noisy, redundant, or conflicting demonstrations. We address this gap by targeting token-budgeted context construction: how to select and compress demonstrations so the prompt carries maximal task-relevant signal with minimal predictive distortion. We propose RDCO, a deterministic, training-free optimizer that scores demonstrations by marginal task information per token, penalizes redundancy and prefix-conditioned conflicts, and compacts the selected context under a bounded predictive divergence constraint to control drift. Across a 10-dataset ICL suite spanning classification and structured generation, RDCO achieves the best average performance (63.26 Acc. on classification and 56.26 EM on generation) and improves the overall average by +2.20 points over the strongest baseline under the same budget. Our results suggest that viewing prompts as finite-capacity messages yields a principled and effective path to more reliable and token-efficient ICL.

Applications · Time Series

Junkai Lu, Peng Chen, Xingjian Wu, Yang Shu, Chenjuan Guo, Christian S Jensen, Bin Yang

Time series reasoning demands both the perception of complex dynamics and logical depth. However, existing LLM-based approaches exhibit two limitations: they often treat time series merely as text or images, failing to capture the patterns like trends and seasonalities needed to answer specific questions; and when trained on a mix of simple and complex tasks, simpler objectives often dominate the learning process, hindering the development of deep reasoning capabilities. To address these limitations, we propose the Pattern-Aware Alignment and Balanced Reasoning model (PATRA), introducing a pattern-aware mechanism that extracts trend and seasonality patterns from time series to achieve deep alignment. Furthermore, we design a task-aware balanced reward to harmonize learning across tasks of varying difficulty, incentivizing the generation of coherent Chains of Thought. Extensive experiments show that PATRA outperforms strong baselines across diverse Time Series Question Answering (TSQA) tasks, demonstrating superior cross-modal understanding and reasoning capability.

Deep Learning · Algorithms

Jianfeng Lu, YuZhao Xiang, Yue Chen, Gang Li, Shuqin Cao, Guanghui Wen

Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneity-induced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead.