论文检索

Social Aspects · Security

Yanyun Wang, Yu Huang, Zi Liang, Xixin Wu, Li Liu

The integration of audio modality into Large Audio Language Models (LALMs) significantly expands their attack surface. Existing jailbreak paradigms predominantly treat audio as a carrier for malicious payloads, relying on semantic optimization, acoustic parameter control, or additive perturbation to embed harmful content into the audio signal. In this work, we challenge this necessity and propose a new paradigm in which the role of audio shifts from content injection to safety alignment interference. We reveal that LALM safety alignment can be compromised solely by specific **Acoustic Latent Semantics (ALS)**, the underlying paralinguistic features intrinsic to the priors of audio generative models. Distinct from previous works that leverage explicit acoustic parameters to merely style malicious audio, we demonstrate that interference audio, benign in content but infused with specific ALS, can serve as a universal jailbreak trigger. Leveraging this insight, we propose the **Acoustic Interference Attack (AIA)**, which decouples the attack payload from the audio. Specifically, AIA employs a set of universal, instruction-neutral interference audio, enabling standard malicious text queries to bypass safety alignment without instance-specific optimization. Extensive experiments on 10 LALMs across five datasets demonstrate that AIA achieves the state-of-the-art attack success rate. Furthermore, our interpretability analysis uncovers the inference path drift induced by AIA and identifies the inherent effective patterns within ALS, revealing the fundamental vulnerability of cross-modal alignment in LALMs.

General Machine Learning · Transfer, Multitask and Meta-learning

Guodong Zheng, Enneng Yang, Xiaoyan Wang, Feihong He, Yihan Chen, Quan Zheng, Peng Wang, Li Shen

Continual learning (CL) seeks models that acquire new knowledge while avoiding catastrophic forgetting. However, many methods that mitigate forgetting constrain parameter updates and thereby reduce model plasticity. We revisit the singular value spectrum of gradients in representative CL methods and show that they commonly exhibit singular value collapse, where only a small subset of gradient directions drive parameter updates. Motivated by this observation, we propose \textbf{P}lasticity \textbf{A}ctivation via \textbf{P}olar \textbf{O}perator (PAPO), a plug-in that preserves the dominant directions that mitigate forgetting while activating previously suppressed directions to enhance plasticity. Concretely, PAPO modifies the gradient $\mathbf{G}$ as $\mathbf{G}\leftarrow \mathbf{G}+\lambda \cdot \operatorname{polar}(\mathbf{G})$, which uniformly increases near-zero singular values without changing the singular vectors. To avoid the cost of explicit singular value decomposition, we approximate the polar factor using the iteration-dependent Polar Express scheme, which relies only on matrix multiplications and additions. In our empirical evaluation on both vision and language benchmarks, incorporating PAPO yields consistent improvements. In particular, on MiniImageNet, integrating PAPO into ER, MAS, GPM and TRGP produces substantial accuracy gains of $9.01\%$, $4.76\%$, $8.90\%$ and $9.19\%$, respectively.

Deep Learning · Large Language Models

Yiran Wu, Mauricio Velazco, Andrew Zhao, Manuel Luján, Srisuma Movva, Yogesh Roy, Quang Nguyen, Roberto Rodriguez, Qingyun Wu, Michael Albada 等

We present \textbf{ExCyTIn-Bench}, the first benchmark to \textbf{E}valuate an LLM agent \textbf{X} on the task of \textbf{Cy}ber \textbf{T}hreat \textbf{In}vestigation through security questions derived from investigation graphs. Real‑world security analysts must sift through a large number of heterogeneous security logs, follow multi‑hop chains of evidence to investigate threats. With the developments of LLMs, building LLM-based agents for automatic threat investigation is a promising direction. We construct a benchmark from a controlled Azure tenant including a SQL environment covering 57 log tables from Microsoft Sentinel and related services, and 7542 generated questions. We leverage security logs extracted with expert-crafted detection logic to build threat investigation graphs, and then generate questions with LLMs using paired nodes on the graph, taking the start node as background context and the end node as answer. Anchoring each question to these explicit nodes and edges not only provides automatic, explainable ground truth answers but also makes the pipeline reusable and readily extensible to new logs. Our comprehensive experiments on the test set with different models confirm the difficulty of the task: the best model so far can achieve a reward of 0.606, leaving much headroom for future research.

Social Aspects · Alignment

Advait Yadav, Sidney Black, Oliver Sourbut

Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation failures may arise. In many real-world coordination problems, from knowledge sharing in organizations to code documentation, helping others carries negligible personal cost while generating substantial collective benefits. However, whether LLM agents cooperate when helping neither benefits nor harms the helper, despite being given explicit instructions to do so, remains unknown. We build a multi-agent setup designed to study cooperative behavior in a frictionless environment, removing all strategic complexity from cooperation. We find that capability does not predict cooperation: OpenAI o3 achieves only 17\% of optimal collective performance while OpenAI o3-mini reaches 50\%, despite identical instructions to maximize group revenue. Through a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, tracing their origins through agent reasoning analysis. Testing targeted interventions, we find that explicit protocols double performance for low-competence models, and tiny sharing incentives improve models with weak cooperation. These results demonstrate that even when helping is free and strategically trivial, many LLMs fail to follow the instructed cooperative objectives, requiring interventions based on specific failure modes. Our findings suggest that scaling intelligence alone will not solve coordination problems in multi-agent systems and will require deliberate cooperative design, even when helping others costs nothing.

Social Aspects · Security

Ye Zhu, Ai Zhao, Jinwei Wang

Image forgery is rapidly evolving, rendering forensic traces increasingly subtle and readily attenuated by post-processing. Although vision--language prompting can inject priors, open-ended LLM-generated prompts are difficult to constrain, and naive language description can introduce semantic perturbations. To address these challenges, we propose Forensic Prompting with Dual-Action policy optimization (FPDA) for vision-language forgery detection and localization, where Forensic Prompting Module (FPM) constructs a structured forensic prompt bank and supports optional text input to provide lightweight stable conditioning via a reliability gate. Moreover, a Dual-Action Policy Optimization (DAPO) is applied to adaptively route prompts and schedule refinement strategies on a per-image basis, stabilizing discriminative cues and improving mask spatial consistency. Extensive experiments are conducted on multiple public datasets of manipulations, diffusion content, face forgeries, and text-enabled settings (e.g., CASIA/NIST16/Coverage, CocoGlide, OpenForensics and SID_Set_description), which demonstrates superior detection and localization performance over state-of-the-art methods.

Deep Learning · Robustness

Tianyi Xu, Cheng'an Wei, Yue Zhao, Kai Chen

Automatic Speech Recognition (ASR) systems, such as those in intelligent assistants, are vulnerable to adversarial examples (AEs). Benign audio clips like music, when embedded with small perturbations, can trick ASR models into recognizing attacker-specified commands. Prior studies focus on minimizing perturbation magnitude to craft AEs. However, they fails to achieve high attack stealthiness against black-box ASR systems in the physical world. In this paper, we introduce the first music carrier selection algorithm and an attention-aware stealthiness loss function to generate stealthy AEs. Extensive evaluations on five commercial ASR APIs and three widely-used voice assistants demonstrate that our method significantly outperforms state-of-the-art techniques in both effectiveness and stealthiness. Notably, in a user study involving 200 participants, 55.6\% of participants perceived our physical adversarial examples as benign audio, which is an improvement of over 20\% compared to existing methods.

Vincent Siu, Jingxuan He, Kyle Montgomery, Zhun Wang, Chenguang Wang, Dawn Song

Existing definitions of agent security are ambiguous because they do not fully capture the holistic view across agent components. For instance, current work fails to distinguish between potentially legitimate administrative tasks and malicious exploitation of the same command. A command to "delete user data" could be either instruction following to reset a sandbox or a prompt injection attacking production systems. We argue that agent security must be redefined through a holistic framework including four core components: identity (who: authority and authentication), task (what to do: authorized objectives), trajectory (progress: action-observation boundaries), and memory (what can be retrieved: information access control). Our framework redefines existing security violations (e.g., reframing prompt injection as an identity violation), enables discovery of new attack vectors, and distinguishes legitimate capabilities like instruction following from security violations like prompt injection attacks. Critically, we demonstrate that temporal aspects are essential: attacks can be misdefined or unnoticed without accounting for how security components in our framework evolve over time. Our framework further identifies that agentic task decomposition and data and control flow patterns are crucial to rigorous security definitions, aspects previous frameworks fail to address, and provides a new foundation for future agent security work.

Applications · Computer Vision

Hao Tan, jun lan, Senyuan Shi, Zichang Tan, Zijian Yu, Huijia Zhu, Weiqiang Wang, Jun Wan, Zhen Lei

The growing capability of video generation poses escalating security risks, making reliable detection increasingly essential. In this paper, we introduce **VideoVeritas**, a framework that integrates fine-grained perception and fact-based reasoning. We observe that while current multi-modal large language models (MLLMs) exhibit strong reasoning capacity, their granular perception ability remains limited. To mitigate this, we introduce *Joint Preference Alignment* and *Perception Pretext Reinforcement Learning (PPRL)*. Specifically, rather than directly optimizing for detection task, we adopt general spatiotemporal grounding and self-supervised object counting in the RL stage, enhancing detection performance with simple *perception pretext tasks*. To facilitate robust evaluation, we further introduce **MintVid**, a light yet high-quality dataset containing 3K videos from 9 state-of-the-art generators, along with a real-world collected subset that has factual errors in content. Experimental results demonstrate that existing methods tend to bias towards either *superficial* reasoning or *mechanical* analysis, while **VideoVeritas** achieves more balanced performance across diverse benchmarks.

Deep Learning · Large Language Models

Tianyu Li, Dongchen Han, Zixuan Cao, Haofeng Huang, Mengyu Zhou, Ming Chen, erchao.zec, xiaoxi jiang, guanjunjiang, Gao Huang

Modern Transformers predominantly adopt the Pre-Norm paradigm for its optimization stability, foregoing the superior potential of the unstable Post-Norm architecture. Prior attempts to combine their strengths typically lead to a stability-performance trade-off. We attribute this phenomenon to a structural incompatibility within a *single-stream* design: Any application of the Post-Norm operation inevitably obstructs the clean identity gradient preserved by Pre-Norm. To fundamentally reconcile these paradigms, we propose SiameseNorm, a *two-stream* architecture that couples Pre-Norm-like and Post-Norm-like streams with shared parameters. This design decouples the optimization dynamics of the two streams, retaining the distinct characteristics of both Pre-Norm and Post-Norm by enabling all residual blocks to receive combined gradients inherited from both paradigms, where one stream secures stability while the other enhances expressivity. Extensive pre-training experiments on 1.3B-parameter models demonstrate that SiameseNorm exhibits exceptional optimization robustness and consistently outperforms strong baselines.

Social Aspects · Safety

Zezhong WANG, Xueyang Tang, RUI LIAN, Yang Lou, Heqing Huang

As Large Language Model (LLM) agents are increasingly deployed in complex environments, multi-turn interaction attacks have become a significant security challenge. Existing detection methods typically rely on historical context. However, this retrospective logic struggles to identify deep malicious intents that are split across turns to hide future risks. Inspired by speculative decoding, we propose the Speculative Safety Honeypot (SSH) framework. SSH uses a multi-agent simulation system composed of small LLMs to build an action-level speculate-and-verify workflow. In the speculation stage, SSH predicts future behaviors of the target agent and asynchronously builds a trajectory tree to expose potential risks in advance. In the verification stage, the system uses the target agent's real actions to calibrate and prune the trajectory tree, effectively reducing false positives. As a plug-and-playable component, SSH provides existing detectors with rich decision redundancy beyond the current interaction slice. By judging risk based on the evolution of the entire trajectory tree rather than a single point in time, the system reduces the reliance on the absolute precision of individual detection components. This improves the defense resilience and the warning lead-time of agent systems against complex temporal attacks.

Social Aspects · Security

Javad Dogani, Devriş İşler, Nikolaos Laoutaris

Federated parameter-efficient fine-tuning (PEFT) enables customizing large language models on private data, yet it is vulnerable to backdoor poisoning—especially when privacy constraints prevent inspection of per-client real-valued updates. We exploit the intuition that poisoning leaves a similar backdoor imprint in which adapter coordinates become salient, so overlap in salient-index supports remains informative even without values. We introduce IndexGuard, an unsupervised index-only vetting primitive in which clients send only Top-$K$ salient update indices and the server operates on the induced overlap geometry, clustering clients and filtering cohesion-outlier groups before aggregation. We analyze support stability under bounded rescaling and separability under shared-trigger poisoning under non-IID drift. Across attacks, backbones, and PEFT variants, IndexGuard provides end-to-end mitigation, preserving clean accuracy while achieving performance comparable to centralized methods.

Social Aspects · Accountability, Transparency, and Interpretability

Charles Ye, Jasmine Cui, Dylan Hadfield-Menell

Language models remain vulnerable to prompt injection attacks despite extensive safety training. We trace this failure to role confusion: models assign roles based on how text sounds, not where it actually comes from. We design novel role probes to capture how models internally identify “who is speaking.” These reveal why prompt injection works: untrusted text that imitates a role inherits that role’s authority. We test this insight by injecting fabricated reasoning into user prompts and tool outputs, achieving average success rates of 60% on StrongREJECT and 61% on agent exfiltration, across multiple open- and closed-weight models with near-zero baselines. Strikingly, the degree of internal role confusion strongly predicts attack success before generation begins. Our findings reveal a fundamental gap: security is defined at the interface but authority is assigned in latent space. More broadly, we introduce a unifying, mechanistic framework for prompt injection, demonstrating that diverse prompt-injection attacks exploit the same underlying role-confusion mechanism. Code available at: https://anonymous.4open.science/r/role-science-B522.

Social Aspects · Safety

Nivya Talokar, Ayush Kumar Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

LLM-based agents increasingly execute real-world workflows via tools and memory. Granting LLMs such powers enables ill-intended adversaries to likewise use these agents to carry out complex misuse scenarios. Existing agent-misuse benchmarks largely test single-prompt instructions, leaving a gap in measuring how agents end up helping with harmful or illegal tasks over multiple turns. We introduce **STING** (*Sequential Testing of Illicit N-step Goal execution*), an automated red-teaming framework that constructs a step-by-step illicit plan grounded in a benign persona and iteratively probes a target agent with adaptive follow-ups, using judge agents to track phase completion. We further introduce an analysis framework that models multi-turn red-teaming as a time-to-first-jailbreak random variable, enabling analysis tools like discovery curves, hazard-ratio attribution by attack language, and a new metric: Restricted Mean Jailbreak Discovery. Across AgentHarm scenarios, STING yields substantially higher illicit-task completion than single-turn prompting and chat-oriented multi-turn baselines adapted to tool-using agents. In multilingual evaluations across six non-English settings, we find that attack success and illicit-task completion do *not* consistently increase in lower-resource languages, diverging from common chatbot findings. Overall, STING provides a practical way to evaluate and stress-test agent misuse in realistic deployment settings, where interactions are inherently multi-turn and often multilingual.

Social Aspects · Alignment

David Africa, Arathi Mani

Consistency training encourages a model to produce similar outputs across related inputs or sampling procedures. Such methods are simple, scalable, and improve reasoning in language models, but their effects on model alignment haven't been well studied. Could the self-bootstrapping nature of these methods amplify undesired behavior in models? We test seven consistency training methods on 108 ``model organisms'': open-source models (7B--70B) fine-tuned to exhibit various forms of controlled misaligned behavior. We find that outcomes vary significantly: consistency training generally suppresses reward hacking and emergent misalignment but amplifies sycophancy. We present evidence that distribution shifts induced by the consistency labeling process, rather than variation in the selection operators, may be the primary driver of systematic alignment effects. Finally, we present a unifying theoretical framework to derive conditions under which consistency training will amplify or suppress misalignment. In total, our study establishes that consistency training is not alignment-neutral, and that its use in critical systems should be carefully audited.

Social Aspects · Security

Zhenxiong Yan, Suhang Yao, Yu Liu, Wenqiang Jin

Large language models (LLMs) are trained at significant computational and data cost, making them valuable intellectual property (IP). Existing IP verification methods primarily rely either on invasive watermarking that degrades model utility, or on superficial behavioral signatures disrupted by fine-tuning and model merging. This apparent trade-off between model utility and IP protection has constrained practical deployment. We challenge this trade-off and propose CircuitPrint, a non-invasive IP fingerprinting framework that enables robust verification through standard model queries by leveraging stable internal computational circuits of LLMs. We show that these circuits function as a persistent computational backbone across model derivatives, allowing them to serve as stable fingerprints for distinguishing LLMs. Building on this stability, CircuitPrint constructs IP signatures by identifying mechanistically essential supernodes that causally produce specific predictions within these circuits. Specifically, trigger queries are synthesized to replicate the internal suppression of these supernodes, thereby inducing distinctive and observable output shifts. Experimental results demonstrate that CircuitPrint substantially outperforms existing baselines while remaining robust under aggressive fine-tuning and model merging, effectively resolving this trade-off without altering model parameters.

Applications · Language, Speech and Dialog

Chenyang Shao, Jiahe Liu, Fengli Xu, Yong Li

Scientific illustration figures are essential for depicting research works' conceptual designs, methodology, and experimental workflows, playing a pivotal role in communicating complex academic insights. However, creating high-quality scientific illustrations remains a labor-intensive task for human scientists. While recent generative image models have advanced prompt-based editing, the synthesis of fully **editable** figures remains a fundamental challenge. Valid editability involves structured transformations of graphical elements, scales, attributes, and text, rather than simple pixel-level changes. Existing models generate raster outputs that do not support manual correction or layout adjustment, limiting their utility in scientific publishing, where editable vector figures are typically required for submission. To address this challenge, we introduce **LiveFigure**, an agentic framework driven by VLM agents that imitates the multi-step drawing workflow of human researchers. It first plans figure blueprints by drawing inspiration from high-quality references in previous works, then generates executable scripts that produce figures via the PowerPoint interface based on skills and experience, and finally refines the outputs with targeted visual diagnostics, producing fully vectorized, editable figures that meet publication standards. Extensive experiments demonstrate that LiveFigure generates inherently editable figures that are both visually clear and aesthetically appealing, achieving 80% publication-readiness within just 17 manual edits—far surpassing the 24% rate of the strongest baseline, NanoBanna. Human preference studies further validate this advantage, with LiveFigure securing a 60% win rate against NanoBanna. Our code is available at [https://anonymous.4open.science/r/LiveFigure](https://anonymous.4open.science/r/LiveFigure).

Social Aspects · Security

Ali Ebrahimpour-Boroojeny, Yian Wang, Hari Sundaram

In this paper, we reveal a significant shortcoming in class unlearning evaluations: overlooking the underlying class geometry can cause information leakage about the forgotten class. We further propose a simple unlearning strategy to mitigate this issue. We introduce Class Membership Inference Attack (CMIA) that uses the probabilities the model assigns to neighboring classes to detect unlearned samples. We find that existing unlearning methods are vulnerable to CMIA across multiple datasets. We then propose a new fine-tuning objective that mitigates this privacy leakage by approximating, for forget-class inputs, the distribution over the remaining classes that a retrained-from-scratch model would produce. To construct this approximation, we estimate inter-class similarity and tilt the target model’s distribution accordingly. The resulting Tilted REWeighting (TREW) distribution serves as the desired distribution during fine-tuning. We also show that across multiple benchmarks, TREW matches or surpasses existing unlearning methods on prior unlearning metrics. More specifically, on CIFAR-10, it reduces the gap with retrained models by $19\%$ and $46\%$ for U-LiRA and CMIA scores, accordingly, compared to the SOTA method for each category.

Social Aspects · Security

Mingrui Liu, Sixiao Zhang, Cheng Long, Kwok Yan Lam

Large Language Models (LLMs) are increasingly vulnerable to *Prompt Injection (PI)* attacks, where adversarial instructions hidden within retrieved contexts hijack the model's execution flow. Current defenses typically face a critical trade-off: *prevention-based* fine-tuning often degrades general utility via the "alignment tax", while *detection-based* filtering incurs prohibitive latency and memory costs. To bridge this gap, we propose **RedVisor**, a unified framework that synthesizes the explainability of detection systems with the seamless integration of prevention strategies. To the best of our knowledge, RedVisor is the first approach to leverage fine-grained reasoning paths to simultaneously *detect* attacks and *guide* the model's safe response. We implement this via a lightweight, removable adapter positioned atop the frozen backbone. This adapter serves a dual function: it first generates an explainable analysis that precisely localizes the injection and articulates the threat, which then explicitly conditions the model to reject the malicious command. Uniquely, the adapter is active only during this reasoning phase and is effectively muted during the subsequent response generation. This architecture yields two distinct advantages: (1) it mathematically preserves the backbone's original utility on benign inputs; and (2) it enables a novel **KV Cache Reuse** strategy, eliminating the redundant prefill computation inherent to decoupled pipelines. We further pioneer the integration of this defense into the vLLM serving engine with custom kernels. Experiments demonstrate that RedVisor outperforms state-of-the-art defenses in detection accuracy and throughput while incurring negligible utility loss.

Social Aspects · Security

Leyi Qi, Yiming Li, Siyuan Liang, Zhengzhong Tu, Dacheng Tao

Large-scale text-to-image (T2I) diffusion models have enabled unprecedented creative applications, but their unauthorized use and reproduction have raised serious intellectual property concerns, making model ownership verification (MOV) increasingly critical. We find that existing backdoor-based diffusion watermarking methods often (implicitly) assume a `faithful' verification process, namely, that the verifier can query a suspicious model and obtain the faithful watermark response to complete MOV. However, in practice, adversaries may intentionally or unintentionally damage potential watermark signals, significantly degrading verification reliability. To address this issue, we propose Cert-LAS, the first certified MOV method for T2I models based on layer-adaptive smoothing. In general, Cert-LAS embeds specified watermarks using diffusion classifiers and an LFS-guided layer-wise noise allocation, and verifies ownership by examining whether the suspected model exhibits significantly stronger watermark responses compared to unwatermarked references through hypothesis testing. We further prove that, under certain conditions, our Cert-LAS can still achieve reliable verification even in the presence of malicious removal attacks. Extensive experiments validate the effectiveness of Cert-LAS and its resistance to adaptive attacks.

Social Aspects · Security

Jiacheng Lu, Yiming Li, Tao Song, Weijian Wang, Wenjie Qu, Haibing Guan, Jiaheng Zhang

Large Language Models (LLMs) with proprietary Chain-of-Thought (CoT) capabilities constitute high-value intellectual property, yet protecting them against unauthorized theft and unlicensed commercialization remains a critical challenge. Existing watermarking paradigms are ill-suited for safeguarding these models: direct logit perturbations inevitably fracture the fragile logical consistency required for complex reasoning, or remain superficial enough to be erased by fine-tuning. In this paper, we propose BiCoT, a framework that embeds ownership directly into the reasoning representations via bi-level variational alignment. Instead of adding external perturbations, our method optimizes the model's internal states to collapse onto a signature subspace. This creates a functional entanglement where the watermark becomes a prerequisite for the model's reasoning utility: removing the signature destroys the capability. To handle representation drift in stolen models, we further introduce a Robust Subspace Registration (RSR) verifier. Experiments demonstrate that BiCoT achieves negligible fidelity loss while maintaining strong robustness against diverse attacks on both in-domain and out-of-distribution data.