论文检索

Social Aspects · Privacy

Martin Van Waerebeke, Giovanni Neglia, Kevin Scaman, Marco Lorenzi, El-Mahdi El-Mhamdi

In machine unlearning, $(\varepsilon,\delta)-$unlearning is a popular framework that provides formal guarantees on the effectiveness of the removal of a subset of training data, the \emph{forget set}, from a trained model. For strongly convex objectives, existing first-order methods achieve $(\varepsilon,\delta)-$unlearning, but they only use the forget set to calibrate injected noise, never as a direct optimization signal. In contrast, efficient empirical heuristics often exploit the forget samples (e.g., via gradient ascent) but come with no formal unlearning guarantees. We bridge this gap by presenting the Variance-Reduced Unlearning (*VRU*) algorithm. To the best of our knowledge, *VRU* is the first first-order algorithm that directly includes forget set gradients in its update rule, while provably satisfying $(\varepsilon,\delta)-$unlearning. We establish the convergence of *VRU* and show that incorporating the forget set yields strictly improved rates, *i.e.*, a better dependence on the achieved error compared to existing first-order $(\varepsilon,\delta)-$unlearning methods. Moreover, we prove that, in a low-error regime *VRU* asymptotically outperforms any first-order methods that ignores the forget set. Experiments corroborate our theory, showing consistent gains over both state-of-the-art certified unlearning methods and over empirical baselines that explicitly leverage the forget set.

Social Aspects · Robustness

Gina Wong, Drew Prinster, Suchi Saria, Rama Chellappa, Anqi Liu

Calibration aligns a model's predictive uncertainty with the frequencies of its empirical outcomes and is important toward understanding and trusting reported probabilities. Recent work shows that enforcing calibration at the level of individual predictors can substantially improve ensemble performance, with mixture-of-calibrated-experts (MoCE) models showing strong empirical improvements in particular. However, the conditions under which calibration helps MoCE are not well understood. In this work, we study MoCE models from a distributional robustness perspective, focusing on how routing mechanisms interact with expert-level calibration. We show that for hard routing, expert calibration is sufficient to ensure calibration of the overall model under a broad class of distribution shifts but is insufficient for calibrating a soft-routed model. We characterize the conditions that must hold for a soft-routed MoCE to be calibrated, and we discuss how reframing calibration as a distributionally robust objective helps recover robustness guarantees for soft-routed mixtures.

Social Aspects · Accountability, Transparency, and Interpretability

Zijin Yang, Yu Sun, Kejiang Chen, jiawei zhao, Jun Jiang, Weiming Zhang, Nenghai Yu

Digital watermarking is essential for securing generated images from diffusion models. Accurate watermark evaluation is critical for algorithm development, yet existing methods have significant limitations: they lack a unified framework for both residual and semantic watermarks, provide results without interpretability, neglect comprehensive security considerations, and often use inappropriate metrics for semantic watermarks. To address these gaps, we propose **WMVLM**, the first unified and interpretable evaluation framework for diffusion model image **w**ater**m**arking via **v**ision-**l**anguage **m**odels (VLMs). We redefine quality and security metrics for each watermark type: residual watermarks are evaluated by artifact strength and erasure resistance, while semantic watermarks are assessed through latent distribution shifts. Moreover, we introduce a three-stage training strategy to progressively enable the model to achieve classification, scoring, and interpretable text generation. Experiments show WMVLM outperforms state-of-the-art VLMs with strong generalization across datasets, diffusion models, and watermarking methods.

Social Aspects · Security

Changyue Li, Jiaying Li, Youliang Yuan, Jiaming He, Zhicong Huang, Pinjia He

Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics. This paper investigates a novel threat: Semantic-Aware Hijacking. We explore the feasibility of hijacking multiple stateless decisions simultaneously using a single universal perturbation. We introduce the Semantic-Aware Universal Perturbation (SAUP), which acts as a semantic router, "actively" perceiving input semantics and routing them to distinct, attacker-defined targets. To achieve this, we conduct theoretical and empirical analysis on the geometric properties in the latent space. Guided by these insights, we propose the Semantic-Oriented (SORT) optimization strategy and annotate a new dataset with fine-grained semantics to evaluate performance. Extensive experiments on three representative MLLMs demonstrate the fundamental feasibility of this attack, achieving a 66% attack success rate over five targets using a single frame against Qwen.

Social Aspects · Safety

Siming Huang, Yuliang Xu, Mingmeng Geng, Yao Wan, Dongping Chen

In this paper, we present a comprehensive analysis and monitoring framework for the impact of Large Language Models (LLMs) on Wikipedia, examining the evolution of Wikipedia through existing data and using simulations to explore potential risks. We begin by analyzing article content and page views to study the recent changes in Wikipedia and assess the impact of LLMs. Subsequently, we evaluate how LLMs affect various Natural Language Processing (NLP) tasks related to Wikipedia, including machine translation and retrieval-augmented generation (RAG). Our findings and simulation results reveal that Wikipedia articles have been affected by LLMs, with an impact of approximately 1% in certain categories. If the machine translation benchmark based on Wikipedia is influenced by LLMs, the scores of the models may become inflated, and the comparative results among models could shift. Moreover, the effectiveness of RAG might decrease if the knowledge has been contaminated by LLMs. While LLMs have not yet fully changed Wikipedia's language and knowledge structures, we believe that our empirical findings signal the need for careful consideration of potential future risks in NLP research.

Social Aspects · Safety

Hanbo Huang, Yiran Zhang, Hao Zheng, Xuan Gong, Yihan Li, Lin Liu, Zhuotao Liu, Shiyu Liang

Large language model (LLM) watermarking has shown promise in detecting AI-generated content and mitigating misuse, with prior work claiming robustness against paraphrasing and text editing. In this paper, we argue that existing evaluations are not sufficiently adversarial, obscuring critical vulnerabilities and overstating the security. To address this, we introduce the adaptive robustness radius, a formal metric that quantifies the worst-case resilience of watermarks against adaptive adversaries. By lifting the paraphrase space into a KL-divergence ball, we approximate this radius and theoretically demonstrate that optimizing the attack context and model parameters can significantly reduce the approximated radius, making watermarks highly vulnerable to paraphrase attacks. Leveraging this insight, we propose RLCracker, a reinforcement learning (RL)–based adaptive attack that erases watermarks while preserving semantic fidelity. RLCracker requires only limited watermarked examples and zero access to the detector. Despite weak supervision, it empowers a 3B model to achieve 98.5\% removal success with minimal semantic shift on 1,500-token Unigram-marked texts after training on only 100 short samples. This performance dramatically exceeds 6.75% by GPT-4o and generalizes across five model sizes over ten watermarking schemes.

Social Aspects · Everything Else

Resmi Ramachandranpillai, Thulasi Tholeti, Tomo Lazovich, Ricardo Baeza-Yates

Many failures of deployed machine learning systems stem not from insufficient accuracy, but from neglecting responsibility as a core design requirement. While responsibility principles are widely studied, they are often treated as post-hoc checks rather than as integral factors of system design. This framework has reinforced the perception that responsible practices inherently trade-off with model performance. In this position paper, we challenge that assumption and argue that responsibility and performance are not inherently at odds. We adopt a lifecycle-oriented perspective, identifying which responsible AI principles are most critical at each stage, from problem formulation and data curation to training, deployment, and monitoring. Drawing on real-world instances, we show how misaligned choices at specific stages can compound downstream risks and how alternative design choices could have mitigated these failures. We argue that responsible AI should be understood as a system design challenge rather than a constraint, and we offer operational guidance for integrating responsibility into mainstream machine learning workflows in a way that supports, rather than undermines, real-world performance.

Social Aspects · Robustness

Jingyuan Zhou, Yuexuan Wang, Kaidi Yang

Ensuring scalable input-to-state stability (sISS) is critical for the safety and reliability of large-scale interconnected systems, especially in the presence of communication delays. While learning-based controllers can achieve strong empirical performance, their black-box nature makes it difficult to provide formal and scalable stability guarantees. To address this gap, we propose a framework to synthesize and verify neural vector Lyapunov-Razumikhin certificates for discrete-time delayed interconnected systems. Our contributions are three-fold. First, we establish a sufficient condition for discrete-time sISS via vector Lyapunov-Razumikhin functions, which enables certification for large-scale delayed interconnected systems. Second, we develop a scalable synthesis and verification framework that learns the neural certificates and verifies the certificates on reachability-constrained delay domains with scalability analysis. Third, we validate our approach on mixed-autonomy platoons, drone formations, and microgrids against multiple baselines, showing improved verification efficiency with competitive control performance.

Social Aspects · Security

Haowen Xu, Xue Tan, Lei Ma, Zhihao Zhang, CHAO WANG, Qingze Wang, Ping Chen, Jun Dai, Xiaoyan Sun

While enabling effective collaboration on complex tasks, LLM-based Multi-Agent Systems (MAS) face critical security challenges due to vulnerabilities at the agent and interaction levels. Most existing MAS security defenses are built upon two core assumptions: semantically-explicit malicious attacks and explicit graph-based modeling of the MAS topology and agent-level interactions. In practice, real-world attacks are becoming more semantically stealthy, while MAS execution is typically asynchronous without the temporal alignment assumed by graph-based propagation models. To address these limitations, we propose AcMAS, an activation-based framework for malicious-behavior detection in MAS. By analyzing internal reasoning states in the activation space of local agents, AcMAS detects even stealthy attacks in a synchronization-robust fashion, without relying on explicit interaction graphs. Moreover, our activation analysis provides critical signals to guide AcMAS in restoring the functionality of compromised agents, rather than the disruptive agent isolation commonly used by the state-of-the-art methods. Comprehensive evaluation demonstrates that AcMAS significantly outperforms graph-based baselines against stealthy attacks, by +0.22 F1 in synchronous settings (0.94 vs. 0.72) and by +0.55 F1 in asynchronous settings (0.93 vs. 0.38), with generalization across diverse open-source LLM backbones, attack intensity, and MAS scale.

Social Aspects · Security

Qing Wen, Haohao Li, Zhongjie Ba, Peng Cheng, Miao He, Li Lu, Kui Ren

Advances in AIGC technologies have enabled the synthesis of highly realistic audio deepfakes capable of deceiving human auditory perception. Although numerous audio deepfake detection (ADD) methods have been developed, most rely on local temporal/spectral features or pairwise relations, overlooking high-order interactions (HOIs). HOIs capture discriminative patterns that emerge from multiple feature components beyond their individual contributions. We propose HyperPotter, a hypergraph-based framework that explicitly models these synergistic HOIs through clustering-based hyperedges with class-aware prototype initialization. Extensive experiments demonstrate that HyperPotter surpasses its baseline by an average relative gain of 22.15\% across 11 datasets and outperforms state-of-the-art methods by 13.96\% on 4 challenging cross-domain datasets, demonstrating superior generalization to diverse attacks and speakers.

Social Aspects · Alignment

Edward Chen, Sanmi Koyejo, Carlos Guestrin

Large language model (LLM) alignment relies on complex reward signals that often obscure the specific behaviors being incentivized, creating critical risks of misalignment and reward hacking. Existing interpretation methods typically rely on pre-defined rubrics, risking the omission of "unknown unknowns", or fail to identify objectives that comprehensively cover and are causal to the model behavior on some dataset. To address these limitations, we introduce Obj-Disco, a framework that automatically decomposes an alignment reward signal into a sparse, weighted combination of human-interpretable natural language objectives. Our approach utilizes an iterative greedy algorithm to analyze behavioral changes across training checkpoints, identifying and validating candidate objectives that best explain the residual reward signal. Extensive evaluations across diverse tasks, model sizes, and alignment algorithms demonstrate the framework's robustness. Experiments with popular open-source reward models show that the framework consistently captures > 90\% of reward behavior, a finding further corroborated by human evaluation. Additionally, a case study on alignment with an open-source reward model reveals that Obj-Disco can successfully identify latent misaligned incentives that emerge alongside intended behaviors. Our work provides a crucial tool for uncovering the implicit objectives in LLM alignment, paving the way for more transparent and safer AI development.

Social Aspects · Security

Lijing Ren, denghui zhang

Generative Image Steganography (GIS) embeds complex secrets within stego-images that are indistinguishable from the stochastic synthesis process itself. It achieves this by exploiting the reversible probability flow between Gaussian noise and the natural image manifold. However, existing steganography faces a key-dependency paradox: precise secret extraction usually requires an external private key or random seed to synchronize the denoising path. In this paper, we present KFStego, a training-free framework substituting cryptographic secrecy with structural redundancy for key-free, high-resolution secure distribution. Our dual-guidance mechanism utilizes structural latent guidance to project secrets into a bipartite manifold via downsampling and halftoning, yielding self-synchronizing shares. While measurement posterior sampling leverages these shares as discrete invariants to steer a differentiable restoration. By backpropagating through a differentiable measurement surrogate, KFStego reconstructs high-fidelity continuous-tone details from sparse binary observations, mitigating fidelity loss from inversion drift. KFStego offers an endogenous secure image distribution paradigm by connecting discrete structural invariants with high-fidelity generative reconstruction.

Social Aspects · Accountability, Transparency, and Interpretability

Anand Shah, Tobin South, Talfan Evans, Hannah Kirk, Jiaxin Pei, Andrew Trask, Glen Weyl, Michiel Bakker

Personalization underpins the modern digital economy. Today, personalization is largely implemented through provider-managed infrastructure that infers user preferences from behavioral data, with limited portability or user control. However, large language models (LLMs) are increasingly being used to perform tasks on users' behalf. The age of LLMs for the first time provides a path to a more controllable and interpretable personalization paradigm, grounded in user-expressed natural language preferences and context. In this position paper, we argue that to provide robust and user-centric personalization, we need a new Human Context Protocol (HCP) to represent and share personal preferences across AI systems. HCP treats preferences as a portable, user-governed layer in the personalization stack, enabling interoperability, scoped access, and revocation. Along with a working prototype to ground discussion, we consider counterarguments along adoption dynamics and market incentives, high-stakes use cases, and outline novel paths via the HCP towards trustworthy personalization in the human-AI economy.

Social Aspects · Fairness

Nripsuta Saxena, Abigail Horn, Wenbin Zhang, Cyrus Shahabi

Despite location being increasingly used in decision-making systems deployed in sensitive domains such as mortgages and insurance, little attention has been paid to the unfairness that may seep in due to the correlation of location with characteristics considered protected under anti-discrimination law, such as race or national origin. This position paper argues for the urgent need to consider fairness with respect to location, termed $\textit{spatial fairness}$. It outlines the harms perpetuated through location's correlation with protected characteristics, which may be particularly consequential due to its treatment as a neutral or purely technical attribute, abstracted from its historical, political, and socioeconomic context. This interdisciplinary work connects knowledge from fields such as public policy, economic development, and geography to highlight how existing fair-AI research falls short in addressing spatial biases, and fails to consider challenges unique to spatial data. Furthermore, we identify limitations in the small body of prior work on spatial fairness work, and propose guidelines to inform future research aimed at mitigating spatial biases in data-driven decision-making systems.

Social Aspects · Accountability, Transparency, and Interpretability

Praneet Suresh, Jack Stanley, Sonia Joseph, Luca Scimeca, Danilo Bzdok

Pre-trained transformers have demonstrated remarkable generalization abilities, at times extending beyond the scope of their training data. Yet, real-world deployments often face unexpected or adversarial data that diverges from training data distributions. Without explicit mechanisms for handling such shifts, model reliability and safety degrade, urging more disciplined study of out-of-distribution (OOD) settings for transformers. By systematic experiments, we present a mechanistic framework for delineating the precise contours of transformer model robustness. We find that OOD inputs, including subtle typos and jailbreak prompts, drive language models to operate on an increased number of fallacious concepts in their internals. We leverage this device to quantify and understand the degree of distributional shift in prompts, enabling a mechanistically grounded fine-tuning strategy to robustify LLMs. Expanding the very notion of OOD from input data to a model’s private computational processes—a new transformer diagnostic at inference time—is a critical step toward making AI systems safe for deployment across science, business, and government.

Social Aspects · Safety

Liyan Chen, Zoe Xi, Yael Kalai

As AI models continue to develop powerful capabilities, it becomes critical that we are able to verify that their output is aligned with our intentions. A recent line of work focuses on verification via debate, a model of interactive proofs where two competing powerful provers, or AI models, debate each other to convince a weak verifier, or a human, of the correctness of their claim. However, debate assumes that the two AI models possess equal abilities and that one of them is truthful, which may not be realistic. In this work, we show *how to avoid debate*: we initiate the study of *single-prover* interactive proofs for AI safety. Prior results in single-prover interactive proofs do not immediately carry over to the AI safety setting because they do not work when the computation has access to an oracle, such as to human judgment or an external database such as the web. We present doubly-efficient single-prover interactive proofs for oracle-aided computations (also known as relativizing proofs), in the settings where (1) the computation is robust, in the sense that the output does not change if at most a small fraction of the answers to oracle queries are incorrect, or (2) the oracle is a low-degree polynomial. These results suggest that interactive verification is possible even without debate, under structured or noise-tolerant oracle access.

Social Aspects · Accountability, Transparency, and Interpretability

Christina Lu, Jack Gallagher, Jonathan Michala, Kyle Fish, Jack Lindsey

Large language models can represent a variety of personas but typically default to a helpful Assistant identity cultivated during post-training. Across several different models, we find an “Assistant Axis" in their activation space, which captures the extent to which a model is operating in its default Assistant mode. Steering towards the Assistant direction reinforces helpful and harmless behavior; steering away increases the model’s tendency to identify as other entities. Measuring deviations along the Assistant Axis predicts “persona drift,” a phenomenon where models slip into exhibiting harmful or bizarre behaviors that are uncharacteristic of their typical persona. We find that persona drift is often driven by conversations demanding meta-reflection on the model’s processes or featuring emotionally vulnerable users. We show that restricting activations to a fixed region along the Assistant Axis can stabilize model behavior in these scenarios—and also in the face of adversarial persona-based jailbreaks. Our results suggest that post-training steers models toward a particular region of persona space but only loosely tethers them to it, motivating work on training and steering strategies that more deeply anchor models to a coherent persona.

Social Aspects · Accountability, Transparency, and Interpretability

Yoshihiro Izawa, Gouki Minegishi, Koshi Eguchi, Sosuke Hosokawa, Kenjiro Taura

Activation steering offers a computationally efficient mechanism for controlling Large Language Models (LLMs) without fine-tuning. While effectively controlling target traits (e.g., persona), coherency degradation remains a major obstacle to safety and practical deployment. We hypothesize that this degradation stems from intervening on the residual stream, which indiscriminately affects aggregated features and inadvertently amplifies off-target noise. In this work, we identify a sparse subset of attention heads (only three heads) that independently govern persona and style formation, which we term *Style Modulation Heads*. Specifically, these heads can be localized via geometric analysis of internal representations, combining layer-wise cosine similarity and head-wise contribution scores. We demonstrate that intervention targeting only these specific heads achieves robust behavioral control while significantly mitigating the coherency degradation observed in residual stream steering. More broadly, our findings show that precise, component-level localization enables safer and more precise model control.

Social Aspects · Everything Else

Won Ik Cho, Seong-hun Kim, Geunhye Kim

This position paper argues that **adopting AI in organizational practice does not guarantee productivity gains, because human and environmental factors critically moderate the relationship between AI deployment and realized productivity improvements**. Following the advent of high-performance generative models, AI use has been rapidly encouraged in some sectors while being restricted in others. Most practitioners assume that AI brings productivity boosts owing to enhanced technical capabilities, but regardless of apparent performance advances in AI technology, human and environmental factors of the organization may substantially attenuate---or even negate---the effective productivity benefits. We identify five key moderating factors: human resource composition, baseline capability of individuals, learning curve of practitioners, incentives for fair use, and flexibility of objectives. Drawing on the partial equilibrium model of Gries and Naudé (2022), we argue that existing economic frameworks may inadvertently overlook these factors. We revise the existing framework to redefine effective organizational determinants and shed light to practical implications including industry and education, responding to alternative views and calling for action of stakeholders.

Social Aspects · Safety

Sudarshan Regmi

The ability of the deep learning model to recognize when a sample falls outside its learned distribution is critical for safe and reliable deployment. Recent state-of-the-art out-of-distribution (OOD) detection methods leverage activation shaping to improve the separation between in-distribution (ID) and OOD inputs. These approaches resort to sample-specific scaling but apply a static percentile threshold across all samples regardless of their nature. In this work, we propose AdaSCALE, an adaptive scaling procedure that dynamically adjusts the percentile threshold based on a sample's estimated OOD likelihood. This estimation leverages our key observation that OOD samples exhibit significantly more pronounced activation shifts at high-magnitude activations under minor perturbation compared to ID samples. AdaSCALE enables stronger scaling for likely ID samples and weaker scaling for likely OOD samples, creating highly separable energy scores. Our approach achieves state-of-the-art OOD detection performance, outperforming the latest rival OptFS by **14.94%** in near-OOD and **21.67%** in far-OOD datasets in average FPR@95 metric in the ImageNet-1k benchmark across eight diverse architectures.