论文检索

Social Aspects · Alignment

Rakshit Trivedi, Natasha Jaques, Logan Cross, Alexander Vezhnevets, Joel Z Leibo

AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful agents under stationary-environment assumptions, treating the world as an exogenous source of feedback. This position paper argues that a solipsistic superintelligence---an extremely capable solver of stationary problems---is unlikely to be cooperative. Deployment induces endogenous nonstationarity: other agents adapt, producing best-response dynamics that reshape the environment the AI was trained to navigate. The result is a train--test--deploy gap where historical distributions diverge from deployment realities; the more aggressively a solipsistic superintelligence exploits historical regularities, the faster it renders them obsolete. Cooperation is therefore not an added capability but an equilibrium property that solipsistic superintelligence cannot guarantee. We call for a multi-agent-first research paradigm treating strategic interdependence as a core design principle, alongside dynamic evaluation: testbeds where distributions are generated by adaptive counterparties, and metrics prioritizing equilibrium stability over single-score task success.

Social Aspects · Alignment

Pushkar Mishra, Charvi Rastogi, Stephen Pfohl, Alicia Parrish, Tian Teh, Roma Patel, Mark Diaz, Ding Wang, Michela Paganini, Vinodkumar Prabhakaran 等

Ensuring the safety of Generative AI requires a nuanced understanding of pluralistic viewpoints. In this paper, we introduce a novel data-driven approach for analyzing ordinal safety ratings in pluralistic settings. Specifically, we address the challenge of interpreting nuanced differences in safety feedback from a diverse population expressed via ordinal scales (e.g., a Likert scale). We define non-parametric responsiveness metrics that quantify how raters convey broader distinctions and granular variations in the severity of safety violations. Leveraging publicly available datasets of pluralistic safety feedback as our case studies, we investigate how raters from different demographic groups use an ordinal scale to express their perceptions of the severity of violations. We apply our metrics across violation types, demonstrating their utility in extracting nuanced insights that are crucial for aligning AI systems reliably in multi-cultural contexts. We show that our approach can inform rater selection and feedback interpretation by capturing nuanced viewpoints across different demographic groups, hence improving the quality of pluralistic data collection and in turn contributing to more robust AI alignment.

Social Aspects · Alignment

Sarah Ball, Phil Hackemann

This position paper argues that modern alignment methods – originally designed to prevent harmful output – are dual-use technologies that may easily be misused by malicious actors for censorship and manipulation. By mapping current alignment techniques to the possibility and actual cases of misuse, we show that the quest for a ''perfectly aligned'' model inadvertently also provides malicious actors with an ever-improving tool for informational dominance. We need to discuss this dual-use potential *now*, as its risk is exacerbated by rapid user adoption of AI as information provider and a political landscape that increasingly shifts towards authoritarianism. We conclude by urging the community to consider the intentional misuse of safety mechanisms and propose mitigation strategies to safeguard against this dual-use potential.

Social Aspects · Accountability, Transparency, and Interpretability

Kiho Park, Todd Nief, Yo Joong Choe, Victor Veitch

This paper concerns the question of how language models and other AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation of this paper is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. We argue that the natural geometry is information geometry, and then show how this interacts with semantic encoding and the linear representation hypothesis. It turns out that the duality structure of information geometry plays a critical role. As an illustrative application, we develop *dual steering*, a method for robustly steering representations to exhibit a particular concept using linear probes. We formally prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. We empirically find that dual steering enhances the controllability and stability of concept manipulation.

Social Aspects · Security

Leyi Qi, Yiming Li, Siyuan Liang, Zhengzhong Tu, Dacheng Tao

Large-scale text-to-image (T2I) diffusion models have enabled unprecedented creative applications, but their unauthorized use and reproduction have raised serious intellectual property concerns, making model ownership verification (MOV) increasingly critical. We find that existing backdoor-based diffusion watermarking methods often (implicitly) assume a `faithful' verification process, namely, that the verifier can query a suspicious model and obtain the faithful watermark response to complete MOV. However, in practice, adversaries may intentionally or unintentionally damage potential watermark signals, significantly degrading verification reliability. To address this issue, we propose Cert-LAS, the first certified MOV method for T2I models based on layer-adaptive smoothing. In general, Cert-LAS embeds specified watermarks using diffusion classifiers and an LFS-guided layer-wise noise allocation, and verifies ownership by examining whether the suspected model exhibits significantly stronger watermark responses compared to unwatermarked references through hypothesis testing. We further prove that, under certain conditions, our Cert-LAS can still achieve reliable verification even in the presence of malicious removal attacks. Extensive experiments validate the effectiveness of Cert-LAS and its resistance to adaptive attacks.

Social Aspects · Security

Jiacheng Lu, Yiming Li, Tao Song, Weijian Wang, Wenjie Qu, Haibing Guan, Jiaheng Zhang

Large Language Models (LLMs) with proprietary Chain-of-Thought (CoT) capabilities constitute high-value intellectual property, yet protecting them against unauthorized theft and unlicensed commercialization remains a critical challenge. Existing watermarking paradigms are ill-suited for safeguarding these models: direct logit perturbations inevitably fracture the fragile logical consistency required for complex reasoning, or remain superficial enough to be erased by fine-tuning. In this paper, we propose BiCoT, a framework that embeds ownership directly into the reasoning representations via bi-level variational alignment. Instead of adding external perturbations, our method optimizes the model's internal states to collapse onto a signature subspace. This creates a functional entanglement where the watermark becomes a prerequisite for the model's reasoning utility: removing the signature destroys the capability. To handle representation drift in stolen models, we further introduce a Robust Subspace Registration (RSR) verifier. Experiments demonstrate that BiCoT achieves negligible fidelity loss while maintaining strong robustness against diverse attacks on both in-domain and out-of-distribution data.

Social Aspects · Accountability, Transparency, and Interpretability

Gal Pomerants, Yaniv Nikankin, Anja Reusch, Tomer Tsaban, Ora Schueler-Furman, Yonatan Belinkov

Protein sequences are abundant in repeating segments, both as exact copies and as approximate segments with mutations. These repeats are important for protein structure and function, motivating decades of algorithmic work on repeat identification. Recent work has shown that protein language models (PLMs) identify repeats, by examining their behavior in masked-token prediction. To elucidate their internal mechanisms, we investigate how PLMs detect both exact and approximate repeats. We find that the mechanism for approximate repeats functionally subsumes that of exact repeats. We then characterize this mechanism, revealing two main stages: PLMs first build feature representations using both general positional attention heads and biologically specialized components, such as neurons that encode amino-acid similarity. Then, induction heads attend to aligned tokens across repeated segments, promoting the correct answer. Our results reveal how PLMs solve this biological task by combining language-based pattern matching with specialized biological knowledge, thereby establishing a basis for studying more complex evolutionary processes in PLMs.

Social Aspects · Everything Else

Arjhun Swaminathan, Anika Hannemann

The Model Context Protocol (MCP) standardizes AI agent-tool interaction, accelerating agentic AI adoption through interoperability. This presents an opportunity to embed trustworthiness: As a standard and an interface between agents and tools, MCP becomes a natural enforcement point; any improvements to it automatically propagate to all systems using it. Analyzing MCP through EU Commission’s Ethics guidelines for trustworthy AI, we identify three things: fundamental shifts in how trustworthiness works, critical challenges these shifts create, and strategic intervention points where protocol-level mechanisms can achieve ecosystem-wide impact. We argue how MCP’s architecture provides a foundation for trustworthiness and propose practical improvements to strengthen it. This position paper posits that building trustworthy MCP enables responsible agentic AI deployments.

Social Aspects · Privacy

Xian Chen, Ruobing Bai, Pan Peng

Subgraph counting is a fundamental problem in graph analysis. Motivated by practical scenarios where graph analytics are performed on subgraphs induced by selected vertices -- rather than on the entire graph -- and by growing privacy concerns, we initiate the study of *differentially private range subgraph counting (DPRSC)*. The goal is to privately count occurrences of a fixed pattern graph within induced subgraphs defined by multi-dimensional attribute ranges. Unlike classical point counting, subgraph counting is inherently nonlinear and exhibits high sensitivity: a single edge modification can affect many subgraph occurrences. We present the first efficient algorithms for DPRSC with small additive error. Our approach introduces a subgraph projection that reduces DPRSC to weighted orthogonal range counting, enabling the use of range trees and local sensitivity estimation to achieve accurate private query answering. We complement our algorithms with matching lower bounds, obtained by reducing reconstruction attacks to DPRSC and leveraging discrepancy theory. In particular, we show that any differentially private algorithm for DPRSC must incur additive error exponential in the dimension. Empirical evaluations demonstrate that our algorithms significantly outperform baseline methods in accuracy while maintaining strong privacy guarantees.

Social Aspects · Accountability, Transparency, and Interpretability

Jinyang Liu, Munir Hiabu

Interpretable machine learning requires models that are accurate and structurally faithful to the data. Existing explainability methods rely heavily on additive representations (e.g., GAMs, SHAP, functional ANOVA), which can suffer from signal cancellation and extrapolation in presence of strong interactions. We propose Tensor Separation Learning (TSL), a regression model that learns a sum of separable (rank-1) tensor products via an orthogonal greedy algorithm. By enforcing separability, TSL avoids the information loss inherent in additive projections caused by marginalizing higher-order interactions. The learned TSL model can be fully reconstructed from first-order partial dependence functions of its fitted factors. We establish approximation-rate guarantees for functions with bounded mixed $ p $-th order partial derivatives and demonstrate that TSL competes with black-box models on regression benchmarks. Crucially, TSL improves interpretability by factorizing interactions, allowing users to explicitly disentangle the magnitude of an effect from its direction directly via the fitted factors.

Social Aspects · Safety

Humzah Merchant, Bradford Levy

Large Language Models (LLMs) frequently memorize sensitive training data, creating significant privacy and copyright risks. We present a novel unlearning framework rooted in the principle that learning is easier than forgetting. We first introduce \textbf{Divergence Decoding (DD)}, a mechanism that uses small, efficiently trained auxiliary models to steer the logits of the LLM away from specific data during inference. We then demonstrate this steered distribution can be trivially distilled back into the base model. Our method decisively outperforms \textbf{state-of-the-art (SOTA)} baselines on TOFU and MUSE benchmarks, and we find evidence of generalization in the domain of images. \href{https://anonymous.4open.science/r/targeted_unlearning_icml2026/}{Code is available at this anonymous link. }

Social Aspects · Safety

Neil Kale, Rebecca Portnoff, Pratiksha Thaker, Michael Simpson, Robertson Wang, Kevin Kuo, Chhavi Yadav, Virginia Smith

Modern artificial intelligence (AI) systems have transformative potential across many domains, but also present profound new risks to child safety. AI is increasingly being misused to create AI-generated child sexual abuse material, facilitate child sexual exploitation, and reduce barriers to harm. In this position paper, we argue that protecting children from AI-facilitated abuse requires new approaches to AI safety. Existing safety techniques assume data accessibility, transparency, and evaluation practices that are incompatible with the ethical and legal constraints surrounding child sexual abuse material. We examine how these constraints create new technical challenges, such as limitations on dataset auditing, red teaming, and fine-tuning prevention. In turn, we outline *15 open problems* in child safety across the AI development lifecycle---from dataset curation and model design to deployment and long-term maintenance. We propose targeted recommendations for researchers, developers, and policymakers to bridge the gap between theoretical AI safety and the realities of child protection. Our work aims to reframe child safety as a central, safety-critical dimension for AI research, motivating new work that translates responsible AI principles into concrete safeguards against the exploitation of children.

Social Aspects · Alignment

Yavuz Faruk Bakman, Duygu Nur Yaldiz, Eleni Triantafillou, Peter Kairouz, Salman Avestimehr, Sai Praneeth Reddy Karimireddy

Large Language Models (LLMs) are rarely static and are frequently updated in practice. A growing body of alignment research has shown that models initially deemed ``aligned'' can exhibit misaligned behavior after fine-tuning, such as forgetting jailbreak safety features or re-surfacing knowledge that was intended to be forgotten. These works typically assume that the initial model is aligned based on static black-box evaluation, i.e., the absence of undesired responses to a fixed set of queries. In contrast, we formalize model alignment in both the static and post-update settings and uncover a fundamental limitation of black-box evaluation. We theoretically show that, due to overparameterization, static alignment provides no guarantee of post-update alignment for \emph{any} update dataset. Moreover, we prove that static black-box probing cannot distinguish a model that is genuinely post-update robust from one that conceals an arbitrary amount of adversarial behavior, which can be activated by even a single benign gradient update. We further validate these findings empirically in LLMs across three core alignment domains: privacy, jailbreak safety, and behavioral honesty. We demonstrate the existence of LLMs that pass all standard black-box alignment tests, yet become severely misaligned after a single benign update. Finally, we show that the capacity to hide such latent adversarial behavior increases with model scale, confirming our theoretical prediction that post-update misalignment grows with the number of parameters. Together, our results highlight the inadequacy of static evaluation protocols and emphasize the urgent need for post-update--robust alignment evaluation.

Social Aspects · Security

Chaolei Han, Hongsong Wang, Jie Gui

Detecting AI-generated music is crucial for preserving artistic authenticity and preventing the misuse of generative music technologies. However, existing discriminative detectors typically rely on generated samples during training and often suffer from severe performance degradation when confronted with music produced by unseen generators, which limits their real-world applicability. To address this issue, we formulate a zero-shot setting for AI-generated music detection, where the detector is trained exclusively on real music without access to any generated samples. Under this setting, we propose MusicDET, a generator-agnostic detection framework based on frequency-guided normalizing flows that probabilistically models the distribution of real music features. By evaluating the likelihood of an input sample under the learned real-music distribution, MusicDET enables effective detection of out-of-distribution music signals. Experiments on the FakeMusicCaps and SONICS datasets show that MusicDET consistently outperforms conventional discriminative detectors, particularly when detecting music generated by previously unseen models.

Social Aspects · Safety

Xinyu Zhao, Rana Khan, Zhen Xu, Zhen Tan, Tianlong Chen

The formal integration of Large Language Models (LLMs) and Multimodal LLMs (MLLMs) into scientific peer-review workflows introduces novel and significant risks. Their safety against adversarial manipulation remains critically underexplored, especially given the multimodal nature of scientific papers where figures, not just text, convey core evidence. This creates a significant gap: current robustness studies are overwhelmingly text-only, and the problem is distinct from standard jailbreaking, as a peer-review attack seeks to induce a domain-specific, targeted failure (e.g., "inflate this score") rather than a general safety policy violation, for which no practical defenses exist. To address this, we introduce PaperGuard, the first comprehensive benchmark designed to systematically evaluate and defend AI-generated peer review against these domain-specific, cross-modal attacks. Our framework is built on three pillars: (1) a new multimodal peer-review dataset spanning multiple scientific domains; (2) a unified suite of attacks, including black-box prompt injections and white-box perturbations, specifically designed to target both text (GCG) and figures (PGD); and (3) a practical defense, motivated by the long-context challenge of academic papers, that uses chunk-based embedding search to efficiently localize and mitigate harmful instructions. Our extensive experiments, conducted across state-of-the-art models, confirm that these systems are pervasively vulnerable. PaperGuard establishes the foundational benchmark, protocols, and actionable defense necessary to pioneer trustworthy, attack-resilient AI-assisted scholarly reviewing.

Social Aspects · Safety

Huanli Gong, Zhipeng Wei, Yu Fu, Haz Shahgir, Ananya Gupta, Yue Dong, N. Benjamin Erichson

Multi-turn jailbreak attacks have emerged as a powerful threat to LLM safety, leveraging feedback from auxiliary judge models to iteratively refine harmful queries. Existing defenses mainly focus on detecting or blocking harmful content at the final turn, leaving the judge-driven refinement loop intact and allowing attackers to extract informative feedback from intermediate interactions. We introduce **D**ecoy for the **J**udge (DJ), a semantics-preserving output rewriting approach that intervenes directly in this feedback loop by modifying the victim LLM’s responses before they are evaluated by the attacker’s judge. By misaligning the judge’s feedback signal, DJ perturbs the judge-driven refinement loop and degrades the attacker’s optimization process while preserving the semantic content of the original response. To enable robust and transferable rewriting, we construct a dataset that captures fine-grained distinctions among semantically similar responses with differing harmfulness signals. Leveraging this dataset, DJ is trained using a combination of supervised fine-tuning (SFT) and direct preference optimization (DPO), allowing it to reliably manipulate judge feedback across diverse judge LLMs. Experiments on HarmBench show that DJ significantly reduces the success rate of state-of-the-art multi-turn jailbreaks, while preserving performance on benign multi-turn benchmarks.

Social Aspects · Privacy

Chang Ma, Jun Li, Kang Wei, Yipeng Zhou, Ming Ding, Yiyang Ni

Federated learning (FL) faces significant challenges from modality heterogeneity, which motivates multimodal federated learning (MFL) to leverage complementary modalities across decentralized clients for improved performance. However, modality imbalance introduces a new attack surface, making MFL more vulnerable to membership inference attacks (MIAs), an issue that remains largely unexplored. In this work, we present the first systematic study of MIAs against MFL and propose a modality-aware attack framework. We show that multimodal models are inherently more susceptible to MIAs due to heterogeneous modality contributions, and existing attacks are suboptimal as they treat multimodal parameters as a whole. By performing MIAs on individual modalities, we find that (i) attacking the dominant modality achieves comparable accuracy with lower overhead, and (ii) different modalities expose distinct membership patterns. To identify members with different patterns, we propose a modality-aware framework that exploits cross-modal performance gaps to adaptively select attack modalities and calibrate inference results. Experiments on three datasets show our approach outperforms baselines across multiple metrics.

Social Aspects · Privacy

Guangnian Wan, Gongfan Fang, Xinyin Ma, Xinchao Wang

Gradient Inversion Attack (GIA) poses a significant threat to federated learning, enabling adversaries to reconstruct private training data from the information shared during training. Prior research has predominantly focused on the vanilla SGD, where the server or an eavesdropper can directly observe true gradients. In practical deployments, however, models may be trained with adaptive optimizers (e.g., Adam, RMSProp, and AdaGrad), for which the observable signal is not raw gradients but momentum-based parameter updates. This setting remains underexplored and undermines traditional gradient-matching strategies, which struggle to recover labels and images from non-gradient updates. To address this gap, this paper explores attacks tailored to modern adaptive optimizers. We present an analytical rule for recovering labels from optimizer updates and propose an update-matching objective that optimizes dummy inputs to reproduce the observed updates. The proposed approach is general and can be directly applied to various optimizers such as Adam, AdaGrad, and RMSProp. Furthermore, we find that, despite being introduced for adaptive optimizers, the proposed objective function also yields stronger attacks in the standard SGD setting. Experiments on datasets such as ImageNet and PACS highlight the effectiveness of our method over existing gradient matching techniques.

Social Aspects · Privacy

Xun Ran, Qingqing Ye, Xin Huang, Jianliang Xu, Haibo Hu

Cross-silo recommendation from implicit feedback is a key task in modern recommender systems, where user-item interaction data are distributed across multiple parties and cannot be centrally collected. Unlike explicit feedback, which provides fully observed real-valued ratings, implicit feedback is one-class and extremely sparse, recording only users' actions or inactions (e.g., clicks, visits, or bookmarks), yet it is far more prevalent in real-world applications. Such behavioral data are often highly sensitive, raising significant privacy concerns when used for collaborative model training. Although differential privacy (DP) has been widely applied to explicit feedback-based models, the resulting utility degradation makes it difficult to apply DP effectively to implicit feedback learning. In this work, we propose DPIMF, a differentially private implicit matrix factorization framework for cross-silo recommendation based on objective perturbation. To improve utility, we redesign the loss function and adopt an importance sampling scheme to reduce the noise scale required for privacy preservation. We further provide formal utility guarantees for the proposed techniques and characterize the conditions under which utility improvements are maximized. Experiments on three benchmark datasets validate our theoretical analysis and demonstrate that DPIMF achieves a better privacy-utility trade-off than state-of-the-art methods.

Social Aspects · Privacy

Puwei Lian, Yujun Cai, Songze Li, Bingkun BAO

Diffusion models have achieved tremendous success in image generation, but they also raise significant concerns regarding privacy and copyright issues. Membership Inference Attacks (MIAs) are designed to ascertain whether specific data were utilized during a model's training phase. As current MIAs for diffusion models typically exploit the model's image prediction ability, we formalize them into a unified general paradigm which computes the membership score for membership identification. Under this paradigm, we empirically find that existing attacks overlook the inherent deficiency in how diffusion models process high-frequency information. Consequently, this deficiency leads to member data with more high-frequency content being misclassified as hold-out data, and hold-out data with less high-frequency content tend to be misclassified as member data. Moreover, we theoretically demonstrate that this deficiency reduces the membership advantage of attacks, thereby interfering with the effective discrimination of member data and hold-out data. Based on this insight, we propose a plug-and-play high-frequency filter module to mitigate the adverse effects of the deficiency, which can be seamlessly integrated into any attacks within the general paradigm without additional time costs. Extensive experiments corroborate that this module significantly improves the performance of baseline attacks across different datasets and models.