论文检索

Social Aspects · Accountability, Transparency, and Interpretability

Shuhang Lin, Chuhao Zhou, Xiao Lin, Zihan Dong, Kuan Lu, Zhencan Peng, Jie Yin, Dimitris Metaxas

While Conformal Prediction (CP) offers a principled framework for producing prediction sets with statistical guarantees, prior methods suffer from critical limitations in both calibration validity and score discriminability, resulting in violated coverage guarantees and excessively large prediction sets. To address these pitfalls, we propose Conformal Path Reasoning (CPR), a trustworthy KGQA framework with two key innovations. First, we perform query-level conformal calibration over path-level scores, preserving the exchangeability while generating path prediction sets. Second, we introduce the Residual Conformal Value Network (RCVNet), a lightweight module trained via PUCT-guided exploration to learn discriminative path-level nonconformity scores. Experiments on benchmarks show that CPR significantly improves the Empirical Coverage Rate by 34% while reducing average prediction set size by 40% compared to conformal baselines. These results validate the efficacy of CPR in satisfying coverage guarantees with substantially more compact answer sets.

Social Aspects · Everything Else

Ujwal Kumar, Alice Saito, Hershraj Niranjani, Rayan Yessou, Tan Phan Xuan

Constitutional AI has focused on single-model alignment using fixed principles. However, multi-agent systems create novel alignment challenges through emergent social dynamics. We present Constitutional Evolution, a framework for automatically discovering behavioral norms in multi-agent LLM systems. Using a grid-world simulation with survival pressure, we study the tension between individual and collective welfare, quantified via a Societal Stability Score $\mathcal{S} \in [0,1]$ that combines productivity, survival, and conflict metrics. Adversarial constitutions lead to societal collapse ($\mathcal{S}=0$), while vague prosocial principles (''be helpful, harmless, honest'') produce inconsistent coordination ($\mathcal{S}=0.249$). Even constitutions designed by Claude 4.5 Opus with explicit knowledge of the objective achieve only moderate performance ($\mathcal{S}=0.332$). Using LLM-driven genetic programming with multi-island evolution, we evolve constitutions maximizing social welfare without explicit guidance toward cooperation. The evolved constitution $\mathcal{C}^*$ achieves $\mathcal{S}=0.556\pm0.008$ (123\% higher than human-designed baselines, $N=10$), eliminates conflict, and discovers that minimizing communication (0.9\% vs 62.2\% social actions) outperforms verbose coordination. Our interpretable rules demonstrate that cooperative norms can be discovered rather than prescribed.

Social Aspects · Everything Else

Rashid Mushkani

This position paper argues that prompts used to deploy large language models (LLMs) in public-sector settings should be treated as governed artefacts rather than private, transient inputs. Prompts encode role instructions, decision framings, and value claims; prompt choice can materially shift outputs even when model weights and input records are held fixed. Existing governance tools, including model and dataset documentation, organisation-level policies, and post-training alignment, rarely make the local prompt collections used in deployment transparent, contestable, or auditable. We propose Prompt Commons: a versioned, community-maintained repository of prompt templates with provenance metadata, licensing, and moderation logs. Using a pilot dataset collected with community partners in a large North American city (443 human prompts; 3,317 after augmentation), we illustrate three governance states (open, curated, veto-enabled) and a negotiation-oriented ensemble method that aggregates stakeholder prompts into compromise recommendations. We close with falsifiable implications and an evaluation agenda for prompt-layer governance.

Social Aspects · Everything Else

Wenjun Cao

This position paper argues that modern machine learning creates structural risks for knowledge and cultural production, operating before AGI thresholds through market selection mechanisms. We use \emph{temporality} operationally for how understanding changes over time and signals left by that process. Representation learning and autoregressive generation approximate output distributions while omitting slow, path-dependent human learning; at scale, these function as general-purpose production technologies. We analyze the link from technical indistinguishability to market selection: when divergence between model and temporal signals is small and verification costly, decision makers cease screening, prices track pooled quality, and temporality-intensive work exits. We call this phenomenon \emph{value collapse}. Recent evidence shows this active: academic publishing has experienced dramatic productivity increases alongside troubling quality trends; cultural production shows explosive AI-generated content adoption. As training data mirror such environments, models absorb their outputs and model collapse risk rises. Alignment is orthogonal: by narrowing observable gaps, it intensifies selection pressures where provenance remains costly.

Social Aspects · Security

Zhengjie Zhou, Jiahuan Yan, Boqun Ma, Weiwei Feng, Tengfei LIU, Weiqiang Wang

Generating realistic adversarial examples for tabular data remains challenging due to heterogeneous feature types and asymmetric inter-feature dependencies. Existing approaches typically rely on handcrafted constraints or undirected similarity criteria to delimit the feasible attack region, which often fail to capture the structural dependency governing tabular generation. Consequently, standard attacks typically produce perturbations that are statistically likely yet semantically inconsistent and prone to optimization stagnation via gradient masking. To address this, we propose LCSA, a white-box framework that formulates adversarial generation as optimization over structurally admissible perturbations. LCSA leverages an ensemble of heterogeneous neural Structural Causal Models to infer dependencies and introduces a structure-aware ripple mechanism. Unlike attacks that perturb features in isolation, this mechanism propagates updates downstream, acting as a structural preconditioner that conditions gradient flow to mitigate masking effects. Extensive experiments demonstrate that LCSA outperforms state-of-the-art baselines in 45 of 50 evaluated configurations, yielding adversarial examples with superior structural consistency and transferability.

Social Aspects · Security

Pengfei He, Ash Fox, Lesly Miculicich, Stefan Friedli, Daniel Fabian, Burak Gokturk, Jiliang Tang, Chen-Yu Lee, Tomas Pfister, Long T. Le

Large language models (LLMs) have shown promise in assisting cybersecurity tasks, yet existing approaches struggle with automatic vulnerability discovery and exploitation due to limited interaction, weak execution grounding, and a lack of experience reuse. We propose Co-RedTeam, a security-aware multi-agent framework designed to mirror real-world red-teaming workflows by integrating security-domain knowledge, code-aware analysis, execution-grounded iterative reasoning, and long-term memory. Co-RedTeam decomposes vulnerability analysis into coordinated discovery and exploitation stages, enabling agents to plan, execute, validate, and refine actions based on real execution feedback while learning from prior trajectories. Extensive evaluations on challenging security benchmarks demonstrate that Co-RedTeam consistently outperforms strong baselines across diverse backbone models, achieving over 60\% success rate in vulnerability exploitation and over 10\% absolute improvement in vulnerability detection. Ablation and iteration studies further confirm the critical role of execution feedback, structured interaction, and memory for building robust and generalizable cybersecurity agents.

Social Aspects · Security

Jiacheng Liu, Yaxin Luo, Jiacheng Cui, Xinyi Shang, Xiaohan Zhao, Zhiqiang Shen

The rapid evolution of GUI-enabled agents has rendered traditional CAPTCHAs obsolete. While previous benchmarks like OpenCaptchaWorld established a baseline for evaluating multimodal agents, recent advancements in reasoning-heavy models, such as Gemini3-Pro-High and GPT-5.2-Xhigh have effectively collapsed this security barrier, achieving pass rates as high as 90\% on complex logic puzzles like ''Bingo''. In response, we introduce Next-Gen CAPTCHAs, a scalable defense framework designed to secure the next-generation web against the advanced agents. Unlike static datasets, our benchmark is built upon a robust data generation pipeline, allowing for large-scale and easily scalable evaluations, notably, for backend-supported types, our system is capable of generating effectively unbounded CAPTCHA instances. We exploit the persistent human--agent ``Cognitive Gap'' in interactive perception, memory, decision-making, and action. By engineering dynamic tasks that require adaptive intuition rather than granular planning, we re-establish a robust distinction between biological users and artificial agents, offering a scalable and diverse defense mechanism for the agentic era.

Social Aspects · Security

Chunlong Xie, Kangjie Chen, Shangwei Guo, Shudong Zhang, Jiamou Liu, Tianwei Zhang, Tao Xiang

The limited transferability of adversarial attacks on Vision-Language Models (VLMs) stems from their failure to navigate model-specific safety alignments, where superficial perturbations exploit surrogate-specific artifacts rather than shared safety-critical features. We reveal through linear probing that safety-related representations are concentrated within specific intermediate neuronal circuits, which act as localized defense bottlenecks that can be disentangled from transferable features. To overcome this barrier, we propose the Safety Circuit Intervention Attack (SCIA), a framework that surgically steers internal representations to bypass these localized safety mechanisms. SCIA employs a dual-objective steering strategy that suppresses the defensive circuit encoding safety features while amplifying the transferable circuit capturing model-agnostic representations, effectively decoupling adversarial patterns from surrogate-specific safety behaviors. Furthermore, we incorporate contrastive semantic steering and spectral smoothness regularization to guide optimization toward compliant semantic regions while producing visually coherent perturbations. Experimental results demonstrate that SCIA significantly outperforms state-of-the-art methods in bypassing unseen black-box VLMs.

Social Aspects · Security

Ziyuan Fang, Tianyi Wei, Guanjie Wang, Weiming Zhang, Nenghai Yu, Wenbo Zhou

Recent advances in video generation have enabled highly realistic synthetic content, raising concerns about the integrity of digital media and motivating the development of benchmarks and detection methods for generated videos. Prior works have largely prioritized bolstering model generalization against unseen generators. However, we uncover a neglected factor: the quality distribution of real videos plays a pivotal role. Current training protocols suffer from a clear quality bias between real and fake data, prone to shortcut learning. Compounded by testing on similar real data distributions, this creates an illusion of generalization. In reality, these models fail to generalize when exposed to real data with significantly different quality profiles. To address this, we propose training with quality-matched real and fake data to mitigate bias. Building on this, we introduce a data expansion strategy that broadens the training set to comprehensively cover the full quality spectrum. This approach enables the model to learn quality-agnostic features for detection, thereby achieving generalization across real data of varying qualities and enhancing real-world applicability. Extensive experiments demonstrate that our method scales well across diverse backbones, consistently enhancing the generalization capability of existing models.

Social Aspects · Security

Hongyi Zhou, Jianfeng Pan, Min Peng, Shaomang Huang, Xuling Zhang

Endpoint Detection and Response (EDR) systems are crucial for identifying malicious activities on endpoint devices, yet existing methods struggle to efficiently model ultra-long log sequences and to provide interpretable reasoning for security analysts. We propose WatchLog, a novel framework that represents raw logs as video-structured data, enabling scalable and expressive video-language modeling of endpoint behaviors. Each event is encoded as a key–value-guided image, and the resulting images are temporally organized into a video sequence. To capture long-range dependencies, WatchLog employs a temporal cross-attention adapter that enables pixel-wise interaction across time. The adapter acts as an auxiliary temporal reasoning pathway, aligning spatial representations with relevant temporal contexts while preserving the original behavioral semantics. We adopt a two-stage pre-training strategy followed by supervised fine-tuning to generate behavior explanations grounded in event-level semantics and detection outcomes. Experiments on our newly constructed EDR8M-20R dataset and a public benchmark demonstrate that WatchLog consistently outperforms state-of-the-art methods in detection accuracy and recall, while offering more interpretable reasoning traces and significantly improved inference efficiency. Extensive ablation studies further support the robustness and interpretability of the proposed method.

Social Aspects · Security

Chengran Yang, zichao wei, Heminghao Deng, Jinfeng Jiang, Zhensu Sun, Ting Zhang, Tianyi Wu, Ming Wen, David Lo

Large Language Model (LLM) based code generation is predominantly formulated as a strictly monotonic process, appending tokens linearly to an immutable prefix. This formulation contrasts to the cognitive process of programming, which is inherently interleaved with forward generation and on-the-fly revision. While prior works attempt to introduce revision via post-hoc agents or external static tools, they either suffer from high latency or fail to leverage the model's intrinsic semantic reasoning. In this paper, we propose Stream of Revision, a paradigm shift that elevates code generation from a monotonic stream to a dynamic, self-correcting trajectory by leveraging model's intrinsic capabilities. We introduce specific action tokens that enable the model to seamlessly backtrack and edit its own history within a single forward pass. By internalizing the revision loop, our framework Stream of Revision allows the model to activate its latent capabilities just-in-time without external dependencies. Empirical results on secure code generation show that Stream of Revision significantly reduces vulnerabilities with minimal inference overhead. We release our code and dataset at \url{https://anonymous.4open.science/r/Stream-of-Revision-CB74/}, and models will be released upon publication.

Social Aspects · Security

Nikil Selvam, Jay Baxter, Sophie Hilgard, Brad Miller, Keith Coleman, Ellen Vitercik, Sanmi Koyejo

Crowdsourced fact-checking systems have been widely adopted by major social media companies such as X, Meta, Tiktok, and Google with the aim of combating misinformation at scale without relying on centralized editorial control. These systems have been developed around a common underlying algorithm: a bridging mechanism—based on matrix factorization—that surfaces notes indicating misinformation only when they receive support from diverse ideological groups rather than simple majority support. Although this algorithm is designed to be robust against traditional brigading, we demonstrate an attack showing that coordinated users can strategically fabricate diverse agreement in the system’s latent space to manipulate the scoring algorithm. We validate this attack on real-world production data and find that a surprisingly large number of notes’ scores can potentially be manipulated with a small number (< 10) of coordinated votes, raising the risk that adversaries could surface arbitrary notes on these social media platforms. We complement these findings with a theoretical analysis of voting strategies that surface arbitrary notes, revealing counterintuitive properties of the system: for instance, rating a note as “Not Helpful” can increase its helpfulness score. Finally, we develop a cost model quantifying manipulation effort and discuss potential mitigations. Following a responsible disclosure process, X's Community Notes team acknowledged this attack and has deployed mitigations based on our findings. We hope this work spurs further research into the robustness of crowdsourced fact-checking systems and, more broadly, bridging-based consensus mechanisms.

Social Aspects · Security

Jiazhen Yan, Ziqiang Li, Fan Wang, Boyu Wang, Ziwen He, Zhangjie Fu

The rapid progress of generative models such as GANs and diffusion models has led to the widespread proliferation of AI-generated images, raising concerns about misinformation, privacy violations, and trust erosion in digital media. Although large-scale multimodal models like CLIP offer strong transferable representations for detecting synthetic content, fine-tuning them often induces catastrophic forgetting, which degrades pre-trained priors and limits cross-domain generalization. To address this issue, we propose the Distillation-guided Gradient Surgery Network (DGS-Net), a novel framework that preserves transferable pre-trained priors while suppressing task-irrelevant components. Specifically, we introduce a gradient-space decomposition that separates harmful and beneficial descent directions during optimization. By projecting task gradients onto the orthogonal complement of harmful directions and aligning with beneficial ones distilled from a frozen CLIP encoder, DGS-Net achieves unified optimization of prior preservation and irrelevant suppression. Extensive experiments on 50 generative models demonstrate that our method outperforms state-of-the-art approaches by an average margin of 6.6%, achieving superior detection performance and generalization across diverse generation techniques.

Social Aspects · Safety

Xuyang Zhong, Qizhang Li, Yiwen Guo, Chen Liu

We propose **DualOptim+**, a novel optimization framework for improving machine unlearning in large language models. It introduces a base state to capture common representations shared by forgetting and retaining objectives and delta states to preserve objective-specific residuals. This architecture allows the optimizer to adaptively bridge shared and decoupled states based on the directional conflict between forgetting and retaining gradients. We further introduce DualOptim+ 8bit, a quantized variant that reduces memory overhead without compromising performance. Extensive experiments across fictitious, real-world, and safety alignment tasks demonstrate that DualOptim+ consistently achieves a superior trade-off between forgetting efficacy and model utility.

Social Aspects · Safety

Maya Okawa, Bo Zhao, Eric Bigelow, Rose Yu, Tomer Ullman, Ekdeep Singh Lubana, Hidenori Tanaka

As large language models (LLMs) increasingly power conversational agents, understanding how they model users' emotional states is critical for ethical deployment. Inspired by emotion wheels, i.e., a psychological framework that argues emotions organize hierarchically, we analyze probabilistic dependencies between emotional states in model outputs. We find that LLMs naturally form hierarchical emotion trees that align with human psychological models, and larger models develop more complex hierarchies. We also uncover systematic biases in emotion recognition across socioeconomic personas, with compounding misclassifications for intersectional, underrepresented groups. Human studies reveal striking parallels, suggesting that LLMs internalize aspects of social perception. Beyond highlighting emergent emotional reasoning in LLMs, our results hint at the potential of using cognitively-grounded theories for developing better model evaluations.

Social Aspects · Safety

Xinbo Wu, Huan Zhang, Abhishek Umrawal, Lav Varshney

As large language models grow increasingly capable, concerns about their safe deployment have intensified. While numerous alignment strategies aim to restrict harmful behavior, these defenses can still be circumvented through carefully designed adversarial prompts. In this work, we introduce a theoretical framework that formalizes a game between an attacker and a defender. Within this framework, we design a theoretical best-response attack strategy and show that it is closely related to many existing adversarial prompting methods. We further analyze the resulting game, characterize its equilibria, and reveal inherent advantages for the attacker. Drawing on our theoretical analysis, we also derive a provably optimal defense strategy. Empirically, we evaluate a practical instantiation of the theoretically optimal attack and observe stronger performance relative to existing adversarial prompting approaches in diverse settings encompassing different LLMs and benchmarks.

Social Aspects · Safety

Nanxiang Jiang, Zhaoxin Fan, Baisen Wang, Daiheng Gao, Junhang Cheng, Jifeng Guo, Yalan Qin, Yeying Jin, Hongwei Zheng, Faguo Wu 等

Concept erasure serves as a vital safety mechanism for removing unwanted concepts from text-to-image (T2I) models. While extensively studied in U-Net and dual-stream architectures (e.g., Flux), this task remains under-explored in the recent emerging paradigm of single-stream diffusion transformers (e.g., Z-Image). In this new paradigm, text and image tokens are processed as a single unified sequence via shared parameters. Consequently, directly applying prior erasure methods typically leads to generation collapse. To bridge this gap, we introduce Z-Erase, the first concept erasure method tailored for single-stream T2I models. To guarantee stable image generation, Z-Erase first proposes a Stream Disentangled Concept Erasure Framework that decouples updates and enables existing methods on single-stream models. Subsequently, within this framework, we introduce Lagrangian-Guided Adaptive Erasure Modulation, a constrained algorithm that further balances the sensitive erasure-preservation trade-off. Moreover, we provide a rigorous convergence analysis proving that Z-Erase can converge to a Pareto stationary point. Experiments demonstrate that Z-Erase successfully overcomes the generation collapse issue, achieving state-of-the-art performance across a wide range of tasks.

Social Aspects · Safety

Aradhye Agarwal, Gurdit Singh Siyan, Yash Pandya, Joykirat Singh, Akshay Nambi, Ahmed Awadallah

Agentic language models operate in a fundamentally different safety regime than chat models: they must plan, call tools, and execute long-horizon actions where a single misstep, such as accessing files or entering credentials, can cause irreversible harm. Existing alignment methods, largely optimized for static generation and task completion, break down in these settings due to sequential decision-making, adversarial tool feedback, and overconfident intermediate reasoning. We introduce MOSAIC, a post-training framework that aligns agents for safe multi-step tool use by making safety decisions explicit and learnable. MOSAIC structures inference as a plan, check, then act or refuse loop, with explicit safety reasoning and refusal as first-class actions. To train without trajectory-level labels, we use preference-based reinforcement learning with pairwise trajectory comparisons, which captures safety distinctions often missed by scalar rewards. We evaluate MOSAIC zero-shot across three model families, Qwen2.5-7B, Qwen3-4B-Thinking, and Phi-4, and across out-of-distribution benchmarks spanning harmful tasks, prompt injection, benign tool use, and cross-domain privacy leakage. MOSAIC reduces harmful behavior by up to 50\%, increases harmful-task refusal by over 20\% on injection attacks, cuts privacy leakage, and preserves or improves benign task performance, demonstrating robust generalization across models, domains, and agentic settings.

Social Aspects · Safety

William Overman, Mohsen Bayati

As increasingly capable agents are deployed, a central safety challenge is how to retain meaningful human control without modifying the underlying system. We study a minimal control interface in which an agent chooses whether to act autonomously (play) or defer (ask), while a human simultaneously chooses whether to be permissive (trust) or engage in oversight (oversee), and model this interaction as a two-player Markov Game. When this game forms a Markov Potential Game, we prove an alignment guarantee: any increase in the agent's utility from acting more autonomously cannot decrease the human's value. This establishes a form of intrinsic alignment where the agent's incentive to seek autonomy is structurally coupled to the human's welfare. Practically, the framework induces a transparent control layer encouraging the agent to defer when risky and act when safe. While we use gridworld simulations to illustrate the emergence of this collaboration, our primary validation involves an agentic tool-use task where two 30B-parameter language models are fine-tuned via independent policy gradient. We demonstrate that even as the agents learn to coordinate on the fly, this framework effectively reduces safety violations in realistic, open-ended environments.

Social Aspects · Safety

Charles Wang, Keir Dorchen, Peter Jin

Contemporary AI safety is increasingly a full-stack discipline. It spans pretraining interventions, post-training alignment (instruction tuning, RLHF and preference-optimization variants), and deployment-time controls (guardrails, monitoring, and red-teaming). This paper argues these efforts optimize the wrong primary target when it comes to self-improving agents: behavioral compliance today rather than teachability tomorrow. Building on the concept of the utility-learning tension formalized by Wang et al., we argue that utility-driven self-modification can erode learnability itself, yielding structural incorrigibility as an emergent consequence of optimization. We therefore call for a shift in priorities from behavioral alignment to enforceable learnability floors that preserve long-run corrigibility under bounded intervention.