论文检索

Social Aspects · Privacy

Hengliang Wu, Jiale Yang, Youming Tao, Shuzhen Chen, Di Wang, Dongxiao Yu

Decentralized Unlearning (DU) aims to remove the influence of specific clients from a collaboratively trained global model. However, existing methods suffer from strong reliance on static, problem-specific hyperparameters or restrictive convexity assumptions, limiting their general applicability. To overcome these limitations, we propose **TRA**jectory-aware **CE**rtified **D**ecentralized **U**nlearning (**TRACE-DU**), a generic unlearning framework for decentralized training. **TRACE-DU** introduces a fine-grained sensitivity analysis that leverages local SGD updates and decentralized training dynamics, thereby eliminating the need for convexity assumptions and reducing dependence on manually tuned parameters. By integrating strategic checkpoint selection with calibrated noise perturbation, the proposed framework enables efficient certified unlearning. Moreover, we exploit historical model trajectories to extend this framework, enabling it to naturally support sequential unlearning requests from an arbitrary number of clients. We provide theoretical guarantees for certified unlearning and derive sensitivity bounds under both convex and non-convex loss functions. Experimental results demonstrate that our framework outperforms state-of-the-art baselines across diverse metrics.

Social Aspects · Safety

Mohammed Alshaalan, Miguel Rodrigues

Optimization-based adversarial suffixes can jailbreak aligned large language models (LLMs) while remaining fluent, weakening detectors based on static global or windowed perplexity statistics. We cast adversarial suffix detection as an \emph{online change-point detection} problem over the token-level next-token entropy stream. Using the fixed system prompt to estimate a robust baseline via the median and median absolute deviation, we standardize user-token entropies and monitor them with a one-sided CUSUM statistic. The resulting detector is model-agnostic, training-free, operates online, and localizes the onset of adversarial suffixes. On a benchmark of $724$ optimization-based suffix attacks (GCG, AutoDAN, AdvPrompter) and $765$ benign prompts from a TyDiQA+OpenOrca mixture with controlled post-prefix perplexity, CPD consistently outperforms perplexity baselines; on LLaMA-2-7B it reaches AUROC $0.90$ and F1 $0.82$. At an operating point with $\approx 10\%$ benign false-positive rate, CPD detects $74\%$ of suffix attacks and concentrates $87\%$ of its triggers inside the adversarial suffix. By comparison, windowed perplexity detects $35$--$43\%$ and frequently fires on boundary-straddling windows. Finally, we show CPD Online can act as a lightweight gate for LLaMA Guard, reducing guard invocations by $17$--$22\%$ on a high-volume stream dominated by benign prompts while preserving guard-level detection quality.

Deep Learning · Large Language Models

Nhi Nguyen, Shauli Ravfogel, Rajesh Ranganath

Large language models (LLMs) are increasingly deployed in high-stakes domains, where free-text explanations such as chain-of-thought and post-hoc rationales are used to justify model outputs. Yet it remains unclear whether these explanations are _sufficient_, i.e., if they contain enough information to explain the model’s output-generating process. We generalize classical sufficiency from feature attributions to arbitrary explanations and prove that explanation sufficiency is inherently relative to an input distribution, which must be explicitly defined for LLM explanations. We propose using the LLM itself to generate alternative inputs conditioned on an explanation, capturing its beliefs about possible inputs. We formalize _self-consistent sufficiency_ as a goal for free-text explanations and introduce an information-theoretic metric, SCSuff, that enables evaluation of free-text explanations without relying on predefined biases or shortcuts. Our experiments show that SCSuff aligns with targeted perturbation tests where applicable and demonstrate that explanation sufficiency can vary with the input distribution. We further find that SCSuff is uncorrelated with model size, accuracy, or uncertainty, suggesting that improving self-consistent sufficiency requires approaches beyond scaling or standard performance optimization.

Deep Learning · Generative Models and Autoencoders

Nan Bao, Yifan Zhao, Wenzhuang Wang, Jia Li

The layout-to-image (L2I) task enables fine-grained control over image generation via object categories and spatial layouts. However, existing L2I methods yield fragmented and distorted generations under few-shot atypical settings. We term this failure as representation fragmentation, arising from a granularity mismatch that entangles semantic identity with visual details. To address this issue, we propose a representation-driven framework that disentangles semantics from primitives for robust few-shot adaptation. Specifically, Semantic Anchoring aggregates categorical semantics into anchors for stable identity, while Primitive Imbuing models recomposable primitives for robust local detail modeling. Conceptual Steering further regulates optimization with a saliency-aware objective to preserve foreground semantic consistency. Extensive experiments demonstrate consistent improvements in the 5-shot regime over state-of-the-art L2I methods in both visual fidelity and alignment across diverse atypical domains.

Applications · Computer Vision

Jiahao Wang, Fang Liu, Licheng Jiao, Shuo Li, Hao Wang, Lingling Li, Xinyi Wang, Xu Liu

Real-time satellite video tracking poses distinct challenges, including accommodating high spatial-temporal resolution, dynamic backgrounds, and constrained onboard computational resources. While Discriminative Correlation Filter (DCF)-based methods offer high-speed inference, they suffer from limited accuracy. In contrast, Vision Transformer (ViT)-based trackers achieve strong performance by unifying representation and aggregation in a single-stream design, yet their heavy computational footprint limits practical deployment in real-time satellite scenarios. In this work, we present FATrack, a novel tracking framework that effectively balances tracking accuracy and computational efficiency. At its core is FA-ViT, a lightweight Vision Transformer backbone that introduces foreground-aware token routing, enabling the model to concentrate computation on target-relevant regions while suppressing redundancy. To mitigate semantic degradation caused by token sparsification, we propose the Adaptive Scatter Module (ASM), which selectively reinforces informative tokens via joint spatial-channel attention and sparse structural propagation, thereby enhancing both semantic fidelity and spatial coherence. By synergistically integrating FA-ViT and ASM, FATrack forms a unified architecture that delivers real-time performance with significantly improved tracking precision. Extensive evaluations on multiple satellite video benchmarks demonstrate that FATrack surpasses existing real-time trackers in accuracy and achieves inference efficiency comparable to DCF-based methods, highlighting its potential for practical deployment in large-scale aerial video tracking systems.

Applications · Everything Else

Alberto Alfarano, Eshika Saxena, Emily Wenger, Francois Charton, Kristin Lauter

ML attacks on Learning with Errors (LWE) with binary or small secrets only succeed on LWE settings with very simple secrets. For example, they can recover secrets with up to three non-zero bits when models are trained on not-reduced LWE data, and three non-zero bits in the ''cruel region'' [9] when BKZ pre-processing is applied. We show that larger training sets and the use of repeated examples in the training data allow the recovery of denser secrets. We empirically observe a power-law relationship between model based attempts to recover the secrets, dataset size and repeated examples. We introduce a stepwise regression technique to recover the ``cool bits'' of the secret. Overall, these techniques allow for the recovery of denser binary secrets: up to Hamming weight $70$ (and $8$ cruel bits) for dimension $256$ $\log_2 q=20$ and $75$ (and $7$ cruel bits) for dimension $512$ $\log_2 q=41$ (vs $33$ and $63$ Hamming weight and $3$ cruel bits in previous works). We also demonstrate our methods' effectiveness on denser ternary secrets, showing a substantial improvement over prior work.

Applications · Chemistry, Physics, and Earth Sciences

Yunyang Li, Lin Huang, Luojia Xia, Wenhe Zhang, Mark Gerstein

Generative models for 3D molecular conformations must respect Euclidean symmetries and concentrate probability mass on thermodynamically favorable, mechanically stable structures. However, E(3)-equivariant diffusion models often reproduce biases from semi-empirical training data rather than capturing the equilibrium distribution of a high-fidelity Hamiltonian. While physics-based guidance can correct this, it faces two computational bottlenecks: expensive quantum-chemical evaluations (e.g., DFT) and the need to repeat such queries at every sampling step. We present Elign, a post-training framework that amortizes both costs. First, we replace expensive DFT evaluations with a faster, pretrained foundational machine-learning force field (MLFF) to provide physical signals. Second, we eliminate repeated run-time queries by shifting physical steering to the training phase. To achieve the second amortization, we formulate reverse diffusion as a reinforcement learning problem and introduce Force--Energy Disentangled Group Relative Policy Optimization (FED-GRPO) to fine-tune the denoising policy. FED-GRPO includes a potential-based energy reward and a force-based stability reward, which are optimized and group-normalized independently. Experiments show that Elign generates conformations with lower gold-standard DFT energies and forces, while improving stability. Crucially, inference remains as fast as unguided sampling, since no energy evaluations are required during generation.

Theory · Game Theory

Achref Doula, Otthein Herzog, Siegfried WU, Max Mühlhäuser

AI-assisted decision-making is subject to AI model uncertainty. Prior works proposed to make this uncertainty explicit for increasing trust and transparency, but its behavioral role was rarely treated. This position paper argues, from a game-theoretic perspective, that human–AI decision support should be viewed as a repeated mechanism in which AI uncertainty functions as a strategic signal that shapes how users adopt reliance policies over time. We formalize a framework in which the interface specifies uncertainty signals, user response such as accepting versus verifying, and the resulting policy-shaping consequences. These repeated steps are used to characterize near-separating reliance regimes. A first pilot study conducted with 180 participants supports our proposition: Our game-theoretic mechanism increased verification and sharply reduced blind acceptance of wrong AI outputs. These initial results support treating human–AI interaction as a game-theoretic mechanism with uncertainty as a strategic signal, rather than a static model property or purely informational label.

Theory · Probabilistic Methods

Achref Doula

Conformal prediction converts point predictions into set-valued predictions with coverage guarantees under exchangeability between calibration and deployment data. We study *conformal calibration transfer*, where this requirement fails because labeled calibration is available only in a source space, while prediction sets are needed in a target space linked to the source through *unlabeled paired* observations (e.g., paired modalities or sensor changes). We propose Transported Conformal Calibration (TCC): we transport labeled source calibration into the target space using the paired data, and then correct residual post-transport mismatch using only unlabeled target inputs. We instantiate this correction with two complementary methods: **TCC-KS**, which uses a label-free uncertainty surrogate to detect mismatch and adjust calibration conservatively, and **weighted-TCC**, which reweights transported calibration toward the target domain for improved efficiency when weights are stable. We provide finite-sample target-domain coverage guarantees that adapt to an observable measure of mismatch. Across CIFAR-100-C, Tiny-ImageNet-C, and SEN12MS, we show reliable target-domain coverage transfer without labeled target calibration data, with label-free diagnostics that predict when correction is needed.

Deep Learning · Generative Models and Autoencoders

Jiankai Zuo, Yang Zhang, Yu Zhang, Jiarui Liang, YAYING ZHANG

Modeling dynamic dependencies from irregularly sampled event sequences is a fundamental challenge in modern machine learning. In many real-world systems, individual-level states evolve continuously over time while being simultaneously influenced by population-level distributional dynamics. However, existing methods typically model these processes in isolation or rely on discrete-time approximations that fail to capture long-range temporal irregularities and sparse observations. This paper studies the problem of learning coupled continuous-time latent dynamics from irregular events, where individual event sequences and global distributional processes evolve asynchronously and interact over time. We propose a Coupled Continuous-Time Latent Dynamics (CoCLD) framework that jointly models individual latent dynamics and population-level distributional shifts, and aligns them in a continuous-time latent space. CoCLD integrates a Diffusion-based Latent Interpolator with Neural Ordinary Differential Equations (Neural ODEs), enabling principled interpolation, generation, and alignment of latent states across arbitrary time points. We show that the proposed coupling mechanism yields a consistent estimator of continuous-time latent dynamics under sparse and irregular observations. Empirical evaluations demonstrate that CoCLD effectively captures dynamic dependencies and generalizes across diverse tasks, including next-event prediction, mobility trajectory generation, and sequential behavior modeling. These results suggest that learning coupled continuous-time latent dynamics provides a powerful paradigm for irregular event sequence modeling.

Social Aspects · Alignment

Woojin Kim, Sieun Hyeon, Jusang Oh, Jaeyoung Do

Aligning Large Language Models (LLMs) with the diverse spectrum of human values remains a central challenge: preference-based methods often fail to capture deeper motivational principles. Value-based approaches offer a more principled path, yet three gaps persist– extraction often ignores hierarchical structure, evaluation detects presence but not calibrated intensity, and therefore, the steerability of LLMs at controlled intensities remains insufficiently understood. To address these limitations, we introduce VALUEFLOW, the first unified framework that spans extraction, evaluation, and steering with calibrated intensity control. The framework integrates three components: (i) HIVES, a hierarchical value embedding space that captures intra- and crosstheory value structure; (ii) the Value Intensity DataBase (VIDB), a large-scale resource of valuelabeled texts with intensity estimates derived from ranking-based aggregation; and (iii) an anchorbased evaluator that produces consistent intensity scores for model outputs by ranking them against VIDB panels. Using VALUEFLOW, we conduct a comprehensive large-scale study across ten models and four value theories, identifying asymmetries in steerability and composition laws for multi-value control. This paper establishes a scalable infrastructure for evaluating and controlling value intensity, advancing pluralistic alignment of LLMs.

Probabilistic Methods · Bayesian Models and Methods

Sherman Khoo, Dennis Prangle, Song Liu, Mark Beaumont

Simulation-based inference (SBI) enables amortized Bayesian inference by first training a neural posterior estimator (NPE) on prior-simulator pairs, typically through low-dimensional summary statistics, which can then be cheaply reused for fast inference by querying it on new test observations. Because NPE is estimated under the training data distribution, it is susceptible to misspecification when observations deviate from the training distribution. Many robust SBI approaches address this by modifying NPE training or introducing error models, coupling robustness to the inference network and compromising amortization and modularity. We introduce minimum-distance summaries, a plug-in robust NPE method that adapts queried test-time summaries independently of the pretrained NPE. Leveraging the maximum mean discrepancy (MMD) as a distance between observed data and a summary-conditional predictive distribution, the adapted summary inherits strong robustness properties from the MMD. We demonstrate that the algorithm can be implemented efficiently with random Fourier feature approximations, yielding a lightweight, model-free test-time adaptation procedure. We provide theoretical guarantees for the robustness of our algorithm and empirically evaluate it on a range of synthetic and real-world tasks, demonstrating substantial robustness gains with minimal additional overhead.

Deep Learning · Generative Models and Autoencoders

Liu Yu, Xingjiao Wu, Ziang Liu, Jiabao Zhao, Daoguo Dong, Liang He

Semantic Typography aims to visualize the meaning of an input word through the form of a character, while preserving its legibility. Existing vector-based methods, which primarily rely on text-driven optimization like Score Distillation Sampling (SDS), often produce glyphs that lack rich semantic details. Furthermore, these approaches struggle to maintain the overall structural integrity of the glyphs and frequently suffer from visual artifacts caused by intersections, compromising both readability and aesthetic quality. To address these challenges, we propose VecDesigner, a novel optimization-based method for vector semantic typography. Specifically, we introduce Visual-Guided Score Distillation Sampling (VGSDS), which leverages text-related reference images as visual guidance to infuse the glyphs with richer and more concrete semantic details. To preserve legibility and structural integrity, we design a vector-based Procrustes loss to constrain the overall deformation of the glyph. Concurrently, we effectively mitigate the intersection problem by imposing positional relationship constraints on the control points. Comprehensive experiments demonstrate that VecDesigner outperforms existing methods in both semantic expression and structural preservation, generating high-quality, expressive, and clearly legible semantic glyphs.

Theory · Learning Theory

Junjie Yu, Zhuoli Ouyang, Haotian Deng, Chen Wei, Wenxiao Ma, Jianyu Zhang, Zihan Deng, Quanying Liu

Deep neural networks often generalize well despite heavy over-parameterization, challenging classical parameter-based analyses. We study generalization from a representation-centric perspective and analyze how the geometry of learned embeddings controls predictive performance for a fixed trained model. We show that population risk can be bounded by two factors: (i) the intrinsic dimension of the embedding distribution, which determines the convergence rate of empirical embedding distribution to the population distribution in Wasserstein distance, and (ii) the sensitivity of the downstream mapping from embeddings to predictions, characterized by Lipschitz constants. Together, these yield an embedding-dependent error bound that does not rely on parameter counts or hypothesis class complexity. At the final embedding layer, architectural sensitivity vanishes and the bound is dominated by embedding dimension, explaining its strong empirical correlation with generalization performance. Experiments across architectures and datasets validate the theory and demonstrate the utility of embedding-based diagnostics.

Deep Learning · Graph Neural Networks

Likang Wu, Zihao Chen, Jianxin Zhang, Sangqi Zhu, Yuanyuan Ge, Haipeng Yang, Lei Zhang

With the rapid emergence of multi-behavior learning in recommender systems, leveraging auxiliary user behaviors has proven effective for mitigating target-behavior data sparsity. Yet auxiliary behavior graphs frequently contain noisy or irrelevant interactions that do not align with the target task, impeding the learning of accurate user and item embeddings. Moreover, the scarcity of direct supervised from the target behavior complicates the extraction of informative collaborative signals. In this paper, we introduce GCIB Graph Contrastive Information Bottleneck, a novel framework that denoises auxiliary behavior information and enriches target behavior representations at both the structural and feature levels. At the structural level, GCIB employs a Graph Information Bottleneck (GIB) objective to maximize mutual information between the denoised auxiliary graph and the target-behavior graph while minimizing mutual information with the original auxiliary graph. This formulation preserves task-relevant structural patterns and suppresses spurious interactions. At the feature level, we propose a cross-behavior Graph Contrastive Learning (GCL) scheme in which denoised auxiliary features and target-behavior features serve as complementary views for both users and items. By contrasting these views, GCIB enriches sparse target-behavior representations with semantics distilled from auxiliary behaviors. Extensive experiments on multiple real-world datasets demonstrate that GCIB outperforms state-of-the-art baselines, highlighting its ability to learn noise-resilient and target-aware representations for multi-behavior recommendation.

Optimization · Stochastic

Yuan Zhou, Yidan Ou, Xinli Shi

Federated Multi-Objective Learning (FMOL) enables collaborative training of conflicting objectives but faces a compounded challenge: the recursive coupling between intra-task client drift and inter-task aggregation bias. We propose DREAM, a unified framework that jointly corrects these two coupled error sources through drift-aware control variates and momentum-smoothed local updates. On the server side, DREAM formulates multi-objective aggregation as a regularized quadratic program parameterized by a task correction matrix, which provides a generalized formulation that can flexibly adapt to scalarization, prioritization, and gradient manipulation strategies. Theoretically, we establish a linear speedup convergence rate of $\mathcal{O}(1/\sqrt{NT})$ for non-convex objectives. We further provide theoretical guarantees for the conflict-avoidant descent direction. In the strongly convex setting, DREAM achieves convergence in weighted sub-optimality and admits a unified Lyapunov analysis showing linear convergence to a regularization-dependent neighborhood. Numerical experiments validate the superior performance and effectiveness of DREAM in practice.

Optimization · Discrete and Combinatorial Optimization

Hongyu Cheng, Amitabh Basu

Machine learning is increasingly used to guide branch-and-cut (B&C) for mixed-integer linear programming by learning score-based policies for selecting branching variables and cutting planes. Many approaches train on local signals from lookahead heuristics such as strong branching, and linear programming (LP) bound improvement for cut selection. Training and evaluation of the learned models often focus on local score accuracy. We show that such local score-based methods can lead to search trees exponentially larger than optimal tree sizes, by identifying two sources of this gap. The first is that these widely used expert signals can be misaligned with overall tree size. LP bound improvement can select a root cut set that yields an exponentially larger strong branching tree than selecting cuts by a simple proxy score, and strong branching itself can be exponentially suboptimal (Dey et al., 2024). The second is that small discrepancies can be amplified by the branch-and-bound recursion. An arbitrarily small perturbation of the right-hand sides in a root cut set can change the minimum tree size from a single node to exponentially many. For branching, arbitrarily small score discrepancies, and differences only in tie-breaking, can produce trees of exponentially different sizes, and even a small number of decision differences along a trajectory can incur exponential growth. These results show that branch-and-cut policies trained and learned using local expert scores do not guarantee small trees, thus motivating the study of data-driven methods that produce policies better aligned with tree size rather than only accuracy on expert scores.

General Machine Learning · Evaluation

Aocheng Shen, Boyu Zhang, Jiaze Li, Ruixuan Ma, Qiankun Zhang, Wang, Bin Yuan, Shenghao Liu, Xianjun Deng

Large language models (LLMs) have revolutionized research in software engineering, and among various tasks, LLM-based code synthesis is promising. A recent line of benchmarks aims to evaluate LLM-generated codes in time efficiency, beyond their correctness. However, *space*, another vital aspect of code efficiency, is rarely evaluated in prior benchmarks. To fill in the gap, this paper introduces *BEST*, the first benchmark for evaluating the efficiency of LLM-generated codes in *both time and space*. It comprises $440$ coding tasks that are rigorously constructed by experts. In addition, we propose a fine-grained *subtask-based* evaluation scheme by dividing each task into multiple subtasks, with different input scales and difficulties. Each subtask is then accompanied by an expert-crafted standard implementation as the efficiency baseline, which achieves the *Pareto optimum*. Building on BEST, we introduce a unified and novel dual-indicator (time and space) metric, named dual@${k}$, generalizing the notion of the standard pass@${k}$ metric and building on a careful and novel construction of a *weight matrix* of subtasks. Through extensive experiments with dual@${k}$ across $47$ LLMs on BEST, our evaluation demonstrates that while LLMs exhibit weak capabilities in generating time-efficient code, their capabilities in space-efficient code generation are even worse. The benchmark is provided in the supplementary material.

Deep Learning · Large Language Models

Yeqiu Chen, Ziyan Liu, Zhenxin Huang, Runquan Gui, Hong Wang, Lei Liu

Recent progress in LLM reasoning has increasingly shifted from single-pass generation to explicit search over intermediate reasoning states. Tree-of-Thoughts (ToT) organizes inference to tree-structured search with branching and backtracking, but it substantially amplifies the key--value (KV) cache: retaining KV states for a frontier of partial trajectories quickly becomes a memory bottleneck that limits throughput and constrains search depth and width under fixed hardware budgets. We address this challenge by observing that KV reuse in ToT-style inference is governed by search dynamics: near-term decoding depends primarily on the active branch and its ancestors, whereas inactive subtrees have low short-term reuse probability yet must remain recoverable for backtracking. Motivated by this, we propose **ArborKV**, a structure-aware eviction framework that couples a lightweight value estimator with a tree-aware allocation policy, and performs purely token-extractive eviction with lazy rehydration to support revisits. Experiments on ToT-style reasoning benchmarks show that ArborKV achieves up to $\sim4\times$ peak KV-memory reduction while preserving near-full-retention accuracy, enabling larger search configurations under fixed device budgets that would otherwise run out of memory.

Deep Learning · Generative Models and Autoencoders

Zhaokai Wang, Penghao Yin, Xiangyu Zhao, Changyao Tian, Yu Qiao, Wenhai Wang, Jifeng Dai, Gen Luo

Exams are a fundamental test of expert-level intelligence and require integrated understanding, reasoning, and generation. Existing exam-style benchmarks mainly focus on understanding and reasoning tasks, and current generation benchmarks emphasize the illustration of world knowledge and visual concepts, neglecting the evaluation of rigorous drawing exams. We introduce GenExam, the first benchmark for multidisciplinary text-to-image exams, featuring 1,000 samples across 10 subjects with exam-style prompts organized under a four-level taxonomy. Each problem is equipped with ground-truth images and fine-grained scoring points to enable a precise evaluation of semantic correctness and visual plausibility. Experiments on 17 text-to-image and unified models demonstrate the great challenge of GenExam and the huge gap where open-source models consistently lag behind the leading closed-source ones. By framing image generation as an exam, GenExam offers a rigorous assessment of models' ability to integrate understanding, reasoning, and generation, providing insights on the path to intelligent generative models. Our benchmark and evaluation code will be released.