论文检索

Reinforcement Learning · Deep RL

Yiming Wang, Yiheng Zhang, Kaiyan Zhao, Xingjie Zuo, Xingyu Liu, Xuetao Li, Furui Liu, Bo An, Leong Hou U

Efficient exploration remains a key challenge in deep reinforcement learning, especially for embodied agents operating in realistic environments with high-dimensional observations and complex dynamics. Recent latent exploration methods define bonuses in a learned latent space, but often struggle in these settings where (i) representations can be noisy or policy-dependent, and (ii) common strategies such as randomized latent objectives or fixed directional spanning are brittle and fail to improve global coverage. We propose Covariance Volume Maximization (CVM), a coverage-driven latent exploration framework with two key components. First, we learn a behavioral state encoder using a *policy-mixture* objective to reduce representation drift under rapidly changing exploration policies, yielding stable and behaviorally meaningful latent displacements. Second, CVM rewards each transition by its exact increase in the log-determinant of the covariance of recent latent displacements, explicitly expanding the explored region and prioritizing under-covered directions. This objective coincides with the classical *D-optimal* design criterion, providing an information-efficiency justification. Extensive experiments on embodied navigation and manipulation tasks demonstrate that CVM substantially improves exploration efficiency and robustness, and scales effectively to different environments.

Probabilistic Methods · Variational Inference

Giacomo Borghi, Jose Carrillo

Motivated by variational inference methods, we propose a zeroth-order algorithm for solving optimization problems in the space of Gaussian probability measures. The algorithm is based on an interacting system of Gaussian particles that stochastically explore the search space and self-organize around global minima via a consensus-based optimization (CBO) mechanism. Its construction relies on the Linearized Bures–Wasserstein (LBW) space, a novel parametrization of Gaussian measures we introduce for efficient computations. We establish well-posedness and study the convergence properties of the particle dynamics via a mean-field approximation. Numerical experiments on variational inference tasks demonstrate the algorithm’s robustness and superior performance with respect to gradient-based method in presence of non log-concave targets.

Jusheng Zhang, Jing Yang, Kaitong Cai, Ziliang Chen, Yongsen Zheng, Kwok Yan Lam, Liang Lin, Keze Wang

Workflow optimization for tool-using LLM agents is often cast as global search over candidate graphs, scored by a scalar metric. This collapses rich, multi-step failure traces into binary outcomes, obscuring recurring failure structure and making refinement inefficient. We reframe optimization as \emph{distributional refinement}: each workflow induces a density over a \textbf{Failure Signature Space} $\mathcal{F}$, and the goal is to minimize its \textbf{Expected Failure Mass}. We propose \textbf{CE-Graph}, which maintains a counterexample pool, estimates dense failure modes, and applies operator-constrained graph edits via a \textbf{Propose-and-Verify} loop with a convergence-aware stopping rule. Across math, code, and QA benchmarks, CE-Graph improves robustness while reducing optimization cost compared to strong workflow-search baselines, suggesting reliability emerges from learning and reshaping failure landscapes rather than merely maximizing aggregate success rates.

Social Aspects · Safety

Daizong Liu, Xiaowen Cai, Junhao Dong, Zhongliang Guo, Xiaoye Qu, Runwei Guan, Xiang Fang, Dengpan Ye

Large vision-language models (LVLMs) have demonstrated remarkable capabilities across a wide range of multimodal reasoning tasks. However, recent research shows that they are susceptible to adversarial examples. Existing LVLM attack methods are generally deployed in the white- or black-box setting, which severely rely on full-model gradients or elaborated transfer strategies, resulting in large resource costs. To this end, this paper focuses on a more efficient gray-box attack setting by solely accessing LVLM's vision encoder. Instead of using target images as the adversarial guidance, our main goal is to perturb the visual feature to best match more natural attacker-chosen target texts. Specifically, we develop a global semantic alignment module to project the visual features onto the SVD-structured subspace spanned by the textual semantics. We also propose to align detailed visual features with multi-context semantic texts extended by LLMs over discrete distributions via optimal transport. Extensive experiments demonstrate the superiority of the proposed method, while our attack is further proven to achieve great transferability across various LVLMs with CLIP-aware transfer designs.

Qi Yu, Ruizhong Qiu, Zhichen Zeng, My T. Thai, huan liu, Hanghang Tong

Alignment plays a fundamental role in many machine learning problems, such as multi-network analysis, multimodal learning, and point cloud registration. Recent works increasingly leverage optimal transport (OT) for distributional alignment, whose effectiveness largely depends on sparse supervision that is hard or costly to obtain in practice. Existing works, however, largely overlook how to actively acquire high-quality supervision to improve their alignment performance under OT frameworks. In this paper, we propose a principled active alignment framework for optimal transport alignment called AvAtar. We quantify the informativeness of a candidate by measuring its gradient-based impact on the global alignment result, computed as the gradient propagation from the global alignment result to all possible supervisions of the candidate through the entropy-regularized OT formulation. While differentiating through OT is challenging given its constrained nature, we leverage the adjoint-state method to reformulate the computation to a linear system solvable by the conjugate gradient method with linear complexity and guaranteed convergence. By encoding the global alignment result via effective utility functions, AvAtar is applicable to general alignment problems under the OT framework. Extensive experiments on three representative alignment tasks demonstrate the effectiveness, scalability, and generalizability of the proposed AvAtar.

Deep Learning · Generative Models and Autoencoders

An Zhao, Shengyuan Zhang, Zhongjian Sun, Yixiang Zhou, Zejian Li, Ling Yang, Tianrun Chen, Lingyun Sun

Flow Matching models have demonstrated strong performance across a wide range of generative tasks. However, their reliance on ODE-based iterative sampling incurs substantial computational overhead, which limits their applicability in real-time scenes. While distillation is a promising solution, existing approaches largely borrow from diffusion-based score matching, often failing to exploit the intrinsic geometric structure of flows and suffering from training instability, high variance, and degraded generation quality. In this paper, we propose Mean Flow Distillation (MFD), a novel distillation framework tailored for flow matching models. We theoretically demonstrate that MFD acts as a temporal low-pass filter, effectively suppressing the high-frequency optimization noise inherent in variational score distillation (VSD) while ensuring global trajectory consistency. We further prove the Mean Flow Matching Theorem, establishing that matching expected average velocities is sufficient for strict distribution alignment. Empirically, on challenging high-dimensional manifolds including 4D occupancy forecasting and text-to-image generation, MFD achieves state-of-the-art performance, enabling high-fidelity single-step generation.

Deep Learning · Large Language Models

Alexander Samarin, Sergei Krutikov, Anton Shevtsov, Sergei Skvortsov, Filipp Fisin, Aleksandr Golubev

Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to suboptimal solutions where minimizing KL does not guarantee maximizing acceptance rate. To address this issue, we propose **LK losses**, special training objectives that directly target acceptance rate. Comprehensive experiments across four draft architectures and six target models, ranging from 8B to 685B parameters, demonstrate consistent improvements in acceptance metrics across all configurations compared to the standard KL-based training. We evaluate our approach on general, coding and math domains and report gains of up to 10\% in average acceptance length. LK losses are easy to implement, introduce no computational overhead and can be directly integrated into any existing speculator training framework, making them a compelling alternative to the existing draft training objectives.

Social Aspects · Safety

Mohammed Alshaalan, Miguel Rodrigues

Optimization-based adversarial suffixes can jailbreak aligned large language models (LLMs) while remaining fluent, weakening detectors based on static global or windowed perplexity statistics. We cast adversarial suffix detection as an \emph{online change-point detection} problem over the token-level next-token entropy stream. Using the fixed system prompt to estimate a robust baseline via the median and median absolute deviation, we standardize user-token entropies and monitor them with a one-sided CUSUM statistic. The resulting detector is model-agnostic, training-free, operates online, and localizes the onset of adversarial suffixes. On a benchmark of $724$ optimization-based suffix attacks (GCG, AutoDAN, AdvPrompter) and $765$ benign prompts from a TyDiQA+OpenOrca mixture with controlled post-prefix perplexity, CPD consistently outperforms perplexity baselines; on LLaMA-2-7B it reaches AUROC $0.90$ and F1 $0.82$. At an operating point with $\approx 10\%$ benign false-positive rate, CPD detects $74\%$ of suffix attacks and concentrates $87\%$ of its triggers inside the adversarial suffix. By comparison, windowed perplexity detects $35$--$43\%$ and frequently fires on boundary-straddling windows. Finally, we show CPD Online can act as a lightweight gate for LLaMA Guard, reducing guard invocations by $17$--$22\%$ on a high-volume stream dominated by benign prompts while preserving guard-level detection quality.

Deep Learning · Algorithms

Jianfeng Lu, YuZhao Xiang, Yue Chen, Gang Li, Shuqin Cao, Guanghui Wen

Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneity-induced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead.

Optimization · Zero-order and Black-box Optimization

Jingzhe Jing, Zheyi Fan, Szu Hui Ng, Qingpei Hu

Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality. We propose **L**ocal **C**onstrained **B**ayesian **O**ptimization (LCBO), a novel framework tailored for such settings. Unlike trust-region methods that are prone to premature shrinking when confronting tight or complex constraints, LCBO leverages the differentiable landscape of constraint-penalized surrogates to alternate between rapid local descent and uncertainty-driven exploration. Theoretically, we prove that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker (KKT) residual that depends polynomially on the dimension $d$ for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially. Extensive evaluations on high-dimensional benchmarks (up to 100D) demonstrate that LCBO consistently outperforms state-of-the-art baselines.

Deep Learning · Graph Neural Networks

Xinya Qin, Lu Bai, Lixin Cui, Ming Li, Hangyuan Du, Ziyu Lyu, Xin Jin

Graph Convolutional Networks (GCNs) are defined based on aggregating the node information of adjacent nodes, that are usually treated as equally important as each other, limiting the representational power of existing GCNs for graph classification. To address this shortcoming, we propose a novel Global Interacted Graph Convolutional Network (GI-GCN), that can leverage the solution vectors maintained during the iterative updates of the Dominant Set to adaptively characterize the global importance distribution of different nodes. Specifically, at each convolution layer, this distribution is adopted to adaptively modulate the importance weights of different node features before performing the local message passing. We show that this convolution strategy can effectively capture the highly correlated information between nonadjacent nodes through the Dominant Set algorithm, not only emphasizing the critical information at the graph level but also enhancing the discriminative power of graph representations. Furthermore, we optimize the spatial complexity of the framework, significantly reducing the memory overhead associated with the global interaction modeling. Experiments demonstrate the effectiveness of the proposed GI-GCN.

Deep Learning · Algorithms

Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Zhiqiang Shen

Dataset pruning remains underexplored for 3D modalities, where inherent class imbalance persists across ***both*** training and test sets. This creates a divergence in evaluation: overall accuracy favors natural frequency, reflecting practical usage; while mean accuracy demands balanced generalization. Instead of forcing a premature trade-off, we advocate for base principles that remain universally robust and beneficial across diverse priors. We cast pruning as a quadrature approximation on population risk and decompose the error bound into *representation error* (fidelity to the underlying manifold) and *prior-mismatch bias* (distribution shift), clarifying what can be improved jointly across priors. To address prior-mismatch bias, we decouple likelihood from prior in the posterior and transfer the structural likelihood via distillation with a calibrated teacher and geometry-preserving constraints. Simultaneously, to reduce representation error, we audit common pruning signals and choose geometric embedding, which exhibits greater robustness given the high inductive bias of 3D models. We also prioritize a safety floor before selection, capturing high-reward regions beneficial across priors. Finally, acknowledging that no single subset optimally satisfies divergent evaluation priors, we augment these principles with a steering wrapper that interpolates between stratified seeding and global selection. Empirical results demonstrate that our framework elevates the performance floor while offering flexibility for different prior preferences.

Applications · Robotics

Zirui Ge, Pengxiang Ding, Yemin Wang, BaoHuaYin, Qishen Wang, Zhiyong Xie, Hengtao Li, Runze Suo, Wenxuan Song, Han Zhao 等

Video action models are a promising foundation for Vision–Language–Action (VLA) because they can learn rich visual dynamics directly from video. However, likelihood-oriented training of diffusion predictors emphasizes globally plausible futures and does not guarantee precision-critical visual dynamics needed for manipulation, so small prediction errors can be amplified by downstream policies. We propose Dyn-VPP, a post-training framework that casts multi-step denoising as policy optimization and aligns predicted future latents with expert visual dynamics via verifiable terminal reward, without modifying any architecture. This enables explicit optimization of dynamics signals that are not captured by likelihood-only training. As a result, Dyn-VPP yields more accurate visual dynamics and improves downstream task execution. Experiments across diverse simulated and real-world manipulation settings show improved dynamics consistency and consistently higher task success.

Deep Learning · Algorithms

Shunxin Guo, JIAQI LYU, Zhiqiang Kou, Shuxia Lin, Xin Geng

Federated Learning (FL) enables privacy-preserving collaboration among distributed clients in open-world environments, but its performance often degrades under data heterogeneity and unpredictable distribution shifts. Test-Time Adaptation (TTA) has recently been introduced into FL to leverage unlabeled data from unseen clients for online adaptation. However, most existing federated TTA methods employ local feature statistics, which can be brittle under diverse and severe distribution shifts. In this work, we observe that despite significant variations in feature distributions, the relational structure among class prototypes—termed *prototype affinity topology*—remains remarkably stable across heterogeneous clients. Building on this insight, we propose FedPAT, a **Fed**erated TTA framework that leverages **P**rototype **A**ffinity **T**opology as a cross-client structural prior. FedPAT learns a global PAT by aggregating class prototypes from source clients, capturing consensus inter-class relationships that are robust to local distribution variations. For unseen target clients, we design a topology-aware mechanism that enhances predictions via diffusion of the global PAT, fuses them with parametric outputs, and performs lightweight optimization for robust test-time adaptation. Extensive experiments demonstrate that FedPAT consistently outperforms advanced federated TTA and classical TTA methods across various distribution shifts.

Deep Learning · Large Language Models

Ning Gao, Wei Zhang, Yuqin Dai, Ling Shi, Ziyin Wang, Yujie Wang, Wei He, Jinpeng Wang, Chaozheng Wang

The rapid evolution of Large Language Models (LLMs) has accelerated the transition from conversational chatbots to general agents. However, effectively balancing empathetic communication with budget-aware decision-making remains an open challenge. Since existing methods fail to capture these complex strategic trade-offs, we propose InteractCS-RL, a framework that reframes task-oriented dialogue as a multi-granularity reinforcement learning process. Specifically, we first establish a User-centric Interaction Framework to provide a high-fidelity training gym, enabling agents to dynamically explore diverse strategies with persona-driven users. Then, we introduce Cost-aware Multi-turn Policy Optimization (CMPO) with a hybrid advantage estimation strategy. By integrating generative process credits and employing a PID-Lagrangian cost controller, CMPO effectively guides the policy to explore Pareto boundary between user reward and global cost constraints. Extensive experiments on customized real business scenarios demonstrate that InteractCS-RL significantly outperform other baselines across three evaluation dimensions. Further evaluation on tool-agent-user interaction benchmarks verify InteractCS-RL robustness across diverse domains.

Social Aspects · Accountability, Transparency, and Interpretability

Dawood Wasif, Terrence Moore, Chang-Tien Lu, Jin-Hee Cho

Federated learning enables on-device training without centralizing data, yet existing systems still struggle to provide explanations that are both locally faithful and globally consistent under strict privacy and bandwidth constraints. Prior approaches either keep explanations siloed across clients, transmit heavy or sensitive artifacts, or replace expressive task models with interpretable surrogates that sacrifice accuracy. We propose xFedAlign, a model-agnostic framework that decouples task optimization in parameter space from explanation coordination in a compact group space. Each client distills a lightweight surrogate to produce private, per-class top-k attribution artifacts, which are robustly aggregated by the server into a Global Explanation Prior that softly aligns client explanations without constraining task learning. Across image, text, and tabular benchmarks with IID and non-IID partitions, xFedAlign matches FedAvg accuracy while consistently reducing explanation drift and improving deletion and insertion AUC relative to Local-XAI, FedAttr-Agg, and Fed-XAI, with only a few kilobytes of additional communication per round. Privacy and robustness evaluations further demonstrate reduced membership inference advantage and increased resistance to attribution poisoning, enabling consistent and trustworthy explanations in federated learning.

Optimization · Everything Else

Arghya Sinha, Aditya Banerjee, Trishit Mukherjee, Kunal Narayan Chaudhury

Trainable denoisers with Lipschitz control have become central to convergent image reconstruction. However, training neural networks that simultaneously offer strong denoising performance and global Lipschitz guarantees is challenging. Existing approaches enforce Lipschitz control only empirically, providing no guarantees beyond the training data. In this work, we show that by exploiting the action of permutations on the image lattice, we can constrain a neural architecture that is globally nonexpansive (Lipschitz bound $\leqslant 1$). We integrate the proposed denoiser with forward imaging operators to develop a reconstruction mechanism that is provably contractive and therefore globally convergent. Experiments on standard inverse problems, such as superresolution and deblurring, demonstrate that our reconstruction performance is competitive with softly constrained baselines while providing Lipschitz guarantees.

Applications · Robotics

Xiaoyu Xiong, Kehan Liu, HuiYi Yan, Shengjie Wang, Yang Gao, Tao Du

Traditional robot co-design approaches typically converge to \textit{one} configuration, which do not explore the flexibility from reconfiguration on heterogeneous environments. On the other hand, existing designs for reconfigurable robots require human-designed configurations. We present Learning to Reconfigure, a holistic pipeline for co-designing the configurations and control of reconfigurable robots in heterogeneous locomotion tasks consisting of several sub-tasks. Our pipeline proposes low-level specialized primitives with a high-level scheduler. To jointly optimize configuration design and control, our primitives employ a multi-tail architecture that disentangles these distinct objectives. Building on this, the scheduler learns to dynamically switch configurations based on global task progress. We evaluate our pipeline on locomotion tasks across walking, flying, and swimming, and compare with the state-of-the-art baselines, including single-robot control and multi-morphology co-design algorithms. Quantitative results based on traversal progress show that our pipeline outperforms single-robot baselines by 5.95x average progress. Compared with the reconfiguration-free design given by the co-design algorithms, our robots also exhibit 9.99x progress on average. These results highlight the critical role of configuration adaptation in achieving versatile robotic autonomy in complex worlds.

Theory · Online Learning and Bandits

Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

This work studies online episodic tabular Markov decision processes (MDPs) with known transitions and develops best-of-both-worlds algorithms that achieve refined data-dependent regret bounds in the adversarial regime and variance-dependent regret bounds in the stochastic regime. We quantify MDP complexity using a first-order quantity and several new data-dependent measures for the adversarial regime, including a second-order quantity and a path-length measure, as well as variance-based measures for the stochastic regime. To adapt to these measures, we develop algorithms based on global optimization and policy optimization, both built on optimistic follow-the-regularized-leader with log-barrier regularization. For global optimization, our algorithms achieve first-order, second-order, and path-length regret bounds in the adversarial regime, and in the stochastic regime, they achieve a variance-aware gap-independent bound and a variance-aware gap-dependent bound that is polylogarithmic in the number of episodes. For policy optimization, our algorithms achieve the same data- and variance-dependent adaptivity, up to a factor of the episode horizon, by exploiting a new optimistic $Q$-function estimator. Finally, we establish regret lower bounds in terms of data-dependent complexity measures for the adversarial regime and a variance measure for the stochastic regime, implying that the regret upper bounds achieved by the global-optimization approach are nearly optimal.

General Machine Learning · Methodology

Junhyoung Chung, Euijong Song, Won Hwa Kim, Gunwoong Park

We introduce Convex Distance Operator Transport (CDOT), the first convex optimal transport framework that aligns distributions across heterogeneous domains by jointly preserving feature correspondence and intrinsic geometric structure. Specifically, CDOT employs an operator-based regularization that aligns aggregated distance structures by introducing distance and conditional expectation operators. Consequently, the proposed regularization improves the robustness to local geometric variations. We further prove that the resulting CDOT discrepancy is a valid pseudometric on the space of attributed compact metric-measure spaces. In addition, we characterize the relationship between CDOT and Gromov--Wasserstien (GW) through a new notion of dispersion gap, formally elucidating the geometric source of non-convexity in GW compared to the convexity of CDOT. In the finite-sample regime, we derive a non-asymptotic risk bound decomposed into optimization and statistical errors, establishing risk consistency under a globally convergent Frank--Wolfe algorithm. Experiments on synthetic point clouds, brain connectomes, and graph classification benchmarks demonstrate better performance over existing methods, with stable and reliable behavior in practice.