论文检索

Deep Learning · Large Language Models

Hongyi Du, Jiaqi Su, Jisen Li, Lijie Ding, Yingxuan Yang, Peixuan Han, Robert Tang, Kunlun Zhu, Jiaxuan You

As large-scale multi-agent systems evolve, the communication protocol layer has become a critical, yet understudied, component affecting system performance and reliability. Despite a range of protocols, such as JSON-RPC, A2A, ANP, and ACP, protocol selection remains ad hoc. To address this, we introduce ProtocolBench, a benchmark designed to evaluate agent communication protocols across task utility, communication overhead, system performance, and resilience under failure. ProtocolBench uses a three-layer architecture with protocol adapters for fair com- parison, diverse scenarios (e.g., document aggregation, collaborative coding), and detailed telemetry. Our results show protocol choice can impact task completion time by up to 36%, communication overhead by 3.5 seconds, and resilience with statistically observable differences. We also propose ProtocolRouter, a learnable protocol routing system that dynamically selects protocols based on runtime con- ditions, improving performance by up to 18% compared to individual protocols. Our findings highlight that hybrid protocol deployments outperform homogeneous ones by approximately 6.6%, with negligible protocol translation overhead. We release ProtocolBench as an open-source framework to standardize protocol evaluation and improve multi-agent system reliability at scale.

Applications · Neuroscience, Cognitive Science

Abdülkadir Gökce, Yingtian Tang, Martin Schrimpf

Task-optimized neural networks are the leading in-silico models of sensory cortex, yet the field lacks a unified understanding of which modeling choices drive improved brain alignment. Prior NeuroAI work is fragmented across datasets and modalities, making it difficult to determine robust scaling trends. Here, we systematically investigate the scaling laws of model-to-brain alignment across 8 neural datasets (spanning electrophysiology, fMRI, EEG, and MEG) and over 600 models with diverse architectures and pretraining configurations. We report three key scaling trends: (1) *Pretraining saturation*: Alignment improves with pretraining compute and data scale but saturates across all recording modalities. (2) *Complementary fine-tuning*: Hybrid task & neural data optimization yields consistent improvements in alignment that generalize across datasets and modalities. (3) *Mapping scaling*: Increasing the number of neural samples to fit model-to-brain mappings yields log-linear gains with the largest impact on alignment. Finally, we propose a novel subject-shared cross-attention mapping which drastically reduces parameter count and improves alignment. Taken together, these results establish multimodal scaling laws that guide resource allocation for next-generation brain models.

Deep Learning · Large Language Models

Ziyao Tang, Pengkun Jiao, Xinhang Chen, LiuWei Liu, Shiyong Li, Jingjing Chen

Given the quadratic complexity of attention, KV cache eviction is vital to accelerate model inference. Current KV cache eviction methods typically rely on instantaneous heuristic metrics, implicitly assuming that score magnitudes are consistent proxies for importance across all heads. However, this overlooks the heterogeneity in predictive fidelity across attention heads. While certain heads prioritize the \textit{instantaneous contribution} of tokens, others are dedicated to capturinglong-horizon utility. In this paper, we propose that optimal budget allocation should be governed by the marginal utility in preserving long-term semantic information. Based on this insight, we propose LU-KV, a novel framework that optimizes head-level budget allocation through a convex-hull relaxation and a marginal-utility-based greedy solver to achieve near-optimal precision. Furthermore, we implement a data-driven offline profiling protocol to facilitate the practical deployment of LU-KV.

Applications · Robotics

Shuaijun Liu, Feiyang You, Xingwei Chen, Ningxin Su

Embodied agents replan frequently to recover from execution drift, partial observability, and coordination hazards. In many LLM-based planners, each replanning call consumes an accumulated textual context that grows over time and across agents (history, failures, summaries, and messages). Once this context becomes large, replanning latency develops heavy tails and can miss real-time deadlines even when task success remains high---a failure mode that is hard to detect from average latency or success alone. We present BRACE, a controller that formulates replanning for embodied agents as a budgeted control loop. At each replanning trigger, BRACE decides whether to replan, selects a replanning mode, and allocates an explicit token budget together with a latency service-level objective (SLO), while accounting for the overhead of optional efficiency modules. As a reusable component, we introduce E-RECAP, a cost-aware progressive token pruning method that predicts token utility and prunes replanning contexts across transformer layers while preserving critical head and tail tokens. On Habitat-Lab navigation with growing multi-agent context, E-RECAP reduces tokens per replanning call by 71-76% and end-to-end replanning latency by 2.1-2.6x with minimal impact on success or SPL. In Meta Habitat, BRACE combined with E-RECAP reduces SLO violation rates from 85.5% to 4.7% without degrading task success. Results across three embodied platforms demonstrate that tail-aware, per-call budgeting is an effective and practical design principle for replanning systems.

Social Aspects · Safety

Nicola Novello, Federico Fontana, Luigi Cinque, Deniz Gunduz, Andrea Tonello

Most current methods for unlearning concepts in text-to-image diffusion models rely on mean squared error-based loss functions to align target distributions with anchors. In this paper, we generalize this idea into a unified $f$-divergence-based framework that recovers the standard mean squared error loss as a specific instance. By generalizing the loss function, we theoretically analyze and numerically validate how different $f$-divergences impact the gradient magnitude and the convergence properties of the algorithm, affecting the quality of unlearning. The proposed unified framework offers a flexible paradigm for selecting the optimal divergence based on the application and user goal, allowing for finer control over the trade-off between unlearning efficacy and generative fidelity.

Deep Learning · Large Language Models

Xiaoxue Zhu, Jilin Hu, Fuyuan Zhang, Jianyu Zhang, Yongwang Zhao

Recent advances in large language models have accelerated neural theorem proving (NTP). Isabelle is a mature and important formal theorem prover that has been widely used in software and hardware verification. However, progress in the Isabelle setting remains limited. Existing approaches either optimize search strategies or train on highly imbalanced raw proof corpora. At the same time, the specialized structure of Isabelle proofs limits the effectiveness of general-purpose data selection methods. To address these challenges, we adopt a data-centric framework for neural theorem proving in Isabelle. We characterize high-quality formal proof data along three complementary dimensions—proof complexity, semantic coverage, and reasoning diversity (PSR)—and propose a PSR-guided data selection pipeline to construct a compact, high-quality training subset. In addition, we leverage verifier feedback as a dynamic data signal during inference, introducing a dynamic feedback-based prompt optimization that iteratively incorporates Isabelle verifier feedback to guide proof generation. We construct and release a 4k high-quality Isabelle dataset based on the PSR criterion. On the miniF2F-test, fine-tuning solely on PSR-selected data achieves 84.8% Pass@64. When further combined with dynamic feedback–based prompt optimization, the full framework improves performance to 90.6% Pass@64, establishing a new state of the art for neural theorem proving in Isabelle.

Social Aspects · Accountability, Transparency, and Interpretability

Hadas Orgad, Fazl Barez, Tal Haklay, Isabelle Lee, Marius Mosbach, Anja Reusch, Naomi Saphra, Byron Wallace, Sarah Wiegreffe, Eric Wong 等

Interpretability aims to explain the behavior of deep neural networks. Despite rapid growth, there is mounting concern that much of this work has not translated into practical impact, raising questions about its relevance and utility. This position paper argues that the central missing ingredient is not new methods, but evaluation criteria: interpretability should be evaluated by actionability—the extent to which insights enable concrete decisions and interventions beyond interpretability research itself. We define actionable interpretability along two dimensions—concreteness and validation—and analyze the barriers currently preventing real-world impact. To address these barriers, we identify five domains where interpretability offers unique leverage and present a framework for actionable interpretability with evaluation criteria aligned with practical outcomes. Our goal is not to downplay exploratory research, but to establish actionability as a core objective of interpretability research.

Evan Dogariu, Anand Brahmbhatt, Elad Hazan

We study the fundamental problem of one-step prediction of a marginally stable unknown nonlinear dynamical system. We describe an algorithm for this problem, based on the technique of spectral filtering, which learns a mapping from past observations to the next based on a spectral representation of the system. Using techniques from online convex optimization, we prove vanishing prediction error for any nonlinear dynamical system that has finitely many marginally stable modes, with rates governed by a novel quantitative control-theoretic notion of learnability. The main technical component of our method is a new spectral filtering algorithm for linear dynamical systems, which incorporates past observations and applies to general noisy and marginally stable systems. This significantly generalizes the original spectral filtering algorithm to both asymmetric dynamics as well as incorporating noise correction, and is of independent interest.

Applications · Language, Speech and Dialog

Marcus Min, Deyuan Mike He, Zhaoyu Li, Zixuan Yi, Sharad Malik, Aarti Gupta, Xujie Si, Osbert Bastani

Autoformalization, translating informal natural language into formal, machine-verifiable languages, has been framed as a tool to generate training data for neural theorem provers, with most work focusing on individual statements. This position paper argues for theory-level autoformalization: formalizing complete theories, including axioms, definitions, theorems, proofs, tactics, and their inter-dependencies as structured libraries. We examine the significance of this shift, address 3 alternative views, identify 5 open challenges, and propose 3 promising paths forward.

Optimization · Large Scale, Parallel and Distributed

Xiuwen Fang, Xuliang Yang, Mang Ye

Federated Learning (FL) enables collaborative training across distributed clients while preserving data privacy. However, fine-tuning large-scale pre-trained models in FL is hindered by resource constraints and communication costs. Although introducing parameter-efficient fine-tuning strategies such as Low-Rank Adaptation (LoRA) effectively reduces trainable parameters, this low-rank constraint exacerbates noise sensitivity, leading to overfitting and aggregation bias. Existing robust federated fine-tuning methods rely on additional proxy data and treat low-rank adapters as generic weight vectors. In this paper, we investigate the structural properties of LoRA and reveal a robustness asymmetry. The down-projection matrix $A$ extracts stable general features, whereas the up-projection matrix $B$ is highly susceptible to fitting noise patterns. Based on this finding, we propose Federated Decoupled LoRA (FDLoRA), which employs a dual-branch mechanism to decouple robust feature learning from noise modeling and mitigates noise interference through noisy branch negative learning. During federated aggregation, we establish global consensus through aggregating $B$ while preserving local feature alignment in $A$. Extensive experiments demonstrate that FDLoRA outperforms existing state-of-the-art methods across various noisy federated scenarios. Our code and models will be released.

Deep Learning · Large Language Models

Leon Chlon, Ahmed Karim, MarcAntonio Awada

Transformers used for evidence-grounded question answering with binary adjudication (e.g., support/refute or yes/no) can be highly sensitive to the order in which exchangeable evidence is presented, producing dispersion across permutations and unreliable attempted answers (“hallucinations” under a Bernoulli predicate). We treat evidence order as a nuisance variable and show that next-token training minimizes expected conditional description length over orderings, which can be close to Bayes-optimal in expectation while deviating under any fixed ordering. We quantify this expectation–realization gap via a Quantified Martingale Violation (QMV) bound that predicts $O(\log n)$ growth in permutation dispersion under harmonic positional sensitivity. We then derive the Expectation-level Decompression Law (EDFL), relating expected information budget to achievable reliability for Bernoulli predicates, and use it to define Bits-to-Trust (B2T), Risk-of-Hallucination (RoH), and the Information Sufficiency Ratio (ISR), with a fixed ISR-gating rule for answer/abstain decisions under permutation mixtures. On 3,059 grounded items from a five-benchmark evidence-grounded QA suite (FEVER, HotpotQA, NQ-Open, PopQA, and Controls), we observe logarithmic dispersion and Jensen gains from uniform permutation mixtures. In a pre-specified held-out audit (528 items), an $ISR=1$ gate attains 0.0–0.7% hallucination with 20.6–27.9% abstention (95% CIs).

General Machine Learning · Evaluation

Walid Durani, Philipp Jahn, Collin Leiber, David B. Hoffmann, Thomas Seidl, Claudia Plant, Christian Böhm

Clustering is commonly compared through leaderboards that collapse performance into a single aggregate ranking. Such summaries obscure why methods succeed, which data properties align with failure, and how conclusions shift under representation changes and realistic tuning constraints. We present CHB, a diagnostic toolkit for hardness-aware clustering evaluation. CHB maps each dataset--representation pair to an interpretable hardness fingerprint capturing (i) separation, (ii) cohesion and scale heterogeneity, and (iii) topology through scalable persistent-homology summaries. Using this diagnostic space, CHB evaluates clustering algorithms under standardized, compute-aware tracks. Conditioning results on hardness coordinates turns comparison into diagnosis: across a broad range of datasets and their representations, CHB reveals reproducible structural regimes, uncovers regime-dependent ranking reversals across method families, and surfaces robustness signatures, including topology-linked breakdowns. CHB further enables representation auditing by attributing gains to measurable shifts in the hardness fingerprint rather than just external performance changes. We release CHB as an open, extensible artifact for evaluating new clustering methods and embeddings within a shared diagnostic framework.

Applications · Computer Vision

Zeyu Jiang, Sihang Li, Siqi Tan, Chenyang Xu, Juexiao Zhang, Julia Galway-Witham, Xue Wang, Scott Williams, Radu Iovita, Chen Feng 等

Most existing 3D assembly methods treat the problem as pure pose estimation, rearranging observed parts via rigid transformations. In contrast, human assembly naturally couples structural reasoning with holistic shape inference. Inspired by this intuition, we reformulate 3D assembly as a joint problem of assembly and generation. We show that these two processes are mutually reinforcing: assembly provides part-level structural priors for generation, while generation injects holistic shape context that resolves ambiguities in assembly. Unlike prior methods that cannot synthesize missing geometry, we propose CRAG, which simultaneously generates plausible complete shapes and predicts poses for input parts. Extensive experiments demonstrate state-of-the-art performance across in-the-wild objects with diverse geometries, varying part counts, and missing pieces. Our code and models will be released.

Deep Learning · Generative Models and Autoencoders

Yuanbang Liang, Zhengwen Chen, Yu-Kun Lai

Latent Diffusion Models (LDMs) achieve high-fidelity synthesis but suffer from latent space brittleness, causing discontinuous semantic jumps during editing. We introduce a Riemannian framework to diagnose this instability by analyzing the generative Jacobian, decomposing geometry into *Local Scaling* (capacity) and *Local Complexity* (curvature). Our study uncovers a **``Geometric Decoupling"**: while curvature in normal generation functionally encodes image detail, OOD generation exhibits a functional decoupling where extreme curvature is wasted on unstable semantic boundaries rather than perceptible details. This geometric misallocation identifies ``Geometric Hotspots" as the structural root of instability, providing a robust intrinsic metric for diagnosing generative reliability.

Reinforcement Learning · Policy Search

Zachary Roch, George Atia, Yue Wang

Robust reinforcement learning (RL) under the average-reward criterion is essential for long-term decision-making, particularly when the environment may differ from its training dynamics. However, most existing studies focus on model-based settings and provide only asymptotic guarantees, hindering their principled understanding and practical deployment, especially in data-limited scenarios. We aim to close this gap by proposing a model-free algorithm, Robust Halpern Iteration (RHI). We first design our algorithm based on a black-box sampling oracle, which can estimate the worst-case performance accurately. We then derive the finite sample complexity of RHI under the generative model setting, assuming the sampling oracle. To concretely design such an oracle, we propose a $K$-order multi-level Monte-Carlo estimator, which is shown to have a lower bias compared to prior methods. We further instantiate our design for multiple uncertainty models, including KL and $\chi^2$ divergence sets, and show that our RHI algorithm achieves an $\varepsilon$-optimal robust policy with a sample complexity of $\tilde{\mathcal{O}}\left( \frac{SA\mathcal{H}^2}{\varepsilon^{(2+o(1))}}\right)$, where $S,A$ are the number of states and actions, and $\mathcal{H}$ is the robust optimal span. Our result asymptotically matches the best complexity in robust average reward RL.

Applications · Time Series

Jiawen Zhu, Shuhan Liu, Di Weng, Yingcai Wu

Non-stationary time series forecasting is challenged by evolving distribution shifts that static models struggle to capture. While Mixture-of-Experts (MoE) architectures offer a promising paradigm for decoupling complex drift patterns, existing approaches are limited by fixed expert pools and memoryless routing, hampering their ability to adapt to abrupt regime shifts. To address this, we propose **Dynamic TMoE**, a framework that unifies architectural evolution with temporal continuity during learning phase. By detecting distribution shifts via Maximum Mean Discrepancy (MMD), we dynamically instantiate heterogeneous experts and prune redundant ones to optimize capacity. Additionally, a temporal memory router leverages recurrent states and an anomaly repository to ensure stable, context-aware expert selection without requiring test-time updates. Experiments on nine benchmarks demonstrate state-of-the-art performance, reducing MSE by 10.4\% and MAE by 7.8\%. Code is available at https://anonymous.4open.science/r/Dynamic-TMoE.

Deep Learning · Large Language Models

Qinghe Ma, Zhen Zhao, Yiming Wu, Jian Zhang, LEI BAI, Yinghuan Shi

Tool-augmented reasoning has emerged as a promising direction for enhancing the reasoning capabilities of multimodal large language models (MLLMs). However, existing studies mainly focus on enabling models to perform tool invocation, while neglecting the necessity of invoking tools. We argue that tool usage is not always beneficial, as redundant or inappropriate invocations largely increase reasoning overhead and even mislead model predictions. To address this issue, we introduce AutoTool, a model that adaptively decides whether to invoke tools according to the characteristics of each query. Within a reinforcement learning framework, we design an explicit dual-mode reasoning strategy with mode-specific reward functions to guide the model toward producing accurate responses. Moreover, to prevent premature bias toward a single reasoning mode, AutoTool jointly explores and balances tool-assisted and text-centric reasoning throughout training, and promotes free exploration in later stages. Extensive experiments demonstrate that AutoTool exhibits outstanding performance and high efficiency, yielding a 21.8\% accuracy gain on V* benchmark compared to the base model, and a 44.9\% improvement in efficiency over existing tool-augmented methods on POPE benchmark. Code is available in the supplementary material.

Applications · Robotics

Xitie Zhang, Aming WU, Yahong Han

Cross-task generalization is a core challenge in open-world robotic manipulation, and the key lies in extracting transferable manipulation knowledge from seen tasks. Recent in-context learning approaches leverage seen task demonstrations to generate actions for unseen tasks without parameter updates. However, existing methods provide only low-level continuous action sequences as context, failing to capture composable skill knowledge and causing models to degenerate into superficial trajectory imitation. We propose Decompose and Recompose, a skill reasoning framework using atomic skill-action pairs as intermediate representations. Our approach decomposes seen demonstrations into interpretable skill--action alignments, enabling the model to recompose these skills for unseen tasks through compositional reasoning. Specifically, we construct a task-adaptive dynamic demonstration library via visual-semantic retrieval combined with skill sequences from a planning agent, complemented by a coverage-aware static library to fill missing skill patterns. Together, these yield skill-comprehensive demonstrations that explicitly elicit compositional reasoning for skill composition and execution ordering. Experiments on the AGNOSTOS benchmark and real-world environments validate our method's zero-shot cross-task generalization capability.

Applications · Computer Vision

Yan Ke, Xin Shen, Jiaying Ying, Xin Li, Xin Yu

People with limb differences often face significant challenges in accessing inclusive AI services, largely due to the lack of structured, high-quality resources centered on disability contexts. In this work, we introduce a limb-deficiency aware body-centric learning and evaluation paradigm that involves (i) a large-scale limb-aware vision–language dataset and evaluation benchmark for multimodal reasoning, and (ii) a model adaptation strategy for Vision-Language Models (VLM) in limb-difference contexts. Specifically, we first collect limb-difference data covering all eight limb-deficiency types across diverse real-world scenarios. The data are systematically organized into 96 limb-affected human action categories and 68 medical-functional classes defined by the World Health Organization (WHO). Then, we curate an expert-annotated vision–language dataset for limb-aware multimodal understanding, named Inclusive VQA for Limb Deficiency (**IVQA-LD**). IVQA-LD comprises 80K VQA pairs spanning eight core tasks including visual grounding, quantitative reasoning, functional semantic classification, and instructional text generation. We benchmark state-of-the-art VLMs on IVQA-LD and find that they consistently struggle across all tasks, exposing substantial deficiencies in limb-aware perception and reasoning. To address this, we further propose a Body-centric Structure-aware Initialization (BSI) strategy that aligns model representations with limb-specific semantics. With BSI, VLMs fine-tuned on IVQA-LD achieve significant performance improvements across all the tasks. We will publicly release the dataset to support future research.

Reinforcement Learning · Inverse

Aleksandar Taranovic, Onur Celik, Niklas Freymuth, Ge Li, Serge Thilges, Huy Le, Tai Hoang, Rania Rayyes, Gerhard Neumann

Preference-based reinforcement learning (PbRL) learns policies from human trajectory-level comparisons, avoiding explicit reward design and expert demonstrations. Existing methods typically train utility functions on trajectory or segment-level preferences while relying on per-step utility estimates during policy optimization. This training and inference mismatch induces a distribution shift that severely degrades temporal credit assignment and limits policy learning. We analyze this issue and propose Preference Learning with Advantage-Weighted Segments(PAWS), a segment-based preference learning method that performs policy updates directly using segment-level advantage functions. By aligning utility training with policy optimization, PAWS preserves trajectory-level preference information and avoids unreliable per-step learning signals. Experiments on simulated robotic manipulation and locomotion tasks demonstrate that PAWS consistently outperforms existing PbRL approaches, highlighting the importance of distribution-consistent preference learning.