论文检索

General Machine Learning · Representation Learning

Jiarong Yang, Yuan Liu

Adopting pre-trained Vision-Language Models (VLMs) in Federated Learning (FL) presents a promising avenue for mitigating data scarcity and heterogeneity. However, existing solutions suffer from high computational complexity or ineffective knowledge aggregation. To address these problems, we propose FedSPA (Federated Adaptation via Semantic-Visual Prototype Alignment). On the client side, FedSPA restricts local optimization to visual prototypes, enabling lightweight personalization. On the server side, we introduce a semantic alignment module that leverages client-uploaded prototypes to minimize a contrastive objective, aligning global semantic prototypes with heterogeneous visual distributions and thereby shifting the paradigm from traditional “learning-to-describe" (optimizing static prompts) to ”learning-to-align". Extensive experiments demonstrate that FedSPA significantly outperforms state-of-the-art methods in both personalized and global benchmarks, while substantially reducing computational overhead.

Applications · Computer Vision

Yong Yang, Xuran Zhang, Shuying Huang, Xiaozheng Wang, Weiguo Wan, Hangyuan Lu

Hyperspectral Image Single Image Super-Resolution (HSI-SISR) faces a conflict between computational efficiency and global non-local modeling. Existing Transformers suffer from quadratic complexity, while window-based methods compromise global capture. To address this, we propose the Spatial-Spectral Dual-Clustering-based Network (SSDCN). Our method introduces three innovations. First, we design a Spatial-Spectral Dual-Cluster Block (SSDCB). Replacing expensive point-to-point attention, it uses content-driven clustering to learn low-rank structural bases, achieving global modeling with linear complexity $\mathcal{O}(KN)$. Second, we propose a pyramid progressive hierarchical architecture with a Feature Reuse Reconstruction Block (FRRB). It reuses the core tensor and spectral factors from coarse levels, updating only spatial factors to minimize redundancy. Third, we propose a Pyramid Hierarchical Reconstruction Joint Loss to supervise intermediate levels, ensuring structural accuracy and preventing error accumulation. Experiments demonstrate that SSDCN surpasses SOTA methods in metrics and visual quality with significantly fewer parameters and FLOPs, achieving an optimal efficiency-performance balance.

Deep Learning · Large Language Models

Xinyi Hu, Yuhao Shen, Zhang Baolin, Hengxin Zhang, Jun Dai, Shuang Ge, Yue Li, Mingcheng Wan

Speculative Decodin promises to accelerate Large Language Model inference, yet its efficacy often degrades in production-grade scenarios. Existing evaluations typically overlook the compute-bound nature of high-concurrency regimes, where verification compute becomes the dominant bottleneck. Consequently, prior methods face a dilemma: static trees incur massive verification waste, while dynamic trees suffer from cumulative misjudgments and kernel incompatibility. To bridge this gap, we introduce ECHO, a high concurrency-oriented framework integrated into SGLang that reformulates speculative execution as a budgeted scheduling problem. Crucially, ECHO employs sparse confidence gating to manage the batch as a unified super-tree, elastically pivoting budget between depth and width to co-optimize the trade-off between reducing global verification steps and maximizing per-step efficiency. Extensive evaluations across diverse model scales—particularly the industrial-grade Qwen3-235B—demonstrate that ECHO consistently outperforms state-of-the-art baselines in both low-load and high-load scenarios, achieving up to 5.35$\times$ walltime speedup and delivering over 20\% relative speedup gain against the strongest baselines.

Deep Learning · Generative Models and Autoencoders

Xuanyi Liu, Deyi Ji, Liqun Liu, Lanyun Zhu, Xuhang Chen, Qianxiong Xu, Peng Shu, Huan Yu, Jie Jiang, Feng Gao 等

Sparse camera-conditioned image-to-video generation presents a pivotal challenge: synthesizing geometrically consistent 3D motion from minimal pose cues. Existing methods, which largely rely on dense supervision or naive interpolation, suffer from severe pose drift and motion discontinuities due to the lack of robust 3D priors. In this paper, we introduce \textbf{CamGeo}, a novel framework that distills rich 3D geometric knowledge from a pre-trained video-to-3D model (VGGT) directly into the diffusion backbone. To achieve this without incurring inference latency, we propose a training-only distillation strategy. Specifically, CamGeo incorporates: (1) keyframe trajectory distillation that enforces cycle-consistency with sparse input poses, (2) cross-frame consistency distillation with both camera trajectory and depth constraints to generate consistent structure across unsupervised frames, and (3) a three-stage coarse-to-fine curriculum learning, progressively scales geometric complexity, from global structure coherence to fine-grained refinement, achieving stable optimization. Extensive experiments demonstrate that CamGeo achieves consistent improvements under various sparsity ratios.

Xinyi Hu, Yuhao Shen, Zhang Baolin, Hengxin Zhang, Jun Dai, Shuang Ge, Chen Lei, Yue Li, Mingcheng Wan

Speculative Decodin promises to accelerate Large Language Model inference, yet its efficacy often degrades in production-grade scenarios. Existing evaluations typically overlook the compute-bound nature of high-concurrency regimes, where verification compute becomes the dominant bottleneck. Consequently, prior methods face a dilemma: static trees incur massive verification waste, while dynamic trees suffer from cumulative misjudgments and kernel incompatibility. To bridge this gap, we introduce ECHO, a high concurrency-oriented framework integrated into SGLang that reformulates speculative execution as a budgeted scheduling problem. Crucially, ECHO employs sparse confidence gating to manage the batch as a unified super-tree, elastically pivoting budget between depth and width to co-optimize the trade-off between reducing global verification steps and maximizing per-step efficiency. Extensive evaluations across diverse model scales—particularly the industrial-grade Qwen3-235B—demonstrate that ECHO consistently outperforms state-of-the-art baselines in both low-load and high-load scenarios, achieving up to 5.35$\times$ walltime speedup and delivering over 20\% relative speedup gain against the strongest baselines.

Deep Learning · Attention Mechanisms

Lijie Yang, Zhihao Zhang, Arti Jain, Shijie Cao, Baihong Yuan, Yiwei Chen, Zhihao Jia, Ravi Netravali

Large reasoning models achieve strong performance through test-time scaling, but this incurs substantial computational overhead due to long decoding from short prompts. While sparse attention can reduce latency and memory usage, existing methods often degrade reasoning accuracy because selection errors accumulate over long generation horizons, or require costly retraining. We introduce LessIsMore, a training-free sparse attention mechanism for long-horizon reasoning. Our key insight is that token importance in reasoning is global and stable: critical tokens are largely shared across attention heads and remain stable over decoding steps. Guided by this structure, LessIsMore enforces cross-head unified token selection and preserves recent context via a stable recency window, yielding a globally consistent token set that can be reused across layers. Across multiple model families and challenging reasoning benchmarks, LessIsMore matches or improves accuracy while attending to substantially fewer tokens. With kernel-level optimizations, LessIsMore achieves up to $1.6\times$ end-to-end decoding speedup and up to $1.72\times$ faster sparse attention computation, with additional long-context results demonstrating the generality of our approach.

Deep Learning · Large Language Models

Gagan Mundada, Zihan Huang, Rohan Surana, Sheldon Yu, Jennifer Zhang, Xintong Li, Tong Yu, Lina Yao, Jingbo Shang, Julian McAuley 等

Group Relative Policy Optimization (GRPO) is effective for training language models on complex reasoning. However, since the objective is defined relative to a group of sampled trajectories, extended deliberation can create more chances to realize relative gains, leading to inefficient reasoning and overthinking, and complicating the trade-off between correctness and rollout efficiency. Controlling this behavior is difficult in practice, considering (i) Length penalties are hard to calibrate because longer rollouts may reflect harder problems that require longer reasoning, penalizing tokens risks truncating useful reasoning along with redundant continuation; and (ii) supervision that directly indicates when to continue or stop is typically unavailable beyond final answer correctness. We propose Weakly Supervised GRPO (WS-GRPO), which improves rollout efficiency by converting terminal rewards into correctness-aware guidance over partial trajectories. Unlike global length penalties that are hard to calibrate, WS-GRPO trains a preference model from outcome-only correctness to produce prefix-level signals that indicate when additional continuation is beneficial. Thus, WS-GRPO supplies outcome-derived continue/stop guidance, reducing redundant deliberation while maintaining accuracy. We provide theoretical results and empirically show on reasoning benchmarks that WS-GRPO substantially reduces rollout length while remaining competitive with GRPO baselines.

Applications · Language, Speech and Dialog

Miria Feng, William H. Tan, Mert Pilanci

Globalization and multiculturalism continue to produce increasingly diverse speech varieties. Yet current spoken dialogue systems frequently fail on under-represented dialects and accents, often misidentifying the input language and causing cascading failures in downstream dialogue tasks. Addressing this dialectal variance under low-resource constraints remains an open challenge, as standard fine-tuning is computationally expensive and prone to overfitting on high-dimensional speech data. We propose Convex Language Detection (CLD), a novel framework that integrates theoretically grounded convex optimization techniques into the spoken dialogue systems pipeline. Our method is efficiently implemented via multi-GPU Alternating Direction Method of Multipliers (ADMM) in JAX, thus providing global optimality guarantees and fast training in polynomial time. Theoretically, we prove that our convex objective induces certified margin stability and provide rigorous guarantees against feature perturbations. Empirically, we demonstrate sample efficiency and robustness to input dialectical variance, to significantly reduce language misidentification rates for low-resource dialects within high-resource languages. Our open-source package is available at \url{https://anonymous.4open.science/r/CLD-845F/README.md}.

Social Aspects · Privacy

Wanjie Wang, Tathagata Banerjee

We study ridge regression and ridge-regularized empirical risk minimization (ERM) under $(\varepsilon,\delta)$-differential privacy via output perturbation. In classical private ERM, the ridge parameter simultaneously controls statistical regularization and the estimator’s global sensitivity. Larger regularization reduces the DP noise scale but increases bias. So choosing the tuning parameter becomes a privacy--accuracy bottleneck. We propose a framework that makes these two roles explicit by decoupling regularization into (i) a statistical penalty $\alpha$, defining the target ridge/ERM solution, and (ii) a privacy-stabilization parameter $c$, used only to enforce a curvature floor and hence a tight sensitivity bound. We apply this framework to ridge regression, where $c$ is used to boost the minimum eigenvalue of the empirical Gram matrix. We derive an explicit bias--variance--DP-variance risk decomposition and characterize optimal $(\alpha,c)$ in several regimes, yielding sharp tuning guidance and improved accuracy relative to single-parameter regularization. Finally, we extend the same decoupling principle to general ridge-regularized ERM. We support the theory with simulations.

Deep Learning · Foundation Models

Liangwei Yang, Shiyu Wang, Haolin Chen, Rithesh Murthy, Ming Zhu, Jielin Qiu, Zixiang Chen, Juntao Tan, Jianguo Zhang, Zhiwei Liu 等

As large language models (LLMs) transition from research prototypes to real-world systems, customization has emerged as a central bottleneck. While text prompts can already customize LLM behavior, we argue that text-only prompting does not constitute a suitable control interface for scalable, stable, and inference-only customization. This position paper argues that model providers should expose \emph{vector prompt inputs} as part of the public interface for customizing LLMs. We support this position with diagnostic evidence showing that vector prompt tuning continues to improve with increasing supervision whereas text-based prompt optimization saturates early, and that vector prompts exhibit dense, global attention patterns indicative of a distinct control mechanism. We further discuss why inference-only customization is increasingly important under realistic deployment constraints, and why exposing vector prompts need not fundamentally increase model leakage risk under a standard black-box threat model. We conclude with a call to action for the community to rethink prompt interfaces as a core component of LLM customization.

Applications · Robotics

Xiaofei Wei, Chun Gu, Li Zhang

Learning robust navigation policies remains a core challenge in robotics. Offline imitation learning suffers from distribution shift and compounding errors at rollout, while reinforcement learning requires reward engineering and learns inefficiently. In this paper, we propose NavOL, an online imitation learning paradigm that interacts with a simulator and updates itself using expert demonstrations gathered online. Built upon a pretrained navigation diffusion policy that maps local observations to future waypoints, NavOL trains in a rollout–update loop: during rollout, the policy acts in the simulator and queries a global planner which has privileged access to the global environment for the optimal path segment as ground truth trajectory labels; during update, the policy is trained on the online collected observation–trajectory pairs. This online imitation loop removes the need for reward design, improves learning efficiency, and mitigates distribution shift by training on the policy’s own explored rollouts. Built on IsaacLab with fast, high-fidelity parallel rendering and domain randomization of camera pose and start-goal pairs, our system scales across 50 scenes on 8 RTX 4090 GPUs, collecting over 2,000 new trajectories per hour, each averaging more than 400 steps. We also introduce an indoor visual navigation benchmark with predefined start and goal positions for zero-shot generalization. Extensive evaluations on simulation benchmarks, including the NavDP benchmark and our proposed benchmark, as well as carefully designed real-world experiments, demonstrate the effectiveness of NavOL, showing consistent performance gains in online imitation learning.

Theory · Deep Learning

Bodie Rubacher

Softmax attention is increasingly treated as a reusable inference primitive in transformers. Much prior theory covers Gaussian/linear models or assumes one shared quadratic query--key metric, which can fail for non-Gaussian exponential-family likelihoods with varying curvature. This risks overreading learned similarities as a global metric or as proof of Bayes-optimality. For discrete latent symbols with exponential-family observations, we show a single head can implement Bayes posteriors (and posterior means) by setting logits to log prior plus log likelihood, and we characterize single-head posteriors as exactly the log-linear (exponential-family) class. Using convex duality, we rewrite log-likelihoods as negative Bregman divergences on mean/sufficient-statistic space, making Bayes rule a soft nearest-neighbor computation; this yields a sharp boundary for globally shared quadratic metrics and a multi-head curvature-atlas approximation with head-count scaling, and we extend these guarantees to in-context estimation with consistency and finite-sample stability bounds. In synthetic Gaussian and Bernoulli in-context estimation, trained minimal attention models validate these predictions: performance approaches a Bayes-oracle baseline as prompt length grows; learned metrics align with noise precision in the Gaussian case, while Bernoulli retains a gap consistent with curvature variation. Together, these results explain when Fisher geometry should emerge, when a single metric is justified, and when multiple heads are necessary for Bayes-like in-context estimators beyond Gaussians.

Applications · Chemistry, Physics, and Earth Sciences

Tianmeng Hu, Biao Luo, Ke Li

Designing RNA sequences that reliably fold into specific secondary structures is essential for understanding their biological functions but remains a challenging computational problem. We propose CocoRNA, a cooperative multi-agent reinforcement learning framework for RNA inverse design. CocoRNA simplifies the design task by decomposing it into smaller sub-problems, each solved collaboratively by multiple agents. This approach reduces the complexity of the problem and improves the exploration of design policies. During training, a centralized critic uses global structural information to guide the agents, enabling them to jointly optimize their design strategies. As a result, CocoRNA learns high-quality RNA design policies that generalize effectively to unseen structures without additional training. Experiments on the Rfam dataset demonstrate that CocoRNA substantially outperforms state-of-the-art methods in both success rate and design speed. Further experiments on other biological sequence design tasks highlight the effectiveness and broad potential of CocoRNA for complex design tasks.

Applications · Chemistry, Physics, and Earth Sciences

Zherui Yang, Haiyang Xin, Tao Du, Ligang Liu

Neural operators have emerged as data-driven surrogates for solving partial differential equations (PDEs), and their success hinges on efficiently modeling the long-range, global coupling among spatial points induced by the underlying physics. In many PDE regimes, the induced global interaction kernels are empirically compressible, exhibiting rapid spectral decay that admits low-rank approximations. We leverage this observation to unify representative global mixing modules in neural operators under a shared low-rank template: compressing high-dimensional pointwise features into a compact latent space, processing global interactions within it, and reconstructing the global context back to spatial points. Guided by this view, we introduce Low-Rank Spatial Attention (LRSA) as a clean and direct instantiation of this template. Crucially, unlike prior approaches that often rely on non-standard aggregation or normalization modules, LRSA is built purely from standard Transformer primitives, i.e., attention, normalization, and feed-forward networks, yielding a concise block that is straightforward to implement and directly compatible with hardware-optimized kernels. In our experiments, such a simple construction is sufficient to achieve high accuracy, yielding an average error reduction of over 17\% relative to second-best methods, while remaining stable and efficient in mixed-precision training.

Applications · Computer Vision

Guangze Shi, Yingjie Mi, Jia Shen, Feixue Shao, Jiarui Cao, Yexin Lai, Xueyu Liu, Rui Wang, Yongfei Wu, Mingqiang Wei

Optimizing prompts for foundation models like SAM represents a challenging high-dimensional black-box optimization problem, fundamentally plagued by the credit assignment ambiguity. To address this, we introduce PromptPilot, a task-agnostic reinforcement learning framework that structurally decomposes the search space into orthogonal semantic and spatial subspaces. Specifically, a centralized manager orchestrates two specialized agents, a feature agent ensuring semantic coherence and a physical agent maximizing spatial coverage, to navigate conflicting optimization objectives. Crucially, our reward mechanism synergizes global segmentation feedback with an efficient approximation of Shapley values, enabling fine-grained attribution of performance gains to individual prompt actions. PromptPilot functions as an inference-time optimization strategy without parameter updates. Extensive experiments demonstrate that our game-theoretic approach significantly improves segmentation performance and generalization, offering a principled solution for automated prompt engineering.

Lin Wang, Zhichao Wang, Ye Shi, Sai Praneeth Reddy Karimireddy, Xiaoying Tang

Federated Learning (FL) often suffers from a trade-off between global model performance and client-level fairness due to data heterogeneity, which often leads to inconsistent performance of the globally trained models, resulting in unfair outcomes among users. Existing fair FL algorithms face a trade-off: they either sacrifice global model performance to promote fairness or fall short of achieving optimal fairness. In this paper, we propose a novel framework that bridge this trade-off by integrating information-theoretic principles with model alignment. Specifically, we leverage the Maximum Entropy Principle to derive an analytic, closed-form solution for fair aggregation weights, ensuring significant fairness enhancements with minimal computational overhead. To maintain the global model performance, we further employ a step-wise model alignment strategy that synchronizes gradient directions across heterogeneous clients, effectively mitigating the drift induced by local updates. Theoretical analysis proves that our method guarantees convergence even in non-convex settings. Importantly, we push the theoretical frontier of federated fairness by extending performance variance analysis to generalized regression, providing broader guarantees. Extensive experiments on five datasets demonstrate that our approach consistently outperforms state-of-the-art methods, achieving superior fairness without sacrificing global accuracy.

General Machine Learning · Transfer, Multitask and Meta-learning

Longtao Xu, Jian Li

Real-world federated systems seldom operate on static data: input distributions drift while privacy rules forbid raw data sharing. We study Federated Domain-Incremental Learning (FDIL), where (i) clients are heterogeneous, (ii) tasks arrive sequentially with shifting domains, and (iii) the label space remains fixed. Two theoretical pillars remain missing for FDIL under partial participation: a guarantee of backward knowledge transfer (BKT) and a convergence rate that holds *uniformly across the task sequence*. We introduce SPECIAL (Server-Proximal Efficient Continual Aggregation for Learning), a simple, memory-free FDIL algorithm that adds a single server-side ``anchor'' to FedAvg: in each round, the server aggregates updates from a uniformly sampled subset of clients and then blends the result with the previous global model via a lightweight proximal step. This anchor curbs cumulative drift without replay buffers, synthetic data, or task-specific heads, leaving communication cost and model size unchanged. Our theory shows that SPECIAL (i) *preserves earlier tasks*: a BKT bound caps any increase in earlier-task loss by a drift-controlled term that shrinks with more rounds, local epochs, and participating clients; and (ii) *achieves task-uniform, communication-efficient convergence* for non-convex FDIL with partial participation: $\mathcal{O}\!\big(\sqrt{E/(NT)}\big)$ in expected gradient norm, with $E$ local epochs, $T$ rounds, and $N$ participating clients, while explicitly separating optimization variance from inter-task drift. Experiments on standard FDIL benchmarks corroborate the theory.

Deep Learning · Everything Else

Yuting Ma, Lechao Cheng, Xiaohua Xu

Federated Learning (FL) with pre-trained Vision-Language Models (VLMs) has emerged as a promising paradigm for various downstream tasks. By leveraging its strong representations, recent studies improve task adaptation under insufficient local data while preserving generalization. However, these methods emphasize fully local optimization with simple parameter aggregation, which can amplify inter-client optimization inconsistency and intra-client over-specialization under heterogeneous and full-data FL settings, making it difficult to balance global task adaptation and generalization. To address these challenges, we propose FedDTL, a novel federated VLM framework that decouples the image encoder and text encoder across clients and the server. Through decoupled encoder training with server-client modality alignment, FedDTL promotes coherent global semantic update and reduces inter-client optimization inconsistency, improving global task adaptation. To further mitigate intra-client over-specialization, we introduce a two-stage local fine-tuning, where a supervised fine-tuning stage enables rapid and reliable warm-start, followed by a reinforcement learning stage that enhances generalization. Extensive experiments on multiple benchmarks, including label skew and feature shift, demonstrate that FedDTL achieves an effective balance between global task adaptation and generalization under various FL data distributions in both few-shot and full-data regimes.

Applications · Chemistry, Physics, and Earth Sciences

Salva Ruhling Cachay, Duncan Watson-Parris, Rose Yu

Global weather forecasting has recently been revolutionized by AI, outperforming traditional physics-based ensembles. However, these state-of-the-art (SOTA) models rely on massive computational resources and increasingly specialized architectures, creating a high barrier to entry. In this work, we demonstrate that such complexity is not a prerequisite for SOTA performance. We introduce U-Cast, a streamlined probabilistic forecaster based on a standard U-Net, trained with Monte Carlo Dropout and the Muon optimizer. By leveraging a novel curriculum—deterministic pre-training followed by probabilistic fine-tuning on the Continuous Ranked Probability Score (CRPS)—our model achieves performance on par with or exceeding GenCast (e.g., up to 15\% CRPS improvement on short-range winds) while reducing training and/or inference compute by an order of magnitude compared to leading baselines. Our $1^\circ$ model trains in less than 15 H200 GPU-days and generates a 60-step forecast in just 12 seconds, suggesting a "Bitter Lesson" for AI weather forecasting: scalable, general-purpose architectures can outperform complex domain-specific designs.

Deep Learning · Large Language Models

Quanxin Liu, Yijun Mo, Ruida Xu, Jianwei Zhong, Changhu Chen, Rui Hao

End-to-end data science agent workflows involve tightly coupled sub-processes with strong dynamic dependencies, posing a challenging long-horizon orchestration problem. Existing frameworks primarily rely on static, chain-like execution plans, which are prone to error propagation from early stages—often causing reasoning chain collapse and task failure, resulting in fragile inference and poor cost-effectiveness. To address these issues, we propose $\text{R}^3$DAO, a reactive data agent orchestration framework based on feedback-driven topology evolution, aiming to build a dynamic evolutionary closed-loop of "hierarchical exploration, iterative recovery, and empirical convergence." First, we introduce a dynamic hierarchical task network that recursively decomposes global intent into macro-logical anchors and micro-operators, enabling low-cost exploration through dimensionality reduction in the logical space. Second, we establish a reactive topology reconfiguration mechanism that leverages semantic reflection to map execution anomalies into diagnostic signals, replacing costly global resets with localized topological optimization for resilient self-healing. Finally, semantic experience distillation implements a dual-loop accumulation that compresses long-horizon trajectories into structured prior, steering execution efficiency toward the optimal regime. Evaluations on the MLE-bench show that $\text{R}^3$DAO achieves a 77.36\% improvement in success rate over advanced R\&D-Agent while maintaining competitive task scores. Notably, $\text{R}^3$DAO compresses the average execution time by 36$\times$ and limits token consumption to just 104k per task, showcasing superior reliability, efficiency, and cost-effectiveness.