论文检索

Reinforcement Learning · Deep RL

Amir Moeini, Minjae Kwon, Alper Bozkurt, Yuichi Motai, Rohan Chandra, Lu Feng, Shangtong Zhang

In-context reinforcement learning (ICRL) is an emerging RL paradigm where an agent, after pretraining, can adapt to out-of-distribution test tasks without any parameter updates, instead relying on an expanding context of interaction history. While ICRL has shown impressive generalization, safety during this adaptation process remains unexplored, limiting its applicability in real-world deployments where test-time behavior is expected to be safe. In this work, we propose SCARED: Safe Contextual Adaptive Reinforcement via Exact-penalty Dual, the first method that promotes safe adaptation of ICRL under the constrained Markov decision process framework. During the parameter-update-free adaptation process, our agent not only maximizes the reward but also keeps the accumulated cost within a user-specified safety budget. We also demonstrate that the agent actively reacts to the safety budget; with a higher safety budget, the agent behaves more aggressively, and with a lower safety budget the agent behaves more conservatively. Across challenging benchmarks, SCARED consistently enables safe and robust in-context adaptation, outperforming existing ICRL and safe meta-RL baselines.

Applications · Health / Medicine

Chunlei Li, Zixuan Zheng, Yilei Shi, Guanglu Dong, Pengfei Li, Jingliang Hu, Xiao Zhu, Lichao Mou

Mislabeled samples in training datasets severely degrade the performance of deep networks, as overparameterized models tend to memorize erroneous labels. We address this challenge by proposing a novel approach for mislabeled data detection that leverages training dynamics. Our method is grounded in the key observation that correctly labeled samples exhibit consistent entropy decrease during training, while mislabeled samples maintain relatively high entropy throughout the training process. Building on this insight, we introduce a signed entropy integral (SEI) statistic that captures both the magnitude and temporal trend of prediction entropy across training epochs. SEI is broadly applicable to classification networks and demonstrates particular effectiveness when integrated with contrastive language-image pretraining (CLIP) architectures. Through extensive experiments on three medical imaging datasets---a domain particularly susceptible to labeling errors due to diagnostic complexity---spanning diverse modalities and pathologies, we demonstrate that SEI achieves state-of-the-art performance in mislabeled data identification, outperforming existing methods while maintaining computational efficiency and implementation simplicity.

Reinforcement Learning · Deep RL

Minjae Kwon, Josephine Lamp, Lu Feng

Safe Reinforcement Learning (RL) algorithms are typically evaluated under fixed training conditions. We investigate whether training-time safety guarantees transfer to deployment under distribution shift, using diabetes management as a safety-critical testbed. We benchmark safe RL algorithms on a unified clinical simulator and reveal a safety generalization gap: policies satisfying constraints during training frequently violate safety requirements on unseen patients. We demonstrate that test-time shielding, which filters unsafe actions using learned dynamics models, effectively restores safety across algorithms and patient populations. Across eight safe RL algorithms, three diabetes types, and three age groups, shielding achieves Time-in-Range gains of 13-14\% for strong baselines such as PPO-Lag and CPO while reducing clinical risk index and glucose variability. Our simulator and benchmark provide a platform for studying safety under distribution shift in safety-critical control domains.

Deep Learning · Large Language Models

Qingyue Zhang, Chang Chu, Tianren Peng, Qi Li, Xiangyang Luo, Zhihao Jiang, Shao-Lun Huang

LoRA has become a widely adopted method for PEFT, and its initialization methods have attracted increasing attention. However, existing methods have notable limitations: many methods do not incorporate target-domain data, while gradient-based methods exploit data only at a shallow level by relying on one-step gradient decomposition. In this paper, we establish a theoretical framework for data-aware LoRA initialization. Starting from minimizing the expectation of the parameter discrepancy between the fine-tuned and target models, we derive an optimization problem with two components: a bias term, which is related to the parameter distance between the fine-tuned and target models, and is approximated using a Fisher–gradient formulation to preserve anisotropy; and a variance term, which accounts for the uncertainty introduced by sampling stochasticity through the Fisher information. Solving this problem yields an optimal initialization strategy for LoRA, based on which we develop an efficient algorithm, LoRA-DA. Empirical results across multiple benchmarks demonstrate that LoRA-DA consistently improves final accuracy over existing initialization methods. Additional studies show faster, more stable convergence, robustness across ranks, and only a small initialization overhead for LoRA-DA. The source code will be released upon publication.

Deep Learning · Algorithms

Jiayu Zhang, Changbang Li, Canran Xiao

In-context learning (ICL) is a practical way to adapt large models, yet under strict context limits it remains unclear how to spend scarce tokens without being misled by noisy, redundant, or conflicting demonstrations. We address this gap by targeting token-budgeted context construction: how to select and compress demonstrations so the prompt carries maximal task-relevant signal with minimal predictive distortion. We propose RDCO, a deterministic, training-free optimizer that scores demonstrations by marginal task information per token, penalizes redundancy and prefix-conditioned conflicts, and compacts the selected context under a bounded predictive divergence constraint to control drift. Across a 10-dataset ICL suite spanning classification and structured generation, RDCO achieves the best average performance (63.26 Acc. on classification and 56.26 EM on generation) and improves the overall average by +2.20 points over the strongest baseline under the same budget. Our results suggest that viewing prompts as finite-capacity messages yields a principled and effective path to more reliable and token-efficient ICL.

Applications · Time Series

Junkai Lu, Peng Chen, Xingjian Wu, Yang Shu, Chenjuan Guo, Christian S Jensen, Bin Yang

Time series reasoning demands both the perception of complex dynamics and logical depth. However, existing LLM-based approaches exhibit two limitations: they often treat time series merely as text or images, failing to capture the patterns like trends and seasonalities needed to answer specific questions; and when trained on a mix of simple and complex tasks, simpler objectives often dominate the learning process, hindering the development of deep reasoning capabilities. To address these limitations, we propose the Pattern-Aware Alignment and Balanced Reasoning model (PATRA), introducing a pattern-aware mechanism that extracts trend and seasonality patterns from time series to achieve deep alignment. Furthermore, we design a task-aware balanced reward to harmonize learning across tasks of varying difficulty, incentivizing the generation of coherent Chains of Thought. Extensive experiments show that PATRA outperforms strong baselines across diverse Time Series Question Answering (TSQA) tasks, demonstrating superior cross-modal understanding and reasoning capability.

Deep Learning · Algorithms

Jianfeng Lu, YuZhao Xiang, Yue Chen, Gang Li, Shuqin Cao, Guanghui Wen

Federated Learning (FL) often suffers from degraded generalization under statistical heterogeneity, where client updates systematically deviate from the global objective. While recent Sharpness-Aware Minimization (SAM) methods promote locally flat solutions, they implicitly assume that local flatness transfers to the global model, which generally does not hold under heterogeneous data distributions. This mismatch gives rise to a flatness discrepancy induced by misaligned loss landscapes. To address this issue, we propose FedScar, a federated optimization framework that explicitly corrects heterogeneity-induced geometric inconsistency. FedScar maintains a history-accumulated geometric bias to capture persistent curvature skew across clients, and employs a variance-aware injection mechanism to steer local updates toward regions that are flat with respect to the global objective. We provide a theoretical interpretation of FedScar as a Split-Dual ADMM formulation, which jointly enforces parameter consensus and geometric alignment. Extensive experiments under severe heterogeneity demonstrate that FedScar consistently reduces flatness discrepancy and improves generalization over state-of-the-art methods, without incurring additional communication overhead.

Applications · Robotics

Kewei Chen, Yayu Long, mingsheng shang

Despite rapid progress in Vision-Language-Action (VLA) models for robotic control, instruction drift remains a persistent failure mode in long-horizon tasks. This paper reconceptualizes this phenomenon, positing that instruction drift is fundamentally a systematic sampling error: local greedy sampling is prone to collapsing into ''Negative Pivotal Windows''—irreversible local optima with high local probability that sever global success pathways. To address this, we propose \textbf{Context-Aware Power Sampling (CAPS)}, a training-free inference-time computation framework. CAPS leverages power distributions to sharpen global trajectory probabilities, effectively activating the model's implicit world model for lookahead planning. Furthermore, we introduce a metacognitive control mechanism based on Signal-to-Noise Ratio (SNR). This mechanism triggers adaptive MCMC search solely when drift risk is detected, enabling a dynamic transition from ''intuitive fast thinking'' to ''rational slow search.'' Experiments on RoboTwin, Simpler-WindowX, and Libero-long benchmarks demonstrate that CAPS significantly outperforms SOTA baselines, such as OpenVLA and TACO, without parameter updates. These results confirm that adaptive inference-time computation is a potent pathway to enhancing embodied long-horizon robustness.

Deep Learning · Algorithms

Gagik Magakyan, Amirhossein Reisizadeh, Chanwoo Park, Pablo A. Parrilo, Asuman Ozdaglar

*Adaptability* has been regarded as a central feature in the foundation models, enabling them to effectively acclimate to unseen downstream tasks. Parameter-efficient fine-tuning methods such as celebrated LoRA facilitate efficient adaptation of large foundation models using labeled, high-quality and generally scarce task data. To mitigate data scarcity in fine-tuning of foundation models, we propose to leverage *task similarity* across downstream users. Intuitively, users with similar tasks must be able to assist each other in boosting the effective fine-tuning data size. We propose *Collaborative Low-Rank Adaptation*, or CoLoRA, which exploits task similarity to collaboratively and efficiently fine-tune personalized foundation models. The main idea in CoLoRA is to train one shared adapter capturing underlying task similarities across all tasks, and personalized adapters tailored to user-specific tasks. We theoretically study CoLoRA on heterogeneous linear regression and provide provable guarantees for ground truth recovery. We also conduct several natural language experiments with varying task similarity, which further demonstrate that when trained together with similar tasks, individual performances are significantly boosted.

Reinforcement Learning · Deep RL

Sarvesh Patil, Mitsuhiko Nakamoto, Shashwat Saxena, Manan Agarwal, Giri Anantharaman, Cleah Winston, Jesse Zhang, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang 等

Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning. Yet there remains substantial debate over how to sample efficiently fine-tune them via reinforcement learning. A prevailing view holds that fine-tuning all GCP steps is unnecessary, motivating approaches that fine-tune only a subset of the generative process: either steering the initial noise distribution or learning residual corrections on top of a frozen base policy. In this work, we introduce Off-policy Generative Policy Optimization (\OGPO{}), a sample-efficient algorithm for finetuning GCPs that maintains off-policy critic networks to maximize data reuse and propagate policy gradients through the full generative process of the policy via a modified PPO objective, using critics as the terminal reward. \OGPO{} achieves state-of-the-art performance on manipulation tasks spanning multi-task settings, high-precision insertion, and dexterous control. To our knowledge, it is also the only method that can \emph{fine-tune poorly-initialized behavior cloning policies to near full task-success with no expert data in the online replay buffer}, and does so with \emph{few task-specific hyperparameter tuning}. We perform extensive empirical investigations on \OGPO{}, finding that its superior performance and sample efficiency lie in its ability to learn beyond the action distribution of the pre-trained base policy, and propose practical implementation details that further boost performance for more complex scenarios.

Optimization · Zero-order and Black-box Optimization

Jingzhe Jing, Zheyi Fan, Szu Hui Ng, Qingpei Hu

Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality. We propose **L**ocal **C**onstrained **B**ayesian **O**ptimization (LCBO), a novel framework tailored for such settings. Unlike trust-region methods that are prone to premature shrinking when confronting tight or complex constraints, LCBO leverages the differentiable landscape of constraint-penalized surrogates to alternate between rapid local descent and uncertainty-driven exploration. Theoretically, we prove that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker (KKT) residual that depends polynomially on the dimension $d$ for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially. Extensive evaluations on high-dimensional benchmarks (up to 100D) demonstrate that LCBO consistently outperforms state-of-the-art baselines.

Deep Learning · Graph Neural Networks

Xinya Qin, Lu Bai, Lixin Cui, Ming Li, Hangyuan Du, Ziyu Lyu, Xin Jin

Graph Convolutional Networks (GCNs) are defined based on aggregating the node information of adjacent nodes, that are usually treated as equally important as each other, limiting the representational power of existing GCNs for graph classification. To address this shortcoming, we propose a novel Global Interacted Graph Convolutional Network (GI-GCN), that can leverage the solution vectors maintained during the iterative updates of the Dominant Set to adaptively characterize the global importance distribution of different nodes. Specifically, at each convolution layer, this distribution is adopted to adaptively modulate the importance weights of different node features before performing the local message passing. We show that this convolution strategy can effectively capture the highly correlated information between nonadjacent nodes through the Dominant Set algorithm, not only emphasizing the critical information at the graph level but also enhancing the discriminative power of graph representations. Furthermore, we optimize the spatial complexity of the framework, significantly reducing the memory overhead associated with the global interaction modeling. Experiments demonstrate the effectiveness of the proposed GI-GCN.

Deep Learning · Theory

Oskar Nordenfors, Fredrik Ohlsson, Axel Flinth

We investigate the optimization of neural networks on symmetric data, and compare the strategy of constraining the architecture to be equivariant to that of using data augmentation. Our analysis reveals that the relative geometry of the admissible and the equivariant layers, respectively, plays a key role. Under natural assumptions on the data, network, loss, and group of symmetries, we show that compatibility of the spaces of admissible layers and equivariant layers, in the sense that the corresponding orthogonal projections commute, implies that the sets of equivariant stationary points are identical for the two strategies. If the linear layers of the network also are given a unitary parametrization, the set of equivariant layers is even invariant under the gradient flow for augmented models. Our analysis however also reveals that even in the latter situation, stationary points may be unstable for augmented training although they are stable for the manifestly equivariant models.

Deep Learning · Algorithms

Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Zhiqiang Shen

Dataset pruning remains underexplored for 3D modalities, where inherent class imbalance persists across ***both*** training and test sets. This creates a divergence in evaluation: overall accuracy favors natural frequency, reflecting practical usage; while mean accuracy demands balanced generalization. Instead of forcing a premature trade-off, we advocate for base principles that remain universally robust and beneficial across diverse priors. We cast pruning as a quadrature approximation on population risk and decompose the error bound into *representation error* (fidelity to the underlying manifold) and *prior-mismatch bias* (distribution shift), clarifying what can be improved jointly across priors. To address prior-mismatch bias, we decouple likelihood from prior in the posterior and transfer the structural likelihood via distillation with a calibrated teacher and geometry-preserving constraints. Simultaneously, to reduce representation error, we audit common pruning signals and choose geometric embedding, which exhibits greater robustness given the high inductive bias of 3D models. We also prioritize a safety floor before selection, capturing high-reward regions beneficial across priors. Finally, acknowledging that no single subset optimally satisfies divergent evaluation priors, we augment these principles with a steering wrapper that interpolates between stratified seeding and global selection. Empirical results demonstrate that our framework elevates the performance floor while offering flexibility for different prior preferences.

Deep Learning · Self-Supervised Learning

Moritz Gögl, Christopher Yau

The Joint-Embedding Predictive Architecture (JEPA) is often seen as a non-generative alternative to likelihood-based self-supervised learning, emphasizing prediction in representation space rather than reconstruction in observation space. We argue that the resulting separation from probabilistic generative modeling is largely rhetorical rather than structural: the canonical JEPA design–coupled encoders with a context-to-target predictor–mirrors the variational posteriors and learned conditional priors obtained when variational inference is applied to a particular class of coupled latent-variable models, and standard JEPA can be viewed as a deterministic specialization in which regularization is imposed via architectural and training heuristics rather than an explicit likelihood. Building on this view, we derive the Variational JEPA (Var-JEPA), which makes the latent generative structure explicit by optimizing a single Evidence Lower Bound (ELBO). This yields meaningful representations without ad-hoc anti-collapse regularizers and allows principled uncertainty quantification in the latent space. We instantiate the framework for tabular data (Var-T-JEPA) and achieve strong representation learning and downstream performance, consistently improving over T-JEPA while remaining competitive with strong raw-feature baselines.

General Machine Learning · Unsupervised and Semi-supervised Learning

Kunling Li, Cunqing Hua, Hongyu Zhu, Tianjie Ju, Pengwenlong Gu

Radio Frequency Fingerprint Identification (RFFI) is a foundational pillar of physical-layer security, providing unclonable identity authentication and lightweight defense mechanisms for zero-trust wireless networks. Its practical deployment, however, often occurs in a source-free open-world (SF-OW) setting, characterized by a continuous influx of unregistered devices and privacy constraints that preclude the retention of historical data. In this paper, we formalize SF-OW RFFI task, which manifests a severe stability-plasticity dilemma: intrinsic signal similarity confuses new classes, while source absence precipitates catastrophic forgetting. To address this, we propose Incremental Orthogonal ETF (IO-ETF), a novel neural collapse-inspired framework utilizing output geometry to actively induce parameter separation and isolation. We further devise a Triple-Level Geometric Alignment (TLGA) strategy via semantic optimal transport, manifold progressive anchoring, and reliable subspace retention to stably align unlabeled streams to this geometric skeleton. Experiments on benchmarks demonstrate a superior trade-off between old-class retention and new-class discovery, offering a promising solution for secure access in dynamic networks.

Deep Learning · Everything Else

Van-Tuan Tran, Hong-Hanh Nguyen-Le, Merim Dzaferagic, Marco Ruffini

Heterogeneous LoRA-rank methods address system heterogeneity in federated fine-tuning of foundation models by assigning client-specific ranks based on computational capabilities. However, these methods achieve only marginal computational savings, as dense feed-forward computations dominate. Sparse Mixture-of-Experts (SMoE) provides a promising alternative through conditional computation, yet we identify that its naive application to heterogeneous federated settings introduces two critical discordances: (i) expert utilization imbalance and (ii) non-differentiability of Top-K routing. Our convergence analysis demonstrates that these discordances lead to degraded convergence, particularly for resource-constrained clients. To address these challenges, we propose Universally Balanced Sparse Mixture-of-Experts (UB-SMoE), which introduces Dynamic Modulated Routing (DMR) to rebalance expert utilization, and Universal Pseudo-Gradient (PG) to reconstruct learning signals for non-activated experts. These mechanisms form a self-reinforcing cycle that maintains expert viability across heterogeneous clients. Experiments on benchmarks show that UB-SMoE achieves up to $45.0\%$ computational reduction on low-resource clients while improving their performance by $8.7 \times$ compared to existing heterogeneous LoRA-rank methods.

Applications · Robotics

Zirui Ge, Pengxiang Ding, Yemin Wang, BaoHuaYin, Qishen Wang, Zhiyong Xie, Hengtao Li, Runze Suo, Wenxuan Song, Han Zhao 等

Video action models are a promising foundation for Vision–Language–Action (VLA) because they can learn rich visual dynamics directly from video. However, likelihood-oriented training of diffusion predictors emphasizes globally plausible futures and does not guarantee precision-critical visual dynamics needed for manipulation, so small prediction errors can be amplified by downstream policies. We propose Dyn-VPP, a post-training framework that casts multi-step denoising as policy optimization and aligns predicted future latents with expert visual dynamics via verifiable terminal reward, without modifying any architecture. This enables explicit optimization of dynamics signals that are not captured by likelihood-only training. As a result, Dyn-VPP yields more accurate visual dynamics and improves downstream task execution. Experiments across diverse simulated and real-world manipulation settings show improved dynamics consistency and consistently higher task success.

Applications · Computer Vision

Jiawei Yu, Zijian Gao, Tianjiao Wan, Xuan Liu, Cheng Yang, Kele Xu

Generalized Category Discovery (GCD) requires models to categorize an unlabeled pool containing both known and novel classes under sparse supervision. We identify a systemic confidence bias inherent in existing parametric methods: while entropy regularization prevents class collapse, it indiscriminately suppresses predictive certainty on all unlabeled instances. This bias drives a distributional wedge between labeled and unlabeled samples of the same category, forcing models to sacrifice their stability on known classes to achieve plasticity for new ones. To resolve this, we propose Reliable Confidence Alignment (RCA), a plug-and-play framework grounded in Evidential Deep Learning. RCA first establishes high certainty anchors on labeled data using a Reliable Anchor for Certainty (RAC) module. Then, we introduce Cross-view Confidence Alignment (CCA) to propagate this grounded reliability to the unlabeled discovery set. Thus, RCA captures the fine-grained geometry of the probability simplex, effectively calibrating the model's epistemic uncertainty. Extensive evaluations on coarse- and fine-grained benchmarks demonstrate that RCA effectively rectifies the confidence landscape, significantly mitigating performance decay on known classes without compromising novel-class discovery.

Deep Learning · Algorithms

Jose Marie Antonio Miñoza

Spiking Neural Networks (SNNs) offer energy-efficient, biologically plausible computation but suffer from non-differentiable spike generation, necessitating reliance on heuristic surrogate gradients. This paper introduces **UltraLIF**, a principled framework that replaces surrogate gradients with *ultradiscretization*, a mathematical formalism from tropical geometry providing continuous relaxations of discrete dynamics. The central insight is that the max-plus semiring underlying ultradiscretization naturally models neural threshold dynamics: the log-sum-exp function serves as a differentiable soft-maximum that converges to hard thresholding as a learnable temperature parameter $\eps \to 0$. Two neuron models are derived from distinct dynamical systems: UltraLIF from the LIF ordinary differential equation (temporal dynamics) and UltraDLIF from the diffusion equation modeling gap junction coupling across neuronal populations (spatial dynamics). Both yield fully differentiable SNNs trainable via standard backpropagation with no forward-backward mismatch. Theoretical analysis establishes pointwise convergence to classical LIF dynamics with quantitative error bounds and bounded non-vanishing gradients. Experiments on six benchmarks spanning static images, neuromorphic vision, and audio demonstrate improvements over surrogate gradient baselines, with gains most pronounced in single-timestep ($T{=}1$) settings on neuromorphic and temporal datasets. An optional sparsity penalty enables significant energy reduction while maintaining competitive accuracy.