论文检索

Reinforcement Learning · Batch/Offline

Tobias Schmähling, Matthias Burkhardt, Tobias Windisch

We propose a data augmentation method for offline reinforcement learning, motivated by active positioning problems. Particularly, our approach enables the training of off-policy models from a limited number of suboptimal trajectories. We introduce a trajectory-based augmentation technique that exploits task structure and the geometric relationship between rewards, value functions, and mathematical properties of logging policies. During data collection, our augmentation supports suboptimal logging policies, leading to higher data quality and improved offline reinforcement learning performance. We provide theoretical justification for these strategies and validate them empirically across positioning tasks of varying dimensionality and under partial observability.

Reinforcement Learning · Everything Else

Xincheng Shi, Changxiao Ma, Yuzhuo Ma, Yongfei Zhang, Rongye Shi

Person re-identification (ReID) models are sensitive to long-tail nuisances (e.g., rare viewpoints, occlusions, complex backgrounds), yet current generative augmentation is largely open-loop: prompts/conditions are sampled heuristically without verifying whether the synthesized samples improve ReID discriminability. We introduce ReasonAug, a closed-loop framework that learns an image-conditioned instruction policy for a frozen generator, turning augmentation into a sequential decision problem over instruction tokens. A Semantic Reasoning Agent (SRA) performs hierarchical planning from global semantics to identity-critical local cues, producing structured edit instructions whose utility is verified by downstream ReID feedback. To make closed-loop optimization reliable, we propose Metric-Aligned Gated Reward (MAGR), which converts metric-learning objectives into a dense reward while gating task shaping by identity preservation to prevent reward hacking, and Structure-Aware Entropy (SAE), which allocates exploration per token to lock identity-critical cues while diversifying nuisance factors. Experiments on Market-1501 and MSMT17 demonstrate state-of-the-art performance, confirming that closing the augmentation loop and learning what to generate yield more discriminative training data than open-loop alternatives.

Applications · Language, Speech and Dialog

Jinhao Duan, Ruichen Zhang, Gengwei Zhang, Huaizhi Qu, Jie Peng, Sijia Liu, Tianlong Chen

Though biological foundation models (Bio-FMs) have delivered strong performance across biomedical tasks, their robustness to small-but-real perturbations is underexplored. In this work, we ask: Are Bio-FMs robust for real-world use? What perturbations compromise their reliability? Our pilot study suggests that due to subtle biological data curation issues and common machine-learning (ML) processing choices, Bio-FMs suffer from two complementary perturbation sources: biologically plausible perturbations (capturing experimental corruptions and curation artifacts) and ML-induced transformations (capturing preprocessing, data augmentation, and embedding choices). Guided by this taxonomy, we design perturbation suites that mimic corruptions frequently encountered in biological experiments, and we systematically probe how transformations in the ML pipeline reshape model behavior. By conducting 2,128 experiments over 11 state-of-the-art Bio-FMs on 7 bio-tasks, we show that most Bio-FMs are vulnerable to both biological perturbations and ML transformations, revealing underappreciated robustness gaps that can directly translate into deployment risk. Interestingly, we find that subtle biological perturbations, which are often imperceptible to current measurement tools, can induce severe discrepancies in Bio-FM outputs and lead to critical failures. We also find that cryo-EM reconstruction models (e.g., CryoDRGN) exhibit a surprising level of robustness even under worst-case adversarial settings. Our study for the first time surfaces critical failure modes and provides a principled perspective for evaluating the robustness of Bio-FMs in real-world biological pipelines.

Deep Learning · Graph Neural Networks

Zhuo Xu, Lu Bai, Jincheng Li, Lixin Cui, Ming Li, Hangyuan Du, Yue Wang

Graph Neural Networks (GNNs) have achieved remarkable successes in graph analysis due to the Message-Passing (MP) mechanism, yet they struggle with heterophilic graphs where connected nodes often have distinct labels or dissimilar attributes. Graph Contrastive Learning (GCL) serves as a promising approach to extract the information beyond neighboring nodes, effectively mitigating the limitations of the MP mechanism in handling heterophilic graphs. Nevertheless, GCL faces two critical challenges when applied to heterophilic graphs, i.e., the potential distribution shift from data augmentation and the loss of robustness caused by high-frequency signals. To address these problems, we propose a novel model, namely the Spectral Fusion Contrastive Learning with Topology-Adaptive Graph Augmentation (SFCLTA) for unsupervised graph representation learning. Our method dynamically adjusts graph structures by a heterophily-aware augmentation strategy, and constrains high-frequency distortions by spectral regularization. We utilize the confidence-weighted fusion to enhance the robustness. Additionally, we introduce a feature reconstruction task as the prerequisites to explicitly mitigate feature-level distribution shifts. Experiments on multiple real-world datasets demonstrate that the proposed SFCLTA consistently outperforms baseline models in multiple tasks.

General Machine Learning · Transfer, Multitask and Meta-learning

Junpeng Ren, Carlos Misael Madrid Padilla, Yanzhen Chen, OSCAR HERNAN MADRID PADILLA

This paper develops a general transfer learning framework for nonparametric regression with heterogeneous data consisting of multiple groups. Under the assumption that groups share a common structure along with group-specific deviations in additive form, the proposed method employs a two-stage offset learning procedure: the first stage pools data from all groups to estimate an overall mean function, and the second stage estimates offsets for each group, yielding final group-level estimators through additive combination. Non-asymptotic upper bounds are established for the proposed framework, covering a broad class of nonparametric estimators under mild complexity and noise conditions. When instantiated with deep ReLU networks, explicit convergence rates are derived under hierarchical composition models, demonstrating the ability to overcome the curse of dimensionality. Conditions that enable positive transfer with faster rates are considered, including learning with simpler functions and data augmentation through pooling samples across groups. Various simulations and real-data experiments further validate the effectiveness of the proposed method.

Matina Mahdizadeh Sani, Nima Jamali, Mohammad Jalali, Farzan Farnia

Pre-trained diffusion models have emerged as powerful generative priors for both unconditional and conditional sample generation, yet their outputs often deviate from the characteristics of user-specific target data. Such mismatches are especially problematic in domain adaptation tasks, where only a few reference examples are available and retraining the diffusion model is infeasible. Existing inference-time guidance methods can adjust sampling trajectories, but they typically optimize surrogate objectives such as classifier likelihoods rather than directly aligning with the target distribution. We propose MMD Guidance, a training-free mechanism that augments the reverse diffusion process with gradients of the Maximum Mean Discrepancy (MMD) between generated samples and a reference dataset. MMD provides reliable distributional estimates from limited data, exhibits low variance in practice, and is efficiently differentiable, which makes it particularly well-suited for the guidance task. Our framework naturally extends to prompt-aware adaptation in conditional generation models via product kernels. Also, it can be applied with computational efficiency in latent diffusion models (LDMs), since guidance is applied in the latent space of the LDM. Experiments on synthetic and real-world benchmarks demonstrate that MMD Guidance can achieve distributional alignment while preserving sample fidelity.

Reinforcement Learning · Planning

Zuyuan Zhang, Zeyu Fang, Tian Lan

Geometric properties can be leveraged to stabilize and speed reinforcement learning. Existing examples include encoding symmetry structure, geometry-aware data augmentation, and enforcing structural restrictions. In this paper, we take a novel view of RL through the lens of order theory and recast value function estimates into learning a desired poset (partially ordered set). We propose \emph{GCR-RL} (Geometric Coherence Regularized Reinforcement Learning) that computes a sequence of super-poset refinements -- by refining posets in previous steps and learning additional order relationships from temporal difference signals -- thus ensuring geometric coherence across the sequence of posets underpinning the learned value functions. Two novel algorithms by Q-learning and by actor--critic are developed to efficiently realize these super-poset refinements. Their theoretical properties and convergence rates are analyzed. We empirically evaluate GCR-RL in a range of tasks and demonstrate significant improvements in sample efficiency and stable performance over strong baselines.

Applications · Health / Medicine

Silas Ruhrberg Estevez, Nicolas Huynh, Tennison Liu, Roderik Kortlever, Gerard Evan, David Bentley, Mihaela van der Schaar

Inferring dynamics from population snapshots is a fundamental challenge in machine learning and biology. In scRNA-sequencing (scRNA-seq), destructive measurements preclude direct tracking of individual cells across time, making trajectory inference underdetermined. Optimal Transport (OT) provides a principled framework for snapshot alignment, but a long-standing modeling question is which cost functions yield biologically meaningful couplings. Standard OT approaches rely on gene-expression distances, implicitly treating cells as independent points and neglecting structured cell-cell communication mediated by ligand--receptor signaling. We introduce $\texttt{CellBRIDGE}$ ($\textit{Cell-Based Regularized Interaction-Driven Gene Expression}$), which augments feature-based OT with a directed, typed interaction cost derived from ligand-receptor activity. By explicitly modeling cell--cell communication, $\texttt{CellBRIDGE}$ improves cross-snapshot couplings and downstream trajectory estimates across synthetic and real scRNA-seq datasets relative to feature-only baselines. Notably, $\texttt{CellBRIDGE}$ enables mechanistically interpretable in silico perturbations: on lung cancer data, silencing specific ligand-receptor pairs induces trajectory shifts that recapitulate expected effects of targeted pathway inhibition.

Deep Learning · Graph Neural Networks

Simon Forbat, Rainer Gemulla

Mixup is a widely used data augmentation technique that constructs new training examples by interpolating between existing ones. While simple and effective in domains like vision and language, applying mixup to graph data is non-trivial and there is no independent empirical evidence for its effectiveness. To fill this gap, we conducted an extensive evaluation study following a unified, established evaluation protocol for graph classification. In contrast to prior results, we found that none of the state-of-the-art mixup methods yielded statistically significant improvements over the no-mixup baseline. To obtain further insights, we analyzed the graphs generated from these mixup methods from an interpolation perspective. We found that (i) many mixup methods failed to interpolate well, (ii) high interpolation error led to performance degradation, and (iii) even good interpolation properties did not lead to performance improvements. Our findings question the efficacy of existing graph mixup methods and highlight the need for a more rigorous exploration and evaluation.

Diego Granziol, Ulugbek Abdimanabov

Backdoor and data-poisoning attacks can achieve high attack success while evading existing spectral and optimisation-based defences. We show that this behaviour is not incidental, but arises from a fundamental geometric mechanism in input space. Using kernel ridge regression as an exact model of wide neural networks, we prove that clustered dirty-label poisons induce a rank-one spike in the input Hessian whose magnitude scales quadratically with attack efficacy. Crucially, for nonlinear kernels we identify a near-clone regime in which poison efficacy remains order-one while the induced input curvature vanishes, making the attack provably spectrally undetectable. We further show that input-gradient regularisation contracts poison-aligned Fisher and Hessian eigenmodes under gradient flow, yielding an explicit and unavoidable safety–efficacy trade-off by reducing data-fitting capacity. For exponential kernels, this defence admits a precise interpretation as an anisotropic high-pass filter that increases the effective length scale and suppresses near-clone poisons. Extensive experiments on linear models and deep convolutional networks across MNIST and CIFAR-10/100 validate the theory, demonstrating consistent lags between attack success and spectral visibility, and showing that regularisation and data augmentation jointly suppress poisoning. Our results establish when backdoors are inherently invisible, and provide the first end-to-end characterisation of poisoning, detectability, and defence through input-space curvature.

Applications · Everything Else

chunyang yu, Ning SUN, Shenyue Wang

Gesture interaction and touchless sensing is a natural and intuitive control method that allows users to control devices through natural hand or body movements, reducing reliance on physical input and enhancing convenience and functional efficiency. The biggest challenge is balancing accuracy and model parameter count for custom gesture tasks, while minimizing the number of data samples required for training. This paper proposes a novel IMU-based BiCrossNet model and two novel data augmentation models (delta-generator and embedding-generator) to address this challenge. Compared with existing methods, the model proposed in this article boosts accuracy by 11.7% and 12.7% in UMAHand (public datasets); and 8.85% and 5.25% in GRHand (self-developed datasets), with decreasing 27.8% pretrained feature-extractor model parameters. This research lays a solid foundation for deploying and implementing custome gesture recognition engineering on intelligent terminal devices.

Reinforcement Learning · Deep RL

Yudong Luo, Erick Delage

Optimizing Conditional Value-at-risk (CVaR) using policy gradient (a.k.a CVaR-PG) faces significant challenges of sample inefficiency. This inefficiency stems from the fact that it focuses on tail-end performance and overlooks many sampled trajectories. We address this problem by augmenting CVaR with an expected quantile term. Quantile optimization admits a dynamic programming formulation that leverages all sampled data, thus improves sample efficiency. This does not alter the CVaR objective since CVaR corresponds to the expectation of quantile over the tail. Empirical results in domains with verifiable risk-averse behavior show that our algorithm within the Markovian policy class substantially improves upon CVaR-PG and consistently outperforms other existing methods.

Applications · Robotics

Chenyu Hui, Xiaodi Huang, Siyu Xu, Yunke Wang, Shan You, Fei Wang, Tao Huang, Chang Xu

Vision-language-action (VLA) models typically rely on large-scale real-world videos, whereas simulated data, despite being inexpensive and highly parallelizable to collect, often suffers from a substantial visual domain gap and limited environmental diversity, resulting in weak real-world generalization. We present an efficient video augmentation framework that converts simulated VLA videos into realistic training videos while preserving task semantics and action trajectories. Our pipeline extracts structured conditions from simulation via video semantic segmentation and video captioning, rewrites captions to diversify environments, and uses a conditional video transfer model to synthesize realistic videos. To make augmentation practical at scale, we introduce a diffusion feature-reuse mechanism that reuses video tokens across adjacent timesteps to accelerate generation, and a coreset sampling strategy that identifies a compact, non-redundant subset for augmentation under limited computation. Extensive experiments on RobotWin 2.0, LIBERO, LIBERO-Plus, and a real robotic platform demonstrate consistent improvements in both task performance and sim-to-real generalization. For example, our method improves RDT-1B by 8% on RobotWin 2.0, and boosts $\pi_0$ by 5.1% on the more challenging LIBERO-Plus benchmark. Code is released in supplementary material.

Applications · Health / Medicine

Marius Knorr, Robert Müller, Jan Bremer, Nils Schweingruber

Fast Healthcare Interoperability Resources (FHIR) is the dominant standard for interoperable exchange of healthcare data. In FHIR, electronic health records form a directed graph of resources. Answering clinically meaningful questions over FHIR requires agents to perform multi-step reasoning, filtering, and aggregation across multiple resource types. Prior work shows that even tool-augmented LLM agents (retrieval, code execution, multi-turn planning) often select the wrong resources or violate traversal constraints. We study this problem in the context of FHIR-AgentBench, a benchmark for realistic question answering over real-world hospital data, and frame reasoning on FHIR as a sequential decision-making problem over a queryable structured graph. We implement a multi-turn CodeAct agent and post-train it with reinforcement learning using a custom harness and tools. A LLM Judge provides execution-grounded rewards. Compared to prompt-based, closed-model baselines, RL post-training improves performance while enforcing data-integrity constraints. Empirically, our approach improves answer correctness from 50% (o4-mini) to 64% on FHIR-AgentBench using a smaller and cheaper Qwen 2.5-7B model. We present an end-to-end post-training pipeline (environment building, harness construction, model training and custom evaluation) that reliably improves multi-turn reasoning over structured clinical graphs.

Deep Learning · Large Language Models

Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj, Manik Bhandari, Mayur Srungarapu, Anqi Liu, Benjamin Van Durme, Li Chen

Aligning large language models (LLMs) to heterogeneous and rapidly evolving safety requirements remains a critical challenge. Existing instruction-tuned LLMs and standalone safety classifiers often fail to generalize to new safety configurations, motivating the need for Reward Models (RMs) that are explicitly configurable to changing specifications. We introduce the Configurable Safety Reward Model (CSRM), which is jointly optimized for calibrated safety compliance and reward modeling. Our approach is supported by configuration-targeted data augmentation that enforces instruction adherence while preserving relative severity structure. The resulting RM is sensitive to fine-grained safety configurations and conversational nuances, substantially improving generalization to previously unseen safety configurations. CSRM achieves state-of-the-art performance on recent configurable safety benchmarks, including CoSApien (94.6\% F1) and DynaBench (75.8\% F1), without requiring additional human annotation. When used for downstream safety alignment, CSRM yields LLMs with a significantly improved helpfulness–safety tradeoff compared to existing baselines.

Reinforcement Learning · Multi-agent

Xuhui Kang, Sung-Wook Lee, Haolin Liu, Yuyan Wang, Yen-Ling Kuo

The ability to adapt to physical actions and constraints in an environment is crucial for embodied agents (e.g., robots) to effectively collaborate with humans. Such physically grounded human-AI collaboration must account for the increased complexity of the continuous state-action space and constrained dynamics caused by physical constraints. However, most existing collaboration benchmarks are discrete or do not consider physical attributes and constraints. To address this, we introduce Moving Out, a human-AI collaboration benchmark that resembles a wide range of collaboration modes affected by physical attributes and constraints, such as moving heavy items together and coordinating actions to move an item around a corner. Moving Out consists of two challenges and human-human interaction data to comprehensively evaluate models' abilities to adapt to diverse human behaviors and unseen physical attributes. To give embodied agents the capability to collaborate with humans under physical attributes and constraints, we propose a novel method, BASS (Behavior Augmentation, Simulation, and Selection), to enhance the diversity of agents and their understanding of the outcome of actions. We systematically compare BASS and state-of-the-art models in AI-AI and human-AI experiments, showing that BASS can effectively collaborate with both unseen AI and humans.

Applications · Time Series

Zongjiang Shang, Chengxi Jin, Binqing Wu, Dongliang Cui, Yue Yu, Haobang Sun, Chuanlin Xu, Ling Chen

Time series forecasting plays a pivotal role in data-driven decision-making across various time series domains. Recently, leveraging their ability to extract semantically rich representations, Large Language Models (LLMs) have achieved promising results in time series forecasting. However, existing LLM-based methods struggle to obtain multi-scale retrieval-augmented representations due to entangled multi-scale representations and redundant multi-scale interference. To address this, we propose TimeMRA, an LLM-empowered Time series forecasting framework via Multi-Scale Retrieval-Augmented representations. Specifically, a scale-aware prompt generation (SAPG) module is designed to decompose time series into multiple scales and generate augmented multi-scale representations. Then, a cross-scale disentanglement constraint (CSDC) mechanism with a router network is designed to obtain the disentangled multi-scale semantic representations while mitigating interference from irrelevant scales. Finally, a cross-modality retrieval module is designed to obtain multi-scale retrieval-augmented representations for time series forecasting. Experiments on 10 real-world datasets demonstrate that TimeMRA achieves state-of-the-art (SOTA) performance.

Deep Learning · Graph Neural Networks

Jialiang Wang, Hanmo Liu, Shimin Di, Zhili Wang, Jiachuan Wang, Lei Chen, Xiaofang Zhou

Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency. Current methods fail to achieve this balance: neural architectural search is computationally expensive, while model retrieval often yields suboptimal static checkpoints. To resolve this dilemma, we model the performance gains induced by fine-grained architectural modifications as edit-effect evidence and build evidence graphs from prior tasks. By constructing a retrieval-augmented model refinement framework, our proposed M-DESIGN dynamically weaves historical evidence to discover near-optimal modification paths. M-DESIGN features an adaptive retrieval mechanism that quickly calibrates the evolving transferability of edit-effect evidence from different sources. To handle out-of-distribution shifts, we introduce predictive task planners that extrapolate gains from multi-hop evidence, thereby reducing reliance on an exhaustive repository. Based on our model knowledge base of 67,760 graph neural networks across 22 datasets, extensive experiments demonstrate that M-DESIGN consistently outperforms baselines, achieving the search-space best performance in 26 out of 33 cases under a strict budget. Code and data are available at: https://anonymous.4open.science/r/M-DESIGN-245/.

Deep Learning · Other Representation Learning

David Méndez, Roberto Confalonieri, Natalia Díaz-Rodríguez

Vision-Language Models (VLMs) excel at tasks like zero-shot classification and cross-modal retrieval by mapping images and text to a shared space, but this requires expensive end-to-end training with massive paired datasets. Current post-hoc alignment methods reduce computational costs by connecting pretrained encoders through lightweight mappings, yet still demand substantial paired data. In this work, we investigate the potential of repurposing the classification heads of pretrained vision models as semantic prototypes. The recycling of these weights, typically discarded after pretraining, unlocks two distinct capabilities: it enables zero-shot alignment by using weights as semantic anchors, and serves as a robust data augmentation strategy by mixing these prototypes with real image-text pairs. We demonstrate that integrating our approach with several state-of-the-art post-hoc alignment techniques consistently boosts accuracy in cross-modal retrieval, zero- and few-shot classification tasks.

Deep Learning · Generative Models and Autoencoders

Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Saining Xie, Jaehong Yoon, Sung Ju Hwang

Modern video generators still struggle with complex physical dynamics, often falling short of physical realism. Existing approaches address this using external verifiers or additional training on augmented data, which is computationally expensive and still limited in capturing fine-grained motion. In this work, we present self-refining video sampling, a simple method that uses a pre-trained video generator trained on large-scale datasets as its own self-refiner. By interpreting the generator as a denoising autoencoder, we enable iterative inner-loop refinement at inference time without any external verifier or additional training. We further introduce an uncertainty-aware refinement strategy that selectively refines regions based on self-consistency, which prevents artifacts caused by over-refinement. Experiments on state-of-the-art video generators demonstrate significant improvements in motion coherence and physics alignment, achieving over 70% human preference compared to the default sampler and guidance-based sampler.