论文检索

James Blossom Eleojo

This paper presents an AI-driven framework for real-time reverberation control in dynamic environments. The system integrates parametric modeling in Grasshopper, Pachyderm acoustic simulation, and machine learning to create a closed-loop controller. A CNN estimates reverberation time from audio signals, while a reinforcement learning agent dynamically adjusts panel absorption coefficients to maintain optimal acoustics. Evaluation showed the system should be able to maintain T60 within 0.15 s of the target under varying occupancy and source positions, outperforming static treatments and enabling self-regulating acoustic environments for improved auditory experiences.

Le Xu, Jiayu Chen

Offline reinforcement learning (RL) is vulnerable to real-world data corruption, with even robust algorithms failing under challenging observation and mixture corruptions. We posit this failure stems from data corruption creating sharp minima in the loss landscape, leading to poor generalization. To address this, we are the first to apply Sharpness-Aware Minimization (SAM) as a general-purpose, plug-and-play optimizer for offline RL. SAM seeks flatter minima, guiding models to more robust parameter regions. We integrate SAM into strong baselines for data corruption: IQL, a top-performing offline RL algorithm in this setting, and RIQL, an algorithm designed specifically for data-corruption robustness. We evaluate them on D4RL benchmarks with both random and adversarial corruption. Our SAM-enhanced methods consistently and significantly outperform the original baselines. Visualizations of the reward surface confirm that SAM finds smoother solutions, providing strong evidence for its effectiveness in improving the robustness of offline RL agents.

Shota Takayama, Katsuhide Fujita

Although centralized training with centralized execution (CTCE) excels at multi-agent coordination, its reliance on global information limits its use in the real world. Conversely, the practical decentralized execution (CTDE) paradigm often struggles with complex coordination. This paper bridges this critical gap by introducing the Centralized-to-Decentralized (CtoD) learning concept: a novel framework for transferring the knowledge of a powerful centralized policy into a robust, practical decentralized policy. Our method, CtoD-MAT, realizes this transition through a curriculum that gradually shifts agents from centralized to decentralized control. A key innovation is our dynamic scheduling mechanism, featuring a mediator module, which ensures a robust and effective knowledge transfer. Using challenging SMAC benchmarks, we demonstrate that CtoD-MAT successfully produces competitive decentralized policies, notably solving complex coordination tasks that are difficult for standard CTDE methods.

Beomdo Park, Hyeonseok Jang, Junseong Park, Minu Baek, Gihun Gil, Minsung Jung, Woohyeon Kwon, Harin Jang, Yeojin Jang, Hoon Jeong 等

Quantum batteries have emerged as a next-generation energy storage solution, leveraging quantum phenomena such as superabsorption to overcome the limitations of conventional energy technologies. However, noise arising from interactions with the external environment degrades the charging efficiency and stability of the battery by disrupting the system's quantum coherence. To address this challenge, this study proposes a robust charging framework for a single-qubit quantum battery based on the Jaynes-Cummings (JC) model. The proposed framework combines the Proximal Policy Optimization (PPO) algorithm with a multi-stage reinforcement learning structure. The agent first learns fundamental control principles in a noise-free, ideal environment and subsequently performs robust learning in progressively noisier and more complex settings. Simulation results demonstrate that the trained agent navigates a stable charging trajectory on the Bloch sphere, thereby achieving high ergotropy even in the presence of noise. These findings suggest that multi-stage reinforcement learning is an effective solution for control problems in noisy quantum systems and provides a theoretical foundation for designing charging protocols for multi-qubit systems.

Narjes Nourzad, Carlee Joe-Wong

In environments with sparse or delayed rewards, reinforcement learning (RL) incurs high sample complexity due to the large number of interactions needed for learning. This limitation has motivated the use of large language models (LLMs) for subgoal discovery and trajectory guidance. While LLMs can support exploration, frequent reliance on LLM calls raises concerns about scalability and reliability. We address these challenges by constructing a memory graph that encodes subgoals and trajectories from both LLM guidance and the agent’s own successful rollouts. From this graph, we derive a utility function that evaluates how closely the agent’s trajectories align with prior successful strategies. This utility shapes the advantage function, providing the critic with additional guidance without altering the reward. Our method relies primarily on offline input and only occasional online queries, avoiding dependence on continuous LLM supervision. Preliminary experiments in benchmark environments show improved sample efficiency and faster early learning compared to baseline RL methods, with final returns comparable to methods that require frequent LLM interaction.

Xiaotao Liu, Prashant Mohit, Arvind Easwaran

In safe reinforcement learning (SRL), there exists an inherent conflict between maximizing reward and minimizing cost. We propose a novel approach that effectively resolve the conflict between maximizing reward and minimizing cost in joint optimization.When the cost exceeds the threshold, we perform cost-reducing updates. Otherwise, we compute policy gradients that maximize expected rewards, while using second-order Taylor approximation to evaluate whether these reward-maximizing gradients would violate the cost constraint. If constraint violation is detected, we adjust the gradient direction to maintain safety compliance; otherwise, we execute standard reward-increasing policy updates. This approach helps ensure that reward-seeking updates do not inadvertently increase costs, thereby reducing the likelihood of constraint violations. Empirical tests show our framework successfully manages reward-cost trade-offs through reward augmentation and cost shaping, improving both performance and safety without switching optimization strategies. Results demonstrate that concurrent treatment of both objectives in one policy gradient update is viable for improving safe reinforcement learning methods.

Pengfei Liu, Rui Kong, Zongzhang Zhang

Hand-craft reward engineering requires domain knowledge with numerous trials and errors, while Preference-based Reinforcement Learning (PbRL) avoids manual reward design but often suffers from limited interpretability and unstable training. To address these issues, we propose a novel preference alignment framework. Our approach leverages large language models to generate sub-reward functions informed by prior knowledge and further align human preferences by optimizing the weights combining these sub-rewards. For policy learning, we introduce Policy Optimization via Pareto Regularization (POPR) which regularizes updates along Pareto-optimal directions. Experiments show that our framework improves reward quality and policy stability, achieving superior performance to expert-designed rewards across most tasks.

Lianghui Liu, Zongzhang Zhang

Offline Meta-Reinforcement Learning (OMRL) leverages pre-collected data to adapt to new tasks. Context-based methods learn task representations from contexts. However, the context is influenced by both the task and the behavior policy. The mismatch between the behavior policy and the testing policy causes a context distribution shift problem, which results in poor task representations and degraded performance. This problem is exacerbated in settings with data limitations. To address this, we propose a novel approach called Meta-Normalizing Flow (Meta-NF). First, it employs a highly expressive and sample-efficient normalizing flow policy. Second, it incorporates a metric for testing-time task representation selection to effectively mitigate the context shift problem. Empirical results demonstrate that Meta-NF outperforms existing OMRL methods, with both components contributing to its strong performance.

Tae Hoon Lee, Joongheon Kim

In adaptive cruise control (ACC), balancing safety, comfort, and sustainability still remains challenging. Accordingly, we propose a hybrid reinforcement learning framework combining proximal policy optimization (PPO) and deep Q-network (DQN) with a multi-objective reward for autonomous carbon-neutral eco-driving. Experimental results revealed the contrasts between eco and non-eco modes, underscoring how reward design shapes driving behaviors.

Soumyajit Dev, Malay Bhattacharyya

Dismantling interdependent directed networks to obtain the largest mutually strongly connected component (MSCC) is an NP-hard problem. To address this, we propose a novel method, Disassembling Directed Interdependent Networks (DDIN), by synergizing Reinforcement Learning (RL) and Graph Neural Networks (GNN). We introduce asymmetric GNNs to capture the asymmetry of in/out-degree and multi-relational attention to model directed inter-layer dependencies, integrated with prioritized RL for efficient node selection in large action spaces. Our contributions include (i) a directed GraphSAGE encoder separating in/out aggregations for asymmetry, (ii) multi-relational attention fusing layer semantics, and (iii) sum-tree prioritized n-step Deep Q-Network (DQN) for efficient policy search. DDIN is evaluated on 5 directed multiplexes from biological, social, and economic domains, achieving 16-23% lower AUDC compared to known baseline heuristics.

Jungho An, Taeyoung Kim, Haeun Kim, Dongsoo Har

Offline Zero-Shot Reinforcement Learning requires an agent to solve unseen tasks using only a fixed offline dataset without explicit rewards. A central challenge is learning representations that capture both high-level long-term planning and low-level physical dynamics. We propose a novel framework, Dynamics-Aware Planning Representation (DAPR), which disentangles these two aspects via complementary contrastive objectives. Specifically, DAPR learns goal-oriented planning directions and local dynamics-consistent directions in the latent space. By jointly enforcing these constraints, DAPR yields representations that balance “where to go” with “how to move.” Experiments on standard locomotion benchmarks (Walker, Cheetah, Quadruped) demonstrate that DAPR consistently improves performance and generalization over strong baselines, achieving substantial gains on precision demanding tasks.

Tanmay Ambadkar, Darshan Chudiwal, Greg Anderson, Abhinav Verma

Ensuring safety in deep reinforcement learning is challenging, as formal methods that provide strong guarantees often fail to scale to complex, high-dimensional systems. We introduce RAMPS, a scalable shielding framework that pairs a general-purpose, learned linear dynamics model with a robust, multi-step Control Barrier Function (CBF) for real-time safety interventions. Experiments show RAMPS significantly reduces safety violations in high-dimensional environments compared to state-of-the-art methods, without sacrificing task performance.

Zijian Zhao

Recently, deep Reinforcement Learning (RL) methods have been widely used in labor management within transportation gig markets, such as ride-hailing, food delivery, and express delivery. Compared to traditional rule-based and optimization-based methods, RL can capture more information about long-term uncertainty and environmental dynamics, leading to better and non-myopic strategies. However, deep learning methods have long been criticized for their low interpretability, raising concerns about algorithmic discrimination in gig markets. Currently, most works focus on this issue from the perspective of statistical analysis and surveys. However, the underlying reasons related to the algorithms remain unclear, as most companies do not disclose their algorithms. This lack of transparency can hinder governments from designing efficient management policies to address these problems. To fill this research gap, this thesis proposal aims to develop appropriate RL methods to mimic the labor management behavior of transportation gig platforms and to propose effective policies that protect the rights of gig workers.

Xinwei Song

Achieving globally desirable outcomes in networked multi-agent systems—such as high social welfare, stable allocations, and widespread cooperation—is a fundamental challenge in AI. This paper outlines a research agenda that explores two complementary pathways to this goal. The first is a top-down approach, where a central mechanism designer proposes rules to guide strategic agents towards theoretically optimal equilibria. The second is a bottom-up approach, where desirable farsighted policies, like cooperation in social dilemmas, emerge from the decentralized interactions of agents via multi-agent reinforcement learning. We argue that the integration of these paths constitutes a promising frontier for creating robust and adaptive multi-agent systems.

Minghong Geng

Multi-agent reinforcement learning enables sophisticated collaborative behaviors in autonomous systems, yet fundamental scalability barriers persist: existing methods struggle to coordinate large agent populations and face challenges with extended decision-making horizons. This research develops hierarchical approaches to scale up multi-agent learning systems through two complementary directions: structural scaling for coordinating increasing numbers of agents and temporal scaling for extending decision-making horizons. This paper presents four integrated contributions: a taxonomic survey establishing hierarchical architectures as the theoretical foundation for scalable multi-agent learning systems, a benchmark for long-horizon multi-objective multi-agent reinforcement learning, a framework integrating self-organizing neural networks with multiple reinforcement learning agents for hierarchical tri-level control, and a framework leveraging large language models for zero-shot multi-agent planning. Through comprehensive validation, this work demonstrates that hierarchical, heterogeneous, modular architectures provide unified, interpretable solutions to multi-agent scalability, bridging theoretical multi-agent reinforcement learning research with real-world deployment requirements.

Oliver Chang

Autonomous driving has shown significant progress in recent years. The combination of advanced sensors, ample data, and machine learning algorithms has led to the deployment of autonomous vehicles (AVs) in cities like Los Angeles, San Francisco, and Phoenix. While not all humans can drive perfectly, AVs should be able to plan, adapt, and react to environmental disturbances, including irrational human drivers. My research focuses on applying reinforcement learning (RL) techniques to validate AV-related cyber-physical systems (CPS) in realistic environments. I develop a custom RL environment that simulates highway driving scenarios with multiple vehicles. This environment includes a CPS model of adaptive cruise control (ACC), a lane-changing model (MOBIL), and an adversarial agent that learns to drive irrationally. My work extends interpretable RL techniques to continuous control tasks like autonomous driving.

Tanmay Ambadkar

While Reinforcement Learning (RL) has demonstrated remarkable success in solving complex sequential decision-making problems, its application in real-world, safety-critical systems is hindered by its reliance on carefully engineered reward functions. Designing effective rewards is notoriously challenging and can lead to unintended or unsafe behaviors, a phenomenon known as reward hacking. Specification-guided RL has emerged as a principled alternative, leveraging formal methods to directly encode high-level objectives, safety requirements, and behavioral constraints. However, the practical utility of this approach is often limited by coarse or under-specified logical formulas and the computational challenge of enforcing safety at scale. This thesis addresses these limitations by developing a unified framework for the automated refinement, scalable enforcement, and flexible adaptation of formal specifications in RL.

Hendrik Baier, Mark T. Keane, Sarath Sreedharan, Silvia Tulli, Abhinav Verma

In this paper, we highlight the field of explainable sequential decision making. We discuss how the problem of explaining sequential decisions gives rise to problems and challenges that are absent from scenarios that focus on explaining single-shot decision making. We provide a short survey of some of the more prominent subareas within explainable sequential decision-making and their unique focuses and blind spots. Here, we argue that we need to go beyond simply focusing on individual subareas like explainable planning, reinforcement learning, or robotics, and move towards studying and tackling the more general problem of explainable sequential decision-making. Such a holistic approach will not only allow us to identify previously ignored problems, but also provide us with the ability to transfer ideas and intuitions from one subarea of explainable sequential decision-making to another. We end the paper with a discussion on future directions and some of the most pressing open questions.

Wanlu Zhang, Jia Zhu, Xi Yang, Weijie Shi, Yue Cui, Xinle Dai, Jiewen Sun

Classroom exercises are imperative for reinforcing learning. However, in conventional instruction, students frequently lack timely and personalized feedback. To address this, we present TacpAgent(Teaching Agent for Classroom Practice),a generative LLM-based agent that delivers detailed, individualized guided feedback to promote self-reflection through a prompt-template framework.In the context of this study, Classroom Practice is defined as the set of structured classroom exercises used for formative assessment.TacpAgent takes as input the teacher-prepared exercises and reference answers, together with students’ submitted responses, confidence levels, and brief explanations, then leverages the DeepSeek LLM with designed prompt templates to generate guided feedback and recommend relevant textbook sections for targeted review. We conducted a three-month quasi-experimental study with two high school classes (N=87). The study compared TacpAgent-supported exercises with traditional paper-based exercises. The experimental group showed significantly higher quiz scores (F=18.516, p＜0.001) and improved emotional (p＜0.001) and behavioral engagement (p=0.003). In contrast, the control group demonstrated no significant changes. The results suggest that TacpAgent may enable scalable, personalized formative assessment in classroom settings and provide practical guidance for integrating generative AI into everyday teaching.

Julian Zhong-Nan Zhang, Yang Yu

The reliable deployment of reinforcement learning (RL) for real-world algorithmic trading is critically hindered by the ``simulation-to-reality gap.'' Standard industry backtesting on static historical data ignores market impact—the feedback loop where an agent's trades influence price dynamics—leading to strategies that are fragile and untrustworthy in live markets. To solve this significant problem, we present a novel and emerging application of AI: a framework for building an interactive, responsive market simulator. Our system first uses imitation learning (IL) to automatically train an ensemble of agents, each learning a distinct trading strategy from a different historical market regime (e.g., bull, bear). This creates a data-driven proxy for a diverse population of real-world traders. We then deploy an innovative Action Synthesis Network to synthesize the actions of this ensemble, generating a realistic, synthetic price trajectory that endogenously models the market's reaction to trades. This interactive environment is then used to train a final RL policy. We evaluate our system on NASDAQ-100 (QQQ) data, and the results demonstrate strong potential for deployment. The RL policy trained in our responsive simulator achieves significantly more robust performance, exhibiting superior downside protection during market downturns compared to various traditional baselines. This application provides a scalable and technically sound methodology for building more realistic training environments, presenting a clear path toward the development and eventual deployment of more resilient and effective algorithmic trading strategies.