论文检索

Tadeusz Dziarmaga, Tomasz Arczewski, Marcin Mazur, Maciej Wołczyk

Continual reinforcement learning (CRL) is the study of optimal strategies for maximizing rewards in sequential environments that change over time. This is particularly crucial in domains such as robotics, where the operational environment is inherently dynamic and subject to continual change. Nevertheless, research in this area has thus far concentrated on off-policy algorithms with replay buffers that are capable of amortizing the impact of distribution shifts. Such an approach is not feasible with on-policy reinforcement learning algorithms that learn solely from the data obtained from the current policy. In this paper, we examine the performance of proximal policy optimization (PPO), a prevalent on-policy reinforcement learning (RL) algorithm, in a classical CRL benchmark. Our findings suggest that the current methods are suboptimal in terms of average performance. Nevertheless, they demonstrate encouraging competitive outcomes with respect to forward transfer and forgetting metrics. This highlights the need for further research into continual on-policy reinforcement learning. The source code is available at https://github.com/Teddy298/continualworld-ppo.

Haimin Hu

Autonomous robots are becoming more versatile and widespread in our daily lives. From autonomous vehicles to companion robots for senior care, these human-centric systems must demonstrate a high degree of reliability in order to build trust and, ultimately, deliver social value. How safe is safe enough for robots to be wholeheartedly trusted by society? Is it sufficient if an autonomous vehicle can avoid hitting a fallen cyclist 99.9% of the time? What if this rate can only be achieved by the vehicle always stopping and waiting for the human to move out of the way? I argue that, for trustworthy deployment of robots in human-populated space, we need to complement standard statistical methods with clear-cut robust safety assurances under a vetted set of operation conditions. We need runtime learning to minimize the robot’s performance loss during safety-enforcing maneuvers by reducing its inherent uncertainty induced by its human peers, for example, their intent (does a human driver want to merge, cut behind, or stay in the lane?) or response (if the robot comes closer, how will the human react?). We need to close the loop between the robot’s learning and decision-making so that it can optimize efficiency by anticipating how its ongoing interaction with the human may affect the evolving uncertainty, and ultimately, its long-term performance.

Amar Halilovic

As the use of autonomous mobile robots expands into dynamic and complex environments, the need for them to provide understandable explanations for their actions becomes crucial. This thesis addresses the challenge of developing explainability for robot navigation by leveraging a hybrid model that combines machine learning techniques with symbolic reasoning methods. Furthermore, the thesis explores the modeling of human explanation preferences and the impact of different explanation attributes on explanation recipients' understanding, satisfaction, and trust. The goal is to integrate different explanation aspects and approaches into a unified framework to support explainable navigation in robotics.

Kate Candon

My work aims to enable robots to better learn from human feedback in human-robot interactions. The way in which people want to collaborate with a robot can vary person-to-person, interaction-to-interaction, or even within an interaction with a given person. Thus, robots need to be able to adapt their behavior during interactions. Robots typically learn from humans via explicit feedback, such as evaluative feedback, preferences, or demonstrations. We know that humans also provide additional information implicitly through non-verbal behavior that gives clues about their internal states during interactions. My work investigates how we can incorporate both kinds of feedback into robot learning paradigms.

Anssi Lin, Anssi Salonen, Nicolas Pope, Henriikka Vartiainen, Matti Tedre

In the fast-growing field of K–12 AI education, there is an urgent need for accessible, hands-on tools that introduce AI concepts and workflows to novice learners. In recent years, a variety of AI education tools have been introduced, ranging from coding environments to physical kits and robots. To provide an alternative to existing AI education tools, this paper presents a low-cost robotics kit (

Tanushree Burman, Milan Dahal, Geling Xu, Chris Rogers, Jennifer Cross, Jivko Sinapov

With the rise of Artificial Intelligence (AI) systems in society, our children have routine interactions with these technologies. It has become increasingly important for them to understand how these technologies are trained, what their limitations are and how they work. To introduce children to AI and Machine Learning (ML) concepts, recent efforts introduce tools that integrate ML concepts with physical computing and robotics. However, some of these tools cannot be easily integrated into building projects and the high price of robotics kits can be a limiting factor to many schools. We address these limitations by offering a low-cost hardware and software toolkit that we call the Smart Motor to introduce supervised machine learning to elementary school students. Our Smart Motor uses the nearest neighbor algorithm and utilizes visualizations to highlight the underlying decision-making of the model. We conducted a one week long study using Smart Motors with 9- to 12- year old students and measured their learning through observation, questioning and examining what they built. We found that students were able to integrate the Smart Motors into their building projects but some students struggled with understanding how the underlying model functioned. In this paper we discuss these findings and insights for future directions for the Smart Motor.

Marie Absalon, Thomas Deneux

Given the massive transformation of all areas of society by AI, it is becoming essential to integrate AI literacy into the various school curricula from an early age. However, teaching the basic concepts of AI and Machine Learning (e.g. training a model; artificial neural networks) at the K-12 level might seem too abstract, whereas teaching only how to use AI fails to really "open the black box". To overcome these difficulties, we have developed AlphAI, a software resource designed to make the understanding of AI algorithms accessible and attractive to the general public and children as young as 8 years old. This is achieved by making AI very concrete, first by manipulating the learning of educational robots that users train for different behaviors, such as circuit racing, using either supervised or reinforcement learning; second by visualizing in real time in a graphical interface the details of AI algorithms (neural networks, k-nearest neighbors, Q-learning, etc). In addition, the use of the software is not limited to beginners, since it allows to write one's own AI in Python to control the robots. In this paper, we present the basic principles of the software, its graphical interface, how to use it with various educational robots, and example activities with classes from Elementary school to University. AlphAI software and robotic kits are commercially available from Learning Robots.

Rushang Karia, Jayesh Nagpal, Daksh Dobhal, Pulkit Verma, Rashmeet Kaur Nayyar, Naman Shah, Siddharth Srivastava

Understanding how robots plan and execute tasks is crucial in today's world, where they are becoming more prevalent in our daily lives. However, teaching non-experts, such as K-12 students, the complexities of robot planning can be challenging. This work presents an open-source platform, JEDAI.Ed, that simplifies the process using a visual interface that abstracts the details of various planning processes that robots use for performing complex mobile manipulation tasks. Using principles developed in the field of explainable AI, this intuitive platform enables students to use a high-level intuitive instruction set to perform complex tasks, visualize them on an in-built simulator, and to obtain helpful hints and natural language explanations for errors. Finally, JEDAI.Ed, includes an adaptive curriculum generation method that provides students with customized learning ramps. This platform's efficacy was tested through a user study with university students who had little to no computer science background. Our results show that JEDAI.Ed is highly effective in increasing student engagement, teaching robotics programming, and decreasing the time need to solve tasks as compared to baselines.

Kaiqing Zhang

Recent years have witnessed tremendous successes of learning for sequential decision-making, and in particular, Reinforcement Learning (RL). Prominent application examples include playing Go and video games, robotics, autonomous driving, and recently large language models. Most such success stories naturally involve "multi-agents". Hence, there has been surging research interest in advancing Multi-Agent Learning in Dynamic Environments, particularly, multi-agent RL (MARL), to which my research has led and made significant contributions. My work has established both sample and computational complexities of learning in Stochastic Games, the most fundamental model of MARL, and advocated a unique Economics perspective of independent learning in Stochastic Games. My work has also initiated the recent studies of distributed and networked MARL, with applications in robust adversarial RL, offline RL, and Robotics. This paper will survey my notable contributions along this journey of developing the foundations of multi-agent learning in dynamic environments.

Reuth Mirsky

A prevalent assumption in human-robot and human-AI teaming is that artificial teammates should be compliant and obedient. In this talk, I will question this assumption by presenting the Guide Robot Grand Challenge and discussing the components required to design and build a service robot that can intelligently disobey. This challenge encompasses a variety of research problems, as I will exemplify via three challenges: reasoning about the goals of other agents, choosing when to interrupt, and interacting in a tightly coupled physical environment.

Roberto Martín-Martín

Artificial Intelligence (AI) has revolutionized fields like computer vision and natural language processing, yet its impact on robotics remains limited by challenges in long-horizon decision-making and complex physical interactions. My research pioneers robot learning algorithms that exploit (predict, perceive, plan, and reason about) physical interaction as a core component of artificial intelligence, pushing beyond passive solutions in domains such as perception, navigation, and manipulation. By leveraging techniques in imitation learning and hierarchical reinforcement learning, my work empowers robots to learn from human demonstrations, navigate interactively in real-world environments, and gather information through purposeful interactions. In my talk, I will explain how these advances are critical for robots to become useful helpers in human environments, opening the door to the next generation of household robots. I will present several AI algorithmic innovations to integrate physical interactions in computation procedures and outline the path toward developing continually learning robots capable of operating autonomously in unstructured human environments, enhancing their utility as adaptable and intelligent assistants.

Yunzhu Li

Humans have a strong intuitive understanding of the physical world. Through observations and interactions with the environment, we build mental models that predict how the world would change if we applied a specific action (i.e., intuitive physics). My research draws on these human insights to develop model-based RL agents that learn from their interactions and build predictive models that generalize widely across a range of objects made with different materials. The core idea behind my research is to introduce novel representations and integrate structural priors into learning systems to model dynamics at different levels of abstraction. I will discuss how such structures can make model-based planning algorithms more effective, helping robots accomplish complex manipulation tasks (e.g., manipulating an object pile, shaping deformable foam into a target configuration, and making a dumpling from dough using various tools).

Ruohan Gao

The future of Artificial Intelligence demands a paradigm shift towards multisensory perception—to systems that can digest ongoing multisensory observations, that can discover structure in unlabeled raw sensory data, and that can intelligently fuse useful information from different sensory modalities for decision making. While we humans perceive the world by looking, listening, touching, smelling, and tasting, traditional form of machine intelligence mostly focuses on a single sensory modality, particularly vision. Therefore, my research, which I call multisensory machine intelligence, aims to empower machines to emulate and enhance human capabilities in seeing, hearing, and feeling, ultimately enabling them to comprehensively perceive, understand, and interact with the multisensory world. In my AAAI-25 new faculty highlight talk, I will present my research that studies two important aspects of the multisensory world: 1) multisensory objects, and 2) multisensory space. In both aspects, I will talk about how we design systems to reliably capture multisensory data from real-world objects and space, how we effectively model them with differentiable simulation algorithms that build a unified multisensory representation to virtualize real objects, and how we explore creative cross-modal/multi-modal applications with sight, sound, and touch in vision, graphics, and robotics. In the end, I will briefly conclude with my future plans.

Daniel S. Brown

Ensuring that AI systems do what we, as humans, actually want them to do, is one of the biggest open research challenges in AI alignment and safety. My research seeks to directly address this challenge by enabling AI systems to interact with humans to learn aligned and robust behaviors. The way in which robots and other AI systems behave is often the result of optimizing a reward function. However, manually designing good reward functions is highly challenging and error prone, even for domain experts. Consider trying to write down a reward function that describes good driving behavior or how you like your bed made in the morning. While reward functions for these tasks are difficult to manually specify, human feedback in the form of demonstrations or preferences are often much easier to obtain. However, human data is often difficult to interpret, due to ambiguity and noise. Thus, it is critical that AI systems take into account epistemic uncertainty over the human's true intent. My talk will give an overview of my lab's progress along the following fundamental research areas: (1) efficiently maintaining uncertainty over human intent, (2) directly optimizing behavior to be robust to uncertainty over human intent, and (3) actively querying for additional human input to reduce uncertainty over human intent.

Erdem Bıyık

In many human-robot collaboration and multi-agent tasks, it is vital to model the partners and estimate their objectives to efficiently collaborate/interact with them. While learning from demonstrations is the most common approach for this, it is very data-hungry, which we cannot afford in many settings including robotics, and demonstrations are unreliable in a surprisingly large number of domains, including those we think humans perform reasonably well, e.g., driving. In this talk, I will start with introducing comparison-based feedback and explain why it does not suffer from most of the problems that demonstrations have, but is still data-hungry. To address this problem, I will propose comparative language based feedback and active learning techniques, which will result in (1) a new type of human feedback, and (2) an active querying algorithm that optimizes the information the AI agent will elicit from the human. I will conclude the talk by discussing what other types of human feedback exist, e.g., interventions or hand gestures, and how we can incorporate them into the existing learning algorithms.

Chen Tang, Ben Abbatematteo, Jiaheng Hu, Rohan Chandra, Roberto Martín-Martín, Peter Stone

Reinforcement learning (RL), particularly its combination with deep neural networks referred to as deep RL (DRL), has shown tremendous promise across a wide range of applications, suggesting its potential for enabling the development of sophisticated robotic behaviors. Robotics problems, however, pose fundamental difficulties for the application of RL, stemming from the complexity and cost of interacting with the physical world. These challenges notwithstanding, recent advances have enabled DRL to succeed at some real-world robotic tasks. However, state-of-the-art DRL solutions’ maturity varies significantly across robotic applications. In this talk, I will review the current progress of DRL in real-world robotic applications based on our recent survey paper (with Tang, Abbatematteo, Hu, Chandra, and Martı́n-Martı́n), with a particular focus on evaluating the real-world successes achieved with DRL in realizing several key robotic competencies, including locomotion, navigation, stationary manipulation, mobile manipulation, human-robot interaction, and multi-robot interaction. The analysis aims to identify the key factors underlying those exciting successes, reveal underexplored areas, and provide an overall characterization of the status of DRL in robotics. I will also highlight several important avenues for future work, emphasizing the need for stable and sample-efficient real-world RL paradigms, holistic approaches for discovering and integrating various competencies to tackle complex long-horizon, open-world tasks, and principled development and evaluation procedures. The talk is designed to offer insights for RL practitioners and roboticists toward harnessing RL’s power to create generally capable real-world robotic systems.

Satyandra K. Gupta

Many manufacturing companies are facing an acute shortage of qualified workers. Deploying robotic cells is a potential solution to address this challenge. Historically robots have been deployed only in mass production applications in manufacturing. A large fraction of manufacturing is classified as high-mix manufacturing where a large variety of products are produced. Manually programming robots is not a viable solution in high-mix manufacturing applications. Robotic cells need to be powered by embodied AI to make them useful in high-mix manufacturing applications. This paper aims to build a bridge between smart manufacturing and AI communities to enable AI researchers to develop methods and tool that can be successfully deployed to realize smart robotic cells for high-mix manufacturing applications. This paper highlights key requirements for developing embodied AI for powering robotic cells for high-mix manufacturing applications. It also makes the case for approaches that combine model-based and data-driven methods to meet the needs of embodied AI in manufacturing applications and describes the role of generative AI approaches in smart manufacturing applications. Finally, it describes how AI can be used to enhance digital twins and augment human-machine interfaces in manufacturing applications.

Mohan Sridharan

Robots are increasingly being used in different application domains due to rapid advancements in hardware and computational methods. However, state of the art methods for many problems in robotics are based on deep networks and similar data-driven models. These methods and models are resource-hungry and opaque, and they are known to provide arbitrary decisions in previously unseen situations, whereas practical robot application domains require transparent, multi-step, multi-level decision-making and ad hoc collaboration under resource constraints and open world uncertainty. In this talk, I argue that for widespread use of robots, we need to revisit principles such as refinement and adaptive satisficing, which can be traced back to the early pioneers of AI. We also need to make these principles the foundation of the architectures we develop for robots, with modern data-driven methods being just another tool in our toolbox. I then illustrate the potential benefits of this approach in the context of fundamental problems in robotics such as visual scene understanding, planning, changing-contact manipulation, and multiagent/human-agent collaboration.

Sadanand Modak, Noah Tobias Patton, Isil Dillig, Joydeep Biswas

This paper addresses the problem of preference learning, which aims to align robot behaviors through learning user-specific preferences (e.g. “good pull-over location”) from visual demonstrations. Despite its similarity to learning factual concepts (e.g. “red door”), preference learning is a fundamentally harder problem due to its subjective nature and the paucity of person-specific training data. We address this problem using a novel framework called SYNAPSE, which is a neuro-symbolic approach designed to efficiently learn preferential concepts from limited data. SYNAPSE represents preferences as neuro-symbolic programs – facilitating inspection of individual parts for alignment – in a domain-specific language (DSL) that operates over images and leverages a novel combination of visual parsing, large language models, and program synthesis to learn programs representing individual preferences. We perform extensive evaluations on various preferential concepts as well as user case studies demonstrating its ability to align well with dissimilar user preferences. Our method significantly outperforms baselines, especially when it comes to out-of-distribution generalization. We show the importance of the design choices in the framework through multiple ablation studies.

Jingyang Zhao, Mingyu Xiao, Junqiang Peng, Ziliang Xiong

In the Clustered TSP (CTSP), we are given an edge-weighted graph satisfying the triangle inequality property, and a family of pairwise disjoint vertex groups. The goal is to find a minimum weight tour that includes all vertices, ensuring that the vertices within each group appear consecutively on the tour. The subgroup planning problem (SGPP) is an extension of CTSP by relaxing some triangle inequality requirements on edge weights. CTSP and SGPP have plentiful applications in AI and robotics. In this paper, we design three improved approximation algorithms for SGPP and CTSP. First, we propose a polynomial-time 2.167-approximation algorithm for SGPP, improving the previous ratio of 3 (IJCAI 2017). Second, we give an FPT 2.072-approximation algorithm for SGPP parameterized by the maximum group size, improving the previous ratio of 2.5 (IJCAI 2017). Third, we prove an FPT (β