Lifelong Robot Learning: Generalization, Adaptation, and Deployment with Large Models

Workshop @ RSS 2024

Aula Hall A, TU Delft

July 19th, 1:50pm - 6pm

Abstract

In recent years, we have witnessed the tremendous success of large models (so-called foundation models) in computer vision and natural language processing (NLP). Hence, there has been growing interest in repurposing existing large models or learning new ones for robotics problems. However, different from vision and language domains, there is no readily available internet-scale data for robotics to train such a model. As a result, an alternative is to train the large models for robotics in a lifelong fashion, where robots collect data during deployment, and refine the model on the fly. While the concept and formulation of lifelong learning originated from robotics, it has been mostly studied in vision and NLP. Now, in the presence of large models, it is the right time to revisit the lifelong robot learning problems in a scalable manner. 

Discussion Topics

In this workshop, our objective is to unite participants to explore and envision the future paradigm of robot learning and deployment, emphasizing the continuous adaptation of robots to novel scenarios in the era of large models. This proposed workshop is intended for audience of backgrounds in robot learning, foundation models for robots, and lifelong learning in decision-making. We invite speakers and presenters from these aforementioned subfields, aiming to inspire meaningful discussions of generalization, adaptation, and deployment of lifelong robot learning methods, given the availability of large models.

We would like to gather people who are interested in generalist agents, lifelong learning, and robotics to discuss the following topics:

• Is lifelong learning critical in building robots that can be deployed in the real world? Is large-scale multi-task learning enough for robotics (just like large language models that possess the emergent zero-shot ability)?

• How should robots adapt to personal usage? How can robots continually learn from their users while maintaining privacy?

• Will the large foundation model for robotics consist of a set of foundation models specialized in perception, control, and language, or will it be a single end-to-end large model? In other words, will it be more likely to be a compositional model or a single giant transformer-like model?

• What properties an ideal lifelong learning algorithm should possess for efficiently adapting robotics models?

• Do we need paradigm shifts in neural architectures of large models to support lifelong learning?

• Given the discussion above, what type of benchmark problems should researchers work on? What metrics should we focus on?

• How to leverage large foundation models to facilitate lifelong robot learning? What are the new opportunities and challenges of lifelong robot learning in the era of large models?

Contributions

Spotlight:

• Lifelong Autonomous Improvement of Navigation Foundation Models in the Wild. Kyle Stachowicz, Lydia Ignatova, Sergey Levine.

• Pick up the PACE: A Parameter-Free Optimizer for Lifelong Reinforcement Learning. Aneesh Muppidi, Zhiyu Zhang, Heng Yang.

• R+X: Retrieval and Execution from Everyday Human Videos. Georgios Papagiannis, Norman Di Palo, Pietro Vitiello, Edward Johns.

• Grounding Language Plans In Demonstrations Through Counterfactual Perturbations. Yanwei Wang.

• Distillation of Diffusion Models into Fast and Tractable Mixture of Experts. Hongyi Zhou, Denis Blessing, Ge Li, Onur Celik, Xiaogang Jia, Gerhard Neumann, Rudolf Lioutikov. 

• Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction. Aidan Curtis, Nishanth Kumar, Jing Cao, Tomás Lozano-Pérez, Leslie Pack Kaelbling. 

Posters: 

• Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight. Jiaxu Xing, Angel Romero, Leonard Bauersfeld, Davide Scaramuzza.

• Affordance-Guided Reinforcement Learning via Visual Prompting. Olivia Y. Lee, Annie Xie, Kuan Fang, Karl Pertsch, Chelsea Finn.

• MaIL: Improving Imitation Learning with Mamba. Xiaogang Jia, Qian Wang, Atalay Donat, Bowen Xing, Ge Li, Hongyi Zhou, Onur Celik, Denis Blessing, Rudolf Lioutikov, Gerhard Neumann.

• PRIMP: PRobabilistically-Informed Motion Primitives for Efficient Affordance Learning from Demonstration. Sipu Ruan, Weixiao Liu, Xiaoli Wang, Xin Meng, Gregory S Chirikjian.

• Rapid Learning without Catastrophic Forgetting in the Morris Water Maze. Raymond L Wang, Jaedong Hwang, Akhilan Boopathy, Ila R Fiete.

• RAIL: Robot Affordance Imagination with Large Language Models. Ceng Zhang, Xin Meng, Dongchen Qi, Gregory S Chirikjian.

• Trajectory-level Exploration and Step-based Update in Episodic Reinforcement Learning. Hongyi Zhou, Ge Li, Dominik Roth, Serge Thilges, Fabian Otto, Rudolf Lioutikov, Gerhard Neumann.