Roadmap to Sample-Efficient Real-World Reinforcement Learning Workshop (R2RL)

Proposed Format, Interactivity, Encouraging Attendance

This workshop is designed around live audience participation: attendees propose and vote on the most pressing blockers in real-world RL, then directly shape breakout discussions and the final roadmap whitepaper.

Main workshop question: Which bottlenecks are preventing sample-efficient real-world RL, and what should the community prioritize next?
Crowdsourced problem board: submissions are prepared from the call for papers, then refined and voted on by participants at the venue.
Interactive breakouts + panel: top-voted problems drive breakout sessions whose outcomes are synthesized in a closing panel and written roadmap.

Overview

Reinforcement learning (RL) has long promised autonomous, self-improving robots, but a stubborn gap remains between simulation success and reliable real-world deployment. At the center of that gap is sample efficiency: every trial on physical hardware costs both human supervision and wall-clock time.

Central question: What problems preclude making RL algorithms sample-efficient enough for real-world deployment, and what should we focus on over the next few years to solve them?

What makes this workshop distinct?

Early-career speakers (students and scientists): Researchers actively running real-robot RL who can surface practical bottlenecks from direct experimentation.
A crowdsourced problem board: At the entrance, attendees propose and vote on the problems most responsible for poor real-world sample efficiency.
Breakout sessions: Each top-voted problem becomes an interactive discussion where groups define the blocker and propose near-term directions.
Interactive panel discussion: Speakers and audience breakout leads synthesize key takeaways across sessions.
A roadmap whitepaper co-produced during the workshop: Organizers compile talk notes, breakout outputs, and panel synthesis into a shared post-workshop artifact.

Call for Papers

Topics of Interest

We welcome submissions on sample-efficient real-world RL, especially work that identifies practical bottlenecks, reports lessons learned from real-robot experiments, or proposes methods that reduce the cost of learning on hardware. Topics of interest include, but are not limited to:

Supervision and Reward Cost: Methods for obtaining useful learning signals cheaply, including learned reward models, human feedback, human-in-the-loop interventions, and strategies for avoiding supervision as the bottleneck.
Exploration and Safety on Hardware: Algorithms and systems for efficient exploration under safety constraints, reset-free or autonomous RL, safe online adaptation, and approaches that improve wall-clock efficiency during real-world training.
Leveraging Priors for Real-World RL: Approaches that use pretrained policies, VLAs, offline datasets, world models, or sim-to-real transfer to reduce the number of required on-robot interactions.
Post-Deployment Adaptation: Methods for finetuning deployed policies, improving robustness to distribution shift, learning new behaviors from experience, and adapting generalist policies without catastrophic forgetting.
Long-Horizon and Contact-Rich RL: Techniques for improving sample efficiency in long-horizon manipulation, dexterous control, contact-rich tasks, and settings with delayed rewards, compounding errors, or difficult exploration.
Benchmarks, Metrics, and Lessons Learned: Benchmarks, evaluation protocols, shared platforms, negative results, system-level insights, and analysis of what worked, what failed, and why in real-world RL experiments.

Submission Guidelines

Submission portal: all papers must be submitted through our OpenReview portal.
Page limit: submissions should be 4–8 pages, excluding references and appendix.
Format: submissions must follow the official CoRL paper template and style, and must be properly anonymized for double-blind review.
Dual submission policy: we disallow work already accepted to the main CoRL 2026 conference. Accepted workshop papers will be listed on the website but are non-archival and will not appear in formal proceedings.

Spotlight Talks

All accepted papers will be presented at an in-person poster session. A small number of selected papers will additionally give a 5-minute spotlight talk. Camera-ready versions of all accepted papers will be made available on the workshop website.

Workshop Schedule (Tentative)

Location: TBD, Austin, TX

Date: Thursday, November 12, 2026

8:30 - 8:40	Opening, Central Question, and Problem-board Framing
8:40 - 9:50	Keynote Talks
9:50 - 10:00	Problem-board Voting
10:00 - 10:30	Oral Spotlights of Key Contributed Papers
10:30 - 11:00	Poster Session + Coffee Break
11:00 - 11:50	Breakout Sessions on Top-voted Problems
11:50 - 12:20	Panel Discussion with Speakers and Audience Volunteers
12:20 - 12:30	Synthesis, Roadmap, and Whitepaper Next Steps