Antonin Raffin | Homepage
Antonin Raffin | Homepage
Latest posts
Direct Policy Search Tutorial - RL Summer School 2026
DQN Tutorial - RL Summer School 2026
A blog post of this tutorial can be found here: https://araffin.github.io/post/rl102/
Exploration in Continuous Control RL
Recent Advances in RL for Continuous Control (SOTA) - Early 2026 Update
RL103: From Deep Q-Learning (DQN) to Soft Actor-Critic (SAC) and Beyond
This second blog post continues my practical introduction to (deep) reinforcement learning, presenting the main concepts and providing intuitions to understand the more recent Deep RL algorithms. In a first post (RL102) , I started from tabular Q-learning and worked my way up to Deep Q-learning (DQN). In this second post, I continue on to the Soft Actor-Critic (SAC) algorithm and its extensions.…
Stable-Baselines3 (SB3) Tutorial: Getting Started With Reinforcement Learning
PhD Defense: Enabling Reinforcement Learning on Real Robots
Designing (Robot) Software That Is Easy to Use
RL102: From Tabular Q-Learning to Deep Q-Learning (DQN)
This blog post is meant to be a practical introduction to (deep) reinforcement learning 1 , presenting the main concepts and providing intuitions to understand the more recent Deep RL algorithms. For a more in-depth and theoretical introduction, I recommend reading the RL Bible by Sutton and Barto. The plan for this post is to start from tabular Q-learning and work our way up to Deep Q-learning…
Getting SAC to Work on a Massive Parallel Simulator: Tuning for Speed (Part II)
This second post details how I tuned the Soft-Actor Critic (SAC) algorithm to learn as fast as PPO in the context of a massively parallel simulator (thousands of robots simulated in parallel). If you read along, you will learn how to automatically tune SAC for speed (i.e., minimize wall clock time), how to find better action boundaries, and what I tried that didn’t work. Part I analyzes why…
Recent Advances in RL for Continuous Control
Automatic Hyperparameter Tuning - In Practice (Part 2)
This is the second (and last) post on automatic hyperparameter optimization. In the first part , I introduced the challenges and main components of hyperparameter tuning (samplers, pruners, objective function, …). This second part is about the practical application of this technique with the Optuna library , in a reinforcement learning setting (using the Stable-Baselines3 (SB3) library).…
Getting SAC to Work on a Massive Parallel Simulator: An RL Journey With Off-Policy Algorithms (Part I)
This post details how I managed to get the Soft-Actor Critic (SAC) and other off-policy reinforcement learning algorithms to work on massively parallel simulators (think Isaac Sim with thousands of robots simulated in parallel). If you follow the journey, you will learn about overlooked details in task design and algorithm implementation that can have a big impact on performance. Spoiler alert:…
Enabling Reinforcement Learning on Real Robots
Ingredients for Learning Locomotion Directly on Real Hardware
Designing and Running Real-World RL Experiments
This talk was also given at the RL Symposium organized by Boston University on May 10, 2024.
Practical Tips for Reliable Reinforcement Learning
This talk was also given at the Industrial RL Workshop organized in Paris Saclay in November 2023.
An Open-Loop Baseline for Reinforcement Learning Locomotion Tasks
Knowledge Guided Reinforcement Learning for Robotics
DQN Tutorial
A blog post of this talk can be found here: https://araffin.github.io/post/rl102/
Automatic Hyperparameter Tuning - A Visual Guide (Part 1)
When you’re building a machine learning model, you want to find the best hyperparameters to make it shine. But who has the luxury of trying out every possible combination? The good news is that automatic hyperparameter tuning can help you. The trick is to allocate your “budget” (aka time and resources) wisely. You want to try out as many combinations as possible, but you…
SBX: Stable Baselines Jax
Proof of concept version of Stable-Baselines3 in Jax. Github repository: https://github.com/araffin/sbx Implemented algorithms: Soft Actor-Critic (SAC) and SAC-N Truncated Quantile Critics (TQC) Dropout Q-Functions for Doubly Efficient Reinforcement Learning (DroQ) Proximal Policy Optimization (PPO) Deep Q Network (DQN) Twin Delayed DDPG (TD3) Deep Deterministic Policy Gradient (DDPG) Batch…
Datasaurust
Blazingly fast implementation of the Datasaurus paper (500x faster than the original): “Same Stats, Different Graphs: Generating Datasets with Varied Appearance and Identical Statistics through Simulated Annealing” by Justin Matejka and George Fitzmaurice. Github repository: https://github.com/araffin/datasaurust
Learning to Exploit Elastic Actuators for Quadruped Locomotion
Training RL agents directly on real robots
Tutorial: Tools for Robotic Reinforcement Learning
The 37 Implementation Details of Proximal Policy Optimization
Stable-Baselines3: Reliable Reinforcement Learning Implementations
Rliable: Better Evaluation for Reinforcement Learning - A Visual Explanation
It is critical for Reinforcement Learning (RL) practitioners to properly evaluate and compare results. Reporting results with poor comparison leads to a progress mirage and may underestimate the stochasticity of the results. To this end, Deep RL at the Edge of the Statistical Precipice (Neurips Oral) provides recommendations for a more rigorous evaluation of DeepRL algorithms. The paper comes with…
Smooth Exploration for Robotic Reinforcement Learning
Learning to Race in Hours with Reinforcement Learning
Fault-Tolerant Six-DoF Pose Estimation for Tendon-Driven Continuum Mechanisms
Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software. Click the Slides button above to demo Academic’s Markdown slides feature. Supplementary notes can be added here, including [code and math](https://sourcethemes.com/academic/docs/writing-markdown-latex/). -->
RL Tips and Tricks / The Challenges of Applying RL to Real Robots
RL Tips and Tricks Slides SB3 Hands-on Session slides SB3 Hands-on Session github repo
Stable-Baselines3: Reliable Reinforcement Learning Implementations
After several months of beta, we are happy to announce the release of Stable-Baselines3 (SB3) v1.0, a set of reliable implementations of reinforcement learning (RL) algorithms in PyTorch =D! It is the next major version of Stable Baselines . The implementations have been benchmarked against reference codebases, and automated unit tests cover 95% of the code. In this blog post, we give you an…
Stable Baselines3
Stable Baselines3 is a set of improved implementations of reinforcement learning algorithms in PyTorch. It is the next major version of Stable Baselines. Github repository: https://github.com/DLR-RM/stable-baselines3 Documentation: https://stable-baselines3.readthedocs.io/ RL Baselines3 Zoo (collection of pre-trained agents): https://github.com/DLR-RM/rl-baselines3-zoo RL Baselines3 Zoo also…
RL Tutorial on Stable Baselines
SRL - Stable Baselines Presentation
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics
Learning to Drive Smoothly in Minutes
Read the full article on Medium
Learning to Drive Smoothly in Minutes
Learning to drive smoothly in minutes, using a reinforcement learning algorithm – Soft Actor-Critic (SAC) – and a Variational AutoEncoder (VAE) in the Donkey Car simulator. Github repository: https://github.com/araffin/learning-to-drive-in-5-minutes Blog post on Medium
RL Baselines Zoo
A collection of trained Reinforcement Learning (RL) agents, with tuned hyperparameters, using Stable Baselines . Github repository: https://github.com/DLR-RM/rl-baselines3-zoo Goals of this repository: Provide a simple interface to train and enjoy RL agents Benchmark the different Reinforcement Learning algorithms Provide tuned hyperparameters for each environment and RL algorithm Have fun with…
S-RL Toolbox
S-RL Toolbox: Reinforcement Learning (RL) and State Representation Learning (SRL) Toolbox for Robotics. Github repository: https://github.com/araffin/robotics-rl-srl Documentation: https://s-rl-toolbox.readthedocs.io Paper: https://arxiv.org/abs/1809.09369 Main Features 10 RL algorithms ( Stable Baselines included) logging / plotting / visdom integration / replay trained agent hyperparameter…
Stable Baselines
Stable Baselines is a set of improved implementations of Reinforcement Learning (RL) algorithms based on OpenAI Baselines. Github repository: https://github.com/hill-a/stable-baselines You can read a detailed presentation of Stable Baselines in the Medium article
S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning
PythonRobotics: a Python code collection of robotics algorithms
Stable Baselines: a Fork of OpenAI Baselines — Reinforcement Learning Made Easy
Read the full article on Medium
Simple and Robust {Computer — Arduino} Serial Communication
Read the full article on Medium
Autonomous Racing Robot With an Arduino, a Raspberry Pi and a Pi Camera
Read the full article on Medium
Racing Robot
Autonomous toy racing car. CAMaleon team at the Toulouse Robot Race 2017. Humbavision team at IronCar. Medium article: https://medium.com/@araffin/autonomous-racing-robot-with-an-arduino-a-raspberry-pi-and-a-pi-camera-3e72819e1e63 Video of the car: https://www.youtube.com/watch?v=xhI71ZdSh6k