Aidan Scannell
Aidan Scannell
Latest posts
Contextual Latent World Models for Offline Meta Reinforcement Learning
Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking
Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data
Winning the 1X World Model Challenge
Beyond Mamba SSMs: Parallel Kalman Filters as Scalable Primitives for Language Modelling
Forgetting is Everywhere
Generative World Modelling for Humanoids: 1X World Model Challenge
Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report
1X World Model Challenge
Introduction World models equip agents (e.g., humanoid robots) with internal simulators of their environments. By “imagining” the consequences of their actions, agents can plan, anticipate outcomes, and improve decision-making without direct real-world interaction. A central challenge in world modelling is the design of architectures that are both sufficiently expressive and computationally…
Huawei-Edinburgh Joint Lab: Discrete Codebook World Models
Generalist World Model Pre-Training for Efficient Reinforcement Learning
Discrete Codebook World Models for Continuous Control
Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learning
Discrete Codebook World Models
In reinforcement learning (RL), world models serve as internal simulators, enabling agents to predict environment dynamics and future outcomes in order to make informed decisions. While previous approaches leveraging discrete latent spaces, such as DreamerV3, have demonstrated strong performance in discrete action settings and visual control tasks, their comparative performance in state-based…
Nordic AI Meet & AI Day: Sample-efficient Reinforcement Learning with Implicitly Quantized Representations
iQRL: Implicitly Quantized Representations for Sample-Efficient Reinforcement Learning
Model-Based Reinforcement Learning
iQRL - Implicitly Quantized Representations for Sample-efficient Reinforcement Learning
Quantized Representations Prevent Dimensional Collapse in Self-predictive RL
Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning
Function-space Parameterization of Neural Networks for Sequential Learning
Implicitly Quantized Representations for Reinforcement Learning
Learning representations for reinforcement learning (RL) has shown much promise for continuous control. In this project, we investigate using vector quantization to prevent representation collapse when learning representations for RL using a self-supervised latent-state consistency loss.
Function-Space Bayesian Deep Learning for Sequential Learning
Sequential learning paradigms pose challenges for gradient-based deep learning due to difficulties incorporating new data and retaining prior knowledge. While Gaussian processes elegantly tackle these problems, they struggle with scalability and handling rich inputs, such as images. To address these issues, we introduce a technique that converts neural networks from weight space to function space,…
Experience
(Function-space) Laplace Approximation for Bayesian Neural Networks
Neural Networks as Sparse Gaussian Processes for Sequential Learning
Sparse Function-space Representation of Neural Networks
Mode-constrained Model-based Reinforcement Learning via Gaussian Processes
--> --> --> Experiment --> Description --> --> --> --> --> --> Greedy exploitation without mode constraint --> -\-> --> --> --> --> --> --> Greedy exploitation with mode constraint --> --> --> --> --> --> ModeRL (ours) --> --> --> --> --> --> Aleatoric uncertainty (ablation) --> --> always high at the mode boundary. This motivated formulating a dynamics model which can disentangle the sources of…
Model-based reinforcement learning under uncertainty
Investigating Bayesian Neural Network Dynamics Models for Model-Based Reinforcement Learning
This project seeks to evaluate and compare different approaches for learning dynamics models in model-based RL. In particular, we plan to compare different approximate inference techniques (e.g. Laplace approximation, MC dropout, variational inference), as well as ensemble methods, to understand why they either succeed or fail in different environments.
Model-based reinforcement learning under uncertainty: the importance of knowing what you don't know
Mode-Constrained Exploration for Model-Based Reinforcement Learning
PhD Thesis: Bayesian Learning for Control in Multimodal Dynamical Systems
Aalto RL Reading Club: Simplifying Model-Based RL: Learning Representations, Latent-Space Models, and Policies with One Objective
PhD Thesis: Bayesian Learning for Control in Multimodal Dynamical Systems
Trajectory Optimisation in Learned Multimodal Dynamical Systems via Latent-ODE Collocation
Identifiable Mixtures of Sparse Variational Gaussian Process Experts
One Keyboard to Rule Them All - I Built a Dactyl Manuform
After the first lockdown here in the UK, I decided that building a keyboard would make a good lockdown 2.0 project. I’ve had my eye on the for a while so I took the plunge and ordered all of the parts. Thanks to for the high quality SLA 3D printed case. So… 6 months after starting the build, I’ve finally finished it, woo! And guess what, it’s surpassed expectations……
GPJax - Gaussian Processes in Jax
I am developing a minimal Python package for implementing Gaussian process models in Python using JAX. I have spent a lot of time using GPflow and I like how they implement their GP library, in particular, their focus on variational inference and how they implement GP conditionals. As such, this package takes a similar approach but offers the benefits (and ease) of having JAX under the hood. GPJax…
Trajectory Optimisation in Learned Multimodal Dynamical Systems via Latent-ODE Collocation
Synergising Bayesian Inference and Probabilistic Geometries for Robotic Control
Creating a CV/Resume in Org-Mode using LaTeX Templates
Over the last few years I have been trying to find the best tools for managing my CV/resume. Previously I was maintaining a JSON file that I could export to pdf/html using and . They provide a well structured format (JSON Schema) for storing your CV data and there are a range of that you can use to style your CV when exporting to pdf/html. I personally think it’s a great idea and I have been…
Trajectory Optimisation in Learned Multimodal Dynamical Systems
This work presents a two-stage method to perform trajectory optimisation in multimodal dynamical systems with unknown nonlinear stochastic transition dynamics. The method finds trajectories that remain in a preferred dynamics mode where possible and in regions of the transition dynamics model that have been observed and can be predicted confidently. The first stage leverages a mixture of Gaussian…
Identifiable Mixtures of Sparse Variational Gaussian Process Experts
This work derives a novel variational lower bound for the mixture of Gaussian process experts model with a GP based gating network based on sparse GPs. The model (and inference) are implemented as a package ( ) written in / . Mixture models are inherently unidentifiable as different combinations of component distributions and mixture weights can generate the same distributions over the…
Setting Up an Emacs Playground on MacOS - Emacs Mac Port | Chemacs | Emacsclient | Spacemacs
This is a short post detailing how I installed Emacs and configured an environment for maintaining multiple configurations (on my MacBook Pro). I wanted to write the post because I have been on a roller coaster getting an Emacs install that provides all of the functionality that I want (maybe even need!). Some Emacs installs (e.g. from homebrew) are not recognised by the yabai tiling window…
How RSI Made Me a Better Developer
I’ve wanted to write a blog post about my journey with repetitive strain injury (RSI) for quite a while now. I’d been putting it off because the amount of content that I wanted to include seemed to be increasing exponentially with time. Anyhow, I’ve decided to go for it and I’m hopeful that I’ve selected the most important parts of my recovery, especially for a fellow…
Gaussian Process Regression
In this post I want to walk through Gaussian process regression; both the maths and a simple 1-dimensional python implementation. I have been working with (and teaching) Gaussian processes for a couple of years now so hopefully I’ve picked up some intuitions that will help you make sense of GPs. I’m focusing more on providing intuition here than I am mathematical rigor so please…
Welcome to Emacs Anonymous, Sorry, My Blog
Hello and welcome to my blog. I am an aspiring researcher (aka phd student) with interests at the intersection of robotics and machine learning. I am particularly interested in uncertainty quantification in ML and as a result spend a significant amount of time with Bayesian non-parametric methods, specifically Gaussian processes. I spend some of my time staring at equations and questioning what I…
Cluster log in with SSH
TL;DR Don’t enter your password every time you SSH into a cluster Configure SSH Add your cluster login details to your ssh config ~/.ssh/config so that you can log in with shh CLUSTER_NAME Importantly, you should not need to enter your password. Assuming you log into the cluster with ssh USERNAME@HOSTNAME then your ssh config ~/.ssh/config should have an entry like this Host CLUSTER_NAME…