Recent content on Gerardo Duran-Martin
Introduction Getting in the door was the hardest part of the whole ordeal. I did an undergrad at a university that doesn’t even show in the global rankings. At the time, I didn’t know about the importance of attending a good uni , so I focused on learning as much as I could during my undergrad, and a bit beyond what I was taught. After finishing my undergrad, I applied for…
Abstract We develop a predictive-first optimisation framework for streaming hidden Markov models. Unlike classical approaches that prioritise full posterior recovery under a fully specified generative model, we assume access to regime-specific predictive models whose parameters are learned online while maintaining a fixed transition prior over regimes. Our objective is to sequentially identify…
Introduction A classical method to tackle the problem of time-series regime detection and segmentation is the hidden Markov model (HMM). 1 At the core of the HMM are two sub-models: a model for latent regimes and a predictive model for observations (conditioned on a regime). In this post, we consider the problem of learning a parametric HMM when neither the regimes nor the model parameters are…
Introduction In this note, we introduce the BONE framework 1 for learning and adaptation in non-stationary environments. We present various strategies to adapt the one-dimensional learning procedure we saw on the first note , under the assumption of structural changes in the reward process. A recap We assume an agent is given a sequence of actions $a_t \in {\cal A}$, and rewards $r_t \in \reals$.…
Abstract This paper develops a framework to predict toxic trades that a broker receives from her clients. Toxic trades are predicted with a novel online learning Bayesian method which we call the projection-based unification of last-layer and subspace estimation (PULSE). PULSE is a fast and statistically-efficient Bayesian procedure for online training of neural networks. We employ a proprietary…
Introduction The algorithms we studied in the previous note assumed a stationary environment: the data-generating-process (DGP) does not depend on time. Under this assumption, the learning algorithms gradually converge to a point estimate, which means that they either stop exploring, stop learning, or both. In various real-world scenarios, we do not want our algorithms to converge. The underlying…
Introduction In this note, we provide an introduction to the problem of sequential learning and decision making from a Bayesian perspective. With this perspective, we tackle two problems under a same framework: on the one hand, balancing exploration and exploitation, and on the other, learning a parametric model of rewards. As we will see, the first problem can be tackled with the Thompson…
Introduction Deep neural networks (DNNs) are great at fitting data, but terrible at predicting when they don’t know the right answer. Tackling this latter challenge (often called uncertainty quantification) is at the core of various problems in AI/ML. For example, reducing hallucinations in large language models, 1 or the classical balancing of exploration versus exploitation in…
Introduction I was first introduced to Gaussian Processes (GPs) as the quintessential example of Bayesian reasoning in machine learning. This perspective is cemented by the seminal textbook on the subject, which states in its foreword: “Gaussian Processes for Machine Learning presents one of the most important Bayesian machine learning approaches.” 1 And yet, the formal definition of a…
Abstract We introduce scalable algorithms for online learning of neural network parameters and Bayesian sequential decision making. Unlike classical Bayesian neural networks, which induce predictive uncertainty through a posterior over model parameters, our methods adopt a predictive-first perspective based on martingale posteriors. In particular, we work directly with the one-step-ahead posterior…
Online learning and sequential decision making Observe sequence of features $\vx_t\in\reals^M$ and observations $\vy_t \in \reals^o$: $$ {\cal D}_{1:t} = \{(\vx_1, \vy_1), \ldots, (\vx_t, \vy_t)\}. $$ We model a mapping from $\vx_t$ to $\vy_t$ through a function $h: \Theta \times \reals^M \to \reals^o$ that maps model parameters $\vtheta \in \Theta$ and features $\vx_t \in \reals^M$ to outputs…
Abstract In this thesis, we introduce Bayesian filtering as a principled framework for tackling diverse sequential machine learning problems, including online (continual) learning, prequential (one-step-ahead) forecasting, and contextual bandits. To this end, this thesis addresses key challenges in applying Bayesian filtering to these problems: adaptivity to non-stationary environments, robustness…
Part III of the filtering notes series: Part I: signal plus noise models Part II: state-space-models Part III: the Kalman filter Introduction In this post, we derive the Kalman filter (KF) algorithm and explore how it can be applied to time series forecasting and sequential (online) learning. As we will see, the KF algorithm arises naturally from the covariance properties of state-space models…
Part II of the filtering notes series: Part I: signal plus noise models Part II: state-space-models Part III: state-space-models Introduction Whether modelling the trajectory of a rocket, forecasting economic trends, or powering large language models, discrete linear state-space models (SSMs) provide a unifying framework for estimation, prediction, and learning. At their core, a linear SSM models…
Abstract We propose a unifying framework for methods that perform probabilistic online learning in non-stationary environments. We call the framework BONE, which stands for generalised (B)ayesian (O)nline learning in (N)on-stationary (E)nvironments. BONE provides a common structure to tackle a variety of problems, including online continual learning, prequential forecasting, and contextual…
Part I of the filtering notes series: Part I: signal plus noise models Part II: state-space-models Part III: state-space-models Introduction From tracking epidemics and predicting financial markets to controlling complex systems like rockets and personalising online experiences, many real-world problems rely on understanding and processing sequential data. In machine learning, this includes…
(B)ayesian (O)nline learning in (N)on-stationary (E)nvironments Paper: arxiv/2411.10153 Slides: bone/talk Repo: gerdm/BONE How do we create agents that can continually learn and adapt to their environment? Two requirements: The ability to learn continually from streaming datasets. The ability to adapt when the data-generating process changes. We present a Bayesian perspective to tackle these two…
Introduction In this post, we introduce the Bayesian Online Changepoint Detection (BOCD) model and its application for estimating the probability of heads in a sequence of coin tosses where the underlying probability changes over time. Flipping coins Suppose the following sequence of coin tosses arrive in a stream: 1 2 3 4 5 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,…
Introduction Our weighted-observation likelihood filter (WolF) got accepted at ICML 2024. Our method robustifies the Kalman filter (KF) by replacing the classical Gaussian likelihood assumption with a loss function. As we show, this modification results in a provably-robust and easy-to-implement variant of the KF. Despite its simplicity, the terminology behind WoLF (similar to the terminology of…
Abstract We derive a novel, provably robust, efficient, and closed-form Bayesian update rule for online filtering in state-space models in the presence of outliers and misspecified measurement models. Our method combines generalised Bayesian inference with filtering methods such as the extended and ensemble Kalman filter. We use the former to show robustness and the latter to ensure computational…
Abstract We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream. The method is based on the extended Kalman filter (EKF), but uses a novel low-rank plus diagonal decomposition of the posterior precision matrix, which gives a cost per step which is linear in the number of model…
The Kalman filter Suppose $\forall t=1,\ldots,T.{\bf x}_{t}\in\mathbb{R}^n, {\bf y}_t\in\mathbb{R}^m$, $$ \begin{aligned} p({\bf x}_t \vert {\bf x}_{t-1}) &= {\cal N}({\bf x}_t \vert {\bf A}_{t-1}{\bf x}_{t-1}, {\bf Q}_{t-1})\\ p({\bf y}_t \vert {\bf x}_t) &= {\cal N}({\bf y}_t \vert {\bf H}_t{\bf x}_t, {\bf R}_t) \end{aligned} $$ then, the Kalman filter equations are given by $$ \begin{aligned}…
Abstract In this paper we present a new algorithm for online (sequential) inference in Bayesian neural networks, and show its suitability for tackling contextual bandit problems. The key idea is to combine the extended Kalman filter (which locally linearizes the likelihood function at each time step) with a (learned or random) low-dimensional affine subspace for the parameters; the use of a…
Do you know what inm,kij,jnm->knm is all about? For an interactive version of this post, see this Colab notebook . Introduction Linear combinations are ubiquitous in machine learning and statistics. Many algorithms and models in the statistics and machine learning literature can be written (or approximated) as a matrix-vector multiplication. Einsums are a way of representing the linear interaction…
Introduction In this post I report my work during the Google Summer of Code (GSoC) 2021 program. During the summer I contributed to Volume 2 of the Probabilistic Machine Learning book by Kevin P. Murphy. The GSoC 2021 TF/Pyprobml program was divided into in two Github repositories: a public repo probml/pyprobml where the final code was pushed and a private repo pyprobml/hermes where discussions…
Introduction On July 10, 2018 I committed myself to read and do all the exercises of the book Pattern Recognition and Machine Learning from Christopher Bishop. After seven hundred and ninety two days, I can finally say I accomplished my goal. One of the main reasons why I decided to study the book from cover to cover was to demystify machine learning (ML). Reading through the book taught me to…
Statistical machine learning About me g.duran@me.com My background is in Bayesian inference, Kalman filtering, time-series modeling, state-space models, and online learning. Talks You can find many of the slides here . April 2026 | MRC Biostatistics Unit, University of Cambridge Learning and adapting in non-stationary environments slides September 2025 | MatheEXLab Webinar @ NUS | Online From…
Experience Quantitative Researcher — Tower Research Capital 2026 — Present Postdoctoral Researcher — Oxford-Man Institute, University of Oxford 2025 — 2026 Summer Intern – Quantitative Researcher — Man AHL Summers 2022 — 2024 2024: Validation and calibration of the internal risk model. 2023: Developed systematic trading strategies that adapt to non-stationary financial time series. 2022: Built…