Recent content on Magnus Ross
My friends and I were discussing England’s world cup fortunes at lunch the other day, and there was a strong feeling that our boys would have no chance against France, should that end up being the final. I wanted to check the odds for this, but there is of course not a market since the teams in the final are not yet confirmed. I had a feeling that you could piece it together from existing…
This is nothing to be proud of, but I have never really studied optimisation in depth. Oh sure, I know my Adam from my AdaGrad and I even used L-BFGS one time, but when people start talking about dual spaces and convergence for L ∞ L^\infty L ∞ continuous functions, I tend to glaze over a bit. For some reason I think in my head a lot of it seemed a bit old fashioned, whatever that means; why do I…
Or, understanding the Cradle-1 pipeline. Lead optimisation is the step in drug design where you take a molecule that sort of works and try to make it actually work, and it’s arguably the step where most real-world design campaigns succeed or die. Due to the influence of a couple of my pals , I have recently become interested in using machine learning to do lead optimisation for proteins.…
Recently I’ve been thinking a bit about foundation models for time series, and I can’t work out how to feel about them. Are they the right approach? Has everyone just got foundation model fever? This post provides a semi-structured set of these thoughts, and unfortunately raises more questions than it provides answers. Time series foundation models (FMs) are trained on large…
The Mind is a collaborative card game that is designed to be fun to play with your friends. I had fun playing it with my friends a couple of weekends ago. The basic rules of the game are pretty simple: 1 There are 100 cards in the deck, numbered 1 to 100. In round R R R each player is dealt R R R cards. The object of the game is to play 2 the cards in order, such that a player never plays a card…
It’s good to read and I like doing it. Since 2022, I’ve been keeping track of all the books I read for fun, taking a few notes on each. I don’t use Goodreads because I’m not really into giving Bezos my hot takes for free. Unfortunately, this, along with the fact I use an e-reader, means it’s impossible for me to performatively show off all the worthy titles I’m…
Evaluating LLMs is hard. Many popular benchmarks have become saturated, leading to artificial inflation of the ability of models to solve complex problems. These issues are exacerbated by the incentive that exists for AI people to constantly hype the field. A recent example of an OpenAI exec claiming that ChatGPT had solved ten open Erdős problems when it had in fact just found existing papers…
The Cape Wrath Trail is known, in a somewhat tongue-in-cheek way, as the UK’s hardest trail. I just got back from walking/running it last week, 1 and by and large had a wonderful time. Right at the start of the first stage, there was a snatched conversation with a guy fixing a house: “Off on the Cape Wrath Trail?” “Yes!” “Lucky you!” Lucky me indeed! There…
Machine learning for health (ML4H) has a problem with deployment in the clinic. Each year, thousands of papers are published describing ML systems for all kinds of medical problems, from antimicrobial resistance to radiology. Unfortunately, only a tiny proportion of these models— estimates put the number at around 1%—are ever prospectively evaluated, meaning deployed and tested in the real world.…
This is a guide that details how to use the Routegadget website to practice route selection for a mountain marathon. While Routegadget’s interface is a little old-school, it is a useful and powerful tool for analysing past races. This guide will focus on past races from the SLMM , but this practice is applicable to any mountain marathon. A first map First, let’s look at a past course…
Unless you’ve been under a rock the past few weeks, you’ve probably heard about Elon Musk’s gaggle of DOGE lads who have been let loose inside the US Treasury payments system . Since they have apparently been given read/write access to some highly sensitive databases and are likely pushing hastily coded “efficiency improvements” to main as we speak, it’s in the public…
Benchmark tasks are a cornerstone of ML research, and a large proportion new research in pretty much every subfield follows the following template: Introduce new layer, architecture, optimiser, inference scheme… Run the model on a set of benchmark tasks Make a big table with your model/method and a load of other similar ones 1 Although this paradigm has some issues, it can be a useful way…
The EM algorithm for me is one of those things that I feel I should know back to front, since it’s a pretty foundational algorithm in probabilistic ML. Unfortunately though I’ve never actually used it explicitly in a model I have built, despite reading about it in various textbooks, so I never properly got to grips with it. If you feel the same way, then hopefully this question will…
This is a nice problem from the book “Cut the knot”, which is a compilation of probability riddles and brain teasers. I like the book because many of the problems are about intuitive reasoning, and don’t really involve that much technical knowledge of probability theory, so they can be fun to do with people who don’t have a formal maths background. Each problem can also…
This problem from the GPML book relates to the effect of the choice of measure when using Mercer’s theorem to compute kernel eigenfunctions on the resulting norm in the RKHS induced by that kernel. In the problem we show that in the finite dimensional case (this also applies in the ∞ \infty ∞ dimensions case but it is then harder to show), the RKHS norm is independent of the measure chosen.…
It is only the third week and I have already failed to manage my goal of posting one solution per week, not a good sign for the future… In my defense, I was busy last week attending the Probabilistic Numerics Spring School , at which I learned lots about probabilistic ODE solvers, on which this weeks late question is based. Probabilistic ODE solvers work by placing a Markovian GP prior…
Problem Note: I was brushing up on my SVD using the brilliant “Mathematics for Machine Learning” book, but the exercises listed in the book for SVD were a bit basic, so I decided to try use ChatGPT to generate a question. The problem below is what came out, quite impressive, although there were quite a few errors in the question that I had to fix (e.g the dimensionalities of the…
I spent the summer of 2022 visiting Markus Heinonen at Aalto University in Finland. Together we worked on energy conserving GP models, and I am happy to say our paper on the work was recently accepted to TMLR. Check out the paper , the code , or some visualizations . We place a GP prior over the Hamiltonian and, using a set of inducing points, map function samples through Hamilton’s…
Problem Take an objective function f ( x ) f(x) f ( x ) with GP prior and i.i.d. Gaussian noise with variance σ 2 \sigma^2 σ 2 , so y i = f ( x i ) + ϵ i y_i = f(x_i) + \epsilon_i y i = f ( x i ) + ϵ i with ϵ i ∼ N ( 0 , σ 2 ) \epsilon_i \sim \mathcal{N}(0, \sigma^2) ϵ i ∼ N ( 0 , σ 2 ) . Assume two locations x 1 , x 2 x_1, x_2 x 1 , x 2 are sufficiently separated that their function…
Update: This paper was published at NeurIPS 2021 check out the final version or my presentation . This work, with Mauricio Alvarez and Mike Smith, has been the main focus of the first year of my PhD, check it out on arXiv . The sampling process for the model described in the paper, which is a non-parametric, and can represent data generated by non-linear operators. There will be a blog post…
There are many interesting things that can be learned about probabilistic modelling from the world of trading and finance, where, perhaps due to the direct connection to very large sums of money, attitudes to problems are generally very different to those of the typical statistician or ML practitioner. Two books I have enjoyed recently in the area are The Education of a Speculator by Victor…
Near the end of last year I had the privilege of being invited at the 3rd Sheffield Workshop on Structural Dynamics . I spoke for 15 minutes about some work I have been doing on the application of Gaussian processes to the non-parametric learning of Volterra kernel functions. The event as a whole was super interesting, and I would fully recommend the other talks . You can find my talk here .
There are a lot of podcasts out there, and this includes loads that are related to machine learning in some way. Over the summer I’ve worked my way through a fair few of them, so I thought I’d compile a list of my favorites. There are a couple of lists like this out there already, but most of them are out of date, either they include podcasts that are now inactive, or leave out ones…
(Note: This series of posts is closely related to and inspired by this paper from Miles Cranmer, Sam Greydanus, Stephan Hoyer and others. To accompany the paper they also wrote a brilliant blog post about the work which I would encourage you to read) I really enjoy Lagrangian mechanics, in fact, I would go so far as to say that studying it was one of the best parts of my physics degree. We will…