A blog about statistics, I guess.
Acknowledgements The code in this post is indebted (and in some cases wholly ripped off from) work by the glorious Finn Lindgren , who emailed me some code to do this probably a decade ago. Yes I am behind on my emails. Finn’s code can be found here as part of the glorious INLAbru project. Code availability The code from this post can be found in my github repo . The time has come once more to…
Code availability The code from this post can be found in my github repo . I’ll be honest with y’all. I was writing something else. It was really long and was getting annoying to edit and was probably never going to be finished. So instead of doing that, I am just going to post this. It’s about making a block matrix in a Stan-compatible way. Why?? Because I wanted to be able to do this. There is…
I am, once again, in a bit of a mood. And the only thing that will fix my mood is a good martini and a Laplace approximation. And I’m all out of martinis. To be honest I started writing this post in February 2023, but then got distracted by visas and jobs and all that jazz. But I felt the desire to finish it, so here we are. I wonder how much I will want to re-write 1 The post started as a…
The other day I went to the cinema and watched M3GAN, a true movie masterpiece 1 about the death and carnage that ensues when you simply train your extremely complex ML model and don’t do proper ethics work. And that, of course, made me want to write a little bit about something relatively hip, hop, and happening 2 in the ML/AI space. But, like, I’m not gonna be that on trend 3 because fuck that…
Gaussian processes are lovely things. I’m a big fan. They are, however, thirsty. They will take your memory, your time, and anything else they can. Basically, the art of fitting Gaussian process models is the fine art of reducing the GP model until it’s simple enough to fit while still being flexible enough to be useful. There’s a long literature on effective approximation to Gaussian Processes…
The time has come once more to resume my journey into sparse matrices. There’s been a bit of a pause, mostly because I realised that I didn’t know how to implement the sparse Cholesky factorisation in a JAX-traceable way. But now the time has come. It is time for me to get on top of JAX’s weird control-flow constructs. And, along the way, I’m going to re-do the sparse Cholesky factorisation to…
Just the other day 1 I was chatting with a friend 2 about MCMC and he asked me a fundamental, but seldom asked, question: What happens my acceptance probability is a bit off? . This question comes up a bunch. In this context, they were switching from double to single precision 3 and were a little worried that some of their operations would be a bit more inexact than they were used to. Would this…
Sometimes it’s the parable of the barren fig tree. Sometimes you’re just pissed at a shrub. Paradoxes and counterexamples live in statistics as our morality plays and our ghost stories. They serve as the creepy gas station attendants that populate the roads leading to the curséd woods; existing not to force change on the adventurer, but to signpost potential danger. 1 As a rule, we should also…
Long time readers will know that I bloody love a Gaussian process (GP). I wrote an extremely detailed post on the various ways to define Gaussian processes . And I did not do that because I just love inflicting Hilbert spaces on people. In fact, the only reason that I ever went beyond the standard operational definition of GPs that most people live their whole lives using is that I needed to.…
Eliza knows a little something about monkeys. This will become relevant in a moment. In about 2016, Almeling et al. published a paper that suggested aged Barbary macaques maintained interest in members of their own species while losing interest in novel non-social stimuli (eg toys or puzzles with food inside). I’d never come across the concept of a Graphical Abstract before, but here is the one…
At some point in the distant past, I wrote three posts about prior distributions. The first was very basic, because why not. The second one talked about conjugate priors. The third one talked about so-called objective priors. I am suddenly 1 of a mood to write some more on this 2 topic. The thing is, so far I’ve only really talked about methods for setting prior distributions that I don’t…
Imagine you have a target probability distribution and you want to estimate the expectation . That’s lovely and everything, but if it was easy none of us would have jobs. High-dimensional quadrature is a pain in the arse. A very simple way to get an decent estimate of is to use importance sampling , that is taking draws , from some proposal distribution . Then, noting that we can use Monte Carlo…
Welcome to part six!!! of our ongoing series on making sparse linear algebra differentiable in JAX with the eventual hope to be able to do some cool statistical shit . We are nowhere near done . Last time , we looked at making JAX primitives. We built four of them. Today we are going to implement the corresponding differentiation rules! For three 1 of them. So strap yourselves in. This is gonna be…
This is part five of our ongoing series on implementing differentiable sparse linear algebra in JAX. In some sense this is the last boring post before we get to the derivatives. Was this post going to include the derivatives? It sure was but then I realised that a different choice was to go to bed so I can get up nice and early in the morning and vote in our election. It goes without saying that…
This is the fourth post in a series where I try to squeeze autodiffable sparse matrices into JAX with the aim to speed up some model classes in PyMC. So far, I have: Outlined the problem Post 1 Worked through a basic python implementation of a sparse Cholesky decomposition Post 2 Failed to get JAX to transform some numpy code into efficient, JIT-compileable code Post 3 I am in the process of…
This is part three of an ongoing exercise in hubris. Part one is here. Part two is here. The overall aim of this series of posts is to look at how sparse Cholesky factorisations work, how JAX works, and how to marry the two with the ultimate aim of putting a bit of sparse matrix support into PyMC, which should allow for faster inference in linear mixed models, Gaussian spatial models. And…
This is part two of an ongoing exercise in hubris. Part one is here. The Choleksy factorisation So first things first: Cholesky wasn’t Russian. I don’t know why I always thought he was, but you know. Sometime you should do a little googling first. Cholesky was French and died in the First World War. But now that’s out of the way, let’s talk about matrices. If 1 is a symmetric positive definite…
Back in the early days of the pandemic I though “I’ll have a pandemic project”. I never did my pandemic project. But I did think briefly about what it would be. I want to get the types of models I like to use in everyday life efficiently implemented inside Stan. These models encapsulate (generalised) linear mixed models 1 , (generalised) additive models, Markovian spatial models 2 , and other…
Every day a little death, in the parlour, in the bed. On the lips and in the eyes. In the curtains in the silver, in the buttons, in the bread, in the murmurs, in the pauses, in the gestures, in the sighs. Sondheim The most horrible sound in the world is that of a reviewer asking you to compare your computational method to another, existing method. Like bombing countries in the name of peace, the…
But I got some ground rules I’ve found to be sound rules and you’re not the one I’m exempting. Nonetheless, I confess it’s tempting. – Jenny Toomey sings Franklin Bruno It turns out that I did something a little controversial in last week’s 1 post. As these things always go, it wasn’t the thing I was expecting to get push back from, but rather what I thought was a fairly innocuous scaling of the…