Recent content on Phillip Haeusler
I was curious to see a complete picture of Australia’s timber industry: where the raw materials come from, where they are processed, what they become, and where they eventually end up. So I took a deep dive into the industry and built this interactive atlas. It brings together Australia’s plantation regions, processing facilities and ports, then connects them into a model of how timber flows…
Last October I was the headline speaker - on right after the founders - at the Canva World Tour keynote . We shipped AI video: a workflow that takes you from an idea to a finished video, fast. Standing on that stage to put it in front of the world was a career highlight.
“Noisy gradients are ok, actually.” I’ve noticed something interesting about the nature of product development - it’s a lot like machine learning. In machine learning, training progress comes down to two things: Finding the right direction (gradient) : You take a batch of examples, calculate the gradient to improve, and then move in that direction. Taking frequent steps : You…
Wrote up a piece on orthogonal linear layers, this one is hosted on github pages https://phaeusler.github.io/orthogonal_layers/
Imagine this… It’s Sunday morning, and you’ve turned on the strategy game Civilization 5. A computer game where you build up a nation, turn-by-turn. By some luck, you start in Australia. Surrounded by an ocean that serves as a natural defense, you control almost 29% of the world’s iron ore, 20% of the world’s Aluminum ore, 16% of the world’s gold ore, 25% of the world’s lithium, 28% of the…
A few days ago I saw that @davidad had posted that GPT instruct can’t play tic tac toe very well. So I tried, and it’s true, it’s not so hard to beat. Unsurprisingly, GPT-4 is very good. This got me thinking… What is the smallest transformer that could learn the optimal strategy for tic tac toe? And how could we train this quickly and efficiently? Let’s see……
This week, curiosity got the better of me. I spent quite a few early hours of the morning with an A100… It started when I saw this paper trending on paperswithcode . Their project site is impressive (at first glance). Your browser does not support the video tag. Imagine being able to re-watch entire movies in a different visual style, or change the weather or location of your social media…
Can LLMs be funny? The answer is obvious to AI engineers - yes, but. The problem with GPT (and friends) is that RLHF reinforces the human preference - which is boring and not funny. Comedy has a style, it’s a combination of ideas, there is surprise, contrast, it’s explicit. To make an LLM funny, we need to vibe -shift it from a boring distribution to a funny one. We can do this with…
Ok, so tinystories is a fantastic paper that shows how a small transformer model can be trained to produce coherent stories. Their trick was to carefully curate training data by synthetically generating it (using GPT). It worked! This got me thinking, is data still necessary when training LLMs? Probably not. All we really need are gradients - data is just a easy way to get them, but there are…
AI is here, and it’s been here for a while. Calculators, spreadsheets, your phone. When a machine thinks for you - it’s AI. AI is silicone compute. And now, we have a new kind of AI. A language AI. So what is different? The capability is different. Consider what we think about. We have an aversion to some kinds of thinking and an obsession with other kinds. For example; most of us…
LLMs are intelligence trapped in a box. They don’t remember anything. They can’t interact with the world. They can’t do work autonomously. In isolation, they are a technological marvel. A text processor. A natural language program. A new engine. However, in their current state, they are limited. Our first use case is poke at the box. We ask GPT to summarize text, phrase an email,…
It’s true, I love productivity, but sometimes I can work too much, and it’s a mistake to miss life. We only get one. I also love adventure, and earlier this year that’s exactly what I needed. A lot of changes were going on in my life. I had just turned 30, finished a job in San Francisco, my wife had taken a medical training position back in Melbourne, and it was time to move. I…
A few years ago, I anxiously trekked through a web of bureaucracy to obtain a US working visa. A journey of paperwork, waiting, and anxiety. The goal to work in San Francisco, a city brimming with innovation and opportunity. I was stressed. To make matters worse, COVID (and Trump) upended the immigration processes, stalling consulate interviews and bringing USCIS services to a grinding halt. But I…
Lets try a simple optimization problem. Fitting a circle to 2D data $$ r^2 = x^2 + y^2 $$ The measurement residual is defined as the distance to the circle boundary for each point i $$ f(x, y, r, x_i, y_i) = \sqrt{(x - x_i)^2 + (y - y_i)^2} - r $$ Taking the partial derivatives for each parameter we get $$ \frac{\partial f}{\partial x} = \frac{ x - x_i }{\sqrt{(x-x_i)^2 + (y-y_i)^2}} $$
A few weeks ago I had an idea. It was motivated by what started as the perfect opportunity to catch up on reading - the christmas holidays. Perched on a couch in a shack by the beach I spend countless, uninterrupted, guilt free hours pouring through text. A handful of novels moved across my lap along with a few newspapers (and of course an endless stream of online articles). In the newspapers…
As I sit down to write this article I would like to acknowledge that the land that I reside is the traditional lands for the Wurundjeri people. I respect their spiritual relationship with their country. I acknowledge the Wurundjeri people as the custodians of the Melbourne region and recognise that their culture and heritage is still important today. When I pass by a cemetery, a war memorial, or a…
Builder Staff Machine Learning Engineer & Tech Lead @ Canva - building agents Headline speaker (after the founders) at the Canva World Tour keynote , Oct 2025 - shipping AI video Created Video Highlights at Canva https://x.com/canva/status/1800166205224898584 Background Highly technical career, mostly in C++, full stack, hardware, software, and AI self-driving trucks @ YC Scale-up Embark - Head of…