In Memory We Trust

Hadrian’s Wall as the Memory Wall

Over the past five years, I have been working in AI policy. In that time, the domain has gone from being the subject of esoteric blogging forums and conferences to the centre of international trade agreements. Compute power to train the largest models has increased by at least 1000x. Basic counting errors have been replaced by International Mathematics Olympiad gold medals. Driverless cars cruised from a distant fantasy to Waymo’s rider-only service resulting in 91% fewer serious injury crashes than human benchmarks. Image and video generation went from blurred outputs to redefining art, scams, and disinformation.

The last few years in particular have been a tremendous privilege for me, trying to help the UK get AI right. I joined the Tony Blair Institute in September 2022, during the collapse of the Truss administration, and a few months before ChatGPT was released and the world as we know it permanently changed. Somehow, I had found work at an influential think tank, getting to develop a highly sought-after policy agenda.

In that time, I got to work on a range of high ambition policy projects that aimed to shift the Overton window within the AI policy ecosystem.

I’ve also been very blessed with the chance to work on implementing some of these ideas, as part of my role in delivering the AI Action Plan for DSIT. When done right, public service can be one of the most epic experiences. I hope that more people take up the opportunity to do their own stint inside government.

Throughout these five years, I have always felt like I am in on a secret. In 2020, the secret was that AI was going to be a big deal (though I must admit, my view back then was that it might only be as big as the internet). Today, the world somewhat recognises the importance of the technology. But the real secret still persists.

First, the scale of AI impacts, in my opinion, are still not priced in. Governments, markets, and institutions are not treating AI like the non-linear innovation that it is. The bigger secret still lies in who the future companies and founders that stand a chance of redefining the world are.

These secrets lead me to a conclusion: the way to try and make a mark in the next phase of my career is to join a company that, if they succeed, will transform the world, led by a generational founder.

That is why I am extremely excited to announce that I am joining Fractile.

Share

Fractile is a UK startup building new AI inference chips. They are producing hardware that removes the memory-bandwidth bottleneck that holds back today’s GPUs during inference, so data centres can serve far more tokens per second at much lower latency and cost.

This challenge is heightened in our current constrained resource environment. According to Goldman Sachs, power demand from AI data centres is set to increase by 165% between 2023 and 2030. The scramble for spare gigawatts on any grid is turning the wheels of a new gold rush. AI chips and low energy costs are the new agents of change.

It is to AI model inference – running the trained model to serve users – that the majority of this compute is now going. Inference is of course the monetisable end-goal of all AI development. Today, we tend to measure inference volumes in ‘tokens’ (roughly, a word of text read or generated by an LLM is one token, for instance). Between April 2024 and April 2025, Google 50x’d their tokens processed.

Over the past year, scientific breakthroughs in inference compute scaling have seen methods that benefit from burning through more inference compute to produce more intelligent AI. The dawn of the reasoning models, driven by reinforcement learning methods in language modelling has shifted compute design priorities. When more reasoning happens at inference, or more tree search, we are both processing more inference tokens per AI-driven answer, and care more than ever about the speed to output these tokens. The memory bottleneck in today’s GPUs forces a trade-off between cost and speed, in turn hugely constraining how we serve these reasoning models.

This is where Fractile’s stroke of inspiration emerges. Over the past few years, they’ve made breakthroughs in more closely integrating compute and memory, developing chips that are able to get ~100x the bandwidth to memory as their GPU counterparts. This allows them to satisfy both key inference needs at once: low per-user latency and very high overall throughput.

This design choice stems from a critical assumption: it is memory bandwidth, rather than FLOPs, that is the computational bottleneck to AI progress. Nowadays, advanced AI chips spend most of their time just waiting for data.

If Fractile are right: if the memory wall is the greatest bottleneck to progress, if their approach to in-memory compute ends up being correct, then their hardware could not just provide up to 25 times faster generation and 10 times better tokens per megawatt compared to current leading GPU systems, but a roadmap to more computing power that unlocks an entire new world of AI performance.

It doesn’t take a visionary to realise that this would mark a pretty seismic shift in the chip market, and a transformation to the UK’s position within the AI supply chain.

Fractile emerged from stealth in July 2024 with seed funding (backers include the NATO Innovation Fund, Stan Boland, Kindred VC and Oxford Science Enterprises), later received an ARIA grant, and has won support from angel investors including former Intel CEO Pat Gelsinger and ARM founder Hermann Hauser.

They also have phenomenal leadership in their CEO, Walter Goodwin. I first saw Walter speak in the summer of 2024 when he presented at Sam Cash’s REALTECH Conference, where he explained the early bets that Fractile had made on inference, in particular that the world would discover ‘inference scaling laws’. OpenAI’s o1 model had been released the week prior, firmly validating the company’s bet, and making the case for new inference hardware innovations even more seductive.

After that presentation, I believed Walter to be a really smart guy working on an exciting product. Now I understand the real detail: Walter is a generational founder building one of the UK’s best chances at restoring national prosperity.

It is perhaps no wonder then that multiple VCs have said to me that Fractile is one of a small cadre of UK startups that stand a good chance of becoming a trillion-dollar company.

The role itself is also a new challenge to me. I won’t be in a policy gig, but will be in a ‘growth’ role. This means ‘growth’ of:

• Business development

• Ecosystem and talent

• Brand and communications (which will still include some aspects of policy)

Although I will still be able to use the knowledge I developed in my policy work, this is an opportunity for me to develop a new set of skills and experience, helping to build a company through a range of different special projects.

This choice was a pretty easy one. When an opportunity like this comes along, you do not say no. But the fact that I had begun pursuing the startup path is probably, deep down, a reflection that I am less bullish on my short term ability to impact policy inside government than previously.

I intend on continuing to post here, including writing up on my time in government. I want to say a massive thank you to my TBI colleagues who supported me throughout my time working there. Every week was full of excitement, interesting meetings, and the chance to have impact.