Jumping in

When I was a kid my parents decided I should learn to swim. So they splashed out for swimming lessons (sorry). In those days the approach was to spend 30 minutes explaining theory at the shallow end of the pool. Then we’d be made to climb out of the pool, walk to the deep end and instructed to jump in it - to see if we’d learnt anything.

I vividly remember sinking like a stone, only to be rescued by a large pole which I’d grab - and then be pulled back to the surface. It wasn’t the most enjoyable experience.

But, one day, I discovered I could swim. And suddenly jumping in became fun rather than terrifying.

Caution, watersplash

Last week I got the chance to take in a LeadDev panel discussion about code review. It’s always interesting to meet new people and learn what they’ve been up to - and this was no different.

But as we talked it reminded me how cautious many people are on adopting AI. So many engineers and companies are diligently trying to add it to their existing processes. And they are finding, well, not a lot of success. I was talking to a project manager friend the other day who told me he reckoned AI had helped them deliver a project in more-or-less the same time as if they hadn’t used AI. Another told me how they were pivoting to use Sonnet to save money. Another knew of Fable and Sol but hadn’t used them.

Then there’s the friend who connected Sol to Blender and made a model of a teacup. It was an impressively nice teacup - you can see it above. So they asked it to create a model of a battleship and, well, the inevitable happened. It was a pretty terrible battleship. At which point they concluded AI was rubbish and gave up.

I find this confusing.

These are intelligent, capable people. And yet they are acting, to me at least, strangely. They seem to be trying to learn to swim. But doing it by only dipping their fingers and toes in. Some occasionally submerse a leg. On rare occasions someone will jump in at the deep-end. But then their lungs fill with water, they need rescuing and they decide to never do it again.

I think there are two problems.

First people aren’t greedy enough. You need to set your sights high. You need to have outlandish goals. Take my PC emulator. Earlier this year I was amazed when it was able to run Doom (1993). Achieving that felt incredible - it’s emulating all a PC - CPU, memory, disks, graphics, BIOS, DOS, interrupts, keyboard, mouse. There’s a lot. And all done by Claude and Codex.

But now it runs Colin McRae Rally (1998). This is one of the first proper 3D games. It has a proper 3D graphics engine - texture mapping, rasterization. DirectX 5. Win32 emulation. A 3dfx Voodoo 2. The CPU emulation is much stronger - it’s now as fast as a 1998 CPU. That has been hard - an incredible amount of work has gone into a dynamic recompiler which works on both x86 and ARM native processors. I’ve learnt a lot about how they work. The incredible parallelization in ARM and x86 processors. And my dislike of x86 and its variable length instructions has continued to grow…

But I digress. Here is it running. The first part of the clip is the original 320x240 resolution game. Then from 0:22 it is at 640x480. But better than the original, because the original textures are dynamically enhanced by an AI image upscaler in the background. And there is anti-aliasing to get rid of jaggy lines too (this still needs a bit more work as you can see on the fences at the side of the track).

Under the covers all 3D scenes are a mesh of triangles; the image you see is textures - little images - stretched over this triangle framework. That’s always intrigued me - so I added a mode to see the raw triangles without any textures. You can see that from 0:46 onwards.

My silly little emulator is now better for the games I want to play than anything on the market.

But it’s not just greed that you need. You also need to be used to failure. AI fails all the time. It can’t do what you ask. It does the wrong thing. The thing you ask might be beyond it.

But when it fails you need to think through how you can reframe the problem so that it has a better chance of success. Can you break it up into chunks? Provide a feedback loop?

Take the battleship. Turn the work into a loop. Add feedback: ensure Sol can see the output of its work. Set an ambitious goal; be clear what success looks like. Add adversarial reviews so independent third parties give the worker honest feedback about the quality of their output. And discuss the brief - what do you actually want - upfront. It’s our strange friend Oracle Driven Development (ODD) again - how does the AI agent test what its done against what you want?

Even then it might not work - and you’ll have to iterate. But as you do you learn. And that helps you the next time.

And so?

I’d never have got the emulator to this point if I’d followed the path of adding AI to a conventional software development process. That’s too slow. Instead I set ridiculous challenges and then watched to see if AI could build them. And I was happy to throw stuff away. The core of the emulator - the x86 emulation - has been rewritten at least five times. As the models have improved, we’ve been able to do more and more. Take dynamic recompilation. That was beyond Opus 4.5. And 4.6. And 4.7. But 4.8 cracked it (although I’d like to think my improved skills and knowledge helped too).

Controversially I haven’t looked at the code. I don’t do human code-review (although we do lots of AI code-review). Everything is a black box and I measure the results. I care about the overall high-level architecture. But I don’t go deeper than that. I haven’t needed to.

Yet…

The emulator project isn’t the real world. I don’t have to deliver to deadlines. It doesn’t matter if the models aren’t able to solve a problem yet. I don’t have a team to keep busy. There’s just me, and flexible money and time budgets.

And that’s where the gap lies. Existing organizations can’t have all their engineers throwing themselves in at the deep end. They have commitments to meet. Customers to support. Profits to make. They are not research institutions.

And then it becomes clear: people are not acting strangely. They are acting sensibly. They know they need to learn to swim, have spotted jumping in is very risky, so are doing the best they can. I’m the weird one. Again.

So how do organizations adapt? I’ve no idea. What I’m doing works for me. But it’d be a brave business strategy. Ultimately we need folk to jump in. But we also need to provide them with a pole. That’s what rescued me when I was learning to swim. My AI pole has been incredible flexibility. But in the real world?