The 5-Step Debugging Framework I Learned by Watching Anthropics Fable Solve a Bug I Couldn’t

My grandpa used to do this whimsical little trick where he made a coin disappear and then reappear behind my left ear. I was five years old and my whole mind was blown. It felt like magic and I was seriously impressed.

As I grew older I found out that it was all a trick. A sleight of hand. Misdirection. But it was my grandpa so it was still cool.

I had a similar feeling when using Fable for a particular tricky bug I had run into — this was before the model was banned by the US government. It spent about 5 minutes going through the code (not only my code but the library code I was using too) and it eventually found a fix. Just like with my grandpa I was seriously impressed by this and it felt a little like magic. But it turns out it was very far from magic and it was all quite logical.

I broke down the approach into 5 logical steps that you can apply to any bug.

The problem involved the Reactflow library. This is a superb frontend library for rendering out Nodes and Edges. In the app I was working on I was rendering out a bunch of nodes and edges at a click of the button. When I pressed the button again all the nodes and edges disappeared from the Reactflow canvas. It was an unusual bug and I was stumped.

I ultimately debugged it using Anthropics Fable model (before the model was banned by the US government).

I want to share the learnings from this experience as the general approach can be applied to any bug.

If you can’t reproduce the bug consistently, then it makes finding the bug much harder. ˜

This is the first step in any debugging exercise — Reproduce-ability. Don’t even start debugging until you can do this — it will make the next steps infinitely easier.

If a bug seemingly happens randomly then your job is to rule our variances. Once you can recreate the bug through a consistent sequence of steps then 90% of your work is done (usually). You want to make it deterministic because this will really help you isolate the possibilities of which part of the code are causing the issue.

Only start using the debugging tools once you can reproduce the bug consistently.

In this particular instance, once I found that I pressed a certain button twice and the nodes just disappeared then the zoning in on specifics can begin. This is where the debug tools come in useful.

It was a web based app so they come with a whole suite of useful tools in the DevTools window. From inspecting the DOM it was quickly obvious that the nodes were actually still there on the DOM but their visibility had been sit to visibility: hidden — another big breakthrough because now we know that the nodes are not disappearing completely but their styles are being updated. But why?

Keep pulling on the string and see where it leads you.

You’re very much in detective mode by this stage. Narrowing down the scope of potential issues by asking good questions. ‘Why are the styles of the nodes being changed to visibility: hidden ?’

This is where you can follow the code. In this case the flow of data went:

“API → Redux → processedNodes / processedEdges props → ReactFlow internals → DOM”

When digging deeper looking at React DevTools, it could be seen that the edges prop going into the ReactFlow library was there — there were 36 of them but nothing was on the canvas for these edges. Yet again, further progress — there was something going on in the library to cause this!

Compare what works vs what doesn’t.

In this instance when I pressed the button once it was all working fine. When I pressed it again the nodes and edges vanished.

The key insight was when pressing the button the first time all the node IDs were new. However, when I pressed it a second time the IDs were all the same. It was the same IDs but a new array that was triggering this issue to occur.

Compacting the problem into one sentence forces your brain to really understand the root cause.

This point is not only useful in explaining bugs but in all sorts of areas. If you can’t explain something in one sentence then you don’t understand it enough.
By concentrating on condensing all the information from the bug into a succinct sentence you’ve very likely know the source of the issue. In this case all it took was a Google search with this sentence to find an existing issue where someone else had experienced exactly the same issue. There were details in the issue about how to address this specific scenario and with that I was able to solve the bug.

Even though it may seem it sometimes, AI Models are not conjuring up magic to solve bugs. They are going about things in a very logical manner. The combination of the available tools they have at their disposal coupled with the amazing speed at which they can digest and interpret text joins together to make a very powerful and potent tool.

As you can see from the example provided the model did nothing more (or less) than take logical steps to hone in on the real cause of the bug and then systematically address it.

The biggest take away for me though is the speed at which AI Models can now do this. Personally I would have been hesitant to go digging into the Reactflow source code and it would have taken me much longer to reach that point anyway.

On the flip side, prior to AI finding the bug it spent another 5 minutes going down a fruitless rabbit hole so by no means is it a silver bullet. The more I work with AI the more I see it as a partner and not a replacement. It still needs a tight leash but at the same time it can get you out of some tricky situations. It’s very much like a dog on a long walk through some wilderness.