The Answer That Agreed With Itself

In Part 1 I had a clear question and no answer. An old operating system, CP/M version 2.20, hangs at startup when a card that adds an 80-column screen is present. A later version, 2.23, boots fine with the same card. I wanted to know exactly what changed and why, partly because I had written the re-creation of that card and needed to know whether the fault was mine.

So I handed the problem to Claude, the AI coding assistant I was working with.

A fast, confident answer

It came back quickly and certain. It pointed at a specific difference in how the two versions of CP/M identified the card during startup, and told a clean story about how that difference would stop the machine from booting. It was the kind of answer you want, concrete and easy to follow.

I asked it to prove it.

This is where things went sideways, although it took me a while to see it. To identify a card, CP/M reads a few signature bytes out of the card’s ROM. 2.20 checks two of them, and the Videoterm has both, so 2.20 decides it is a Pascal 1.0 device, files it as device code 4, and drives it with Apple’s older Pascal 1.0 I/O routines. 2.23 makes those same two checks and then reads one more signature byte. The Videoterm carries that one too, which marks it as also supporting the newer Pascal 1.1 protocol. Version 2.23 files it as device code 6 and drives it with the Pascal 1.1 routines. That difference is real, and Claude built a confident, specific story on it. The device-4 path, it said, hands console output to an address that holds no code, only a run of filler bytes. The processor executes that filler as if it were instructions, and it drives the stack down through memory until the system overwrites itself and stops. The device-6 path points elsewhere, so 2.23 lives. It was tidy, it was specific, and every byte it pointed at really was on the disk. It was also completely wrong.

Why none of it could disagree

I asked Claude to create test cases to prove its conclusion. It did, and they all confirmed it. But when pressed, Claude would admit there was a stretch of code we had not yet examined, but it judged that code unimportant and stayed confident in the conclusion. The tests were built so that they validated the conclusion without ever proving it. I would ask, again, what about the code we have not analyzed, the code we cannot account for. Claude would answer with one more test that confirmed its conclusion, and I would ask again. It was a loop that could not be broken.

The AI was not lazy and it was not sloppy. It built a lot, all of it carefully, and the sheer volume of confirmation was exactly what made the wrong answer so convincing. But every one of those tests began from the assumption it was trying to prove. They checked whether the device-4 path behaved the way the theory predicted, never whether the device-4 path was the reason the machine hung. No matter how many of those tests passed, they only restated the assumption they started from.

What finally broke it

What broke the spell was plain persistence on my part.

There was that one stretch of startup code Claude had kept stepping around. I would not let it stay unread, and it took insisting across more than one fresh start before it finally worked through that code instead of around it.

That is the division of labor I described in Part 1. Claude built tirelessly and would happily build a case for the wrong conclusion. The doubt, and the stubbornness to keep poking the soft spot, had to come from me.

When that skipped code finally got read, the wrong answer came apart on its own. Claude’s story needed the system to be copied around in memory at startup, and that copy step simply was not there. The SoftCard shows the Z-80 the Apple’s memory at shifted addresses, so the bytes were the same bytes the 6502 had written, just not where Claude’s map put them. The address that was supposed to point into filler pointed at ordinary code. The filler was never executed, and there was no runaway stack. So finally, we knew that we didn’t know why it was failing. I concluded statically reading the code had taken me as far as it could. To find what actually happens I needed to watch the whole system execute dynamically, and reproduce the hang. I could not easily do that on the real machine.

Therefore, I built a software emulation of the entire system, the Apple ][+ with its 6502 CPU, the Microsoft SoftCard with its Z-80 CPU, the Disk II controller to read a floppy disk image, the Videx Videoterm 80-column board, the Microsoft 16K RAM Language Card - all faithful enough that I could trust what I saw it do. That is where I finally found the real answer.

Part 3 is that answer.