I’ve been trying to adapt my coding workflow to LLMs for a while. The way I use LLMs has definitely changed over time and I’ve written about it here and here.
In my experience, around late 2025/early 2026, SOTA models crossed some threshold and made a significant leap in coding performance. When Opus 4.5 and then later GPT-5.3-Codex came out, suddenly you could trust the output of LLMs a lot more. Sure they still made some mistakes (and still do) but these were significantly rarer than before. You could expect the code to be right, whereas I felt like before then my expectation was that we’d have to iterate quite a bit.
Over the months following that big leap, I started shipping so much more, especially my own stuff. It’s incredibly easy to build whatever you want now with LLMs. At work, the speedup was a little less dramatic since you have to be careful with production software that’s actually being used by a ton of people, and then there are reviews, etc. But still, LLMs became a huge part of my workflow. I rarely wrote code by hand anymore.
Until, that is, a few weeks ago when suddenly our access to modern LLMs was cut at work. We were told that we could no longer use anything except an in-house harness that routes us to a pool of older models. The most capable of the models is Sonnet 4.5. We can’t select the models: the harness decides which model to route requests to, so it’s hard to tell how often we actually even get to use Sonnet.
Needless to say, this was a bit of a shock and many an argument was had. Regardless, it is what it is and we’ve had to adapt. And this is where that big leap comes into play. Turns out we got used to workflows that are no longer viable. I can no longer have an LLM write a feature. It just can’t.
These LLMs now like to rewrite things like this:
for _, f := foos {
if bar(f) {
return baz(ctx, f)
}
}
to
for i := range foos {
f := foos[i]
if bar(f) {
result := baz(ctx, f)
return result
}
}
I can guess why it’s changing the for variable: it’s trying to protect against
the old Go gotcha in which the reference of the loop variable was kept after the
end of the iteration, but this hasn’t been the case for years and certainly isn’t
an issue on the Go version we require in the codebase. Why the result variable
introduction is beyond me. None of these rewrites were needed or requested, the
harness or the LLM simply thought this has to change.
Even things like code reviews are more challenging. I got used to having LLMs assist in reviewing PRs, but now with these models these reviews are much less useful. I realized I had to stop relying on them after the available models went 0 for 7: 0 actual flaws in 7 findings.
The example that brought painful memories of the past was when one of the models found a critical issue in a code snippet like this (not the actual code):
if req.Foo == nil {
return nil, status.Error(codes.InvalidArgument, "missing foo")
}
bar.Baz = req.Foo.Baz
It reported a critical issue: if req.Foo was nil, then
referencing req.Foo.Baz would cause a nil-pointer dereference panic. Obviously
that is not true here because if req.Foo was nil, the function would have
returned before the dereference. I pointed it out to the LLM which happily responded
“You’re absolutely right…”
I rarely see a modern LLMs miss a validation/initialization right above a use. The other false positives were mostly like this: easy to disprove at a glance. But others would be less obvious to someone not familiar with the system. The claims were clearly wrong to me, but proving them wrong involved some investigation, which I had to do just in case I was wrong.
All of this takes time. Individually, these false positives don’t look like much, but they all add up. And then you have people on the team trusting the LLM reviews and posting their findings in your PRs, and then you need to spend time proving that no, that’s not right. It’s a lot of wasted time for something that’s supposed to accelerate you.
So, while for my personal projects LLMs feel like magic, at work I’m back in time by maybe a year or so. I can no longer trust the output of my LLM. It stopped accelerating me and started slowing me down. I can only safely use LLMs for small and very specific changes that I have to overexplain, review line by line, and iterate.
It’s hard to see how much things have evolved in a year until you have to go back in time.
