Generally, I am an opponent of using AI in coding. I believe in such way you are losing the ability to think in terms of code, architecture, and logic. People overestimate their ability for knowledge retention; you cannot retain stuff you are not using daily. So I am big opponent against AI usage in code and pretty much everything that involves human brain.
But then… legacy code. It is like haunted house. Nobody wants to go in, but sometimes you have no choice. And when my team suggested using AI to fix it, I realized we might be making a deal with the devil.
A couple weeks ago, I was consulting a company. There was this legacy piece supported by a new team. The original team who wrote the functionality? Long gone. Imagine an admin panel built when Obama was still president - AngularJS 1.x, a backend equally ancient, a spaghetti-code monster with zero, or maybe useless, documentation. It is really difficult to find professionals for these technologies today; they have all switched to React or something newer.
The new team is mostly juniors. No one really knows how the application works, or what the thinking was behind it.
This was classic product problem. A low-value task demanding high-value investment. The choice was bleak. We could either:
Invest heavily: Waste weeks, maybe a month, of an engineer’s time just to understand the application. They would learn basics of a dead framework, make few small changes, and then this knowledge would be thrown away. The outcome is minimal, it does not move the needle.
Do nothing: Leave broken panel as it is, which was causing internal problems.
Then, a third option appeared.
The team proposed using ChatGPT. And it was surprisingly good at this one specific thing: understanding old, forgotten code. AI is like a genie - it’ll grant your wish today, but you might not like the fine print later. Sure, ChatGPT fixed our problem in hours instead of weeks.
The process was extremely cheap. Instead of two weeks of learning, developers could go through technicalities and come up with solutions they could test very fast. They could make the tiny tweaks to functionality needed to make it fulfill goals of today.
For internal tools like admin panels, things not available to everyone outside your company, this is good solution. You do not have to invest lot of time to learn about old thing, unless you want to rewrite it completely.
The choice was clear. We allowed them to use ChatGPT to come up with fast solution to make that internal panel work.
But there is a price. The price you are paying is not obvious.
When you use an AI tool like this, you must understand the trade-off. It is not just about code.
You get speed.
Iterations are fast. Testing is fast. The problem is solved today.
You pay with knowledge.
This is a critical piece. Your people are not learning.
The knowledge that was "acquired" to make the fix was lost about 10 minutes after code was deployed. It does not remain in their hats. They cannot apply this logic to new problems, because they never truly understood the old one. They just followed instructions from machine.
We had this temporary fix. It may hold up for quite some time. But we did not learn anything about functionality. If something bigger breaks in that old panel tomorrow, we are back at start. We have no skills, because we did not invest anything into learning phase.
Working with LLMs is fast, yes. It is cheap, yes. But over time, your engineers will be losing skills.
Let me explain it - you just asked a calculator to solve a math problem. You have the answer, but you did not learn the formula. This is fine for one problem. But if your job is to be a mathematician, you are now worse at your job.
So next time you are staring down a legacy mess, ask yourself: is this a quick patch or a long-term investment? If it is the latter, maybe skip the AI band-aid - your engineers will thank you later.
For the first, AI is a brilliant, cheap tool. For the second, it is a deal with the devil. You get a quick fix today in exchange for fragility and ignorance tomorrow. As Product Manager, you must know which deal you are making.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.