Trust, Safety, and the Black Box

Let’s talk about the elephant in the room when it comes to using AI to write code: TRUST.

There’s been plenty of debate around what AI might mean for developers—questions about job loss, skill erosion, and the philosophical implications of handing over the keyboard to a machine. But honestly, those aren’t the things that keep me up at night. The real concerns? Security risks. Code quality. Scalability. These are the practical issues that surface the moment you stop marveling at what AI can do and start looking closely at what it actually did.

Because if no one’s checking, you don’t really know what you’re shipping. And that’s a problem.

Working with tools like Replit has been genuinely exciting. You give it a prompt, and it doesn’t just spit out code—it sings. It’s like having an extremely optimistic developer on your team who never says no, never gets tired, and delivers full-stack applications in record time. It’s fun. It’s fast. It’s honestly kind of addictive.

But then comes the hard part: testing.

You start finding bugs—sometimes minor, sometimes serious—and realize the AI had no idea they were even there. I’ve tried having it set up automated test suites, but the results have been inconsistent. What’s clear is that you still need to use real developer tools to investigate what's going wrong. The AI is enthusiastic—it’ll gladly help you identify the bug, even suggest a fix. But whether that fix actually worked or quietly broke something else? It has no clue. That burden is still on you.

And when you're not carefully reviewing the code it generates—especially under pressure or in fast-moving sprints—technical debt can pile up quickly. AI is great at producing working solutions on the surface, but beneath that, it may be duplicating logic, skipping validation, or introducing subtle architectural inefficiencies. If you're not keeping a close eye on what’s being added to your codebase, you can easily find yourself buried in fragile, unmaintainable code down the line. Quick wins now can become major problems later.

What I’ve started doing in my own workflow is splitting responsibilities. We use Replit to generate the frontend—it’s great at mocking up interfaces quickly, scaffolding components, and even setting up data flows. But the backend? That’s entirely handled by human developers. It’s a completely separate engine, protected by strict API permissions and security protocols. That way, even if the frontend has vulnerabilities (and it probably does), they’re not going to compromise the core logic or data layers of the application.

This kind of architectural separation—where the AI-built layers can’t directly affect the more sensitive systems—isn’t just smart. It’s necessary.

Because here’s the thing: AI is convincing. It presents its work with confidence. It organizes your code, adds schema tables on the fly, structures everything neatly, and makes it all look beautifully intentional. But it doesn’t know when it’s wrong. And unless you test it extensively—every time, even after small aesthetic changes—you’re at risk of breaking things without realizing it.

One example: I made a simple design tweak recently—purely visual, just a layout shift—and it unexpectedly impacted functionality. That’s the kind of thing a human would catch. But the AI didn’t. And unless you're watching closely, you might not either.

So yes, it’s less work to use AI as a development partner. But it’s more work to ensure the safety and correctness of the final product. You can’t outsource responsibility. Not yet.

That said, I do think this will evolve. As the models improve, as context-awareness deepens, and as integrated validation layers become more robust, AI will get better at understanding the intent behind changes and adjusting accordingly. Eventually, I think we’ll reach a point where the abstraction layer is strong enough—trustworthy enough—that you can rely on it to build fully functional applications for most use cases.

We’re not there yet. But that’s exactly why I’m experimenting with this now: to see where the current limits are. To push the envelope. Because if we don’t explore those edges ourselves, we’ll never really understand what’s safe, what’s risky, and what’s possible.

AI might be the most powerful junior dev you’ve ever worked with. But for now, it still needs a lead engineer—you—to double-check the work and make the calls.

-moki