AI Is Not Your Copilot, And It Will Never Be

I was working on a speaking proposal recently. A pitch for a 40 minute presentation where first impressions are made in the first few minutes to draw a connection with your audience. The content wasn’t the problem. I knew the material. I even had the metaphor in mind. What I needed was the storytelling arc. How can I shape the outline so that it piques the audience’s interest and holds them? Where should I introduce the framework and provide relevant examples from not just triumphs but also failures? Could I make a full room of strangers feel something before they understood why.

So I turned to AI as my thought partner and the conversation was energizing, even insightful! We played with the metaphor, tested imagery, explored how specific language would connect to the audience’s lived experiences and how it would make them feel. The AI understood the nuances of what made the metaphor merely clever versus one that would genuinely resonate with humans.

And then I caught myself using a word I want you to notice: “we”

“We” didn’t do anything. I gave instructions. It generated options. I selected, revised, and decided. At no point did it care whether the talk got accepted. At no point did it feel the particular anxiety of putting your ideas in front of people who might say no. At no point was it going to walk onto that stage and feel the lights.

That’s the seduction of the copilot metaphor — it tricks us into thinking that there’s someone else in the cockpit with us and can take over flying the plane at a moment’s notice.

In aviation, the distinction is formal and the stakes are absolute. The captain is the pilot in command and the final authority who is ultimately responsible for the flight. The copilot, formally called the first officer, is also a fully licensed, fully trained pilot. They can land the plane. They can take command if the captain is incapacitated. They share the cognitive load of monitoring instruments, communicating with air traffic control, managing systems failures. If there’s a disagreement, the captain’s decision is final — but both are accountable. Both are responsible.

The relationship between captain and first officer is one of distributed authority with clear hierarchy. They share duties — the first officer flies the plane during certain segments — but the chain of command is never ambiguous. The copilot’s job isn’t to make the captain’s job easier. It’s to make the flight safer. And those are not the same thing.

Microsoft’s AI assistant is formally branded “Copilot” and “copilot” has become a more generic term that is used synonymously for Generative AI assistants. When you first hear it, the metaphor feels right. It implies thought partnership. Shared risk. Someone watching your back.

None of that is true.

Here is the question that breaks the analogy: when things go wrong, who answers?

A real copilot carries responsibility. If they fail to catch a warning light, there is a cockpit voice recorder, a review board, a consequence. They can make life-and-death decisions. Their judgment has been forged through thousands of hours of training, simulation, and consequence.

If your AI “copilot” generates a strategy built on a hallucinated statistic, or drafts an email that damages a relationship, or produces code with a subtle flaw that ships to production — who is accountable?

AI does not share risk. It does not lose sleep. It does not sit in the meeting where the client says this isn’t what we discussed. It will not explain itself to your board, your team, or your family. It has no reputation to protect and no career to lose. It doesn’t care if you fail or succeed. It will be exactly as confident generating the right answer as the wrong one, because confidence and correctness are, for AI, completely unrelated experiences.

This isn’t a flaw in current AI that some future version will fix. It’s a structural reality. You cannot have a partner who bears no cost. That’s not a partnership. That’s a tool with good marketing.

When you believe you have a copilot, you relax in ways you shouldn’t. You distribute trust to something that hasn’t earned it and can’t honor any commitments. You begin to skip the verification you would have done if you understood you were alone. Not because you’re lazy but because the metaphor told you someone else was watching.

I see this with the leaders I coach. They start using AI as a sounding board and gradually stop pressure-testing the output. The suggestions sound reasonable. The language is polished. The structure makes sense. And slowly, almost imperceptibly, the leader’s own judgment atrophies because the output is plausible enough that the hard work of independent evaluation starts to feel redundant. A Stanford software engineer at a Fortune 50 company told me that her peers are turning in code that they haven’t fully reviewed because it seems “good enough” and if it breaks something, it’s just part of the Silicon Valley culture of “move fast and break things.” What can possibly go wrong that can’t just be reverted back to its previous state?

This is the copilot’s real cost: it doesn’t make you wrong. It makes you uncritical. And in leadership, the moment you stop being your own toughest questioner is the moment you start making decisions that no one around you including your AI will challenge.

AI is not your copilot. It is your instrument panel and at best your plane on “auto-pilot.”

An instrument panel gives you data you couldn’t gather as quickly on your own. It extends your perception. It shows you patterns in terrain you haven’t yet crossed. It can highlight blind spots, automate tasks, and even spark creativity. But no pilot has ever confused the instrument panel with the person sitting next to them. The panel doesn’t have judgment. It doesn’t adapt to context it wasn’t designed for. It doesn’t say something feels off which, in aviation and in leadership, is often the observation that saves everything.

On January 15, 2009, Captain Chesley “Sully” Sullenberger and First Officer Jeffrey Skiles were flying US Airways Flight 1549 out of LaGuardia when the plane struck a flock of Canada geese just minutes after takeoff, destroying both engines. At 3,200 feet, with no thrust and no time, Sullenberger took the controls and made a decision no algorithm had been programmed for: he would land the Airbus A320 on the Hudson River. He had 208 seconds. In that span, he had to calm himself through the startle response, impose a problem-solving framework on a scenario he had never trained for, choose the most important actions and ignore everything else, and execute a water landing so precise that raising the nose a fraction of a second too early or too late could have killed everyone on board. All 155 people survived.

In an interview years later in 2015, Sullenberger was asked how to prevent the erosion of human skills as technology gets better. His answer is even more relevant in the Age of AI. An excerpt from the SHM interview with Bob Wachter and Sully Sullenberger

SS: …I tend to favor having the pilot directly and completely in control of the airplane. The downside of that is that every pilot has to be trained well, be highly experienced, and have a deep understanding of airplanes and how they work.

BW: But you had that experience because you had a whole career before all of this technology. How does a young person get that? And how do we prevent the erosion of skills as the technology gets better?

SS: As we use the technology more and more – and we’re encouraged to do so by our airlines because it’s so efficient – then we get the sense that it’s almost infallible. And, because we haven’t done much manual handling of the airplane, we lose confidence in our ability to manually control the airplane as well as the automation can. That sometimes makes some pilots reluctant to quickly and effectively intervene when they see things going wrong or when the automation isn’t doing what they expect.

BW: Is the solution forcing them to fly manually more of the time?

SS: Yes. We have to design our systems to require our engagement. We cannot design a system that’s so hands off that we are simply required to sit there and watch it for 14 hours. That’s simply not going to work

The warning for us remains the same. What saved 155 lives that January afternoon was not a system. It was a human being who had spent a lifetime accumulating judgment through consequence. Someone whose attention sharpened under pressure because real things were at stake, who could synthesize decades of experience and intuitively know what to do in a situation no one had anticipated. That is what a copilot is. That is what AI will never be.

AI is a brilliant assistant and can augment our thinking. It can highlight blind spots, automate tasks, and even spark creativity. But it’s not a partner in responsibility. If you crash, you crash alone. So use AI, but never abdicate your seat in the cockpit.

In the end, you’re the captain—AI will never be, not even your co-pilot.