It's Our Turn to Disrupt

There’s a promise buried inside almost every vision of the AI future: eventually, these machines will know us really, really well.

They’ll know how we write. What we like. What we don’t like. How we work. What we’ve done before. What we’re trying to accomplish. They’ll remember our preferences, our projects, our conversations, perhaps years of decisions and reactions.

And the better they know us, the better they’ll be able to help us.

It sounds wonderful doesn’t it?

I’m starting to wonder (yes, that’s just me wondering again) if it might also be a problem.

Now don’t get me wrong. I’m not worried about an AI knowing that I hate certain words or prefer a certain kind of writing. There are obvious upsides to not having to explain yourself from scratch every time you open a new conversation.

What I’ve been noodling lately is…what happens when the machine gets REALLY good at knowing you?

Because eventually there’s a strange question hiding inside all that personalization:

What if I don’t want to keep being that person?

We tend to think of personalization as an unqualified good.

Netflix learns what you watch. Spotify learns what you listen to. Amazon learns what you buy. Social platforms learn what keeps you around. The more data they collect, the better they become at predicting what you’re likely to want next.

Generative AI takes this somewhere considerably more intimate.

It doesn’t merely recommend the next movie or song. It participates in your thinking. You bring it an idea and it helps develop it. You bring it a problem and it suggests solutions. You bring it something you’ve written and it learns how you sound.

As memory and personalization improve, it becomes possible to imagine an AI that develops an extraordinarily detailed model of you.

Your preferences. Your patterns. Your habits. Your goals. Your language. Your previous decisions. The ideas you return to. The arguments you find persuasive. The things you tend to reject.

In other words, an increasingly accurate version of the probable you.

And that’s where things get interesting.

Because the probable you and the ACTUAL you aren’t necessarily the same person.

At least I hope not.

Human beings have always been eager to turn one another into models.

“He’d never do that.”

“She’s always been like that.”

“You’re not the kind of person who…”

We do it to ourselves, too.

I’m not creative. I’m terrible with money. I’m an introvert. I’m bad at technology. I’m not a risk-taker. I could never start over at my age.

An observation becomes a pattern. The pattern becomes an identity. The identity becomes a prediction.

Then the prediction quietly starts making decisions for us.

One of the many weird things about being human is that we’re capable of violating these models. Sometimes spectacularly.

People leave careers they spent decades building. They become fascinated by subjects they once found boring. They change political or philosophical convictions they once considered fundamental. They fall in love with things they previously disliked. They discover talents late in life. They start over.

Sometimes we even become unrecognizable to ourselves.

That isn’t a bug in human identity to be fixed.

It may be one of its most important features, and many times it represents our finest hour.

This gets particularly uncomfortable for me because I’ve spent years arguing that we shouldn’t let AI erase the things that make us recognizable.

I still believe that.

But I’m beginning to think there’s an important distinction I haven’t paid enough attention to.

There’s a difference between preserving yourself and preserving a VERSION of yourself.

We talk about getting AI to learn our voice as though our voice were a finished artifact. Something we can capture, describe, upload and protect.

But your voice isn’t supposed to remain frozen.

Neither are you.

The way I write today isn’t the way I wrote twenty years ago. Thank God. The things I care about have changed. The things I’m willing to say have changed. My tolerance for certain kinds of BS has definitely changed.

Sometimes I look at my memories on Facebook and cringe when I see the things I used to post in an attempt to be funny or clever. Things I wouldn’t dream of posting today.

Some of the ideas I once believed were important no longer interest me at all. But things I would have dismissed ten years ago now fascinate me.

That’s just part of being alive.

So what happens when we surround ourselves with increasingly sophisticated systems designed to preserve continuity with who we’ve already been?

The better AI gets at “sounding like you” as it thinks alongside you and anticipates what you’ll find useful, the more important it may become to occasionally do something that doesn’t sound like you at all.

Because authenticity, at the end of the day, has to be malleable or we’d all go insane.

Imagine this technology a few years from now.

Your AI has years of context about you. It remembers thousands of conversations. It knows the projects you abandoned and the ones you finished. It knows which recommendations you followed. It knows your recurring concerns, your goals, your tastes and the language you use when something excites you.

It may be able to make observations about you that are more consistent than your own memory.

“You’ve historically preferred this.”

“You’ve tried something similar three times.”

“This conflicts with what you said six months ago.”

“This direction doesn’t align with the goals you’ve repeatedly identified as important.”

That could be extraordinarily useful.

It could also be extraordinarily limiting.

Because sometimes the correct response to “you’ve never wanted this before” is:

I know.

Sometimes the fact that something contradicts everything that came before is precisely what makes it worth exploring.

The machine may have overwhelming evidence that you are unlikely to become the person you are trying to become.

And it may be right.

Until you do.

We talk constantly about data privacy, and for good reason.

But perhaps there’s another right we’re going to have to think about in an increasingly personalized world:

The right to disrupt AI in order to prevent becoming the inevitable conclusion of our own data.

Your chat history should inform your future. It shouldn’t own it.

The person you’ve been deserves a vote. Not a veto.

There is something profoundly human about disruption: the strange leap, the inexplicable fascination, the abandoned plan, the changed mind, the sudden conviction that the life or business or identity you’ve carefully constructed no longer fits.

These moments look irrational from the standpoint of prediction.

Of course they do.

Prediction requires continuity.

Transformation requires breaking it.

And maybe that means one of the most valuable human capacities in the AI age will be the ability to become temporarily unpredictable.

Not unpredictable for the sake of novelty. Not randomly changing everything because consistency has suddenly become unfashionable.

I mean retaining the ability to surprise the model.

For decades, the word disruption has mostly belonged to technology.

New technology disrupts an industry. A platform disrupts a business model. An innovation disrupts the way we work. Humans are the ones expected to respond, adjust and adapt.

But perhaps there’s another kind of disruption becoming necessary now.

Disrupting the version of ourselves that AI has become very good at expecting.

Trying the thing that makes no sense according to our previous behavior. Following an idea that doesn’t fit neatly with everything else. Questioning a preference simply because it has become a preference. Changing our mind without worrying that the historical record can prove we once believed the opposite.

Maybe even creating something that an AI trained on everything we’ve ever created would never predict came from us.

That possibility excites me, and I truly hope it excites you too.

Because the more powerful prediction becomes, the more valuable genuine surprise may become.

And maybe that’s one thing we should refuse to automate away: our capacity to become someone who could not have been completely inferred based on the person who came before.

The machines are going to get much better at knowing us.

I hope we retain the ability to occasionally make them wrong.

And ourselves.

Here’s to interrupting our regularly scheduled programming.