Semper Infidelis: The Generative AI Quandary

I launched a new newsletter (and accompanying web app)! It’s called The Convergence Report (yes, that’s the name, no, I’m not changing it).

It’s an AI-enabled data aggregator that sweeps the web for news stories about the convergence between the creator economy and traditional media1.

The Convergence Report covers stuff like:

• Branded content deals

• Creator-led productions

• Partnerships between legacy and (new) new media

• And way more…

Subscribing gets you a weekly digest that summarizes the most compelling stories across the world of media & entertainment.

If you don’t want to miss any updates, subscribe here:

On to it!

| fəˈdelədē |

Accuracy in reproducing or representing something; exactness.

late Middle English: from Old French fidelite or Latin fidelitas, from fidelis ‘faithful’, from fides ‘faith’.

I know it’s a bit cheesy of a start for a Substack article.

My bad.

But I promise the dictionary definition of the word is going to come into play. Just hang tight.

Full disclosure: This was meant to be Part 2 of The Gen AI Ick, but I decided it outgrew the original framing, so here we are; a whole new concept.

📼 The fidelity arrow – Every major entertainment technology for the last 100 years has moved in the same direction. Gen AI is the first to reverse course.

🧠 Errors of kind – The difference between a camera making a mistake and AI making a mistake isn’t degree. It’s category. And that distinction changes everything.

🤥 Deception as a feature – One company’s entire business model depends on you not being able to tell what’s real. What happens when deceit IS the product?

🤝 The broken contract – Gen AI video breaks the promise made by advances in entertainment tech over the past century.

The history of entertainment technology has told a story of increasing fidelity as defined above: “Accuracy in recreating something.”

In the 20’s, synchronized sound made movies feel more real. People spoke, words came out of their mouths instead of in interstitial titles.

In the 30’s, color film processing brought film’s visuals closer to how people perceive the world.

IMAX film expanded the visual canvas of cinema, able to capture greater detail than any film technology before it.

From there, the ability to faithfully reproduce reality in cinema went on a binger:

Sound went from mono to stereo to surround sound to Dolby Atmos which reflects spatial accuracy (where sound is coming from).

During the digital revolution, 480P sensors gave way to 720P, which gave way to 1080P, 4k... You can now shoot in 17k if you’re willing to fork up the $25,000 to buy the Blackmagic Ursa Cine 17k.

True: Up until very recently, video was considered inferior to film2 and the push for digital had visual fidelity take a back seat to convenience, but most theatrical releases were shot on film until digital technology really started giving it a run for its money (see RED and Alexa).

As a general rule of thumb, advances in entertainment technology meant that humans were able to capture the real world with greater clarity, detail and precision.

I understand that the T-Rex in Jurassic Park doesn’t accurately reflect reality. If it did, I doubt many of us would leave our houses so as to avoid that spitty acid dinosaur.

But the entire point of the technology was to make the generated images match reality. CGI was (is) a technology invented to make fictional things look the way they might in real life.

The long arrow of this evolution of technology pointed in one direction: Towards fidelity; a faithful recreation of the real world.

Then came Generative AI.

Generative AI (in its current iteration – LVM) doesn’t capture reality.

In fact, it doesn’t capture anything.

The technology itself is predictive, not reflective.

This means that its output is a statistical approximation of what something might look like based on pattern recognition across training data, not real life. That’s the case whether you provide it an image reference in your prompt or not.

Side Note: If you want to learn a little more about the shortcomings of LLMs – which extend to LVMs – check out David William Silva’s excellent post on why LLMs will (probably) not be what gets us to AGI.

It’s also why when you feed it an image of yourself and tell it to reproduce that image exactly, there is a non-zero chance that it will stray wildly from the provided image and give you extra fingers and possibly a nose job.

What I’m pointing to is the distinction between errors of precision and errors of kind.

When a camera overexposes an image or a CGI dinosaur’s skin doesn’t quite catch the light, those are errors of precision. The technology is trying to be faithful and falling short by a matter of degree. At the risk of anthropomorphizing, the technology is trying to match reality but lacks the precision to do so.

Improving the technology means adding precision to the model so its output more closely matches reality.

When Gen AI attempts to render a dancer and they end up with an extra foot or legs that should NOT bend that way, that is an error of kind. The technology is trying to adhere to the images in its training data. Feeding it more training data won’t (necessarily) get it any closer to an accurate depiction of reality.

Improving the technology means adding training data and weighting guardrails so its output more closely matches the weighted average of its training data.

That’s why the GenAI clips you see in your LinkedIn feed (we’re still using LinkedIn, right?) are all cyberpunk vikings and outlandish fight sequences between Matt Damon and Ryan Reynolds or whatever the fuck.

In many cases, these models are better at producing the outlandish than the realistic.

If we plot Generative AI on a chart alongside other technologies used in entertainment, we see that it is the only entertainment-focused technology whose advancement diverges from the predominant arrow of advancement towards accuracy.

In practice, Gen AI is closer to the phenomenon of confabulation – when a brain produces false memories it believes to be real – than creation or generation.

Jen Topping discussed a company called Arcads AI in a recent post .

Their pitch (according to their website): “Create Winning Ads With AI.”

Their business model is predicated – according to Jen’s post – on the following use cases:

• “AI street interviews (you can’t tell they’re fake)”

• “Podcast-style clips that look 100% real”

• “AI unboxing videos”

• “Viral food videos with zero kitchen”

I would say it’s fair to call this “artificial” or “synthetic influencer marketing.”

The original value prop of influencer marketing is pretty simple.

As an example: A beauty influencer – who has spent 5 years building up the trust of an audience of 2 million followers – partners with a sunscreen brand who is launching a new product.

They give her a sample of their new sunscreen and pay her $15,000 to tout the benefits of the product to her trusting followers. She confers trust. They confer money.

Everyone wins (ideally).

This model is based on trust and fidelity.

A real person with a real, trusting audience stakes their real reputation on this real product, and in exchange, they are paid a real sum of money.

The AI-centric model does exactly the opposite. A fake GenAI character whose following is of questionable value (many might be bots 🤖) touts the benefits of a sunscreen it has never used, will never need, and MAY NOT EVEN BE A REAL PRODUCT.

This is worse than a company acting in bad faith. This is a company whose entire business model is based on fooling people.

It’s as if the deception itself were hard-coded into the company’s mission statement.

And I think it’s going to backfire.

Because humans don’t love to be lied to.

This is evident in the emergence of new laws – notably 2 out of New York State – that require people to disclose when synthetic humans are used in advertising. There’s no reason to doubt this will extend to influencer marketing in an attempt to curb practices that are – by their very design – deceptive.

New laws aside, when the core capability is fabrication rather than capture, the business models that grow from it are naturally oriented toward deceit. Not necessarily because the builders are bad guys, but because that’s what the tech is good at.

Arcads AI isn’t malfunctioning. It’s working exactly as designed.

And that’s the problem.

Previous entertainment tech was almost universally embraced by creatives. Of course there were holdouts who railed against CGI and digital cinema capture, but they proved to be the exception rather than the rule, even in the early days of the tech’s emergence.

AI is different. The reaction by creatives has been overwhelmingly negative, and the response from the general public hasn’t been much better, with 63% of Americans saying products and services using AI make them nervous, vs. 40% saying it makes them excited3.

I think part of this response is due to the fidelity framework discussed above.

Creatives have always been in the business of capturing or expressing truth – even if that truth is fictional.

A technology that moves creatives further away from that truth, from that trust, from that fidelity will remain anathema to the creative process, and – I expect – to audiences who value truth.

Every prior innovation has said to creatives: “Here’s a more powerful tool to help you capture something real.”

Generative AI video says: “Reality is irrelevant.”

The other definition of fidelity that I didn’t include above:

Fidelity (noun) Faithfulness to a person, cause, or belief; loyalty.

Turns out these two definitions aren’t as separate as they look. A technology that isn’t faithful to reality isn’t faithful to the people who’ve spent their careers trying to capture it, either.

Semper fidelis – always faithful.

That’s been the unspoken contract between entertainment technology and the people who have used it for over a century.

Generative AI is the first technology to break it.

Stay faithful.

And stay tuned,

Jon