The Color of Music

Today, I want to talk about music.

It felt like a disservice to write a 3-part course on AI Filmmaking and not talk about music at all, so let’s get into it. Today’s post will present two exercises that you can do right now to level up the way you use music in your work.

Unsurprisingly, it starts with leveling up the way you think about music.

WARNING: This post is 0% about using AI to generate music. It’s about using music as a tool. If you’re here for prompt crafting or a hot take on Seedance, you’ll probably want to keep scrolling. I’ll see you next week.

Here’s the problem with music: it’s a cheater.

Take any sequence – a montage, or an action sequence, or a dialogue scene – and slap some music on it. It doesn’t matter what genre, or whether the music even fits the scene. You’re always going to find some nice moments of synchronicity: a swell in the cue that matches a dramatic head turn, or a beat drop that lands just as a door slams. These moments can help you “find” the emotion of a scene, but nine times out of 10 they’re a mirage.

Over my years as an editor and a showrunner I’ve been handed countless scenes where professional editors had just slapped a cue under the timeline. No sculpting. No choices. The relationship between the shape of the music and the shape of the scene was wholly inconsequential. And the thing is, if you just look at the cut quickly, those cuts often seem “good enough.”

In the AI world, we have a word for that: SLOP.

And whether you’re making AI films or traditional ones, Slop is the enemy. So today I’m going to talk about how to go deeper with your music choices. Some of this is going to be a bit woo-woo, and some of it may be a bit rudimentary, but I promise you that the techniques I’m writing about today will level up your scoring game.

Way back in the silent film era, the Russian filmmaker Sergei Eisenstein was chasing a vision of “total cinema,” where the visuals and soundtrack were perfectly aligned to create what he thought would be an almost magical power.

He wrote about this extensively in his book The Film Sense. He posited that filmgoers “read” a frame the way they read a book: rigidly proceeding from left to right.

As seen in the below storyboard from his film Alexander Nevsky (scored by Prokofiev), Eisenstein extensively diagramed each frame, charting visual intensity from left to right, then used that as a roadmap for Prokofiev’s score. He aimed to make the soundtrack match the intensity of the visuals precisely.

This approach is as adorable as it is totally bonkers.

With 130+ years of movies behind us, we all know we don’t read a film frame rigidly from left to right. Our eyes wander around the frame wildly, only constrained by the filmmaker’s mastery of composition and flow. But sure, in the early days of cinema Eisenstein’s approach seemed valid.

And in fact, Eisenstein was onto something. Music does have a shape. Identifying that shape and calibrating it to (or against) your visuals, pacing, and performances is the editor’s superpower.

Here’s your first exercise. It’s of the woo-woo variety I warned you about. Let’s listen to some tunes. But not just any tunes – some great ones. “Great” being defined by a fantastic combination of talents coming together on one track:

• A writer with a strong POV.

• An orchestration with a strong POV.

• A performer with a strong POV.

(Shout out to my film school screenwriting professor who taught me this exercise approximately one million years ago.)

The Listening Playlist:

Here’s a link to a Spotify playlist featuring the American standard, “Day In, Day Out” (composed by Rube Bloom with lyrics by Johnny Mercer), performed by three very different artists:

1. Billie Holiday

2. Frank Sinatra

3. Sarah Vaughan

Listen to each track twice in a row.

On Your First Listen: Just feel the song. Understand the lyrics. Get a vibe.

On Your Second Listen: Think about colors and tactile sensations. Does the music make you see sparkles? Dirty violets or bright pastels? What shape does the music take? Is it poking at you with knitting needles or blanketing you in velvet? What about images? Does the drum break call to mind a subway rattling through a platform, or footsteps on dirty concrete? Is the vibe oily, or pristine?

The only wrong answer is a vague answer. Get specific.

Then move on to the next track and repeat.

I think you’ll find that the differences between the three versions are profound. Go do some listening before I spoil your fresh ears with my own takes below…

On Billie Holiday’s version, my inner ear hears desperate longing, and I see velvety purples and dusky reds. This is a woman who is rapt and in the thrall of her lover – addicted, you might say. “Addicted” is the safe-for-work word for what I hear on this track.

Sinatra (predictably enough) is the confident man about town, wrapped in chrome and in control. The images are sharp. Where Billie was singing from her bedroom, Frank is hunting… and he knows he’s not going home empty handed.

Then Sarah Vaughan comes at it from a wildly different angle: irrepressible excitement. Her tempo is way up. Sometimes she’s swept along by the music. Other times the music can just barely keep up with her. I see bright stabs of light, lens flares, gold, and silver.

Each version of this song knocks me on my keister in a very different way. And I would use each one to score a very different scene.

I loaded up that playlist with a couple of other songs – and their covers – that I think are useful:

• “There is a War”: The Leonard Cohen original vs. the Nathaniel Rateliff cover. Notice how the Rateliff track is a conversation, a dialogue. That’s a major shift to my ears that completely changes the track.

• “Gimme Shelter”: The Rolling Stones vs. Merry Clayton’s solo version.

The point of this exercise is to drill down and really hear the music. You don’t need any musical ability to figure it out; it’s all about emotion, and intention. Finding the cue that perfectly aligns with the intention of your scene – or sculpting it to create that alignment – that’s where the magic happens.

Now that you’ve learned to identify what a song is doing, it’s time to put it to work.

Sometimes (though rarely) you’ll play a track from beginning to end under your scene. When you do that, more often than not you’re ceding control… and that’s a recipe for slop. A great song will make it seem like it’s not slop, but that’s a trap. Don’t be seduced.

I like to think of music as a table of contents for the flow of a scene. If the emotions are dragging, what you might need is a cue change, a pause, the breath that music can create. Music can land a moment, it can pull the rug out, it can say “ok we’re done here, what’s next?” Sometimes just a key change at the right moment will make your actor’s internal life become crystal clear. It can save a bad performance.

Some of the best film actors do very little, because they know the power of two things: the close-up and the score.

When I assess a track in the edit bay, I have a checklist of questions I’m looking to answer, ranging from the coldly technical to the heartfelt:

• The Intro: Does it feather in so I can ease it under my scene, or do I need to crash in hard?

• The Outro: Does it fade out slowly or does it resolve in a dramatic fashion?

• The Arc: Does it develop over time? A track with a key or tempo change gives you exponential possible uses. You can place the shift at a moment that underscores or subverts your characters’ hopes. That’s power. We like power.

• The Variance: Does it have a quiet part to counter the loud part? A one-note track will only last so long before you need a new track. You need variance. You are the sculptor here, and you’re only as good as your clay.

• The Lyrics: The tone fits the emotion… but do the words? All the pieces matter.

Let’s assume you’ve found the perfect cue. Now it’s time to think about how to use it.

So how do you learn to control the cue? How do you bend the music to your indomitable will?

Here’s a fun exercise that I’ve used to train junior editors (and sometimes senior ones) on how to exert mastery over music.

Imagine you have a 3-minute, 58-second cue. It’s a great song. Perfect for the vibe of your scene. And let’s say that scene is the main titles of an HBO show. Crucially, the sequence has a hard time limit of exactly 60 seconds. So you’ve got to lose 75% of the cue. But the song is expensive to license, so you want to make sure to get all the bang for your buck by making sure all the best parts are on screen: the intro, the creeping build, the blasting chorus, the attention-grabbing bridge, and the final resolve.

That’s a lot to cram into a minute. But just playing the chorus is a disservice to the artist, the viewer, and the budget. You gotta make the most of it…

The specific track and show I’m thinking of is Jane’s Addiction’s “Superhero” as heard in the opening credits of Entourage.

Don’t you dare look the credit sequence up on YouTube. Don’t do it.

Don’t!

(Not yet.)

Instead: Spend a buck-ninety-nine, download the song, drag it into your editing software, and get to work. You’ve got a hard 60 seconds. Lose 75% of the song while maintaining its general awesomeness. Make it work. Know that if you sell the song short, Perry Farrell will egg your house.

The goal of this exercise is not to perfectly match the HBO version. It’s to identify the key elements of the song, figure out how much screen time they require to be “satisfying” to the ear and the heart, what can be lost, and what absolutely must be kept.

Perhaps you start with the intro and the outro and figure out how to fill in the middle. Perhaps you drop the song in your timeline and whittle down increasingly shorter versions to discover how much can go. Or perhaps you totally rethink the track and do some seriously amazing remixing violence to it.

The only wrong answer is an answer that sucks… or is 61 seconds long.

Once you’re happy with your cutdown – and only then – click on this link to watch the expertly cutdown version that HBO used.

The skills you’re unlocking here will carry you well beyond a simple title sequence. Once you learn how to make four minutes fit into three, you’ll see how much power you have, and what compelling results that power can create. Where do you need the key change to land? Where does the pace of your scene desperately need a boost from the soundtrack? How can you turn a crappy AI performance into something with soul?

Once you experience the power of bending a track to the constraints you place upon it, a world of cinematic control will open up for you.

And the emotional force you can impose on your timeline – and your viewer – will explode.

Share

Thanks for reading. I know a lot of you are extremely talented producers, editors, and filmmakers… I’d love to hear your thoughts on how you use music in the comments.