Showing posts with label predictinator. Show all posts
Showing posts with label predictinator. Show all posts

Friday, May 19, 2023

Cinefex: Predictinating the Oscars with Todd Vaziri (from 2014)

illustration by Graham Edwards

update, May 2023: Since Cinefex shut down, this article had been inaccessible. I've resurrected it and reprinted it here.


Predictinating the Oscars with Todd Vaziri

by Graham Edwards, originally published on Cinefex.com, February 4, 2014


As a movie-mad Chicago kid, Todd Vaziri dreamed of being a stuntman. He never did get to ride a horse along the top of a moving train, but he did get to work in the movies – as a visual effects artist.

Todd began his career at Banned From The Ranch, under mentors Van Ling and Casey Cannon. “I got my feet wet in the crazy world of compositing and rotoscoping,” Todd told me, “using a brand new tool at the time called Commotion, which was developed by Scott Squires.”

Todd eventually moved to Industrial Light and Magic, where he’s worked for the past thirteen years in a job he describes as “absolutely a dream come true.” Recently, he was a sequence supervisor on Star Trek Into Darkness, and handled a number of shots on The Lone Ranger. Both films are Oscar-nominated for their visual effects in the 86th Academy Awards, which brings us neatly to Todd’s secret obsession: devising a foolproof method of predicting VFX Oscar winners. It’s called ... The Predictinator!


So, Todd, the Predictinator – what is it, and what does it do?

It’s a formula that my wife and I came up with. Taking the Academy Award for Best Visual Effects nominees, and based on quantifiable criteria, it accurately predicts the winner of the visual effects Oscar.

What inspired you to create it?

It all started out of an attempt to investigate how and why The Golden Compass beat Transformers in 2007. Transformers was very near and dear to my heart – I spent a year of my life on that movie, and thought it deserved to win. But the Academy voters thought otherwise. After many discussions with colleagues, we wondered how we could get inside the heads of the Academy voters. Why is it that some years it’s a slam-dunk, and other years it’s just weird?

My first attempt was to ask, “Which is a better predictor? Critical acclaim or box office?” Looking at the Rotten Tomatoes aggregator, I used its percentage value as my quantifiable gauge of critical acclaim. Then I took domestic box office as a measure of popularity. I can’t remember exactly, but in maybe two out of three cases, critical acclaim was a slightly better predictor than box office.

I wrote a series of articles about that on my blog FXRant. When I showed them to my wife, she said, “This is nice, but it’s kind of fuzzy. You should make a formula. You know what’s happened in the past, so why not craft a single formula to predict which will win, and see how it works in the future.”

So in late 2009, we came up with this formula. Working out the criteria was fun. We included critical acclaim and box office performance, then we reverse engineered it.

What criteria does The Predictinator use to make its prediction?

Well, the full Academy has something like 6,000 members, and most are actors or retired actors – so we ask what do they look for in a film? Oscar season is typically in the Fall, so do some Academy voters have a shorter memory span? How many additional Academy Award nominations did the films get? Is the film a sequel? Looking over the statistics since 1989 (which was when we decided to start the data) we noticed that sequels, even if they had great visual effects, were not generally winners – especially if a previous film in the series was a winner.

After a lot of trial and error, we got many of the previous winners to “win” with this formula, but there were a couple that really stuck out. It was very difficult to come up with additional criteria to make those films win, particularly The Golden Compass over Transformers, and Babe over Apollo 13. So we came up with what I call the “fuzzy creature” question, which asks, “Are the primary effects for that picture organic character animation?” Not robots, not hard surface stuff – creatures. If the answer’s yes, we then ask, “Does it involve facial acting?” A film gets extra points if it fulfils those two pieces of criteria.

The final difficulty was films like Death Becomes Her and What Dreams May Come. They were somewhat critically acclaimed and got modest box office, but didn’t have the hallmarks of other visual effects Oscar winners. We realised that both of those films had lead actors who had won an Oscar before. So we gave points for that, which allowed them to win. We rationalised that the Oscar-winning star power of a lead actor in a visual effects film pushed the Academy voter to support that film.

All that gave us a formula that worked historically from 1989 to 2008. In the four years since, it has correctly predicted Avatar, Inception, Hugo and Life of Pi.

Do you tweak the formula each year, or is it set in stone?

We’re intending to lock it. It was really, really difficult to come up with this one formula, and it was a point of pride that the same formula we developed back then would work into the future. We had to adjust it a little bit when the Academy finally allowed for five nominees instead of three. Also it’s a lot of work to change it. But it’s all working just fine, and we’re very proud of it.

This year, the five nominees for the Academy Award for Best Visual Effects are Gravity, The Hobbit: The Desolation of Smaug, Iron Man 3, The Lone Ranger and Star Trek Into Darkness. The Predictinator predicts that Gravity will win. Some people might say that, as predictions go, that’s a no-brainer. How would you answer that?

You could say it’s a no-brainer, sure. But that’s an emotional statement. Gravity was a big hit, near universally loved by critics, with two extremely likeable stars. It has ten Academy Award nominations. Alfonso Cuarón just won the DGA Award. That’s not to mention the innovative visual effects; the process that created them is unlike any other, and pretty much everyone would agree they were nearly flawlessly executed.

But those are all subjective statements, and my counterpoint would be that the formula breaks all that down to the level of data. I would also add that while the formula works for the “no-brainer” years – like the Avatar year – it also worked for the Hugo year. A lot of folks weren’t picking Hugo with lots of confidence. The good money was on Rise of the Planet of the Apes or the final Harry Potter film. But no, Hugo was picked by The Predictinator ... and it won!

So what’s next? Will you expand your offering to include other awards? Can we look forward to The Predictinator 2.0?

We’ve considered it. I just pitched to my wife the other day: “You know, maybe we could do this for animated features. Maybe there’s a correlation between box office and ...” and she was like, “We have enough on our plate!”

The other categories where it might be possible to do a Predictinator-type treatment would be the “technical” categories: things like editing, cinematography and sound. I personally don’t have any interest in tackling those – it’s a great deal of work. But I applaud and support anybody who wants to go ahead and do this. Let me know and I’d love to help.

How does it feel to have your work showcased in the Academy Awards nominations? Were you even a little bit tempted to skew the results to Predictinate one of your movies into the top slot?

Of course not! This is a thing of science! But seriously, when I wrote my article about The Predictinator’s results, I didn’t want to mention the fact that I had participated in two out of the five pictures. I didn’t want to give even the remotest semblance of skewing the data.

Could you talk about your work on Star Trek Into Darkness and The Lone Ranger, picking a favourite shot from each film and describing its creation?

I was on Star Trek Into Darkness for almost an entire year, in charge of the space jump that Kirk and Khan do between the Enterprise and the Vengeance, and the Enterprise falling towards Earth. I also worked on some of the early Nibiru volcano stuff, and composited the shot of the Enterprise rising out of the water. Lee Uren was the lighter, and he rendered and simmed all of the water for that. It turned out to be a really, really great shot – a really collaborative effort.

(Among the shots composited by Todd were the shots of the Enterprise regaining power and firing her thrusters, triumphantly reversing her headlong plunge into the Earth’s atmosphere.)

It’s a sister shot to one in JJ Abrams’s original Star Trek movie, where the Enterprise emerges from Saturn’s rings. Roger Guyett and Pat Tubach, the visual effects supervisors, were very open to ideas about how the thrusters turn on, and so was JJ. We saw the thrusters very briefly in the original Saturn shot. But there was no atmosphere, and it was zero gravity, so we decided we could diverge from that look if we wanted to.

(Using practical elements alongside CG, the shot includes details like tiny puffs of smoke that precede the actual firing of the thrusters.)

It’s like a dirty chamber being burned up as the rocket fuel comes out, just to give it a sense of reality and scale. It came out pretty well.

(After Star Trek Into Darkness came The Lone Ranger, which saw Todd working under visual effects supervisor Tim Alexander. One of Todd’s shots had the Lone Ranger riding his horse Silver along the top of a moving train. An instant before the train enters a tunnel, our hero spurs his horse in a spectacular leap down on to a flatbed car, narrowly avoiding being smeared against the tunnel entrance. During the shot, the camera tracks behind the stuntman’s shoulder as the horse performs its jump.)

It was actually quite a brilliant shot design that Gore Verbinski and Tim came up with. It started with a full live action stuntman, in costume, on a horse, galloping on a full bluescreen set, shot outside. That made it look very real.

The jump was only two or three feet, but it was enough to get things going. You could feel the horse pull all of its muscles and tense, so we got that true organic motion. Then we transitioned to a fully CG Lone Ranger and horse for the rest of the leap. I had to do a blend morph from the live action to the CG, with nowhere to hide.

I’m so incredibly proud that both projects got nominated for Oscars, particularly The Lone Ranger. Despite the amount and density of the visual effects work, we’re hoping that people aren’t thinking about the visual effects at all – that the effects are truly invisible.

Finally, Todd, it’s time to come clean. We both know The Predictinator isn’t a formula at all. It’s a machine you’ve built in your basement using old household appliances and bootleg body parts. So tell me, does it run on regular unleaded, or is that sucker nuclear?

No, it’s not nuclear! It’s electrical! But I need a nuclear reaction to generate the 1.21 gigawatts of electricity I need. Besides, the stainless steel construction makes the flux dispersal much more smooth. You know that! I know that!

The VFX Predictinator, 86th Academy Awards Edition – detailed breakdown of the results at Todd’s blog, FXRant

Star Trek Into Darkness images copyright © 2013 by Paramount Pictures. All rights reserved. The Lone Ranger images copyright © 2013 by Walt Disney Pictures. Special thanks to Greg Grusby, ILM.

Monday, January 29, 2018

The VFX Predictinator, 90th Academy Awards Edition


What is The VFX Predictinator? Start here.

Even though The Predictinator failed at predicting "Ex Machina" two years ago (probably because we're now in a post-digital era and our assumptions are no longer rock solid), the formula bounced back last year, correctly predicting "The Jungle Book" to win the visual effects Oscar.

We ran the numbers for The VFX Predictinator with the nominees for Best Visual Effects for the 90th Academy Awards, based on data for January 13, 2018. Here are the results, as promised, but without our typical annual, long-winded accompanying article.
  • 5.31 points for “War for the Planet of the Apes"
  • 4.72 points for "Blade Runner 2049"
  • 4.34 points for "Star Wars: The Last Jedi"
  • 4.19 points for "Kong: Skull Island"
  • 3.63 points for "Guardians of the Galaxy, Vol. 2"


APES ON TOP
The Predictinator chose "War for the Planet of the Apes" to win the visual effects Oscar, based on the classic criteria of having strong critical acclaim and the film's prominent organic character animation. It's a solid choice, and a gut-check confirms this.

THE SPOILER?
Following close behind is "Blade Runner 2049", which could upset "Apes", since it is, arguably the most 'artsy' and 'classy' choice for Academy voters. In fact, in a post-digital world, the most classy choice has been winning the visual effects Oscar more frequently ("Ex Machina", "Life of Pi" and "Gravity", for example).

AND THE REST
In third is "The Last Jedi", which earned points for its huge box office, but suffers from being a sequel. Rounding out the list are "Kong: Skull Island" and "Guardians 2", which are strong hits but will probably not resonate with Academy voters.


We’ll see what happens when the 90th Academy Awards take place on March 4, 2018.

update, 3/4/2018: Nope!

update, 1/1/2019: This effort represents the last time we will run The VFX Predictinator. There are two main reasons why we are retiring the formula and this effort. One, that we are clearly living in a post-digital era; the world has changed, and the newness of digital characters is no longer shiny and exciting, which means the assumptions are no longer accurate. I wrote extensively about the post-digital era here. And two.



Monday, February 27, 2017

"The Jungle Book" Wins the Oscar

Congratulations to the entire visual effects team behind "The Jungle Book" for their Academy Award win for Best Visual Effects! (And, yep, The VFX Predictinator got it right.)

THE JUNGLE BOOK
Robert Legato, Adam Valdez, Andrew R. Jones and Dan Lemmon
This is the third Oscar win for Legato ("Hugo" and "Titanic"), the second for Jones ("Avatar"), and the first for Lemmon and Valdez.


Sunday, February 12, 2017

The VFX Predictinator, 89th Academy Awards Edition


What is The VFX Predictinator? Start here.

After bloviating in two giant posts about the reasons we might have incorrectly predicted the Visual Effects Oscar winner last year, we have decided to run the numbers for the 89th Academy Awards, perhaps for the last time:

  • 5.48 points for “The Jungle Book”
  • 4.19 points for “Rogue One: A Star Wars Story”
  • 3.87 points for “Kubo and the Two Strings”
  • 3.17 points for “Deepwater Horizon”
  • 3.09 points for “Doctor Strange”

The charred, withered, barely-breathing remains of The VFX Predictinator predicts that “The Jungle Book” will with the visual effects Oscar in the 89th Academy Awards. Here’s the full set of scores:


Directed by Jon Favreau, “Jungle Book” is a photorealistic live-action film that is nearly fully computer generated. The live-action portions of the film featuring young actor Neel Sethi were filmed on soundstages in Los Angeles. Computer graphics dominate each “Jungle Book” frame in, arguably, the most photoreal, most extensive, and most immersive use of CG environments and characters in a single film. “Jungle Book” advances upon the success of Oscar-winning “Life of Pi” (2012) to a staggering degree. Oh, and all the animals in the film talk, too. (Some even sing.)

“Jungle Book” took the most points in a competitive field this year; only 1.29 points separate “Jungle Book” from its closest competitor, “Rogue One”. Let’s take a look at the Predictinator criteria and how the race worked out for the five films.

A COMPETITIVE SLATE
On a number of fronts, the five Oscar nominated visual effects films are quite competitive. Most significantly, the five films all scored similar Tomatometer ratings from critics. In fact, these five films were the highest rated movies for critical acclaim for the 27 years of Oscar races that we’ve tracked. “Kubo” took home the most points for Critical Acclaim (with its 97% Tomatometer rating), while the lowest rated film was “Deepwater Horizon” with a very strong 83%. While this criteria was competitive, the scores for this category are relative to one another, so “Jungle Book” earned .75 points more than “Deepwater”.

None of the five films took away more than two Oscar nominations. As a result, no film earned any “Academy” points (a film starts earning “Academy” points only if it earns at least four Oscar nominations). Also, none of the five films were penalized for being a sequel; sequels, historically, are shunned at the Academy Awards. As you may have noticed, we did not label “Rogue One” a sequel. Even though the film is part of a cinematic universe, we decided that since the film follows a new set of characters, rather than a returning set of characters going on another adventure, it does not earn the sequel identity.

As an aside, if we continue running the Predictinator past this year, we are considering clarifying this piece of criteria to possibly include “reboots” and “films in an established cinematic universe”, since we feel like those films are also historically shunned at the Academy Awards.

With $364M domestic box office take, “Jungle Book” was the #2 earner among nominees (after “Rogue One”’s $522M), which decimated the box office point values for “Deepwater” and “Kubo”, since we score box office points relative to one another.


HOW JUNGLE BOOK ENDED ON TOP
“Jungle Book” finished with a score 1.29 points higher than its next closest competitor, “Rogue One”. Most significantly, “Jungle Book” earned an important point for its primary visual effects consisting of organic creatures, and an additional .75 points for facial acting. None of the other four visual effects nominees earned any points in either of these two categories. The magnificent organic character animation in the film gives the movie a leg up, since, historically, the full Academy favors films with synthetic character animation and, particularly, animation that features characters that talk and emote.

The digital human work featured in “Rogue One” we deemed as supporting visual effects, not the primary visual effects created for the film. Space battles, environments, and “the world of Star Wars” are the primary vfx work of the film. We gave the same ruling last year to “The Force Awakens”, which featured computer generated characters Maz and Snoke as supporting visual effects elements.


Similarly, we also deemed the visual effects of “Kubo” to not qualify for ‘organic creature work’ as its primary visual effects. “Kubo”’s nomination for visual effects, the first for an animated film since 1993’s “The Nightmare Before Christmas”, is not without controversy. While live-action and animated films share techniques and concepts, their visually-oriented goals are substantially different. The vast majority of live-action films contain visual effects whose goal is to trick audiences into believing the sequences actually happened in front of real-life cameras, next to real human beings. Animated films do not share the same goal of tricking the audience in this way. Nevertheless, “Kubo” is a nominee, and just as we ruled for “Nightmare”, we deemed the primary visual effects of “Kubo” not to be the character animation in the film, but the creation of the entire world of the film. The art direction, creation, animation and assembly of the entire frame is the visual effect.

You may have noticed that we haven’t yet discussed “Doctor Strange”. Two years ago, we added the ‘is the film based on a comic book?’ criteria, which deducts one Predictinator point. “Strange”’s point value sits it solidly in the middle of the pack, just as most comic book movies typically do. To reiterate our reason for adding this criteria: comic book films have, historically, not been rewarded with visual effects Oscars.


FORGET ABOUT NUMBERS: THE GUT CHECK
We recently went into great detail discussing how our formula may be outdated since we may have already entered a new era of visual effects that renders our old assumptions no longer valid. We also qualified our argument with the “fluke year” defense: “Sometimes the 6000+ members of the Academy think differently than is expected; typically, the very next year, they go back to voting the way that is typically predicted.” This is the main reason we decided to run the numbers for this year’s race; if last year was, indeed, a fluke, then our philosophy remains sound.

Generally speaking, we feel pretty good about “Jungle Book” winning the Oscar; it’s the film that our gut tells us will win. It’s a nearly-universally loved film, with groundbreaking, well-executed visual effects. It was a giant hit, and Academy voters will feel good about rewarding this type of film.

That said, my wife and I have nagging concerns. Putting it mildly, last year’s “Ex Machina” win, which destroyed our formula, burned us pretty badly. As a result, we can’t help but look for potential spoilers, and attempt to pre-explain their potential victories.


POTENTIAL SPOILERS
The two potential spoilers of this race are, in our mind, “Deepwater” and “Kubo”. The two films are the least “Hollywood” of the five films, and considering the “Ex Machina” win from last year, we need to pay attention to these two well-regarded films.

“Deepwater” features a man vs. nature narrative (as opposed to the typical protagonist/antagonist structure of the three other nominees), whose visual effects strongly support the narrative, rather than serve the purpose of pure spectacle. Academy voters sometimes shun typically structured films with “good guys/bad guys”, especially when there’s a worthwhile alternative worth rewarding (“Hugo”, “Benjamin Button”, “Gravity” and “Life of Pi” are recent examples that come to mind).

 “Kubo” is a strong contender as well, considering the film represents a magnificent achievement in hand-created stop motion animation (along with a healthy, significant amount of computer graphics and digital compositing). Academy voters could vote for “Kubo” as a protest against computer graphics; “CGI is ruining movies!”, a misguided trope, still has traction in 2017. Even though a predominantly stop-motion animated film like “Kubo” couldn’t exist without modern technology, average Academy voters could reward the film as the anti-CGI nominee.

And, had “Arrival” made it past the bake-off to become a nominee, I’d be shouting from the rooftops that it could easily become this year’s “Ex Machina”. If “Arrival” earned a Visual Effects Oscar nomination, it would have earned nine Oscar nominations, and been a force to be reckoned with. (No, we are NOT going to run the numbers with “Arrival” as a nominee. It’s a lot of work. Feel free to do so on your own.)


SORTA-INTERESTING DETAILS
For only the second time in the last nine years, none of this year’s visual effects nominees also have a Best Picture nomination. In contrast, last year, three out of the five nominees were nominated for Best Picture (“Revenant”, “Mad Max: Fury Road” and “The Martian”). In addition, only one out of the ten films that participated in the visual effects bake-off was ultimately nominated for Best Picture (“Arrival”).

Marvel Studios earned its seventh visual effects Oscar nomination in nine years, with “Doctor Strange”. As a reminder, Marvel Studios debuted nine years ago with “Iron Man” (2008).

We’ll see what happens when the 89th Academy Awards take place on February 26, 2017.

Update: We got it right.


Friday, February 03, 2017

The VFX Predictinator Was Completely Wrong, Part 2

What is The VFX Predictinator? Start here.


After much thought, we believe that the visual effects industry may have entered into a new era of its storied history. The first chapter of the digital era, launched with 1989’s “The Abyss” and its never-before-seen visual effects of the pseudopod, has ended, and a new chapter of visual effects history, the "post-digital" era, may have already begun. The high expectations of stellar-quality visual effects because of digital effects’ democratization and easier accessibility have exploded; a greater-than-ever diversity of films that can harness the power of visual effects is now a reality. If true, this destroys previous assumptions about the predictability of Visual Effects Oscar winners; the "post-digital" era cannot support the same criteria of The Predictinator, which provided accurate predictions of Best Visual Effects Oscar winners from 1989-2014, the digital era.

In 2015, after the 1.0 version of The Predictinator incorrectly predicted “Guardians of the Galaxy” instead of “Interstellar”, we made a slight tweak to the algorithm. We managed to plug the hole in our formula by giving negative points to comic book movies, as we believe Academy voters think films that are based on comic books are less worthy of award. Ultimately, the formula remained accurate through to 1989 (and even strengthened many historical races). In last year’s article, my wife and I pondered a fundamental question, to determine if a larger change is occurring with how the Academy votes for visual effects winners, one that would ultimately break our formula:

• Has there been a slow change in how visual effects films are perceived by the Academy, causing members to vote differently?

We discussed this at length, and I’d like to expand upon a couple of our original ideas, as well as add new thoughts.

Last year, after examining the changes that have been emerging over the last seven years, I wrote this about digital characters: “Audiences now expect Gollum-quality digital characters in their films, and the Academy may no longer reward a film solely for hitting this quality standard.” I now realize that I was probably too specific with this sentiment, limiting the idea to only digital characters. Academy voters and audiences not only expect stellar synthetic characters and creatures, but expect world-class, quality visual effects in every film--not limited solely to synthetic characters, and not solely from big-budget blockbusters like “Titanic”, “Lord of the Rings” or “Pirates of the Caribbean” films.  Expect smaller, more intimate, less spectacle-based films like “Hugo” and “Ex Machina” winning visual effects Oscars in the future, if this is indeed the case.

We may have also underestimated the impact of the visual effects moving from three nominees to five.  Films that would might not have made it past the bake-off with only three slots are now sliding in with five nominees, and sometimes winning the Oscar. Arguably, “Ex Machina” and “Hugo” would not have made it past the bake-off if our category still only was given three nomination slots. (As a reminder, the nominees are decided upon at a bake-off of films, and are chosen by the visual effects branch of the Academy.) 

In fact, the expansion of the Visual Effects Academy Award from three to five nominees in 2010 represents an indication of the industry’s maturation, potentially marking the start of the post-digital era. No longer is the craft of visual effects a second-class citizen; this change was long overdue and richly deserved.

In addition, three years ago, I wrote about the correlation of Visual Effects nominees and Best Picture nominees. With up to ten Best Picture nominees and the bump to five Visual Effects nominees, we foresee more films traditionally ignored in the visual effects race winning Oscars.


Furthermore, we see a trend of Academy voters actively shunning giant, franchise movies. Without the aid of some kind of visual effects breakthrough, we foresee reboots, sequels, Marvel comic book movies, and Star Wars movies not winning Academy Awards as often as they used to. Just look at the last eight visual effects Oscar winners for proof: “Ex Machina”, “Interstellar”, “Gravity”, “Life of Pi”, “Hugo”, “Inception”, “Avatar”, “The Curious Case of Benjamin Button” and “The Golden Compass”.

The first chapter of the digital age of filmmaking might be over, and a new post-digital chapter may have begun. Needless to say, The VFX Predictinator operates within a series of assumptions and quantifiable data to support those assumptions that may no longer be valid in the post-digital era.

In 1989, “The Abyss” marked the dawn of the digital era of Hollywood visual effects: computer graphics and digital compositing supplanted physical models, traditional matte paintings, and optical compositing and ultimately expanded the filmmaking possibilities. The digital era heralded storytelling of a different kind (a liquid metal Terminator, ultra-realistic dinosaurs, bullet time), and audiences have reaped the benefits for over two decades. The ubiquity of digital filmmaking and the maturation of the visual effects community has turned visual effects into a commodity, a value that is no longer exclusive to a certain type of film.


Our entire philosophy behind the predictability of Oscar voters has changed. With five nominees, the nominated movies frequently represent a greater variety of films. No longer are mind-blowing visual effects relegated to a precious few tentpole films, no longer are digital effects relegated to the few directors (Robert Zemeckis, James Cameron, Peter Jackson) who dared swim the dangerous waters of digital technology and turn it into art. Filmmakers like Ang Lee, Martin Scorsese, David Fincher and Alex Garland have access to the world’s talented visual effects artists and technicians, and the landscape of filmmaking is better off because of their access.

I can hear you asking from across the internet: “Yeah yeah yeah, well, are you going to run the numbers?” As you have just read, we’re not entirely convinced The VFX Predictinator, in its current state, is still valid. But we’ll run the numbers. Stay tuned.



Wednesday, February 01, 2017

The VFX Predictinator Was Completely Wrong, Part 1

"Ex Machina" winning the Oscar® for Visual Effects

What is The VFX Predictinator? Start here.

2016 was a year of earth-shattering upsets: the Chicago Cubs won the World Series. Donald J. Trump won the American presidential election. And “Ex Machina” won the Academy Award for Best Visual Effects.

Needless to say, we were shocked by “Ex Machina”’s win at the 88th Academy Awards over here at The VFX Predictinator Headquarters.  This upset didn’t just shake the pillars of the mighty VFX Predictinator, it shattered the entire philosophy into smithereens.

The visual effects category has seen a few upsets in its day. Typically, the award goes to the behemoth of the category (think “Avatar”, “Jurassic Park” and “Forrest Gump”). Most recently, the most memorable upsets include “Hugo”’s win over “Rise of the Planet of the Apes”, “Babe”’s win over “Apollo 13”, and “The Golden Compass”’s win over “Transformers”. As shocking as those upsets were, our VFX Predictinator accurately predicted those races. (Yes, even “Hugo”.) We understand how those films won, since they each earned their victory with just enough points of criteria as dictated by The Predictinator.

We have no idea how “Ex Machina” won. Our formula, which correctly predicted the winner of the Visual Effects Oscar from 1989-2014, was completely wrong.


Its victory upends our philosophy. There is no model to explain what happened. Alex Garland’s film is imaginative, thought-provoking, and beautiful. The quiet, modest and understated visual effects that support the film are of the highest quality; the depiction of Ava (Alicia Vikander) was both technically flawless and aesthetically gorgeous.

I’ll touch on the highlights of last year’s numbers (to read the entire analysis, read The VFX Predictinator, 88th Academy Awards Edition). By all accounts, it was “The Revenant”’s Oscar to win; Alejandro Inarritu’s film was nominated for twelve Oscars (and ultimately won for Director, Actor and Cinematography), and soared with pre-awards Oscar buzz. It was a solid hit, and also earned key points since the primary visual effects for the film consisted of an organic creature. “Mad Max: Fury Road” was close behind in the scores; the film was also a critical darling and ultimately won a staggering six Oscars from its ten nominations. “The Revenant”, “Mad Max: Fury Road” and “The Martian” (another visual effects nominee), were all nominated for Best Picture, as well.

Then there was that universally-loved, gargantuan blockbuster which earned over $2 billion, “The Force Awakens”.



In contrast, “Ex Machina”, while highly respected and a critical darling, was barely on the radar. Yes, it earned two Oscar nominations (Visual Effects and Original Screenplay); in contrast, the other visual effects nominees earned five, seven, ten and twelve nominations. Surprisingly, “Ex Machina” didn’t even earn a single nomination at the Visual Effects Society Awards, which has eight feature film categories. The film earned $25M in the U.S.; the film with the next lowest box office score was “The Revenant”, with $119M.

From a purely statistical point of view, these numbers simply don’t point to a victory for “Ex Machina”. In fact, the film earned the least number of points from The Predictinator of the five nominees. Looking at the scene from a less numbers-driven perspective, the visual effects for “Ex Machina” were beautifully designed and executed, but one could argue the challenges tackled and achieved for the film did not rise to the level of past visual effects Oscar winners, which made its victory surprising to so many industry watchers. Some voices have expressed a similar sentiment for “Babe”, “The Golden Compass” and “Hugo” as well, but those victories are, for the most part, explainable. I have yet to hear a rational explanation of how the Academy’s 6000+ members cast more votes for “Ex Machina” over its competition, probably because I believe(d) so strongly in the philosophy behind our formula.


Let’s look at some of the theories I've heard that attempt to explain the “Ex Machina” win.

THE SPLIT VOTE
This theory states that the front-runners’ votes (in sum totaling a near-majority) were split right down the middle, allowing an underdog to sneak in and grab a victory. Let’s say the front-runners were “Mad Max: Fury Road” and “The Revenant”, and they each earned precisely 32% of the vote. As front-runners, the two films garnering 64% of the vote seems plausible. That leaves “Ex Machina” to swing in with 35% of the vote and win victory, right? But this only works if zero percent of Academy voters voted for “The Force Awakens” or “The Martian”, which isn’t likely.

Okay, so what if “Fury Road”, “Revenant” and “The Force Awakens” split the near-majority vote? For this to work, each of the three films would have to earn precisely 24.9% each, leaving “Ex Machina” with 25.3% of the vote and the win--assuming zero votes for “The Martian”. As a reminder, with five nominees, the absolute minimum percentage of votes required to win is 20.1%.

IT'S AN ACTOR'S MOVIE
This theory is based on the idea that actors, the largest single branch of the Academy membership, prefer to vote for performance-heavy films, typically shunning visual-effects driven blockbusters, even in the ‘technical’ categories such as Editing, Cinematography and Visual Effects. (Reminder, the full Academy membership votes for the Oscar winners, not individual branches.) Typically, this would mean the actors’ branch would shun films like “Transformers”, “Poseidon”, and “The Avengers”, which, as the theory goes, wallow in spectacle at the expense of human characters.


Giving some credence to this theory is the fact that Alicia Vikander, star of “Ex Machina”, won an acting Oscar Academy Award last year--but for a different film. She took home a Best Supporting Actress award for her work on “The Danish Girl”. Was “Ex Machina” riding the buzz and momentum that ultimately awarded Vikander an Oscar? This idea becomes far less likely once one is reminded that “The Revenant” featured megastar Leonardo DiCaprio, the actor who took home a Best Actor role for the very same film. It’s more likely that “The Revenant” (starring a man who already had four Oscar nominations and ten Golden Globe nominations under his belt) would benefit more from the ‘halo effect’ of momentum than “Ex Machina”, starring the relative-newcomer Vikander.

In addition, The Predictinator attempts to account for a bit of ‘actor prestige’, with a piece of criteria dedicated to acting: ‘has the lead actor of a film previously won an acting Academy Award?’

A FLUKE YEAR
This idea takes the point of view that the grand philosophy of The Predicinator is solid, and the Academy voting block, riding an un-predicted massive wave of goodwill toward "Ex Machina", voted for it in overwhelming numbers. There's no denying the visual effects for "Ex Machina" were beautiful and compelling; it rightfully earned its stellar reviews. Similar to Marisa Tomei's acting win for "My Cousin Vinny", or "Shakespeare in Love" winning Best Picture (over "Saving Private Ryan"), sometimes the 6000+ members of the Academy think differently than is expected; typically, the very next year, they go back to voting the way that is typically predicted.

THE PARADIGM HAS CHANGED
Something has changed in the way visual effects are being evaluated, interpreted and understood. The art and science of digital visual effects have matured to such an extent that, perhaps, a new chapter in the history of filmmaking has begun. Our previous assumptions no longer are true, and therefore, our formula has begun to crumble.

In Part 2, we’ll discuss this idea and illustrate how it affected the results of The VFX Predictinator. Here's Part 2.


Tuesday, January 26, 2016

The VFX Predictinator, 88th Academy Awards Edition


What is The VFX Predictinator? Start here.

Okay, enough small talk. Let’s dispense with the pleasantries and fire up The VFX Predicinator 2.0! We plugged the numbers into our wondrous formula, and here are our results!
  • 7.33 points for “The Revenant”
  • 6.56 points for “Mad Max: Fury Road”
  • 5.85 points for “The Martian”
  • 4.33 points for “Star Wars: The Force Awakens”
  • 3.01 points for “Ex Machina”
The VFX Predictinator predicts “The Revenant” will win the Oscar for visual effects in the 88th Academy Awards. Here’s the full breakdown of the how the competition played out:


“The Revenant” is riding a wave of critical acclaim and surprising box office success--surprising for a 2 ½ hour art house film. The film just won Golden Globes for Picture, Actor and Director, and was nominated for more Oscars than any other movie. In a little over two weeks, the movie has earned over $119M in North America and, in its third weekend of released, clawed to the top spot at the weekend box office. At the time of this writing, it has earned $223M globally.

The epicenter of the visual effects of “The Revenant” is The Scene. And if you’ve seen the film, you know exactly what The Scene is.

The bear attack in “The Revenant” is one of the most extraordinary scenes in motion picture history. The sequence is making audiences jump out of their seats and gasp with horror. Similar to the shocking shower scene from “Psycho”, the bear attack is the film’s pivotal event that moves our protagonists and antagonists into action. The attack is visceral, animalistic, honest and immediate, and could only be realized on film with the extraordinary coordination of performance, stunts, makeup, sound effects, and visual effects. It’s the show-stopping scene that everyone who has seen the film is talking about.

None of the other nominees have such a single, defining “oh, wow!” visual moment. The other four films nominated for visual effects have consistent, beautiful visual effects throughout the entirety of the films. While “The Revenant” has far more visual effects than audiences suspect (like creating and extending the film’s wilderness environment with set extensions, bluescreen work, and stitching multiple shots together into a single shot), the bear attack is the cornerstone of the film’s effects. Audiences may logically understand that the bear was created entirely out of pixels, but the extraordinary execution of the sequence allows the magic to take over. The attack feels spectacularly real.

THE REVENANT

THE TOP THREE
“The Revenant” earned a boatload of Predictinator points from its Academy Score; the film earned a monstrous 12 Academy Award nominations (more than any other VFX nominee since “Benjamin Button” in 2008). It also earned a key point for its primary visual effects being organic creatures (the bear attack), but didn’t earn that second point for facial animation, because the bear didn’t talk. It also earned a full point for being a December release.

Interestingly, the top point-earner has the lowest Tomatometer score (81%) of the five nominees--this happened last year, too, when “Interstellar” won the Oscar with the lowest Tomatometer score among VFX nominees. But, like last year, it’s anomalous that the five visual effects films are so highly regarded with critics. Usually, there’s one big critical turd in the bunch (“The Lone Ranger” at 31%, “Transformers 3” at 35%, “The Golden Compass” at 42%, for example).

MAD MAX: FURY ROAD

Closely behind, in second place, is the other strong contender for the visual effects Oscar, “Mad Max: Fury Road”. For most of the year, I thought “Mad Max” would certainly win the Oscar, and its strong score illustrates why it might upset “The Revenant”. George Miller’s fourth entry in the "Max Max" series shocked the world with its energy-bursting, nearly feature-length chase sequence. Stunningly elaborate and beautiful stunt and physical effects work were augmented and extended by computer graphics across the entirety of the film. The car chases simply could not exist without the tight cooperation between the digital and physical effects teams, and the results speak for themselves.

"Mad Max" had the highest Tomatometer score: an astonishing 97%, the highest critical acclaim of a visual effects nominee since “Gravity” (97%) and “Babe” (98%). “Mad Max” earned a boatload of Oscar nominations (10), and also earned a key point for Actor Prestige; Charlize Theron, a lead in the film, has previously earned an Oscar for acting (“Monster”). “Mad Max” was the only film this year to earn an Actor Prestige point. [update: Yes, "The Martian" star Matt Damon has an Oscar, for writing. When designing the formula, we specifically stated the criteria as "has the lead actor won an acting Oscar." So, no points for "The Martian".] However, the action-packed spectacle was penalized for being a sequel, and didn’t earn any points for organic creature animation, since its visual effects were environmental in nature and not character-based.

In third is “The Martian”, which was the #2 box office earner of the bunch, and was tied for second for critical acclaim. Ridley Scott’s crowd-pleasing space drama featured hundreds of effects shots transforming the Jordan location set into the Martian landscape, Martian storms, spacecraft liftoffs, and a dramatic space rescue. The Matt Damon film earned a lot of Oscar nominations (7), and was released late in the year, which helps its chances. Like “Mad Max”, however, the film didn’t earn any crucial creature points, since it didn’t have organic creatures.

STAR WARS: THE FORCE AWAKENS

AND THE REST...
In fourth is the phenomenon “Star Wars: The Force Awakens”. The film contains over 2100 visual effects shots, and required the tight collaboration between digital effects, the brilliant makeup & creature team and the physical effects team. Equal parts spectacle (planets being destroyed by a giant laser beam, the Millennium Falcon flying inside a crashed Star Destroyer) and solemn, understated moments (Rey resting outside her home, a downed AT-AT, the expressive animation of a rolling volleyball robot), the film's visual style has been praised for its aesthetic fidelity and consistency.

"The Force Awakens" has earned 3.5x as much money at the domestic box office as the next nearest film (a whopping $879M when we ran the numbers), giving it 1.04 points in Box Office score. And since we calculate Box Office as a relative value, its nearest competitor (“The Martian”) only earned .27 points. A strong Tomatometer score and 5 Oscar nominations certainly helped the “Star Wars” cause; however, very little else helped the film. It took a double hit on the sequel criteria (being a sequel, and being a sequel to a previous VFX Oscar winner), and didn’t earn any points for organic creatures (Maz and Snoke are not considered the ‘primary’ visual effects of the film), nor did the film earn any Oscar Prestige points (Harrison Ford has been nominated for an Oscar, but has never won). Of course, for pure nostalgia reasons, Academy voters might feel like awarding the film a visual effects Oscar as a token gesture, since the film may not win any other awards. Perhaps voters might want to acknowledge the visual effects behind the behemoth that has earned nearly $2 billion at the box office with a trophy.

EX MACHINA

Finally, the haunting and beautiful film “Ex Machina” rounds out the scores. With its relatively paltry box office take ($25M), it garnered only .03 points in that criteria. With only 2 total Oscar nominations, it didn’t earn any points in Academy Score (we only award Predictinator points to films with 4 nominations or more), and earned the lowest Month of Release score, as it was released in April. The film features flawless and exquisite visual effects design and execution. The work in "Ex Machina" revolves around a humanoid robot (played by Alicia Vikander), featuring a patchwork of human parts and exposed robotic mechanics. Many were surprised that the film made it past the bake-off, considering its modest, non-spectacle-based imagery, but no one should be surprised at its meager Predictinator score.

OTHER OBSERVATIONS
This is the first time in modern history that three nominees for visual effects are also nominated for Best Picture. We’ve been tracking the Predictinator since 1989, and only three years had two nominees for Best Picture in the visual effects category (1995, 2003, 2009). By the way, six out of the last seven VFX winners were also nominated for Best Picture.

Ironically, after we added the Comic Book criteria to The Predictinator 2.0, the alteration to the formula doesn’t make any difference in this, its first year of use. “Ant-Man” and “Avengers 2” would have been the films that would been affected by the minus one point if based on a comic book criteria, but they aren’t nominated for the Oscar.

The resulting point values for each of the films pretty much matches our gut instincts. Before running the numbers, my wife and I agreed that “The Revenant” has clearly earned the most awards-season momentum of any film this year. We think that Academy voters will be swept up in the momentum of “The Revenant”’s Oscar campaign. The movie is still making a lot of money at the box office, and will probably win award after award in the lead up to the Oscars.


POTENTIAL SPOILERS
This is a potentially tricky year for our visual effects prediction; “Mad Max”, “The Martian” and “The Revenant” are all certainly strong competitors in this year’s contest. All three are nominated for Best Picture, so there’s a ton of ‘prestige’ in the visual effects category, which is unusual. Also, “The Revenant” might be dismissed with the reductive thought: “well, that one scene was brilliant, but it was just one scene.”

Ironically, the single biggest stumbling block for “The Revenant” winning a visual effects Oscar might be the filmmakers themselves. While director Alejandro Innaritu and Twentieth Century Fox have been loudly trumpeting the challenges of shooting the film, there has been near-complete radio silence on promoting the visual effects of the film. Dozens upon dozens of articles have been written in The Hollywood Reporter and Variety on Leonardo DiCaprio’s performance in the film, and cinematographer Emmanuel Lubezki’s difficulty of shooting the film with natural light only. Fox even has an entire YouTube playlist of ‘behind the scenes’ videos they’ve produced. Where are the articles and videos about the challenges of creating the visual effects for bear attack in “The Revenant”?

The director is apparently actively blocking any details about the process of creating The Scene, ostensibly to protect the magic tricks that went into creating The Scene. At the time of this writing, only a scant few articles have actually exposed how the bear attack was created with any kind of detail. In December, DiCaprio has said he was "not at liberty" to talk about how the bear scene was accomplished. Soon, a Cinefex article will be published. But will it be too late for Academy voters to understand the scope of the visual effects work? Do Academy members even realize the extent of the visual effects in the film?

Even as late as the Golden Globes awards ceremony, the filmmakers were still actively suppressing information about the film’s visual effects work:
If this active suppression of the film's visual effects techniques continues through the Academy voting period (February 12-23), “The Revenant”’s chances of winning the visual effects Academy Award become severely diminished. But if information about the creation of the bear attack makes its way to Academy voters, we feel the film has a solid chance of taking home the gold.

This is a tricky year; no film is a slam-dunk, or has the 'totally gonna win' vibe of films like "Inception", "Gravity", or "Avatar". While not as wild as the "Hugo" year, this year's competition will once again test the foundations of The Predictinator; it will hopefully confirm our main thesis, that Academy voters typically get swept up in awards-season momentum, and want to reward films that have a (perceived) 'prestigious' reputation.

The Academy Awards ceremony takes place on February 28, 2016.


Wednesday, January 13, 2016

Making The Predictinator Right

What is The VFX Predictinator? Start here.

In the last article, we discussed how The Predictinator got it wrong. Before deciding if we should continue predicting the Oscar winning visual effects film with our current formula, alter it, or (gasp!) abandon it, we examined two questions that might help us make a decision:

• Has there been a slow change in how visual effects films are perceived by the Academy, causing members to vote differently?
• Are we missing something that has been in front of our eyes the whole time?

First, let’s take a look at the potential changes that have occurred over the last decade.


WHAT HAS CHANGED?
In the early days of digital visual effects, only a few filmmakers were confident and comfortable in helming multi-million dollar visual effects blockbusters. Oscar winning visual effects films were dominated by directors like Robert Zemeckis, Peter Jackson, Steven Spielberg, James Cameron and the Wachowski siblings, largely making wondrous, successful crowd-pleasing blockbuster visual effects films.

But as time went on, complicated visual effects became more accessible to a larger pool of filmmakers at greatly reduced costs. In the modern era, visual effects became a tool of art-house, ‘important’ and ‘prestigious’ filmmakers like Martin Scorsese, Ang Lee, and Alfonso Cuaron, filmmakers who regularly make Oscar-caliber films.

Incidentally, all three of those directors have directed Oscar-winning visual effects films within the last decade. And six out of the last seven visual effects Oscar winners were also nominated for Best Picture.

With breakthroughs created for films like “The Abyss”, “Terminator 2” and “Jurassic Park”, audiences (and Academy voters) were amazed by the fidelity of digital creatures, something audiences had literally never seen before.  In fact, the reason The Predictinator’s data set begins in 1989 is because that was the year “The Abyss” was released, which ushered in the new era of digital effects.

The next two decades saw films that had a heavy presence of digital creatures win Oscars. Year after year, films featuring Gollum, Davy Jones, King Kong, a talking pig, and talking polar bears were winning Academy Awards. This is why our formula gives so much weight to two criteria: organic creatures and organic creatures that talk. However, the full Academy seems to be valuing these breakthrough digital creatures less each year, particularly noticeable in the disregard of the two recent “Planet of the Apes” films featuring astoundingly realistic all-digital apes. While audiences still make blockbusters out of films with digital creatures, the Academy seems less likely to award a visual effects Oscar to a film solely for the presence of its quality digital characters.

Audiences now expect Gollum-quality digital characters in their films, and the Academy may no longer reward a film solely for hitting this quality standard. The bar for the “wow” factor has been raised. In fact, four out of the last five Oscar winning visual effects films had no significant digital characters.


WAS THERE SOMETHING ESSENTIAL MISSING?
Our formula couldn’t differentiate between “Interstellar” (a prestige-leaning ‘classy’ Oscar film) and the well-reviewed, popular crowd-pleasing comic book film “Guardians”. Were there characteristics of “Interstellar” that could be quantified to boost its point value? Were there essential characteristics of a film like “Guardians” that could be quantified to reduce its point value? And would these be universal truths, correctly impacting past and future contests?

Like we mentioned in the previous article, “Interstellar” had a respected, auteur director (could we quantify that, somehow?). The film was touted as a heavy ‘practical effects’ film (is there a way to quantify miniatures and physical sets?). Are we hurting ourselves by using the Tomatometer as a gauge for critical acclaim, since crowd-pleasing popcorn films can garner higher Tomatometer scores than prestige pictures like “Interstellar”?

As for “Guardians”, it had a strong comedic bent (how on earth does one assign a point value to humor? I mean, even the esteemed Golden Globes doesn’t have any idea how to classify a comedy. Oh, look, “The Martian” just won the Golden Globe for Best Comedy!). “Guardians” is part of the Marvel Cinematic Universe, but isn’t a sequel (so do we just ding it for being a Marvel film? That seems too specific).

WHAT DO WE DO?
We were stuck.

We were not comfortable with abandoning our little formula, just because it broke once. We would not let it die; we’re too proud of it. And we were not ready to accept that it would occasionally be wrong. That’s no fun at all.

We could have fractured the Predictinator into two distinct time periods, breaking it up into two separate formulas, to account for our perceived changes to the industry we outlined above. That option seemed inelegant; our goal was a single, polished formula to rule them all. We could have tinkered with the criteria and how each piece of data is weighted against one another, but, even with the slightest tweak, we quickly realized the entire formula would fall down like a house of cards. This thing is pretty well-calibrated, and can’t withstand much tinkering without destroying itself.

The only option remaining, we thought, was to add a new magic piece of criteria that would solve our problem, without breaking the rest of the formula. And we wanted to add something that had a universal truth, not a statistical hack. For example, we weren’t going to add the criteria “Does the title of the film include the word ‘Guardians’? Minus one point!”

So we thought about it. For nine months. (Hey, we were busy!)


THE “I’VE GOT IT” MOMENT
One day, my wife was in the kitchen, I was in the living room. She just blurted out, “I’ve got it.”

And it was staring us in the face every time we looked at the full spreadsheet of films from 1989-2014. She had figured out a new piece of easily quantifiable criteria that would simultaneously de-emphasize digital characters as well as bolster the power of a ‘prestige’ film.

“Is the film based on a comic book? Minus one point.”

Duh. It seems so obvious now. Historically, and for the foreseeable future, Academy members simply do not give Oscars to films based on comic books. Peering into their headspace, I suspect they consider these films to be pure popcorn and not worthy of the honor of receiving an Academy Award. We posit that subtracting a Predictinator point for films “based on a comic book” will not only slightly de-emphasize digital characters, but will strengthen point values of prestigious films that frequently win visual effects Academy Awards.

My wife executed a simple masterstroke that not only fixed the “Interstellar” year, but also bolstered the historical data. Since 1989, only one film based on a comic book has won the Oscar. “Spider-Man 2” is the lone victor, with 12 comic book films going home empty handed. Even our most difficult year, 1992, was made easier with this tweak. Instead of “Death Becomes Her” squeaking past “Batman Returns” by a narrow margin, it now crushes the caped crusader with the new criteria. (By the way, the single comic book winner, “Spider-Man 2”, still commands its victory over its next contender, “Harry Potter 3”.)

Here is what the data looks like. Again, the only thing that we changed to the formula is a single line item, “Is the film based on a comic book?” If so, one point gets deducted. No other criteria values were changed. “Interstellar” now wins the Academy Award for visual effects. Presenting The VFX Predictinator 2.0.

(((RT Score/ Sum of all noms' RT Score) X 5)^2) + (BO (millions)/ BO Total of all noms) + (Academy Noms (only if 4 or more) X .25) + (((Month of Release / Total Month of Release) X 2.5)^2)* + (Sequel = -.5) + (Prior Sequel won Oscar = -1) + (Primary FX organic creatures = 1) + (Primary organic creatures include facial acting = .75) + (Lead Actor an Academy Award Winner = 1) + (Film based on comic book = x(-1)) = Final VFX Predictinator Score

*value has an upper limit of 1




Looking toward the future, the frequency of comic book films is increasing. Marvel films are a cinematic juggernaut- the twelve Marvel movies have grossed over $9 billion worldwide, and every movie studio is building their own universes, flooding the future movie market. The vast majority of these films are aimed at young audiences, consistently hitting explosive levels of risk-averse, family-friendly PG-13 mayhem. These films will continue to earn visual effects Academy Award nominations, because the best visual effects facilities in the world are creating outrageous, spectacles and otherworldly characters for them. The films will continue to perform well at the visual effects bake-offs, and will continue to earn Oscar nominations.

Prestige pictures, however, they are not, and I think the Academy will continue to resist rewarding comic book films.

We’re going to stick with this, The Predictinator 2.0, for a while. Well, at least for this year.

The nominations for the 88th Academy Awards will be announced Thursday, January 14, 2016. We will run the numbers through The VFX Predictinator 2.0 sometime after.