Will ‘Planet of the Apes’ Finally Win a VFX Oscar? If Level of Difficulty Counted Most It Would (Video)

Oscar nominee Joe Letteri tells TheWrap just how hard it was to translate Andy Serkis’ emotion to the screen

Last Updated: February 23, 2018 @ 2:06 PM

It’s been a running discussion as to whether motion capture performances should be nominated for Oscars and other awards. That debate got ever louder when we saw the intense, emotionally wrenching performance from Andy Serkis as Caesar in “War for the Planet of the Apes,” the third and final of the series.

Serkis didn’t get nominated for an acting Oscar this year, but who did was Joe Letteri for Best Achievement in Visual Effects. And while the performance we see on the screen is all Serkis, it was Letteri and his team who were responsible for making sure Caesar reflected all the subtleties and gestures offered up by the actor.

“We talk about performance capture, but that really is just recording the actor’s performance with markers on the body to get what the body movement is and with the video camera to record what the face is doing,” Letteri told TheWrap’s Steve Pond following a screening of “War” at the Dolby Cinema at the Vine in Hollywood Thursday. “What we have to do then is translate that into the character’s performance that you see on screen.”

Letteri is the Senior Visual Effects Supervisor at Weta Digital, and he shares the Oscar nomination for “War” with Daniel Barrett, Dan Lemmon and Joel Whist. He’s won Oscars for the second and third “Lord of the Rings” movies as well as “King Kong” and “Avatar.”

So if you look into the eyes of the “Planet of the Apes” character Bad Ape and see a bit of Gollum, it’s because Letteri was one of the key players behind both mocap characters performed by Serkis. And looking at eyes in particular helped Letteri and his team at Weta Digital capture the realism director Matt Reeves’s film demanded.

“It’s the really small movements that show you the performance. Especially around the eyes, that fine skin around the eyes and eyelids,” Letteri said.

The three “Apes” movies all won the top award at the Visual Effects Society’s VES Awards, and the latest film even won four different VES Awards earlier this month. But the first two “Apes” movies lost the VFX Oscar to “Hugo” and “Interstellar.” It’s an amazing snub considering the groundbreaking work Weta Digital has done on all three films. And Letteri stressed that animating these apes has been no easy task.

“It’s not a one-to-one mechanical performance, because chimps look like us, but they’re not close enough if you really have to do things like dialogue,” Letteri said. “They can very easily look like a big coconut mouth talking. It doesn’t take much for it to fall apart. Those are the kinds of things that we learned over the course of the three films. Animators have to take what we see the actors doing frame by frame and try to judge whether or not we’re getting the same emotional response.”

Letteri would devote countless hours watching Serkis’ performance side-by-side with what they were seeing from Caesar. Reeves conveyed so much of the film through long, sobering close-ups, and it was important that everything Serkis was giving was the same as what they were interpreting for the screen.

“You might be seeing a shot where there’s anger here, but there’s also a touch of sadness. I see the anger, but I’m not getting the sadness. And you try and dissect why that’s happening,” Letteri said. “Oh, it’s because Andy has these extra layers of skin around his eyes that Caesar doesn’t have. Well can we get that in there? Well, chimps don’t have that, so maybe we can add a little of it in and do a little with lighting. We’re always trying to come up with ways to solve the problem of getting the emotional beats to come through.”

All told, Letteri said “War for the Planet of the Apes” has little more than 15-20 shots that weren’t in some way digitally enhanced. Beyond even the realism of the apes, Weta Digital designed remarkably sophisticated models to create the forest devastated by an avalanche at the end of the film. They developed custom software and then simulated how a complete forest might grow over the course of 100 years, all within their digital landscape.

“I don’t sculpt, I don’t paint, I don’t do any of those things you would typically think of that are models to get into filmmaking. The only thing I was interested in is how do I make the picture,” Letteri said.

Given all that is possible, it begs the question why a film like “War for the Planet of the Apes” needs actors at all. But Letteri said real life people are still fundamental to the work they do.

“The idea of working with actors to do this is you get that basis,” Letteri said. “You get what actors know how to do, which is to create the emotion and drama in the moment. And our job is to take that and translate it and create the character from that.”

Watch a clip from the Q&A Thursday above.