Visual effects supervisor Bill Westenhofer came on board Warcraft early to help the director establish a methodology for filling out the fantastical worlds of the film, and their inhabitants.
The feature, which represents Moon and Source Code director Duncan Jones’ leap into major studio fare, is based on Blizzard Entertainment’s franchise of Warcraft video games. Set in the fictional universe of Azeroth, the film, like the games, sees humans battle giant creatures called orcs. Orc actors like Toby Kebbell (who plays leader Durotan) and Anna Galvin (his partner, Draka) conduct themselves entirely via performance capture throughout the film, which was shot in early 2014 and reportedly underwent 20 months of post-production.
Westenhofer has worked in visual effects with studio Rhythm and Hues for over 20 years, and is a two-time Oscar-winner (for The Golden Compass and Life of Pi). MovieMaker spoke to the veteran about making this ambitious game-to-film adaptation—now the highest-grossing film in that category of all time, and one which serves as a showcase for stunningly realized VFX.
Ian Failes, MovieMaker Magazine (MM): In this film we go to a lot of different locations and there are so many complicated characters, many completed in CG. What were some of your initial conversations with Duncan Jones about how all this would be accomplished?
Bill Westenhofer (BW): Well, when I was asked to come on board, it was the first time I’ve had an interview where my addiction to a computer game actually was a dividend. They asked me if I’d played the game, and yes, I’d been playing this thing addictively since it came out in beta test. So I became the technical advisor. Whenever we needed to talk about something I could log on and show Duncan and say, “Well, here’s what Stormwind looks like in the game,” for example.
Duncan’s version of the script was going to be orcs vs. humans, where both sides have protagonists and antagonists, so you needed to relate to the orcs. He knew we were going to need to do motion capture; he wanted to base it on actors and have performances.
But the other thing he said was that he still wanted to try to make this feel as much like a live-action production as possible. He didn’t want to have to do a ton of separate captures and motion control stages, and just shoot blank plates for the orcs and come back and do that as animation later. He wanted to try to make it feel like we’re filming everything together on the set. Which is what we ended up doing, and the success we had with that was due in some small part to being a bit naïve about exactly how hard that was going to be. If I’d known what I know now about how hard it is to set up motion capture on a live-action set with trees, smoke and all that other stuff, I might have been a little more cautious. But we would have missed out on some of the gems that we got from doing what we’re doing.
MM: What kind of sets did you end up building, and how did that still fit in with the motion capture shoot?
BW: The strategy was to build backlots and sound stages where we could build physical things that let everyone engage with them. Even in the sets with orcs, they still had tents, and everything was shamanistic and tribal so it was covered in furs, covered in animal parts, and stuff that is very complicated to create digitally.
We still wanted to do motion capture in that space, so the end result after working with [principal visual effects vendor] Industrial Light & Magic, and motion capture provider Giant, and then another motion capture company called Animatrik in Canada, was that the three of us came together and figured out how to do the motion capture in situ. The biggest example is Elwynn Forest, where we had 120 cameras around the stage, some hidden in trees. But the camera array would allow us up to nine real-time performers to have motion capture suits, and then the A-camera operator filming on an Arri Alexa could, through his viewfinder, see in real-time the people converted into orcs with the proper proportions, and we could see it composited together with the live action while we were filming it.
Anduin Lothar (Travis Fimmel) defends himself against an orc in Warcraft
How that paid off was that the A-camera operator was the first person to really see how successful the technology was. Instead of making a guess with proportions, he got the proper framing and he could be making these tiny free adjustments that a camera operator would do normally—you know, notice head turns and things like that. It was a revelation to him.
And then we also were able to do some fun tricks. If you did several takes with the actors together, if we were happy with the orcs, we could ask them to sit down, and then we could run the capture over again from the camera operator’s viewpoint. We’d get clean plates that were following real action with humans or orcs combined. You could even do coverage, you could go run it again and do close-ups of the action that you just captured already.
Another fun example was several scenes where orcs are inside a tent, and the tents were way too small to work as a true motion capture volume. So we taped out a space outside the tent that was measured to fit exactly. The motion capture performers are outside doing their thing while the camera operator was inside and we just had the software off-set physically, where it would draw the orcs in world space. So to the camera operator they were inside right in front of him, and you could do coverage and film the whole thing.
MM: By the time you were filming there had already been some great facial animation work done for the Planet of the Apes movies, and ILM had also been making Teenage Mutant Ninja Turtles with facial capture cameras and tools. What was the state of play for you on Warcraft in terms of facial capture?
BW: I knew from each of those films, and others I’ve worked on, what we were going to be capable of doing. I had thought, up until that point, that the facial tracking was generally done with only a few little dots and then there’d be a lot of extra animation that would need to be done. So that’s what I was expecting. But then ILM came in and they said they want to put 120 dots on the face. I’m thinking, “Man that’s so overkill; you’re going to be keyframing this anyway; why are we doing this?” And they went and did a test. While we were shooting they brought back the first test. This was the first proof of concept, and it absolutely blew me away.
Not only were they using the facial capture from the head-mounted cameras pretty directly, and had done some animation work around the lips or to slightly enhance a brow movement, say, but for 95 percent of it, it was a straight draw from the motion capture driving the performance, and it was amazing. It literally captured every nuance of the actor’s face. They found a way to track the eyes as well, so they were getting the tiny micro-corrections that eyes do. And it really was as faithful a projection of the actor’s performances as I’ve ever seen. We had the benefit of the fact that the orcs were similar proportions to humans, although much larger, and they were still pretty close to a human face anatomically. That’s work that I think ILM should be very proud of, and it of course still included a lot of extra animation nuances.
MM: That facial capture, was that also always done live as part of the principal photography on set?
BW: Absolutely, we thought that was very important. A performance is a complete thing; it’s not just a body separate from the head. With a few exceptions we really stuck to taking a performance and using the face and the body from the same place. Actually, I thought one other nice by-product of all the virtual production work was that we had no reshoots for data capture purposes. There wasn’t ever a case where we put the stuff together, and put in the scene, and looked at it and said, “Oh, that doesn’t quite fit.”
MM: You’ve mentioned the simulcam [an augmented reality system that utilizes digital cinema cameras and motion capture tracking technology to provide a real-time
composite of live action with CG backgrounds or characters]. Did that mean you had to previsualize everything beforehand so that there would be something to see in the simulcam?
BW: We previs’d all the scenes with [L.A.-based previsualization company] The Third Floor, so we had previs assets, then we actually ran them through ILM. So they sent back a previs-level version of the hero model so it fit proportion-wise, but it was still too detailed for what the motion capture software Motion Builder could handle in real time. We generated the capture on the fly so Duncan could change the shots dramatically, and we had real sets so you didn’t feel like you were beholden to the previs for how we shot the footage after the fact.
Warcraft character posters reveal facial animation. Courtesy of Legendary and Universal Pictures
MM: How did the simulcam and previs impact shot design? Was there a certain approach that Duncan had, or that you had an influence on in terms of keeping virtual cinematography as real as possible?
BW: Duncan likes the Sergio Leone style. That was what he wanted to tailor it after. He wasn’t a fan of the Peter Jackson camera, flying all over the place and coming down. He wanted to ground it in what you felt like you could shoot for real. That held up for a lot of the shots. We’d end up breaking that rule later in some battle shots for some establishing vistas, but for I’d say 98 percent of it is really what physical cameras can do.
MM: What were some of the challenges you had when, say, a live-action actor would be interacting with an orc? How would a typical scene like that be filmed?
BW: Initially we were worried about heights and size, but we were fortunate to find a bunch of performers. We had one particular stunt performer who was 6 foot 10, 300 pounds, and could actually move pretty nimbly and athletically. And then we got a lot of really tall performers who were close enough in height that they got around the eye-line problems. For the fight scenes, we’d be choreographing and the orcs would pretty much be themselves. We had a month of orc training where they had to fight like an orc, and how to move like an orc, and swing like you’ve got tons of mass. We’d assist the humans with wire pulls when they were taken out and smashed by a giant war hammer. But it was a lot of really direct acting between the two.
The exceptions would be if you weren’t physically touching or interacting with someone and it could be simple, in which case we could do several takes with the actors and then let the orcs stand aside so we could do a clean plate.
MM: Once you’d finished with the on-set motion capture, how did cutting the film begin, especially as there wouldn’t have been many finished shots for some time?
BW: Well, the editor was Paul Hirsch, and God bless him, he is one of the most seasoned guys you’re ever going to meet. He cut his teeth on Star Wars. Certainly for almost all the scenes there were guys in the gray tracking suits so you could get a sense of cutting stuff, and there was also the simulcam. We had that saved so in many cases you’d get dailies with a rough version of the orcs in place.
But many other times we had to get postvisualization to put things together, and immediately Third Floor started working on that while we were still shooting. Their post-production process could take the actual motion capture data—we’d select the takes and give it to them, and they’d give us back plates with the orcs in the appropriate place for that. But that was quite a bit of work. The director’s cut was tough because we weren’t able to replace everything in time and there were a lot of guys in pajamas, which does make for a challenging part of the process. If we’re lucky enough to do Warcraft 2 we do have to figure out how to get the simulcam as good as possible so we can use it right off the bat in editorial.
MM: In terms of final animation, what was ILM doing that really helped sell a final, convincing look for the orcs in particular?
BW: It was just complete attention to detail. I mean, when we were making [tiger protagonist] Richard Parker for Life of Pi at Rhythm and Hues, success for that was just studying hours and hours and hours of tiger footage. For the orcs, although they would be completely CG, we still built the armor practically and used it for reference. We thought there would be fidelity in having a seamstress and a costume person put it together so they’d know how things tie together in appropriate way.
So we physically built those things, and we put it on a bodybuilder so he would walk on set in every environment. We as human beings are evolutionarily capable. For many reasons, we can look at something and say that ball’s not travelling through the air quite right, because, say, to catch a ball we know what the pendulum arc’s gonna be. We can judge how much something weighs by looking at how it moves. We can tell a person’s emotions and if someone’s lying by subtle cues in the face. So we’re all experts at it, but it’s those things that if you don’t get right, there’s just something about it that screams fake. Infidelity. You gotta get those right.
MM: I’d heard there were some deleted scenes. Was there anything you can highlight that didn’t make the film?
BW: There’s a good 20 minutes of material that was cut out of the beginning of the film, much to my dismay, and there were some awesome, awesome scenes in there of the orcs walking through their home planet of Draenor, walking by. Hopefully it will make it to a director’s cut, because I thought looked really great.
Bill Westenhofer
MM: You’ve worked on a lot of big visual effects films, but was there anything that surprised you this time around?
BW: Well, for the first time in about nine years, I did get to do some hands-on stuff. There’s a little creature called a murloc which is a fan favorite in the film. For people playing the game, it’s this little low-level character and it’s the first thing that wipes you out in the game. They run off and they get all their friends and they come back to attack you. There weren’t any in the movie and Duncan and I thought, “We really should put a murloc in this.”
It was at a time when we were trying to get everything to fit into a budget, and rather than go to ILM I said, “I think I can do this.” So I got a copy of Blender and After Effects and I got a low-level murloc model from Blizzard, rez’ed it up, textured it, animated it, did the whole thing myself, and it’s in the movie. And people are pointing it out in their reviews, so it makes me happy.
MM: It’s obvious in this film that so much of the world and the characters had to be created with visual effects, and of course so many other big films require the same. Do you think that’s changing the role of a visual effects supervisor at all?
BW: I think sometimes not everyone can appreciate just how much visual effects can bring to a production. They can think of visual effects as just another department, and don’t realize the amount that it can truly add for the story process of the film. It was really rewarding to share that process with Duncan. We both collaborated with each other on ideas and things that go beyond just executing greenscreen. And with visual effects being such a big part of movies these days, it’s time that the endeavor gets the respect that it deserves as much as any cinematographer, production designer or other person on the film. MM
Warcraft opened in theaters on June 10, 2016, courtesy of Universal Pictures.
Unlike many other magazines, MovieMaker is completely independent. We rely on our readers for support to continue providing top-quality filmmaking guidance. If you liked this article, consider tipping us $1.
Share:
