http://www.fxguide.com/featured/a-world-away-john-carter/
Edgar Rice
Burroughs’ early 1900s tales of a fictional Mars (known as Barsoom) have been
brought to the screen with modern day visual effects in John Carter. The film,
directed by Pixar’s Andrew Stanton, sees the titular Civil War character,
played by Taylor Kitsch, transported to the red planet where he meets the nine
feet tall Tharks and all manner of civilizations. Visual effects artists from
Double Negative, Cinesite, MPC, Nvisible, Legacy Effects and Halon
Entertainment all collaborated on the alien world and its inhabitants.
In this
roundtable interview we sit down with the film’s overall VFX supe Peter Chiang,
and Double Negative’s head of animation Eamonn Butler and animation supervisor
Steve Aplin to discuss the Tharks, Thoates, Woola and the white apes. And look
out for more coverage of John Carter in an upcoming fxguidetv with Cinesite
about their work for the mile-long walking city of Zodanga.
Planning
and prep
Original
plate. The actor wore a gray suit and a facial capture camera helmet.
Final shot.
fxg: What
kind of research did you have to do to fill out the fantastical environments
for Mars?
Peter
Chiang (VFX supervisor): Well, we spent two-and-a-half years on the picture –
we started in May ’09 – and Andrew Stanton and a series of concept artists
including Ryan Church and the production designer Nathan Crowley had already
been working with Legacy Effects. They did all the ZBrush sculpts on the
characters and created the worlds. So when we arrived on the project there was
already a great portfolio of artwork. We went to the Viking images of Mars and
referenced rocky terrain and it was used as a springboard. In the end, Andrew
wanted to use real locations in Utah and to build sets that resembled Mars but
didn’t take it into that barren, rocky, geological landscape – he wanted more
interesting architecture of past civilizations.
fxg: One of
the things you brought to the screen was some very humanistic qualities to the
Tharks – how did you plan for them as characters and what proof of concept or
testing did you need to do?
Eamonn
Butler (head of animation): One of the first things we did was a test back in
2009. It was really just diving in and creating a small scene with a live
action actor and a Thark, using our tools as they were and our pipeline as it
was. We were trying to find out what the questions were going to be on the
shoot, instead of trying to solve it in one go. We learnt that the Tharks were
nine feet tall – they’d been reduced from 15 feet in the books – just so we
could actually frame them appropriately with a six foot actor. It was still
quite high, so if we had to frame things a certain way then we ended up doing
things like animating the muscles in the abdomen so when the Tharks talked you
would root the voice there. Up until then, we’d always looked at facial
animation as just from the top of the forehead to the chin, in terms of putting
in controls for the animators. On this film, we decided that because the
characters’ necks are so long and they’re such elegant creatures, that our
facial anim started at the top of the head and went down to the chest height.
The context
for believability on these Tharks was always going to be when you had a real
actor standing right next to one. We had to make sure the Tharks didn’t have a
made-up methodology or biology for how those muscles would fire and how those
faces would work. So we talked to Andrew about maybe re-working some of the
muscles on the Thark design so that they made more sense biologically. We
re-worked the sub-structure and muscle form so that they made sense to our
riggers and subliminally made sense to the audience, too, because we just
didn’t want to have everything be fantastical.
John Carter
meets Tars Tarkas for the first time.
Chiang:
Character-wise, as well, we had it set out from the beginning from Andrew that
these guys were like a tribe of vikings. They like to fight, they’re
aggressive. So we did a number of tests before the shoot to explore that avenue
so we could get an idea and visual language about how these guys moved. We
showed this to the principal actors playing the Tharks – we showed Willem
Dafoe, Samantha Morton, Thomas Haden Church and Polly Walker – who brought so
much to the characters themselves which we also able to key off.
Butler:
Andrew also allowed the animators to use alternate takes. So every take was
made available to us. He would give us his selected take of the actor, but then
the animators could look through other takes and pick out nice little details
or nuances that might make the scene more appealing. The animators really
helped to contribute or augment what the actors had done. The actors were on
stilts so their physical performance was a little bit limited. They were on
stilts because Andrew really wanted the eyelines to be correct in frame. We
always shot with our actors at the right head-height whether they were on
stilts or using a backpack with a head extension or on a platform. Part of the
sell of making the audience believe the Thark was really there was that Taylor
Kitsch was always looking in the right place and connecting with that
performance.
Shooting
and stereo
Original
plate.
Final shot.
fxg: John
Carter was shot on film – what were the things you were doing on set to survey
the light and the scenes?
Chiang: On
set we had to really get the methodology to handle the different settings we
were in. We always started off with image-based lighting and HDRIs for every
scene. When you get it back and start lighting the characters, there’s always
additional things you need to do to enhance certain aspects of the scene. There
might be particular parts of the frame that Andrew wanted to highlight, so
you’d put a soft key on that. We always put additional spotlights and played
around with the ambiance. We also tried to use the clean plates as much as
possible too for less clean-up. With that there’s no characters occluding the
principal characters so we had to put in a lot of shadows and re-lighting.
fxg: What
was the lighting and rendering workflow?
Chiang: We
used Maya, RenderMan, Houdini and Nuke for comp’ing.
fxg:
Although this is a post-conversion film, how did stereo impact on the
animation?
Steve Aplin
(animation supervisor): Traditionally in a VFX movie where you’re adding in
characters in a 2D screening, you can kind of cheat your characters a little
bit in space to get a better silhouette and just to read from camera. But we
knew going in with the stereo that the CG characters had to be in the exact
same space as the on-set performers were. If we didn’t, then the stereo shows
it up, so we couldn’t cheat any poses.
Butler:
What compounded the problem for us was using anamorphic. We were using the
entire plate to compose your characters, which gives you very little room to
re-work the plate to add in a digital extension or a Thark. So we had to be
very careful when we shot the movie to make sure the DP knew exactly where to
look. After the fact, it gave us less wiggle room with augmenting the action or
staging the scene. The stereo and anamorphic didn’t make it impossible, but it
was like wearing soft, furry handcuffs.
Chiang:
When they did decide to do post-dimensionalization on the film, we started to
do a lot of lidar on set to help us respect the 3D space and know the volume.
Scenes that you normally wouldn’t worry about, you started to think about the
left eye a lot more and what needed to be done.
On-set
facial capture
On-set
plate.
Final shot.
fxg: What
was the tech behind the facial motion capture on set?
Aplin: We
had a dual camera set-up so that the principal actors would wear a head harness
that had two NTSC cameras pointing at their faces to capture the left and right
side. We had markers on the actors’ faces, so we could take that stream of
video back to Dneg and we had people here tracking it to go onto a digital
version of the actor. Then we wrote a transferring tool which could move that
over to our Thark face which would give a similar performance to what we had
captured on set. From there we were able to animate over the top or just embellish
as we needed to. We also found that in some of the close-up work we needed a
little bit more fidelity in the face. We wanted to read more of the musculature
underneath the skin, so we came up with this idea of trigger shapes, which
worked for a number of measurements and compressions from frame to frame. It
would fire these pre-made tension shapes of tendons underneath the skin, firing
just before a major shape was dialed in. So if you were going to move the
corner of your mouth back, you’d suddenly see a flex shape happen underneath
the skin. On top of that, we’d run a face skin sim, which would add a little
bit of extra jiggle and tie everything together and make it look very solid
close up.
fxg: Did
you do a separate motion capture shoot for the actors at all, or how did you
approach keyframing them?
Butler: The
actors were always on set and we’d also shoot a clean pass without them there
to give us the freedom later on to add in the Tharks and move them around for
composition reasons. But other than facial capture we didn’t capture the
actors’ motion on set. We were outdoors for most of the shoot and we didn’t
want to get in the way of them performing. Often times, using an optical set-up
can be time consuming. We did place a ton of witness cameras on set so we were
able to track their position if necessary. But most of the time Andrew wanted
to explore the performance the actor gave and augment it slightly or make it
more Thark-like.
Watch some
behind the scenes footage from on set.
We reached
a point where we thought it might be cool for the animators to capture their
own performance and feather that in to what the actor did or use it as
reference – much the same way that traditional animators use mirrors to look at
their faces. So we invested in a Xsens MVN suit and we had a special room at
the studio where animators could put the suit on and act stuff out and have it
on their desk by the afternoon. Then they could select different takes of their
own performance and combine it with what the actors gave us. We ended up using
the suit to generate lots of the library animation that we needed to build up a
crowd system. We produced over 800 cycles of animation for that crowd library
which was more than there were shots in the film that we animated on.
Animators
typically don’t like working with motion capture really, but we empowered them
to use it for themselves to be able to try out different takes and get out all
the bad ideas really quickly and get in the good ideas. And the other thing was
we didn’t want to do say mo-cap of the actors on stilts because it looks just
like that – actors with leg extensions – and we didn’t want to kill our
creative time.
The
animation pipeline
Original
plate.
Final shot.
fxg: This
would have to be one of Dneg’s biggest creature shows – what tools did you use
for the animation, crowds and the creature pipeline?
Butler: The
proof of concept test we did at the beginning highlighted the need to have a
strong rigging pipeline. In the past, rigging has generally been engineering an
envelope to move appropriately and then adding fur or cloth to it. In this
case, we needed something a lot more sophisticated. So we had LOD (levels of
detail) rigs. The lowest level of detail was in our layout phase, the lightest
rig. Then the animators had simple rigs eliminating high detail. But they
needed higher detail for facial. We created a whole new department called CFX,
short for creature effects, and they developed skin sliding, tendons that could
be fired automatically or by hand, a muscle-based PSD system to accurately
represent what muscles do under the skin. We used a lot of displacement detail
with dynamic wrinkles, particularly around the face. We created another
department called Shot-sculpt which were used for extreme shots – say when one
of the characters is being punished. Those poses in the rig literally broke the
rig, but Andrew was very keen to make her look like she was suffering, so we
had to push her into a place that was quite extreme. So Shot-sculpt went in and
re-worked the rig into something that was incredibly muscular and compensate
for the natural limitations you often find in CG rigging. We also developed
some crowd tools to allow us to put up to 5,000 Tharks in one scene, say for
the ape arena. We used a multi-layered system for that.
Aplin:
There was also the whole look development process, right through from the
texturing of the skin to the sub-surface scattering. We did a terrific amount
of research into relating it back to human movement and skin. There’s a
subconscious thing where people perceive it to be more real if they can relate
it to humans. Andrew’s idea of having actors on set was a great one because
that was really where everything started. I think synergy between a martian and
a human was important. For instance, the eyes started off with black scleras
and very contrasty irises, but in the end Andrew went for a more white sclera
and blue irises. So the function of the eyes, all the meniscus, all the bump
and sclera and wet look, the caustics – all needed to develop to resemble a
real human eye to give that emotion. It was all about retaining the real-life
performance of the actors and translate that.
See some of
the animators at work on John Carter.
fxg: The
other large component of course was the extra materials on the Tharks’ bodies –
leather, fur, stones. How were these done?
Chiang:
There were a lot of props – we broke it down into stages. We’d matchmove the
first shot, and there’d be a layer of animation to get the performance Andrew
really wanted. So we had to construct the pipeline in such a way that sim’ing
and all the cloth and creature effects weren’t wasted on shots before they were
final’d. So all the caches for the animation were done at an approved stage,
and that was brilliant for Andrew because he was used to signing off on shots
very early through his animation of feature films. What that meant was we could
be very constructive about how and when we would sim cloth and creature effects
later. But there was a lot of high-level detail put into the props for both the
hero characters and the crowds. In the pipeline we had almost check-boxes which
let you add props and turn on and turn off to whatever the scenes demanded.
Aplin: I’d
say animation was fortunate in that we didn’t have to get involved in the cloth
side of stuff too much. If we got the character into a specific pose that we
knew would be problematic for cloth down the pipe, we had limited controls on
some of the props hanging off and compose them in a way that would help the sim
later on.
Butler:
Some of the costumes were actually made for real. In the beginning, Andrew had
commissioned full-size macquettes of the hero Tharks. The costume designer
actually went and made Tars’ costume with a mantle and so forth, so we had
really good reference to start from.
Original
plate with buggies.
Final
Thoates shot.
fxg: The
Thoates which the Tharks use for transport, were filmed with stand-in buggies.
Can you talk about how they worked on set?
Butler:
Usually animation happens after principal photography – in this case we knew
the Thoates had to be animated before principal in order to educate the
building of a rig that we could use on set. So we did a whole bunch of walks
and trots and runs at different speeds. We extracted the saddle information
from that animation and calculated how much rotation and angles and speed. Then
we passed that information onto (special effects supervisor) Chris Corbould and
(special effects coordinator) Scott Fisher, who went and built four rigs for
use on set – only three were on set at one time. They stripped out all of the
petrol engines and put in electric engines to be able to record sound. We
painted them out and retained just the saddles and added our animation back
underneath.
Chiang:
What was great about that was it went back to what Andrew wanted on set – to
retain the actor’s eyelines and respect the 3D space the creatures would
occupy. Having these great big units on set enabled the actors to really feel
like they were on a beast and convey that – it made the job a lot easier to
frame the shot and get a better composition of what it would look like in the
final film. They could go up to certain speeds – we always knew the Thoate run
would go up to 40 miles per hour.
Butler: We
also did explore potentially using camels on set at one point. Andrew was very
excited about it – we even went and motion captured a camel after making a
special suit for it. It was covered in tracking markers – he’s quite a star in
camel circles now – he looked like a ninja, it was incredible. But he was
deemed to be a little bit risky for our actors on set and they can be slightly
impractical as well. But we ended up taking a lot of cues from a camel’s motion
in the way we animated the Thoates, so it was still really useful.
Man’s best
friend on Mars
Woola's
proxy on set.
fxg: Woola
– the dog-like character – seems to have had a big reaction with the audience.
How did you approach him and how did you stay away from it being too cartoony
even though it’s a source of humor and has that ultra-speed?
Butler: I
think our initial take was actually more cartoony. Here you had a ten-legged
creature that can move at the speed of sound. So the animators went naturally
to a slightly more caricatured take on it. Andrew kept calling it the Road
Runner too. But we had an epiphany during a sequence we called ‘Honeymoon’s
over’ which was the last sequence before he returns to Earth. He’s talking to
Woola and he takes his medallion and throws it into the ravine. As he’s talking
to Woola, our initial takes had him listening and responding like he understood
him. Andrew asked us to simplify it, so we took all that out and made it just
like your own cat or dog which looks at you. It doesn’t have a clue what you’re
saying, it just looks at you with love and affection. When we started holding
back on that, and just treating it like a real animal, it started to click and
people would read into that performance.
Aplin: We
had to play up the vacancy – that was where we got the most laughs, certainly
internally. We realized we could make Woola very endearing without doing much
by giving him a vacant expression – like a bulldog which we used for reference
of his movement. One of the physical challenges with the Road Runner elements,
was moving a 400-500 pound creature at such a speed and having it as a
believable thing in that world was always going to be difficult. We really had
to sell that idea on the back-end when he stopped, so we explored a lot of
different ideas for how he could displace his weight when he stops moving so
that you actually buy it.
Butler: We
explored playing with blur a little bit, so could see the mechanics of his
feet, but it just didn’t look like it was in the real plate, so what helped us
sell the speed was interaction and dust pick-ups. The big rooster tail of dirt
behind him was what showed his strength and speed in the real world.
fxg: For
the more intimate scenes with Woola, what kind of stand-in was there on set?
Butler:
Initially, Andrew actually made us audition the animators and we sent him video
of these. We built a big cardboard box head with handles on the back. So all
the animators who wanted to do Woola for the movie videotaped themselves with
someone else, say, throwing a stick. We showed them to Andrew and he felt like
it was good but he really wanted a stronger performance. Animators are
typically kind of shy, so we ended up going with a puppeteer called Todd Jones
who had worked at Hensons. He did a phenomenal job. We built for him a giant
foam head with some simple puppeteering tools to blink the eyes and move left
and right. He would walk through the scene with Taylor Kitsch to help with
eyelines and use that as a jumping off point. We also had an animation lead
back at Dneg, Craig Bardsley, who was a great compliment to Todd. Craig was
very good at naturalistic animation so he kept everything very real for us.
Watch a
scene with Woola.
Chiang: I
have to say Eamonn and Steve made excellent Woola’s too. When Todd wasn’t
available they dressed up in the greenscreen suits.
Aplin: We
didn’t have to, we just chose too…
Chiang: So
there are some great outtakes. Actually watching the plates is just hilarious
seeing them before the animation gets final’d, because you see them just
running around with this head. And I believe you took a tumble on one of them
Eamonn?
Butler: I
did, I took a tumble on my head.
Chiang:
Eamonn had to run at 200 miles an hour and obviously couldn’t. It’s all
captured on camera, though.
White apes
arena fight
Original
arena plate.
Final shot.
fxg: How
did you plan for the white apes fight and how was it shot?
Chiang: We
marked out the volume and there were restrictions in terms of how much
greenscreen we could have. We came up with this idea for containers – very
large 40 foot containers that you could stack up. It was out in the open in
Utah the location. So they were boarded green. Nathan Crowley the production
designer provided us with the dungeon entrance up one end and the stand for Tal
at the other, so that there was some reference to eyelines. Obviously the
ground was the sandy location.
We had a
physical prop that Scott Fisher had made as a prosthetic ape head that was on a
crane rig driven by an electrical buggy. Then there was an ape kite that acted
as a stand-in prop. To build a physical foam white ape is impractical, so we
came up with this idea of making kites – very lightweight cut-outs of the
characters. So when we took it into post there was some scale reference. Then
there were multiple cranes in order to throw Taylor around that Tom Struthers
the stunt coordinator had designed. Then there was a lot of digital takeovers,
set extensions and CG white apes and crowds of Tharks…
Aplin: Andrew
had previsualized that whole sequence extensively with Halon, which nailed down
a lot of his story ideas. When you’re previs’ing you can cheat quite a bit and
change the size of the apes for better compositing, but of course we couldn’t
do that, especially with the stereo. So we did have to re-work a number of the
cameras and compositions.
Butler: One
of the challenges was the geography of the ape arena which was almost perfectly
round. The sequence started out longer and more elaborate – it was one of the
first to go into production for us and one of the last to finish – mainly
because there was every trick in the book. There were lots of short shots,
crowds, interaction, fur – everything.
Chiang: We
also came up with a particle sand solver. When you shoot the real sand and you
need to dimensionalize it, we really had to get rid of all the sand that Carter
was flicking up and replace with CG sand, to create the other eye.
fxg: With
the white apes themselves, what was the brief for their design?
Butler:
Andrew would say to us they’re the fiercest creatures on Mars. They’re blind,
they live in caves. Every time they were on screen, Andrew had this phrase –
‘just push it to 11′ – obviously a Spinal Tap fan. He just wanted to make it
clear that these things were all about destruction and he wanted you to be
afraid the moment you saw them. So when the second ape is released, they just
immediately attack each other. The blind feature helped us elaborate on how
they would be able to find their prey by smelling and sniffing and scratching
at the ground. It was also a dramatic technique to slow down the action – it
gave Carter a chance to figure out how to escape and defeat the creatures.
fxg: What
was the fur solution for the white apes?
Butler: Our
grooming tools have been developing since 10,000 B.C. and we also had sand
falling into it clumping up. And each strand of fur was four feet long so we
had to groom them per shot to make it look believable and behave in a dynamic
way. It just required a lot of tender love and care to make it work.
Chiang:
Because they were jumping around a lot, Andrew wanted to retain the whiteness
within the red arena. Every time the apes fell on the ground, he wanted the
idea that the sand would get in, but obviously get redder and redder. They
would get up and shake off the sand and go white again. So there was a constant
re-generation of the texture in order to retain the whiteness of it.
Original
plate.
Final shot.
fxg: How
did you approach the digi-doubles for Carter in this scene?
Chiang:
We’ve increased the photogrammetry process for that, so we do a lot of image-based
modeling rather than just cyberscans. We have a lot of photographs shot at
different ranges so we can re-light using the existing textures, rather than
having to go in and create separate shaders.
Aplin:
There were also some key expressions scanned for Taylor Kitsch which we could
use for more distant shots. We could transfer the scans to quite simple blend
shapes that were broken up a little bit so we could off-set timings and still
get a reasonable performance.
fxg: Can
you talk about the compositing challenges for the arena?
Chiang:
Well it was a combination of plates that made up the composite. There was a
location called Rainbow Rocks in Arizona that served as a template for the look
of the arena. It’s some natural geometry formed into a horseshoe shape. We flew
around that in a helicopter taking photographs for textures which served the
basis for the main wall. Nathan Crowley built certain sections, but we did
replace most of it rather than blend pieces. So a lot of the time we would take
Carter out of the scene and then do a complete re-build of the background,
using projections but also cheating the volumetric a little bit so that the
Tharks would still read green. The DP Dan Mindel loved to shoot backlit so the
ambient became very important on the textures and the way we lit the whole
arena. We would shoot one way in the morning and when the sun came around we’d
shoot the other way. Then we would need to merge those. It meant that a lot of
the live action set became very flat and ambient. When Andrew looked at those
shots sometimes he would want more definition and we would go to a complete CG
re-build.
An animated
director
Director
Andrew Stanton and actor Willem Dafoe.
fxg: What
kind of sensibilities do you think Andrew Stanton, with his history in
animation, brought to the set and to the visual effects process?
Butler: I
think there might have been some concern about how Andrew would be on set – you
have to make a lot of decisions quickly, it’s very fast-paced and high tension
– whereas feature animation can take years and years. But he was really in his
element on set and people stopped worrying about it very quickly. He’s also an
actor – he took classes as an actor a long time – and he connected with Taylor
and everyone else. I think he really impressed a lot of people with how mature
he was on set. That really comes from Pixar where they’re very story-driven and
budget conscious. They’re one of the most successful studios in the world that
just happens to work in animation. I think people think animation is a
completely different thing but really Andrew is a filmmaker. We were always in
safe hands with him and he could always give an answer or make a creative
decision about how something would push the story forward and why it was even
in the movie in the first place. And he’d dabbled a little with a live-action
DP, Roger Deakins, on Wall-E and he tried to stick to real camera rules. So he
already had a language for what he wanted to see.
Aplin: He
was also very keen to get one-on-one face time with the animation crew. It was
almost like going back to school. He just had an incredible eye and could guide
you back on target. He loved to get up in front of us and act out a certain
part. His energy is also incredible.
Chiang: I
think it was a natural step for him to go into live action – I think he was
ready for it. The challenges that a live action feature brought really required
the same attitude for animation. He could easily dissect what was going on and
methodically work out the best route using the tools we were given. So although
we were introducing him to photorealism and getting all the photography right
and those technical aspects, he had this ability to take all that on and work
it out for himself and really educate us about all the information through all
the heads of department. That really made all the working process with him such
a joy.
