Last week I was at the Siggraph conference in New Orleans. Siggraph is THE place to be if you are at all involved in computer graphics, and this includes animation, scientific visualization and visual effects. With temperatures in the 90s, and around 90% humidity, it was great to be able to walk directly from my hotel to the convention center via the indoor Riverwalk shopping mall that runs along the Mississippi River.

I was there to take in the production sessions, vendor talks, panels, courses and exhibit hall—and of course, to talk up ShotRunner whenever I could.

I can’t go into everything at the conference, there is simply too much to cover, but I thought I would relate details on a couple of interesting presentations I attended that I haven’t yet seen blogged about elsewhere. Another great source of Siggraph news is fxguide where they cover days 1, 2 and 3.

Benjamin Button

First up for me was a keynote by the folks at Digital Domain on the making of Benjamin Button. I didn’t know this, but the head of the ‘older’ Benjamin Button was a complete replacement—a complete ‘digital double’. And I also didn’t know that a feature-length, fully digital human head had never been done before. There have been non-humans like Dobby from Harry Potter, but achieving a realistic human face has been incredibly elusive. In fact, people talk about this phenomenon called the ‘uncanny valley’ whereby the closer to get to realism, the more ‘creepy’ the face seems to people. When the animation is clearly not trying to fool us into perceiving a human (as in Dobby) our minds are able to let go and be drawn into the story. But as we approach a lifelike human, our minds start to fight the image—and this sort of “digital Botox” effect becomes distracting. We could see this in movies like Beowulf.

So the prospect of doing a digital double of a well-known actor like Brad Pitt over the length of a feature film was admittedly terrifying for the folks at Digital Domain. But after a successful screen test, the project was green-lit and over 155 people at DD got to work. Some of the extraordinary steps they took to insure that Brad Pitt’s performance came through to the digital double included covering his face with a phosphorescent paint and literally recording every single movement that his face could make. They also made a series of incredibly lifelike head models showing how he would look at age 60, 70 and 80 and lit them from dozens of angles inside of a automated light dome. They worked for two years to get a twitchy, natural look to the eyes.

Even the lighting was a major accomplishment in itself. With Brad Pitt’s digital head traveling through so many different lighting environments, sometimes within a single scene, getting the lighting right was extremely challenging. They decided to use image-based lighting (a technique that drives the lighting in a 3D scene from high dynamic range spherical panoramas of the set) throughout the project. But because the character was often moving, they had to additionally develop a method of animating the light spheres—a topic of at least 2 other highly technical talks.

The difficulties didn’t end there: sometimes the neck and collar of the body double (for ‘young’ Benjamin Button) did not match the Brad Pitt’s acting in the scene and much reworking of the images would become necessary. To ensure accurate tracking, extra ‘witness cameras’ were brought in to survey the scene from alternate angles—this meant that up to four 3D match move solutions had to be worked on for every shot. Many shots needed well over 100 revisions before final approval.

All in all an extraordinary effort, but the results were equally extraordinary.

Randy Thom

Randy is director of sound design at Skywalker Studios and he presented a keynote on designing a movie for sound. His premise was that sound design is not just adding foley and cool sound effects, great sound needs to start with the script, otherwise its a potentially distracting decoration at best. Planning needs to start in pre-production.

His main example was the opening of ‘Apocalypse Now’, which provides an interesting juxtaposition of stylized helicopter sounds with the image of a spinning ceiling fan. This required quite a bit of experimentation during production to arrive at the final sequence. Francis Ford Coppola’s main character, Willard, morphed from a very active participant in the action to someone very passive, thereby letting us experience the world of Saigon through Willard’s eyes and ears. This opened the door to many creative uses of sound to help tell the story.

In a picture that has space for dramatic sound design, the dialog is sparse. The script will have ‘holes’ and ‘pauses’ for us to experience the sounds of the world the character inhabits. Music, by contrast, sits on top of the action and primarily imparts a mood.

He also spent a little time talking about the process of acquiring sounds for movies. People seem to think that most sounds are somehow synthesized, but that is not true. Most sounds used in films begin as actual recordings of real things. There is just so much complexity and texture to real sounds that can not yet be completely synthesized.

Randy then did a fast-forward to Pixar’s Wall-E. In this animated film, they started to think about sound design early on in the storyboarding process. Perhaps because there was not a lot of dialog in the first 20 minutes of the film, sound design became a crucial part of the storytelling process. Sounds would be created and given to the animators as inspiration, for example, Wall-E’s ‘trash compactor’ sound went through several iterations and sparked several different versions of his animated motion. They would repeatedly screen the movie without dialog, without music and then again with just sound effects—all to be sure that the audio was always in service of the story.

So his message was that sound can be just as effective a storytelling tool as the visuals or music in a film, it just takes a bit of planning and space in the script for the sound to be heard.