All In - Dreamworks Animation Goes All 3D

From The Creative COW Magazine - Stereoscopic 3D

Jim Mainard Los Angeles, California USA ©2009 Jim Mainard and CreativeCOW.net. All rights reserved.

Article Focus: DreamWorks wasn't the first to make a modern-day 3D animated movie. But they were the first to commit to do nothing else, and to commit to start-to-finish 3D authoring for all of it.

• ALL IN
• FROM LEFT TO RIGHT, AND IN SPACE
• BY THE NUMBERS: Monsters vs Aliens
• BY THE NUMBERS: TECH
• WHERE WE GO

When Dreamworks Animation began in 1997, it very much had a start-up mentality, and the company kept that entrepreneurial spirit as it has reinvented itself many times over the years.

We were a relatively small team then, just big enough to focus on one movie at a time. The number today is more like 1500 employees, and we are able to have many more production pipelines open. We are now typically working on three greenlit movies at the same time, with at least three behind those that are ready to fit into the next slot. At any given time, we might have eight or 10 productions in some stage of development.

Our first film, "Prince of Egypt," was released in 1998. It was a 2D animated film with a lot of CG effects in it - things like fire, or the effects around the parting of the Red Sea. People tried to come up with words for it. I think somebody called it a "tra-digital" film! I'm not sure that it fit exactly, but it was an interesting combination of words to describe our hybrid approach.

The kinds of visual effects we were adding were quite uncommon in any kind of animated film at the time, partly because CG was expensive, and partly because the registration of the CG elements with the 2D assets was a challenge.

At the time, there were not many software systems to do those kinds of things, which led to Dream- Works creating its own technology team. I was in the early group of folks added to the software development team, to begin building the tools that could support the kind of filmmaking we wanted to do. We made those kind of films with "Prince of Egypt," through "Eldorado," "Spirit," and "Sinbad."

After "Sinbad," we branched out into all CG films, trained traditional artists to use CG tools, and rewrote our toolset.

Research and development was the arm of the technology organization that was responsible for all the applications that the artists would use to make the films. Some of that was integration with third-party software like Maya, and writing plug-ins. We also did things like determining new algorithms, for example, for creating water solutions for the ocean surfaces in "Sinbad."
The general charter of R&D in those years was to push the envelope, to try to do more in an all-digital film than had been done before, bearing in mind the creative appetite of the show.



"Prince of Egypt"

ALL IN

Research into 3D production began a little over three years ago. The original plan was to do our next film in 3D, and we had chosen as that one film "Monsters vs. Aliens."

The big surprise came when, within just a few months of deciding that, DreamWorks CEO Jeffrey Katzenberg said in 2007 that he wanted to do all the films in 3D, and that the first of those would be released in 2009. We obviously still had to build the same tools to make one film in 3D, but this certainly upped the game quite a bit. We now had to quickly ramp up every part of the company to get all of our shows ready for a new kind of production.

That involved training programs for the artists. It involved rewriting and reformulating tools. It meant going to third party vendors to get them to add functionality, and where they couldn't or wouldn't, we had to build it ourselves. And then we had to bring all of that together into an integrated package.

At the same time that Jeffrey was actively involved in engaging the exhibitors to get more screens, we were working with standards groups to try to help identify and support the light levels that would be required, and try to vet the systems already installed to make sure they were producing similar results.

On the creative side, we had to discover the challenges of color grading and the implications of what that meant for 3D filmmaking. Along the way, we found that none of the cheats we had been using in 2D would work anymore. Once you add depth, they become quite obvious. We now found ourselves required to, for example, turn to volumetric rendering in cases where we might have gotten away with a cheap camera trick or a blur before.

I was pulled out of R&D and asked to to take this on as a studio initiative. Jason Clark ("Stuart Little," "Monster House") from the producing side was also brought on early, to help get the productions thinking in the way that they would have to think to make a 3D film. And then we brought on Phil McNally, formerly with ILM and Disney, as stereoscopic supervisor

My charter was to move forward, to get the studio up and ready for making 3D films - whatever we found out that meant. Our goal was to institutionalize the knowledge of 3D filmmaking into the studio, so that everyone, regardless of their role, understood how 3D filmmaking impacted their job or their role.

That became a way of working. We were not looking to create a specialized team that would post process 3D or anything like that. We wanted to make sure that our best and brightest creative folks in every department understood the implications of 3D, and that as a studio, we were making the most of it.



Monsters vs. Aliens.

FROM LEFT TO RIGHT, AND IN SPACE

There are many implications for working in 3D. As filmmakers, they start right from the beginning, in composing a scene.

In traditional or CG animation, the animation generally occurs from left to right, within sort of a proscenium stage, largely so that you can see characters clearly against their backgrounds. But as soon as you add depth you have opportunity to rotate that space and actually put the characters in depth from each other as opposed to simply left to right on the screen.

3D also impacts the way that you set dress the environment: it is important to add depth cues, to give a sense of the volume of the space. The classic example is that, if you want people to see how long a road is, you show telephone poles going into the distance.

Lighting is even more important. It is important that the eye goes to where you want it to in stereo. Particularly for folks that are newer to looking at a 3D film, the tendency is look around in awe at everything in the entire beautiful environment. "Wow, look at the leaves on that tree. That's amazing!"



Unfortunately, they are doing that as the story is unfolding. That's not what you want to accomplish. You want them focused on where you want them focused! In general, the rules of cinematography still follow. The eye is drawn to brighter places on the screen and drawn away from darker places. That's a general rule, but it's an example of why lighting is so critical.

People also generally look at things in the foreground before they look at things in the background. It's just a human reaction things closer to you are more meaningful than things far away.

With 3D space you can determine whether they are close to you, but with also where they are relative to a space that includes the screen. That is, your eyes are focused on the screen, but the question is, really, where do they converge? Where are they focused within the entire volume presented to them?

The general rule is something like a third of the total volume of the viewing space can occur behind the screen, and about two-thirds can occur in front of the screen. That's important because, if you think about a movie where you never play anything forward of the screen, then you have very little depth to work with in order to compress an entire real world behind that screen.

Think about watching a movie in a theater. There's a black border around the screen, and it functions like a frame - everything happens inside that frame, and behind it.

However, there is a trick in 3D composition that brings that black border out into the audience. If you do that, then everything is still, essentially, playing behind that frame. Everything feels inside the screen, but yet, it is not.

Things work a little differently in most IMAX theaters, where the edge of the screen is at, or near, the edge of your peripheral vision.

As such, the frame is no longer important or relevant. The vanishing point, the maximum depth is actually on the screen, so that EVERYTHING plays forward of the screen,

What we do on our films is to provide mechanisms for adjusting the convergence to allow us to play that movie appropriately. IMAX does the work in making this adjustment to our creative leadership's expectations, setting it to play "in the right place" in the theater. You don't want it playing so far forward that it's necessarily looming over you.



There are things that we can do as filmmakers to make that easier as well. We can over render. "Monsters vs. Aliens" was virtually all rendered at 1920, but we could decide to render 2048 pixels. That would allow some flexibility between the left and right images, to accommodate that difference in the playback scale. It depends on whether we think the shots would need adjustments.






WHERE WE GO

Looking at the numbers for "Monsters vs. Aliens," if I were an exhibitor, I definitely would have wanted a 3D screen in my complex, when the film came out. Or maybe two. So despite all the sort of financial things going on in the world today, I think that we'll see the number of screens probably double this year. By the time James Cameron's "Avatar" comes out by the end of the year, we could certainly be pushing 4000 screens or maybe more.

Audience expectation is clearly high. Our polling showed that people wanted to see it in 3D, and came back later to see it when they couldn't get in. That tells us that there's still capacity to grow out there. Internationally, we're seeing similar patterns, but the numbers of 3D screens is so small - I think its on the order of 1000 screens outside North America - that it's going to take some more time.

Creatively, our objective is to hide the hard science from artists, and let them get back to doing what they do best, just creating art. I think we have mostly accomplished that today. Even our training manuals focus more on space and volume than on the mechanics of the eye, or interocular distance, or the way that the brain forms a 3D image from disparity between the left and the right eye.

I think that the science frontier that remains is more on the display side.

In particular today, there is the idea that there is a left and a right image, and that that's how you create 3D. But practically, for moving in the direction of a truly 3D display, autostereoscopic display that doesn't require any glasses, new development needs to take place. Peopl feel okay about them - but I think that getting rid of them makes 3D ubiquitous.

Currently, there's such a narrow viewing angle on an autostereo display that you can only move your head something like six degrees. Most of us cannot imagine watching a movie sitting on our couch and only moving our heads six degrees. It's just not very practical. But I think that there are technologies coming that will allow autostereoscopic displays to happen.

That's the next place where the big science will done. I think we'll see it. There are big places with smart people working on it. It might be a decade out, but we need to be prepared for it.

There are implications for the studios as well. As we think about archiving a movie for the future of cinema, we don't want to get caught where we are still making black and white movies when color TV's are coming out. We need to be careful that we are thinking about the technologies that will be there in the next decade.

When the 20 year anniversary of your movie comes up, you don't want your library to be dead. If you have perceived that stereo means a left and a right image for 3D viewing going forward, you probably looking at it little short-sighted.

You probably have to think about archiving the movie as 3D geometry plus textures that will be rendered separately. You can even imagine that, in the future, the display systems will receive the geometry, and receive the textures, and will render that movie real time for you.

The right thing to do, and the goal for any of the studios that are making these kinds of films, is to have their technology groups focus on eliminating the scientific implications of decisions, and just allow creativity to rule.

Jim Mainard
Los Angeles, California USA

Jim is the Head of Production Development for DreamWorks Animation, and was responsible for the studio's transition to stereoscopic 3D filmmaking. Before the entertainment business, Jim's background was aerospace, where, among many other things, he was the Principal Systems Engineer for the Hubble Space Telescope's ground station planning systems.

Find more great Creative COW Magazine articles by signing up for the complementary Creative COW Magazine.