Combining motion capture with ray tracing, the forthcoming film could start a VFX revolution.
When filmmaker Kevin Margo isn't stuck at his day job as a leading visual effects supervisor for everything from Hollywood films to video games, he spends his days and nights inventing a revolutionary new process that, he believes, will change the way people think about computer graphics in cinema.
Since his short film, Grounded, debuted to critical acclaim, the gears in Margo's mind have been moving. Absorbing the world of visual effects and tackling the problems presented by grossly inefficient, industry-standard CGI practices, he's out to prove his solutions with his forthcoming short film, Construct.Most of the beautiful, digital worlds seen in today's box offices are generated through a process called ray tracing, in which detailed algorithms usually created by a programmer simulate the behavior of light in the real world and produce high quality realism. It's an intensive, time-inefficient process, however, so motion capture and other types of real-time digital imaging (say, video gaming), rely on rasterization, which converts images from simple vectors into the arrays of fast-moving pixels we know and love.
Margo has developed a new method: digital imaging that ports V-Ray ray tracing software through motion capture software. The results capture the beauty of ray tracing in real time, allowing for the kinds of creative decisionmaking that had, until now, only been possible with rasterization or live action filmmaking. It's a killer combo that could open digital storytelling up for both independent filmmakers and studios, alike.
We spoke to Kevin Margo (K) and Derron Ross (D), the lead motion capture actor who plays Bill the robot, a few questions about making Construct, and the trails they're blazing while the film is still in production.
The Creators Project: At this point, what stage of production are you in?
Kevin Margo: We've captured a ton of performances for the remaining 6-7 minutes of Construct, and now we're blocking out scenes and cameras along with creating a few more unique environments and characters.
How was directing Construct different from a traditional CGI film?
K: As a director and cinematographer, what this new technology allows me to do is to capture the moment in more fidelity and clarity. It is more representational of the final product than other motion capture techniques. It's representational of the real world, using lighting and rendering algorithms that are simulating real world light.
As a visual effects supervisor on an action film, there's this great tension and dynamic interaction between all the elements going on at the same time. The performers are reacting to the distance of the camera, the camera operator is reacting to the performers, but also color and shape and light. It is always in flux through the lens. As soon as that's possible, you can then be framing and composing shots to all this beautiful light in a way that is not traditionally possible in the visual effects workflow.
I was missing that in the CG process. No longer does the visual effects pipeline need to support the concept of infinite flexibility, which is hugely inefficient. In my experience, you are forced to constantly support the possibility that anything can change at any given moment, all the way through delivery. It's extremely expensive, so if you can find ways to enable the director to envision the image in its entirety, especially during production, you're going to save a ton of time, a ton of work, and it's gonna be much more clear.
Derron, as an actor, you have to image the entire setting in a very abstract way. Tell me about how you got into your character's psychological state without seeing anything that he's seeing.
Derron Ross: The technological approach Kevin took with this sort of drove everything to have a lot more pre-planning than we normally do for motion capture shoots. Usually shoots happen before we get the environment mapped out and the set set up. [Kevin] had the set fully designed and laid out. We literally knew exactly the dimensions we'd be playing with in the world of Construct. That was awesome because we were able to first actually have the dimensions literally mapped out on paper, and then we were able to give that to the stunt team. They were able to roughly choreograph with those dimensions and and when we rehearsed the choreography, we could really rehearse.
From a set perspective, we already know what everything looks like in our minds because of what Kevin already showed us, and then physically we had things roughly set up in the space.
Stepping back to Bill [Derron's character], I analyzed, "What is his thought process?" and "How does he act?" Kevin was very open to ideas I had about Bill having a military background prior to Construct, where he's taking these guys out in a very tactical manner.
That gave me a lot to play with because now that makes me think about these things that'll be going through his mind. Kevin wanted me to make him very human. He has a wife. He has a son. So he was cool with us playing it like he just wants to be a regular guy now. He's happy not to be decommissioned, he's happy to have a job and have a wife and have a kid. He doesn't want to rock the boat anymore. But, unfortunately his coworkers are doing some shady things and he still has to call upon old skills to survive.
Since the motion capture didn't record your facial expressions, how did you incorporate the humanity Kevin wanted for Bill into your acting?
D: When Kevin and I were working on the character I kept asking, "Do you want anything that looks stiff or any form of robotic movement?" And he said, "No, I want it to be as human as possible."I had started doing all this research, reading about Michael Fassbender and all the research he did on Prometheus, and I asked Kevin, "Do you want my posture more correct? Do you want me more stiff and all that?" And he said, "No, just be yourself. It doesn't have to be robotic, it can be more normal. We want them all humanlike." So that made it a lot easier, and I didn't have to worry too much about changing physicality.
I will say this, there are certain movements I did try to keep to a minimum. Not too much swagger in a walk, not too much arm movement that's too loose. Humans do move differently because of the way we walk naturally, so there were little things I was doing on a subtle level.
I think that what's great is being able to kind of see how your movement is showing up on your avatar, like posture and stuff, and then we can also see in the environment where things are happening. We can start cheating things a little bit.
K: On a live action set when they're setting up a scene, they're blocking out the scene and they're figuring out roughly where the camera angles are going, where the actors are, the way they need to traverse through the scene. Having that kind of rendering technology on hand enabled us to block out scenes better, with more understanding of everything while we're capturing performances. If we didn't have that rendering technology on hand, we would basically be blocking the scene out days later with these performances captured in a void, with no frame of reference to anything. You can block out the scene, you can see how lighting is going to fall from different camera angles, and then you can look for efficiencies. All that kind of stuff you can be aware of, and this is a way to be efficient about it.
How did you come to decide that Construct was the best story to tell with these techniques?
I'm very interested in the technology process of the creative experience. I'm always toeing the line between technical advances and art. I work at my day job as a visual effects supervisor, so I'm always very attentive to process and how I can increase efficiency in whatever it is we're doing. Given that, I love all CGI stuff, and I've been very attentive to recent developments, knowing that soon we're going to be at a point where you're going to be able to do this real-time path tracing interactively, and apply that to a production. And one of the things that I thought would be really cool would be a motion capture volume. You know, enabling the performers to be rendered with all this great realistic lighting and shading while everything is happening. I knew that seemed like a viable option and I wanted to do that.
I realized that whenever we are to implement this process, there are going to be certain technical limitations, and that drove the content of the film. Robots are much easier to manage from a 3D animation perspective. The subtleties of human performance are kind of absorbed or lost into a robot. So people accept the robots for what they are, in terms of emotion, whereas they're much more critical of human performances. Knowing that there's this process I want to advance, there's gonna be certain technical limitations that I need to embrace creatively. I had this Construct story in mind and was waiting for the opportunity to develop this workflow to support it.
I don't mean to draw a comparison, but I see thinkers that are on the cutting edge of technology, and the stories that they write are influenced by the acknowledged limitations of the technology at the moment. Look at what James Cameron did between The Abyss and Terminator 2. In Abyss, they figured out how to make this floaty, liquid water face. And the he started to write Terminator 2, right? He's like, "Oh, wouldn't it be cool if we took this Abyss technology and turned it into a liquid metal bad guy?" So he was evolving the limitations of the technology at hand to go a step further, to achieve a story element for one bad guy.
We only see a human briefly in the Construct teaser. Tell me about the world you've created: How do humans and robots interact?
Humans in the world of Construct are a threatened and receding population of equal sentience. As robotics and artificial intelligence have advanced, social and political tensions have arisen from the increasing presence of these robots. It's very much a metaphor for any historical immigration or cultural confrontation. Their presence in the short film specifically serve to enlighten the viewer of the robot's psyche. Perhaps human/robot conflict can be avoided given the perceived cold rationalization we have of machines, but as I'm alluding to in the teaser, these robots are not a unified mind, each with their own thoughts, perspectives and emotions influencing their behaviors.
What was the biggest breakthrough that made you realize Construct was possible?
I was proactively trying to make that breakthrough happen. If you follow the 3D rendering stuff, there are all these real time path tracers out there, but they're applied to a specific 3D application. They're meant for a user sitting at a desk to interact or move around a viewpoint, sitting in front of a monitor. Chaos Group http://www.chaosgroup.com/en/2/index.html is the maker of this new technology, and during my day job I use their software. I've kind of developed a professional relationship with them, and about 5 months ago they came to me and said, "We loved what you did with your previous film, Grounded. Let us know how we can support you as an independent filmmaker." It's a new marketing avenue that they're exploring.
That's when I realized that I'd always wanted to do this path tracing in Motion Builder, the motion capture software, only, with performances. I proposed that we develop the short film and they apply V-ray to the Motion Builder. I wanted to see in real time all the amazing beauty that their renderer is capable of doing. I said, "You port your software to Motion Builder, I want to do this in a volume. Let's use Construct for this application."
When they got on board, we started realizing we needed a lot of really high end hardware, GPUs, the fastest processors possible. We reached out to Nvidia and Boxx, who made the towers that house the graphics cards. They were very receptive to the idea that we're trying realize, and also to the short film itself. They were very supportive and chipped in some resources.
What we've got here is a fusion of the visual freedom granted by CGI films and the life-likeness and efficiency of a live-action set. Do you think, down the line, the kind of motion capture technology you're developing could threaten traditional filmmaking?
I think it has the potential to change the way we approach many types of films, but not all types of films. I think this is going to be immediately hugely beneficial to effects heavy, high-end, costly productions, in films where all of the visual effects are intertwined with live-action elements. I'm seeking ways of creating efficiencies in the process and also enabling the visual effects community more creative influence and control in the larger filmmaking process. If the visual effects community, which knows how to use all these tools, can insert their knowledge and skill sets on the front of the pipeline, now you've surrounded the live action community, and you are able to influence how that is executed. In my mind, it could make it more efficient.
So my film buddies who fear the death of cinema are worrying too much?
Well, look at Avatar. It looks essentially like a live-action film, except 75% of it is fully CG. You have the access to these rendering tools that are capable of generating photorealistic imagery, then you introduce a process like mine that can expedite and bridge the gap between the live-action and the computer graphics approach. In films like that, I think it has a potential down the road to become a very appealing way to tell stories and create films. It's not going to be for every type of film. Maybe a hundred years from now you'd be doing love stories with this kind of thing.
Look at Benjamin Button, look at the interest in recreating photorealistic performances virtually and digitally—the interest is there. This is another step in creating these possibilities to a better degree. That's another thing I'm hoping, is to watch this whole production workflow mature and to enable visual effects artists to have access to a range of tools, and to be able to tell their own stories. I want to see this whole process democratized.
I really hope that a mature virtual production workflow—like what's starting to happen here—will do for the visual effects community what the DSLR revolution has done for live-action filmmaking in the last decade; just to see how, once this technology is in the hands of more people that think differently, a ton of really great range of creative ideas and expressions. Visual effects, traditionally, has been only resourced for hugely expensive budgets and productions, but f we can get this kind of workflow efficient enough that it's cheaper to do, now we have what traditionally looks like huge, costly visual effects applied to quirky, independent story ideas. That's really exciting.
I really like the comparison of your work to the DSLR revolution, about democratizing the visual effects process.
I'm a bit of a revolutionary in the sense that I hate people who have control and power, and I want to distribute that power to as many people as possible. I want everybody to be able to create what they want to create. If the drive is there and the ideas are there, there's no reason they shouldn't be able to see their vision turn into reality. This is just trying to facilitate that.
Which filmmakers had the most influence on your vision for Construct?
Well Kubrick, definitely. I like what Alfonso Cuaron did with Gravity, and actually all of his films. The cinematography and the immersion that those really long camera takes convey… David Lynch I love from a psychological perspective. And I like what Neil Blomkamp is doing visually. I dig his aesthetic sensibility. The best sci-fi films are just backdrops to something more substantive, so I love how he's embracing social elements which are relevant to the modern world and inserting them into sci-fi genres.