OUAN504 - The Last of Us, Motion Capture

I chose to look at motion capture for this task because I have never before looked into or had the opportunity to use it myself. However, I recently saw a behind the scenes video of the way the motion capture sessions were conducted during the production of the critically acclaimed video game title, 'The Last of Us'. What shouldn't have been to surprising the first time I saw it but I found interesting was the way in which the actors portraying the characters in the game acted out full scenes using the motion capture suit, which is then translated directly into the in game characters movement and expression. I'm not entirely sure why, it may have just been my unfamiliarity with the way motion capture works and is used, but I almost imagined it's function was for certain movements to be acted out. However, the way in which full scenes were expressed this way as they would be in a film only wearing a motion capture suit instead was eye-opening.

I want to start by looking at motion capture very generally, as before looking into this I didn't really have any prior knowledge to the way it was used and operated. Motion capture, or 'mo-cap' for short, is a process used to record the way objects and people move. It is used in a wide variety of fields and has a multitude of different ways including; entertainment, sports, medicine, and for validation of computer vision and robotics. In terms of what I looked at here, it was focused more around the way motion capture is used in filmmaking and video game development. This instance of motion capture refers specifically to recording actions of human actors, and then using that footage and the information gathered to animate digital character models in 2D or 3D, via commuter generated imagery. There is another term used to identify a specific kind of motion capture which involves subtle facial expressions and finger movements. This is often referred to as performance capture as oppose to motion capture. In motion capture sessions such as the one displayed in the video below, movements of one or more actors are sampled many times per second. In contrast to this, early techniques used images from different cameras to calculate positions. Often the purpose of motion capture is to record only the movements of the actor, not his or her visual appearance. The data containing the details of the movement is mapped to a 3D model so that the model performs the same actions as the actor. I suppose this process and approach to recording the movement of actors could be compared to the more traditional method of rotoscope. There are several instances where the movement of actors was filmed and then used as a guide for the frame-by-frame motion of a hand-drawn animated character sequences.

The way motion capture was used in the video game, 'The Last of Us' was that live acting were performed on a scene to scene basic, in order to produce the cut scenes for the game and thus tell the story in this heavily realistic and authentic way. Professional actors as oppose to just voice actors were hired to play different characters, most notably Troy Baker playing the main character - Joel. As the audience I understand and appreciate the need for scripted movement and dialogue in order to portray this intricate story in a way that it can be conveyed at the highest level possible. The interesting thing about recording scenes for video games using motion capture in this way is that it is incredibly challenging not just for the production team but for the actors themselves. It's difficult at first to understand the complications for video game actors, across all games produced in this way. Having to act a lengthy scene of movement and dialogue without their character's appearances, without the clothing that will be seen in the final version, and a completely non-existent setting. With each year that passes and more and more video games of this calibre being produced to a growing demand and a growing industry, actors some of which are quite well established are crossing over to the world of video game performance, it will be interesting to see how the mobcap will change and develop in this industry over the coming years.

The process involved in making these intricate cinematics include the raw motion capture footage where actors act out the scenes using movement and voice as if it was real life, however like mentioned before they have to wear the motion captures suits, and so lack character clothing or a realistic environment to perform in. Next, the movement is mapped onto the character or object models and a body pass is done. A body pass is essentially the first layer of clothing, colour, and featured are added to the characters - at this point, you can start to see everything coming together. After this a facial pass is conducted which adds in the facial expressions and facial movement of the characters. A lighting pass is then done, where the environments are added and lighting plus shadows are added. This is the part where I felt the scene really came to life and the characters and the environment are combined. A final version is then made with all the adjustments made to create the final piece.

Here is a brief quote from a developer at Naughty Dog talking about the process of storytelling and the overall creation of 'The Last of Us'.

“High tech cameras record the tracking markers on the mocap suits to capture these performances. Raw mocap data is taken to our team, cleaned up, and then the next step in the cinematic process takes place. Six steps in about a minute. Making game cinematics looks pretty easy, right? What you’ve seen is a time lapse showing over two years of planning, writing, performance, scripting, animation, lighting, and artistry. Then apply more tweaking, more adjusting, and more fine-tuning. And then some more before it ever goes out the door.”