Gollum Shmollum: Kinect Hack Does CGI Animation on the Cheap

Under the HUD

An enterprising team of animators has hacked Microsoft's Kinect sensor suite to do something tangibly amazing: They're using the cheap hardware as a motion-replication system to power CGI characters. Remember the millions of dollars needed for Gollum in Lord of The Rings? Yup--like that.

Kinect hacks are coming thick and fast, some clever, some useful, some practical and some just hinting at a rich future for the technology that we're only just beginning to dream up. But this new hack, from creative collective Triangle Productions, is possibly the most directly useful one we've seen yet: The team has used openly available Kinect hacker software to build a motion-capture interface for their animated CGI cartoon series "Under the HUD."

The show itself is a web-based production that follows the everyday lives of two video game characters. Some of the shows have been written and pre-produced, but the core trick is using Kinect to control the motions of the CGI "puppets" that make it all work. This technique is an alternative to carefully manipulating the simulated joints and limbs of the 3-D creations by hand--the kind of painstaking animation trick that Pixar used for the first Toy Story film, for example, and which represents a more traditional way of creating computer animated movies. It works, but it hardly results in totally convincing movements.

Motion-sampling is a different technique that relies on cloning the movements of human actors onto a computer-generated skeleton. It's how Avatar was made--helping you believe in the humanoid CGI characters because their moves seem so fluid--and it's how, famously, Gollum was created in the paradigm-breaking Lord of the Rings movies (see a clip of this technique at the bottom of this article, starting at about 2:30 minutes). But the LoTR motion-capture relied on million-dollar technology, over 20 separate cameras, a specialized sensor-laden "suit" for the actor and some very high-end computer processing: Triangle's solution needs just $150 sensor hardware and a lot of imagination. Plus the team avoids having to pay licensing fees for existing game-engine solutions that could also be used to drive the CGI puppets, albeit with potentially less convincing movements.

Thanks to the Kinect hack, herky-jerky CGI characters will be replaced with fluid, lifelike movement in kids' shows, sci-fi, and whatever else Hollywood--or amateur film buffs--dream up.

To read more news like this, follow Fast Company on Twitter: Click here.

Read More: Most Innovative Companies: Microsoft

Add New Comment

3 Comments

  • wildcatherder

    Wow! I'd like to see "Outer Space Astronauts" come back with this type of animation.

  • n8

    The solution from Triangle is using Brekel software, developed by 3D animation studio Motek Entertainment in Amsterdam.

    While the low-cost solution of the Kinect is certainly interesting, it is highly limited and will not be replacing the motion capture systems used for high-end productions such as games or films any time soon. For example, the Kinect does not track the rotations of the hands, feet or head, so the animators will still need to keyframe all of this animation (referred to as the "Pixar way" in the article).

  • Leon Jackson

    Perhaps before marvelling at the wonders of motion capture, one should also note the numerous failures (Polar Express, A Christmas Carol), due the inability to capture nuanced acting. Also, in the case of Gollum, noted in the article, that motion capture data was not used exclusively for the character, and it was also used as a basis for the animators...that brought the character to life. Also, all the facial animation was done by animators, using Serkis as reference. And, watching the "herky-jerky" movements of the captured actors using Brekel's toolkit (not built by the folks at Triangle Productions), while interesting, has a long way to go before replacing expensive motion capture systems...and even further to replace artists and animators.