Is This the Cheapest, Most Wacky-Looking Gestural Interface in the World?

Robert Wang

Gestural interfaces aren’t quite mainstream yet, partly because they can be ruinously expensive–involving multiple cameras and custom detectors. But a student at MIT has devised one requires only a Webcam, and some crazy looking Lycra gloves that look like they’d be worn by hipsters from the future.

The obvious forbear to this tech is the Sixth Sense system, also out of MIT, which relied on colorful bits of tape attached to your fingertips.

But Robert Wang–an MIT grad student who invented the new system along with Jovan Popovićt, his faculty adviser–points out that tech such as Sixth Sense is really just 2-D: “You’re only getting the fingertips; you don’t even know which fingertip [the tape] is corresponding to.” His system, by contrast, allows the entire range of a hand’s movements to be translated into a virtual 3-D world–meaning that you can have fingers and palms flexing, manipulating objects just like you would in real life.

The big trick of the interface is the colorful pattern on the gloves, which allow a computer to discern the 3-D shape of the hand. A Webcam simply snaps a picture of those gloves; software then crunches the color patterns, and instantaneously compares the color patterns to a database of hand gestures. After finding a match, it generates a virtual 3-D model.

Wang thinks his device could be used in video games–but also by engineers and designers, to manipulate scale models in 3-D (a la that awesome scheme in Iron Man 2, where Tony Stark plays with that model of the New York World’s Fair).