This year was the last Microsoft keynote at CES and by most accounts the bright spot was the expansion of the Kinect line with a new Kinect for Windows. The new SDK and hardware is aimed at commercial development with a new “Near Mode” that takes a technology designed to recognize a body across the livingroom and calibrates it to also recognize objects within a foot or two.
Microsoft hinted at the new development platform last year with the release of a non commercial SDK and a site called Kinect Effect, showing the Kinect being used in a variety of non-Xbox situations like medical therapies. They released a corresponding video in October.
Note the fine print: “Depictions are visionary”. A little while ago, we complained about Microsoft’s lifeless concept videos and watching the Kinect Effect video, I get that same feeling.
Kinect’s voice and gesture input is often referred to as a “natural user interface” but there are very few situations where humans have been able to control tools by waving our hands in the air. A key part of human manipulation of the world is the manual aspect. We anchor ourselves to our environments through our sense of touch. If you buy Bret Victor’s complaint that touchscreen interfaces are Pictures Under Glass then the Kinect is Waving at Pictures Off In the Distance.
Take the parts with the musicians. I’ve played cello and percussion–both activities depicted in the film. In my experience, tactile feedback is a crucial part of playing the instrument. With it, you can play with your eyes closed. Without it, you end up with Theremin 2.0 and the Theremin is not an easy interface to learn. Similarly, when the guy is controlling a robot to disarm the car bomb, it seems clear that he’d be better off with a haptic interface, so he can feel what kind of resistance the plug is giving.
That said, the part with the surgeon swiping in the air to look at X-rays makes a lot of sense, but that’s only because he needs to keep things sterile–he can’t touch anything with his hands.
This is not to say that a lot of amazing Kinect-based applications won’t find their way to market. For all that individuals make their way through the world by grasping and holding things, we interact with one another socially through the Kinect’s two inputs–voice and touch. The Kinect offers a way to live in society with machines. Think HAL 9000 before he went crazy. There will be great applications, but I doubt very much that the successful ones will look anything like the video.
It’s worth remembering the road that brought us here. Writing for Wired, Tim Carmody gives an excellent overview. Back in 2010, Microsoft representatives were making vaguely threatening comments about Kinect hackers as Adafruit sponsored a $3,000 bounty on the creation of open source drivers for the device. Microsoft quickly reversed course and it was later revealed that one of the Kinect’s designers, Johnny Lee had secretly sponsored the bounty after he failed to convince Microsoft to open up their drivers. In effect, Microsoft has had to be dragged, kicking and screaming, into the future.