Microsoft's Kinect Is Now A Sign Language Translator

Researchers in China have turned Microsoft Kinect into a real-time translator for sign language.

A new Microsoft Research project has achieved a major milestone: Real-time machine translation of sign language using Kinect. Researchers in China created the prototype Kinect Sign Language Translator, which understands Chinese sign language and translates it into Chinese in real time using Kinect's sensor set--and vice versa. Professor Xilin Chen of the Chinese Academy of Sciences, who worked on the project, said in a release that "We knew that information technology, especially computer technology, has grown up very fast. So from my point of view, I thought this is the right time to develop some technology to help [the deaf community]. That’s the motivation."

The Chinese team, working with Microsoft Research Asia, combined Kinect's sensors with machine translation technology for the multi-year project. According to Microsoft Research, one long-term goal for the project is to ease communication for sign language speakers in commercial and employment settings.

Add New Comment

1 Comments

  • Yoshi

    Nice research, but very limited in terms of real-world application since Kinect cannot recognise individual finger movement, which is absolutely essential for proper sign language.

    The system's ability to recognise small movements at high speed (which happen a lot during the signing process) is also extremely limited.

    Perhaps when (if?) Microsoft produces its next version of the Kinect hardware, we will see such an application have genuine use in the real world. I hope that the company's researchers continue to work on the project.