If you get frustrated enough while trying to remember a password, you may be tempted to flip off your computer, but pretty soon it just might know what that means. Researchers at Carnegie Mellon University’s Robotics Institute have taught a computer to interpret the body language and movements of multiple people—and even the pose of individual fingers.
“We communicate almost as much with the movement of our bodies as we do with our voice,” Yaser Sheikh, associate professor of robotics, said in a statement. “But computers are more or less blind to it.” To improve human-computer interactions, Sheikh and his team developed a method for tracking 2D human form and motion with the help of the Panoptic Studio, a two-story space outfitted with 500 video cameras. They took the data they gathered and taught the computer how to identify certain movements and gestures, or what they call “real-time pose detection” for an entire group of people at the same time.
While Sheikh and his team have already released the computer code for both multi-person and hand-pose estimation to encourage collaboration and development, they will present their findings at the Computer Vision and Pattern Recognition Conference later this month, explaining how, in the future, human-computer interactions could be more natural, and people may be able to communicate with computers simply by pointing at things.