Microsoft’s new tech demo can look at a photograph of a face and tell you the emotions it shows. You can upload your own photos, and it will scan them, detect the faces, and tell you what your friends were really feeling when you tripped the camera shutter.
Microsoft’s Project Oxford photo research division has come up with all kinds of startling tools in the past, from an app that could tell your age, to the amazing Photosynth, which can grab your vacation photos and arrange them to make a 3-D model of the places you visited. But emotion detection is a whole–perhaps spooky–new level.
The system uses machine learning to process existing images, and builds its predictions on what it sees. And as it is fed more and more images, its suggestions improve.
After processing, a face is broken down into its emotional constituents, with a score card presented for each person recognized in the photograph. The emotions are quantified according to anger, contempt, disgust, fear, happiness, neutral, sadness and surprise. To test it, I uploaded a photograph of the internet’s Grumpy Cat, and the avatar photo of Co.Exist editor Morgan Clendaniel.
The site refused to process the cat photo, but it got straight to work on Morgan’s emotionally ambiguous avatar. The rating engine assigned him close to zero in every category except happiness (0.314) and neutral (0.686). It failed to detect the clear touch of ironic detachment and the subtle air of impatience masked by a professionally impartial smile, but given that human survival has long hinged on our ability to accurately reads the tiniest signs of emotion and intent in the faces of others, it’s not surprise that I can see more in there.
What is interesting is the idea that a machine would ever be able to accurately read emotion at all. As a technology demo it’s an interesting curiosity, but there could be practical real-world uses. In fact, there’s already a number of companies that offer emotion-recognition software of various kinds, such as Affectiva (out of MIT), Eyeris, and Nuance (for voice).
One benign application is in smartphone cameras, which already detect smiles and present you with the best photo selected from a burst. The app could use the emotional content of your snaps to help choose the most appealing images.
The tech might also be deployed to process video in CCTV footage. Airports or camera-blanketed cities might like to flag potential troublemakers based on their emotional state. In an accompanying blog post, Microsoft’s Allison Linn writes that it could be used in marketing, to see what viewers think of a movie for instance, or their reaction to a store display, or to a plate of food.
Or, more likely, people like me will use this advanced tool to analyze photos of their boss.