Orbeus Trains Machines To Recognize Faces, Emotions, Broccoli

Never mind Facebook or, a new visual recognition engine uses a machine-learning algorithm to spot objects and scenes, logos and products, even gender and emotions–all to better target you.

Orbeus Trains Machines To Recognize Faces, Emotions, Broccoli

Facebook has been taking a lot of flack lately over, the startup it reportedly acquired for roughly $60 million for its facial-recognition technology, which is creating a whole new set of privacy concerns for the social network. But a new visual recognition engine developed by startup Orbeus aims to go far beyond facial features.


Today, the founders of Orbeus presented their system at Chicago-based accelerator Excelerate Labs’ demo day, showing off a machine that can detect everything from emotions and gender to objects and scenes, and logos and products. By mining an ever-growing database of images, the Orbeus team can teach its machine-learning algorithm to recognize myriad photo features and categories. Xing Meng, CEO of Orbeus, believes the technology could have a huge impact not only on online social sharing, but also on advertising and content delivery. “Every time you upload photos to the service, we could learn your taste or general patterns from the images, which could allow more interested ads or content based on the context of your photos,” Meng says.

When uploading a service onto Orbeus’ Rekognition tool, an API third-party developers can integrate with their own services, Orbeus scans the photo against its expansive database of categorized images to determine certain characteristics. If the photo is of a person’s face, Orbeus can recognize gender, approximate age, and even certain emotions. It can also recognize scenes, such as if there is a beach or forest present in the photograph, and objects, such as products, logos, or even various types of foods. “Certain categories we can train for a pretty high degree of accuracy,” Meng says. “So instead of recognizing foods as vegetarian because there is a lot of green in terms of the texture, we could actually soon recognize broccoli. But distinguishing between living rooms or bedrooms or living rooms for scene recognition–well, that’s a little more complicated.”

With that level of specificity, Meng imagines the technology could help developers serve up more tailored content. “We tested it on one of our cofounders, and found that almost 50% of the food photos he took were of vegetarian food,” Meng says. “Yet Facebook is still pushing Ruth’s Chris [Steak House] ads? It’s clear he’s likely a vegetarian.”

Of course, it’s still early on in the startup’s life, and its technology is far from perfect. When scanning a few photos–you can test the system yourself here–we ran into a few complications. For example, Orbeus guessed that Apple iOS SVP Scott Forstall was just 22 years old, when he’s actually twice that age. Orbeus was also unable to recognize the Mercedes logo, instead pegging it, bizarrely, as a 31-year-old female. But other tests were more successful, and the Orbeus team touts that its system has shown better results in certain instances than Facebook’s platform.

But the more accurate the technology becomes from Orbeus and Facebook, the more critics are likely to have privacy concerns. It’s one thing for Orbeus to recognize the name of one of my friends, but to then guess how my friend is feeling, his approximate age, the food he’s eating, and the products present in his kitchen? That might be too much information for some.

And it’s quite possible Orbeus hasn’t even dreamed of all the applications of its technology. It demoed an experimental app today to show off what its technology could do in a social setting, such as organizing friends based on similar interests seen in photographs (sailing, swimming, and so forth). It’s also already working with some 200 developers who are taking advantage of its API.


Says Meng, “When Facebook acquired, it left a big hole in the market.”

[Image: Flickr user Michael Glasgow]

About the author

Austin Carr writes about design and technology for Fast Company magazine.