How Are You Feeling Today? Soon Your App Could Know

Emotion-tracking technology–using your phone’s camera to monitor your face–is being used for “neuromarketing” already. And it’s coming soon to health, education, and gaming apps.

How Are You Feeling Today? Soon Your App Could Know
[Image: Abstract via Shutterstock]

Imagine if your exercise tracking app could also tell when you looked tired after a run. Or if the online class you were taking could flag the teacher when you looked confused. Today, websites and apps already track our every click and scroll. Soon they could be tracking our emotions, too.


Affectiva, founded in 2009 out of MIT’s Media Lab, is a startup that has made its name over the last few years with its emotion-tracking technology that analyzes people’s facial expressions through the camera on their computer. Its long-term goal is to be the “emotion data layer” between consumers and their ever-multiplying apps and devices, says Affectiva CEO Nick Langeveld.

So far, the company’s main product, Affdex, has primarily been used in the most obvious application from a business standpoint: by advertisers, market researchers, and brand developers seeking to measure reactions to products or messaging.

More recently, however, with the recent release of a mobile software developer kit, Affectiva has been courting customers in other sectors, including health, education, and gaming apps.

Take education: “Online education has a long way to go,” says Affectiva CEO Nick Langeveld. “One of the ways it could become more useable is by having an ability for the instructor to understand the pupils–just as they would in a normal classroom environment. Are they laughing at my jokes? Do they get it?”

It’s not just teachers that may find emotion-tracking. Soon, everyone from doctors to game designers may find uses in better understanding people’s levels of boredom, stress, or engagement as they watch a video or move through a storyline in a game–not to mention individuals themselves interested in the “quantified-self” movement. “We look at the consumer in the middle and all of these different digital experiences around that consumer. In reality, there’s an emotional layer that really sits in between the consumer and those devices and those experiences,” says Langeveld.

So far, Affectiva is just starting to have discussions with these kinds of developers, and Affectiva isn’t the only company devising new strategies to track people’s emotional reactions while carrying or using devices. Langeveld imagines some additional R&D could be needed to adapt its emotion coding schemes to different sectors.


Privacy, of course, is a growing concern for everyone. Affectiva has it written into its contracts that every developer must get the user’s explicit permission before using the camera to track their emotions. If they don’t abide, the company can cut them off. But he hopes that won’t be too much of a problem because people will want to share that sort of data if they get a useful benefit from it–like many people are willing to share their GPS coordinates today to get localized information on their smartphones.

“We as a company are always very careful to be very sensitive to the user experience … making sure that someone knows what they are giving and what they are getting” says Langeveld. “As this little industry that is growing, I want to make sure everyone approaches it with that level of responsibility.”

About the author

Jessica Leber is a staff editor and writer for Fast Company's Co.Exist. Previously, she was a business reporter for MIT’s Technology Review and an environmental reporter at ClimateWire.