Even dabblers in the world of selfie-taking know how to cheat the system with filtering apps on their phones: Basically, you just use the “Sierra” filter on Instagram, which makes everything look 15% nicer, and then call it a day. But a new app from researchers at MIT, Adobe Systems, and the University of Virginia, has the potential to seriously step up the “style transfer” game–especially when it comes to pictures of faces. As MIT PhD student YiChang Shih and team discovered, it’s possible to create a filter that responds to faces in ways more akin to how a professional photographer would light a subject in a studio. Rather than just apply a “global transfer”–one which applies specified image parameters the same way every time–MIT’s filters adjust each part of the image, each part of the face, differently. It’s a heck of a breakthrough–and Shih explained to us how it works.
“The user provides an input headshot of themselves–clean, not too noisy, no handshake blur. Also, try to avoid hard shadows,” he says. “Cell phone pictures under daylight or room light usually work quite nicely.” Then, he explains, users choose a style by a photographer’s name. Currently, the algorithm is set up to allow for the styles of photographers like Diane Arbus, Richard Avedon, and Martin Schoeller–and then the method Shih devised will perform the filter.
“The method has three steps,” Shih explains. “First, our algorithm starts to search for a good matching portrait among that photographer’s work–it usually has similar features to the input, such as skin color, beard, accessories like glasses. Then our method computes ‘dense correspondence’ between the matching portrait and the input–this means eyes-to-eyes, nose-to-nose between input and the matching portrait. Finally, we use this correspondence to perform ‘local transfer,’ which employs a multi-scale approach to the transfer of the local visual style from the matching portrait to the input.” Additionally, it’s set up to add some additional features, like eye highlights to mimic studio lighting.
In other words, Shih’s algorithm takes the very specific styles of the world’s most famous portrait photographers, and allows anybody with a cell phone the chance to apply those styles to their own selfie–eventually, anyway. A consumer-facing app, which Shih says he hopes to complete, isn’t yet available. But before long, you’ll probably be able to convince people viewing your OKCupid profile that Richard Avedon shot your profile picture–if that doesn’t impress a potential date, what will?