Two years ago, a new, freely distributed AI software called “Deepfakes” enabled the public to melt reality by placing anyone’s head on someone else’s body in any video. Deepfakes is powerful, scary, and just labor intensive enough that our world hasn’t imploded yet. The AI’s biggest challenge is that for the tech to work convincingly, you have to collect hundreds of videos and images to create a digital mold of the person you want to impersonate.
But what if creating a digital clone didn’t require all this work? What if you could fake someone from a single photo? That’s the promise of new research out of Samsung’s AI lab. Starting with just one photo, Samsung’s latest AI technique can animate the 2D image into a convincing, full motion video. They’ve animated Britney Spears, Neil Patrick Harris, Marilyn Monroe, even the Mona Lisa herself.
Their system learns what the core geometry of someone’s face looks like so it can apply that like a mask to someone else’s face in a talking head-style video (like interviews or YouTube selfie monologues).
Unlike Deepfakes, which uses a technique called a GAN (or generative adversarial network), in which one AI draws pictures in attempt to fool another AI until it gets really really good at forgery, Samsung’s system starts by scanning landmarks on someone’s face to understand how they will move about a frame. What it sees is a simple line drawing of a nose, mouth, eyes, eyebrows, and a chin–the sketch looks like a Matisse portrait. And that portrait informs all the digital fakery that ensues next, what the researchers dub “puppeting.” Training the system on a single photo can work well or work terribly. But eight photos works better. And 32 photos looks pretty perfect, to my eye.
Another great paper from Samsung AI lab! @egorzakharovdl et al. animate heads using only few shots of target person (or even 1 shot). Keypoints, adaptive instance norms and GANs, no 3D face modelling at all.
▶️ https://t.co/Xk5D4WccpD
📝 https://t.co/SxnVfY72TT pic.twitter.com/GjVrJbejT0— Dmitry Ulyanov (@DmitryUlyanovML) May 22, 2019
Of course, most of us would never see the puppeting behind this tech. We’d just see the finished product, unaware that one person is “wearing” someone else’s visage like Arya Stark in Game of Thrones.
But for now, Samsung’s research is just that: Research. The techniques are publicly available, but there is no user-friendly software tool to turn that science into an instant forgery machine. Well, yet. Given that you could deploy this technique with a single photo, it seems like just the sort of AI that’s waiting to become the next silly, or uncanny, or just highly memeable app. And if the last year of software has taught us anything, there’s a lot of interest in commercializing the sort of tech that made Deepfakes an internet sensation.
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.