Discussions of AI and machine learning are often buttoned-up–and for good reason, considering their society-shaping impact. But art made by neural networks has become a field in its own right, full of people training algorithms to emulate people’s faces, fireworks, and even naked women. This often silly, irreverent work isn’t just fun–it may help us understand AI.
The Munich-based Google artist-in-residence Mario Klingemann’s Twitter feed is a delightful mess of neural experimentation. He shows, in near real-time, his explorations training neural nets on various types of data. The results of his attempts are often humorous, showing just how bad algorithms are at conjuring something as complex as a human face. That’s the point, though: By watching how these neural nets learn from something as innocent as an old painting, we get a glimpse at the state of the art of AI.
Right now, Klingemann is focusing on oil portraits from before 1900. He’s built a photorealistic face generator, based on NVIDIA’s pix2pixHD algorithm, by training it on a few thousand paintings from predominantly European artists before 1900. The resulting faces, created by a machine trying to see the world as an Old Master might, veer between believable and laughable.
“If you look at art history, it’s clear that faces have fascinated artists since the beginning of culture. I guess one of the reasons is that faces are easy and very hard at the same time–you can draw a recognizable face with just a few lines or you can try to reproduce one down to the last pore,” Klingemann tells Co.Design via email. “The difficult part is that every human is an expert in human faces, we notice the slightest changes in expression or if some proportions are somehow not right. Which means that if you paint or generate a face, slight changes might tell a different story or slight errors will become immediately visible.”
Can you tell the difference between these two images–one of which is an oil painting by a human, and one of which came from a neural network?
It's still possible to tell the fake from the real one, but it does get harder. So, which one is the generated one? pic.twitter.com/uUWXyyQx9V
— Mario Klingemann (@quasimondo) April 4, 2018
At first glance it’s hard to tell. But with a closer look, you can see some strangeness going on with the oddly black left eye and the dark, shadowy goatee of the righthand image. For Klingemann, getting a neural net to produce something this good is a technical challenge–and in this case, it’s still not quite good enough. “It is very easy for me to see how good the model works, especially in the details, since any errors will stand out as uncanny or simply wrong,” he says. He also acknowledges that, because the model is trained on centuries-old images of largely middle-aged European men and younger European women, most of the faces are white–so he’s looking for more source images to diversify his training data.
For Klingemann, training neural networks is also an artistic challenge, a creative experiment that relies both on human and machine. “Having a face generator is like having a story generator,” he says. “Every face or grouping of faces will trigger some associations, question, or even emotions. And of course the aspect that a machine does this gives it an interesting twist.”
In fact, through his experiments, he’s found that generating portraits that look like 19th-century paintings is far easier than creating photorealistic portraits. “When we look at a painting we are much more forgiving about things that do not look right, since we cannot really be sure that this was not the intent of the artist,” he says. After all, how many old portraits have such a bizarre sense of human anatomy (for reference, see Creepy Renaissance Babies) that they look like they could have been generated by a computer?
Take Ecce Homo, a botched restoration attempt of a 1930 painting of Jesus that blossomed into a viral meme in 2012. Klingemann created his own algorithmic version of the painting, and it’s just as creepy–and hilarious–as the original.