In just over a year, a technique called “style transfer” has discombobulated everything I’d figured out about art. By feeding many van Goghs into a piece of software, style transfer can deconstruct his strokes of paint–and then copy them, turning any photo into a conceivable mashup with Starry Night. Artificial intelligence had done something that a century of artists and scholars could not: replicate van Gogh’s unique visual genius.
But does that make style transfer AI an artistic genius? Or is it more the equivalent of a monkey that’s been taught to play pseudo-Bach hymns on a recorder?
Ask Ahmed Elgammal, researcher and computer science professor at Rutgers University, and he’s likely to say style transfer is the latter. Elgammal is going further than training an AI to simply copy art masters. Instead, he trained a system to deviate from what it’s been taught–and to, hopefully, create the next unexpected masterpiece.
“Style transfer systems start with a photo and try to transfer some style into it . . . This is not our goal,” says Elgammal, who worked in collaboration with College of Charleston. “Our system generates novel images from scratch, without any human in the loop who chooses the composition or what style to transfer. Our goal is to simulate the artist’s creative process, or at least some aspects of it, which is the role of deviation from established styles in creating new art.”
In other words, Elgammal wants to create AI that thinks like an artist–or whole schools of artists–by looking to its contemporaries, internalizing what they’re doing, and creating its own unique spin. It’s exactly this process, his team theorizes, that gives rise to new art movements. Realism gave rise to impressionism, which in turn gave rise to expressionism. Art continuously builds upon itself through improvisation and critique.
To teach a machine to do this, Elgammal’s team riffed upon the hottest tool in machine learning today: a Generative Adversarial Network (or GAN).
The best way to understand a GAN is not to think about it as one AI, but two AIs that have been pitted against one another. They’re two virtual players competing or criticizing one another in order to make the virtual brain of the greater neural net smarter.
A normal art-generating GAN works like this: Player One is loaded with thousands of images. Player Two is told to create images from scratch to fool Player One into thinking it’s looking at art. After countless attempts, in which Player One rejects or accepts Player Two’s work, eventually, Player Two might learn to make something that looks a lot like art, consistently–or even copy a specific style like van Gogh. But what you get with this approach is a system that’s good at copying old art styles, not creating new ones.
Elgammal’s GAN works a bit differently. Player One is loaded with art of various styles. Player Two tries to make art from scratch. But in this case, Player One doesn’t just label something art, or not art. It also tells Player Two whether or not the art looks like any known style. Player Two is motivated to create something that passes for art, but also passes as a totally new style of art–something outside of Player One’s ability to pin it as falling under any single art movement.
What did the system manage to generate? You can see its creations throughout this post. “I expected to see weird, ambiguous, scary, and ugly images; or the best scenario, what would be good imitations of typical traditional art,” says Elgammal. “The results were surprising as they show novel yet aesthetically appealing art, in ways that exceed our expectation. I guess, our design was successful to push the machine to explore parts of the creative space that is novel, yet not too shocking in a negative way.”
While the pieces are, predictably, hard to describe, Elgammal is right in that they’ve mostly all achieved an aesthetically pleasing baseline. Some look more like photos, others veer toward abstract expressionism. I see hints of everything from Rothko’s color blocks to lazy Adobe Illustrator filters. Sometimes I see what might be genius. Other moments, idiocy.
But of course, this is exactly how I critique the works inside any real art gallery. My fellow humans, when asked, believed that Elgammal’s neural net was making real art about 75% of the time. And that’s quite a high bar, given that human subjects tested as a control labeled actual abstract expressionist works as real art 85% of the time, while a collection of Art Basel pieces were only labeled as art 48% of the time.
Elgammal’s system certainly seems to be successful in creating novel art. But does it prove that a neural net could create something we might consider to be the next art movement–the evolution of art itself?
“Is what it generates the next art trend? It is hard to answer that,” admits Elgammal. “Definitely, we see a variety in what the algorithm generates. What might constitute a trend or consistent style depends on various other factors. So the algorithm explores novel possibilities.” Indeed, because art’s cultural value rarely comes down to what you or I think about it. Its value is ultimately determined by critics, curators, and billionaires. And, I guess, Generative Adversarial Networks.