What do you do post-New Year’s Eve when you’re an AI artist? You train a neural network to take your smartphone footage of fireworks and generate new ones.
At least, that’s what Google Arts & Culture artist-in-residence Mario Klingemann did. He used a neural network called Pix2Pix that tries to anticipate the next frame in a video, and trained it using pairs of consecutive frames from his fireworks videos. Because the algorithm only knows what occurred in the previous frame, it often doesn’t work very well. For instance, Klingemann has experimented with this technique before using a video of human motion. Instead of creating a visually interesting movie, it “just became a beige soup,” he tells Co.Design in an email. “So every [type of] footage leads to different results–a few work, many don’t,” he continues. “But I had a gut feeling that fireworks might work since there is constant alternation between creation and destruction.”
Klingemann compares the swirls of his generated fireworks to Turing patterns, which occur frequently in nature–for instance, the spots in the fur of a leopard or the complex rippling patterns found on fish or in seashells. “[Turing patterns] are temporary islands of stability in a sea of chaos shaped by opposing forces,” Klingemann says. “Ah, you see I can get quite philosophical when I look at these systems because I believe that in some way these patterns happen in any system, not only physical ones. I am pretty sure human societies and communities show these patterns, too–it is only hard to see them.”
Klingemann plans to take his fireworks experiments and turn them into higher-resolution artworks based on the same algorithmic principles. (One such work will be on display in an exhibition that opens next week at the Boston Cyberarts Gallery.) But he also hopes to investigate how the same algorithmically predictive mechanisms could create what he calls an “inspiration engine” that could function within another neural network. That engine, he says, “could act as a constantly mutating seed that other models can build upon.”