Some art institutions get so wrapped up in preserving the past that they fail to properly engage the future. The Smithsonian American Art Museum (SAAM) is not one of those institutions. Fresh off unveiling a virtual reality Burning Man installation this past August, SAAM turned one of its robotic tour guides, Pepper, into a multimedia installation that transformed paintings into ambient synthesizer music. At an event on Tech Family Day last month, Pepper was found engaging with visitors, capturing their emotions, and interpreting them with a selection of a painting and ambient tunes.
Rachel Goslins, director of the Arts & Industries Building, who kickstarted the Pepper project, said turning Pepper into a synth player grew out of the museum’s work with SoftBank Robotics USA. The company, which created Pepper, offered to donate 100 Peppers to the Smithsonian to experiment with visitor engagement and education. While the Smithsonian had been running various experiments with Pepper, Goslins says she turned to Ian McDermott, a member of the Hirschhorn Museum’s ArtLab, a digital media studio for teens, to create an experience for Tech Family Day.
“[They] coded and built this new way of using Pepper,” says Goslin. “It’s organic, and fun, and extraordinary.”
The Werkstatt processes the signal in dissonant, pulsating tones. Some are pleasant bleeps and blops, while others are higher pitched squeaks. The Werkstatt then sends these pulses to two Moog Mother-32 semi-modular synths, which then route the sounds to an Eurorack modular system, where other effects are applied to create a more complex and melodic mix.
“Things like tempo will be shifted based on the colors perceived by Pepper,” says McDermott. “You end up getting this ambient music that’s running from images from the museum, and the images get translated to sound ultimately through a series of relay points.”
Passing the Pepper
Later this fall, the Smithsonian hopes to roll out accessibility programs with Pepper. The museum’s accessibility team already does verbal description tours, where someone describes what a painting looks like to visually impaired guests. With Pepper, the museum can establish a direct correlation between the painting and sound waves. McDermott’s project for Tech Family Day was thus initiated, in part, to hack a solution for future accessibility programs.
“Where I take this from here is Pepper can now go through a gallery and point its hand, or I can grab the data from her cameras, translate that light data, and run it through the system,” McDermott says. “With that idea, we can have Pepper go through the galleries and remix existing works into a way that they weren’t intended to be absorbed, sonically rather than visually, or open it up to provide accessibility to the visually impaired and give tours that way.”
Beyond the Smithsonian’s efforts in accessibility, education, and making artworks into multimedia experiences, there is another element at play. While Pepper’s ability to read human emotions and turn this data from visual art into sound art is intriguing, the Smithsonian seems to have unexpectedly engaged with one of the chief concerns of the future—how humans and robots will interact as artificial intelligence advances. Figures like Elon Musk, and the recently passed Stephen Hawking, are keen on sounding alarm bells about AI and robots. But these creations, and the programs they run, needn’t be adversarial. Gosling seems to agree.
“On a philosophical level, art is an emotional form of creation, right?” she muses. “It’s a bit poetic to think that we’re lending some of our human emotions to a robot, to create a new form of artistic expression.”
In the future perhaps, humans and robots can learn and create together. And what better way than by interpreting and fashioning new artistic experiences?