Designer Jono Brandel always wondered what it would feel like to have synesthesia, a neurological condition in which people experience multiple senses at once. To try and replicate the experience, he has created Patatap, an “animation sound kit” that fuses the experience of sound and sight. Go to the site and start pressing keys; each one simultaneously produces a sound and an animated shape, from which you can compose a song that is both audible and visual (note: you can also click on the image above and start pressing keys).
“Certain sounds from songs I listen to jump out at me,” explains Brandel. “It’s fleeting, but when a sound pops back into my head I’ll try to draw it. Then I’ll use that as a point of reference when creating an animation.”
To actually create the sounds, however, Brandel worked with the music composers Lullatone. He would send along his animations, and they would come back with music. Then everyone would work together to hone the colors, sounds. and pictures into “distinct musical sets.” (On Patatap you can play multiple alphabets of sounds and shapes–press the space bar to change things up.)
The software is open source under the MIT License, and Brandel has been intrigued by how some people have chosen to use it. “I didn’t expect people would use Patatap to type their name out,” he says. But his real hope: “It would be amazing and hilarious to see or hear Patatap in a professional setting, say a Beyoncé concert.”