Congratulations. That genius idea to set a 30-second video of your baby’s first steps to the tune of Dave Matthews Band’s “Crash” just became the bajillionth baby’s-first-steps video to use the same tune. And now every Facebook friend you shared the clip with is furiously clicking to block or defriend you. (We’re looking at you, too, guy who set the video of his dog hanging out the car window to Louie Armstrong’s version of “What A Wonderful World.”)
You deserve better. Your kid deserves better. The world deserves better.
“There’s no reason in this world that we should have shitty sound anywhere,” says Hans Zimmer, the multiple-Grammy-winning composer for the scores of more than 100 films, including The Lion King, The Dark Knight trilogy, and Inception.
Zimmer co-created an app called VJAM (now in the iTunes store) that uses patent-pending technology to add custom-tailored micro-scores and matching video filters to almost any video.
VJAM is the consumer-facing offshoot of the enterprise-focused UJAM service, cofounded by Zimmer, Pharrell Williams (with whom Zimmer composed and produced music for the 84th Academy Awards); sound engineer, computer scientist, and UJAM CEO Axel Hensen; Chief Visionary Officer Peter Gorges; Chief Music Technology Architect Paul Kellett; and Funny Or Die board member, venture capitalist, and UJAM angel investor Mark Kvamme.
Plenty of apps in the style of Instagram for video target the new mob of iPhone auteurs by letting them slap music and filters on sharable mini movies–Viddy and SocialCam are two of the biggest players in the space. Still others let you edit your videos to fit the length of songs from your iTunes library or let you layer just a piece of a song over your clip (which means you end up with the intro or ending instead of the part that best helps tell your story).
VJAM’s breakthrough involves an algorithm that matches the right musical moments with precise video moments. In other words, it makes the music as important as the images to help make your videos stick–whether you’re hoping to impress grandma with clips of the grandkids during the holidays or hoping to score the next viral hit.
The free version of VJAM comes with 12 free themes (soundtracks plus suggested video filter pairings); five more themes such as Jack Frost (below) or Pulp Retro are available for an additional $.99. Clips can be up to 30 seconds long in the free version. In the pro version, on sale for a limited time for $2, clip length is unlimited (though Hensen recommends keeping it under 10 minutes to avoid long upload and render times). And the paid version comes with three additional themes. Hensen says the company is adding new themes for sale every week–that’s VJAM’s main revenue stream.
What you won’t find in either version of the app are songs by Lady Gaga, Psy, Katy Perry, or any well-known pop star. Songs are instead supplied by third-party partner Extreme Music and are written or chosen to be adaptable to video or film. Zimmer did write the score called “Woad To Ruin” on the “Slo Mo Hero” theme, which comes with the free version.
Hensen says he could see a day when pop artists sign on to create VJAM-specific music. But for now the music for VJAM is a select piece of a larger catalog offered to clients of VJAM’s parent company UJAM–music supervisors and other sound pros already use the service to manually manipulate thousands of songs for their video or film projects. The consumer-facing VJAM app handles the difficult cutting and pasting automatically for its recreational users.
“Basically our algorithms try to build a perfect arc for the music so it creates something meaningful,” Hensen tells Fast Company.
To highlight what’s meaningful, VJAM pairs musical moments with sad, happy, mysterious, or hilarious moments it identifies in video clips. “We’re trying to find out how much energy is contained in a certain section of the song,” Hensen says. “And then we try to rearrange the music so the energy works well over the course of the video.” He wouldn’t reveal exactly how VJAM’s algorithm identified “energy,” but on the music side, he said it has to do with which instruments play at various points and whether they’re playing verses or choruses, for example.
Music might not be able to generate excitement from scratch for inherently bad home movies, but it can make them more engaging for those viewers who are already somewhat interested. The right sound at the right time has the power to transform a dull scene into an emotional experience (try watching the midnight swimming scene in Jaws or the shower scene in Psycho with the sound down). By shuffling around parts of the song or cutting out or beefing up certain instruments during certain parts of the video, a piece of music can also help tease out the beginning, middle, and end of an otherwise meandering (boring) clip. It can help tell a story.
In the clip below, it’s the plucky notes that make you realize there’s some tinkering going on with this snowman, and the song resolves when the job’s done.
Zimmer says the whole idea for VJAM started with the idea that he and his cofounders always have tunes in their heads. “We started off by trying to figure out how we could get anybody who has an idea for a tune to realize it.” He calls the VJAM algorithm a “bit of trickery and magic,” but the result, he says, is that, “suddenly that incredibly dull movie of your child’s first dinner that you’re boring all of your relatives with becomes incredibly exciting, and you’re not going to incur the wrath of people around you anymore.”
Music and emotion are concepts Zimmer has rubbed together for decades. He’s used it to help make the cartoon The Lion King feel like an epic. And he worked closely with director Christopher Nolan to built the sonic landscape of Gotham City in the Dark Knight trilogy. Nolan also relied on Zimmer for Inception. “Music is how you can get away with a very abstract story like that,” Zimmer says. “It was a subliminal way to take you on that journey. It was emotionally comprehensible, even if you somehow missed the odd line or, intellectually, have a problem rationalizing it, which is okay… Who has time to rationalize?!” Even if your video doesn’t involve a dream within a dream within a dream, music can still help it make sense, Zimmer says.
He has put sound to use in apps before he helped create VJAM, too. The Inception app, for example, uses sound to send users on an actual journey in a virtual dream. And sounds and music he composed play heavily in the app for The Dark Knight Rises.
Writing music for smartphones has presented unique challenges that are as complex as those associated with music for big screens, Zimmer says. “Nobody’s embraced the technology and actually started writing music differently–music that takes into account what the tech can do. That’s what we’ve started.”
[Image: Flickr user Murat Livaneli]