Music and video games have always been intertwined.
But in this video, posted by the Stanford Laptop Orchestra, that marriage goes to an all-new level.
During a recent performance in Stanford’s Bing Concert Hall, the Laptop Orchestra, “a large-scale, computer-mediated ensemble” directed by Ge Wang–the founder of the music app powerhouse Smule–played a piece called Carillon, which is “a networked (virtual reality) instrument that brings you inside a massive virtual bell tower.
Carillon was built using the Unreal Engine, a game development toolset. The orchestra used a combination of Oculus Rift VR goggles and Leap Motion’s hands-free gesture control system, which allowed the musicians to “slave each performer’s avatar arms and hands to the controller,” Leap Motion wrote in a blog post.
“The core interaction in Carillon is the control of a set of spinning gears at the center of the Carillon itself,” co-creator Rob Hamilton said in the blog post. “By interacting with a set of gears floating in their rendered HUD-–grabbing, swiping, etc.-–performers speed up, slow down, and rotate each set of rings in three dimensions. The speed and motion of the gears is used to drive musical sounds…turning the virtual physical interactions made by the performers into musical gestures.”