When drummer Jason Barnes lost his arm in an accident, researchers at a Georgia Tech lab built him a robotic prosthesis so he could keep playing. But as they designed the arm–adding features like superhuman speed and the ability to improvise–they realized that something similar could change how any drummer plays.
The result is a third arm for drummers that attaches at the shoulder, listening to the music and adding creative beats of its own.
“I think this can hopefully inspire the drummer to think about music and play music in way that he or she wouldn’t otherwise,” says Gil Weinberg, founding director of the Georgia Tech Center for Music Technology. “That’s because we have this principle we call ‘listen like a human, but improvise like a machine.'”
Using microphones and sensors on the drums, the robotic arm knows what the beat is, and can automatically adjust speed and complexity, or shift to ¾ time. Then it uses an algorithm to improvise.
“It understands music based on how humans perceive music,” says Weinberg. “But when it improvises, it uses algorithms that humans would never use. We use fractals, and genetic algorithms, and all kinds of sophisticated processes that require complicated computation, the kind of thinking that humans are not likely to think. The response will probably be different–things that you never heard before, things that a computer thinks.”
The researchers first developed some of the features when they built the prosthesis for Barnes, and realized they could go beyond a simple robot that could help Barnes grip the drumstick. “Once we started, we thought, if we’re already building a robotic arm, why don’t we extend the capabilities to speed and pitch and timber that humans cannot play?” he says. “The main addition was a second stick. The second stick has a mind of its own, it improvises, builds on what Jason is playing, surprises him sometimes, and inspires him to create. It’s not completely under his control.”
The robotic arm also uses a camera to know where it is in space, so it’s always in the right position to play, and it watches as the human arms move. When the drummer moves to the snare, the robot moves to the tom, and vice versa. The lab is also working on an EEG headband that will communicate the drummer’s thoughts to the robot, so it knows in advance when the music is going to change.
The same technology could also be used on other instruments–or in different fields. “This kind of technology, where the arm is actually smart and it knows what you want it to do, either based on your body or based on analyzing your brain, and it can understand the environment … in the future, can be used for doing other tasks, not just music,” Weinberg says. “Tasks that are more technical, like surgery or technical repairs. Imagine painting the wall when you have two arms that help you.”