Your New Robot Overlord Turns Out To Be A Pretty Good Marimba Player

Looking for a musically sensitive, responsive bandmate? Maybe you should try out Shimon.

Shimon is a jazz-playing robot created by Gil Weinberg and his team at the Georgia Tech Center for Music Technology. Shimon was debuted seven years ago, but the robot gave a performance alongside a human collaborator yestererday at Moogfest in Durham, N.C., a festival concentrating on the intersection of music and technology. The video was captured by Quartz reporter Mike Murphy, who has written about Shimon.

This four-handed robot was designed not just to play music, but to listen and improvise alongside a human — each pushing each other to new creative ideas. In the video, you see the robot responding with something of a human's sense of pacing and phrasing, changing tempos appropriately and seeming to take in his human colleague's material and riffing on it in a way that makes harmonic sense.

In the performance, Shimon's head bobs along with the groove. That particular anthropomorphic touch isn't just for novelty's sake. According to the description of Shimon on the Georgia Tech website, "An embodied anthropomorphic robot can create familiar, acoustically rich, and visual interactions with humans ... the visual connection between sound and motion can allow humans to anticipate, coordinate and synchronize their gestures with the robot." That is, Shimon's head motions are supposed to help the human anticipate the robot's next moves.

Back when Shimon made his debut in 2009, Weinberg said in an interview with NPR's Robert Siegel on All Things Considered, "The whole idea is to use computer algorithms to create music in ways that humans will never create," Weinberg said. "Our motto is, 'Listen like a human, but improvise like a machine.' "

When you close your eyes and listen to them perform together, it feels quite organic, like two human percussionists working together. (Or really three, considering Shimon's extra arms, which the robot starts employing around 2:32.) But visually, the spectacle of human and robot responding to each other's playing is quite marvelous.

Shimon isn't the only music-minded machine that has come out of this Georgia Tech group. Its other creations include Travis, a smartphone-enabled speaker system/"music companion" that responds to the music you play, including swiveling its built-in speakers toward you as you dance around and making new music recommendations.

Another is a robotic drumming arm developed for amputees. Working with a drummer named Jason Barnes, Weinberg and his team built a drumming prosthesis that includes motors and two drumsticks. One stick is controlled by the musician's bicep muscles and electromyography (EMG) muscle sensors; the other stick "listens" and improvises. And as you can see and hear, it can zip right along.