Deaf Jam: Experiencing Music Through A Cochlear Implant
When Sam Swiller used hearing aids, his musical tastes ran to AC/DC and Nirvana — loud bands with lots of drums and bass. But after Swiller got a cochlear implant in 2005, he found that sort of music less appealing.
"I was getting pushed away from sounds I used to love," he says, "but also being more attracted to sounds that I never appreciated before." So he began listening to folk and alternative music, including the Icelandic singer Bjork.
There are lots of stories like this among people who get cochlear implants. And there's a good reason. A cochlear implant isn't just a fancy hearing aid.
When his cochlear implant was first switched on, the world sounded different.
"A hearing aid is really just an amplifier," says Jessica Phillips-Silver, a neuroscience researcher at Georgetown University. "The cochlear implant is actually bypassing the damaged part of the ear and delivering electrical impulses directly to the auditory nerve."
As a result, the experience of listening to music or any other sound through the ear, with or without a hearing aid, can be completely unlike the experience of listening through a cochlear implant. "You're basically remapping the audio world," Swiller says.
A Boy's Audio Life
Swiller is 39 years old and lives in Washington, D.C. He was born with an inherited disorder that caused him to lose much of his hearing by his first birthday. That was in the 1970s, and cochlear implants were still considered experimental devices. So Swiller got hearing aids. They helped, but Swiller still wasn't hearing what other people were.
"The earliest memory I have of that awareness is a family picnic around the time I was 4 or 5," he says. He recalls "feeling isolated and separated from everyone, even though it was a picnic filled with family friends."
Even with hearing aids, Swiller was only understanding perhaps one word in three. He relied on lip reading and creativity to get by.
And he took refuge in music, especially grunge rock that cranked up the low-frequency sounds he heard best. "Isolation was a common theme in my childhood and so Nirvana kind of spoke to me," he says.
Swiller didn't know what most of the lyrics were saying until MTV started closed captioning its music videos. But it didn't matter.
"I just love music," he says. "Not just the sound of music, but the whole theory of music, the energy that's created, the connection between a band and the audience and just the whole idea of just rocking out."
Swiller kept rocking out through high school and college. Then, in his late 20s, the hearing he had left pretty much vanished.
By that time, cochlear implants had been greatly improved and had become a common option for people with profound hearing loss or no hearing at all. So Swiller, after giving it a lot of thought, decided to give the new technology a try.
That meant surgery to place a receiver under the skin behind his ear and electrodes in the cochlea, a hollow tube in the inner ear containing nerve endings that carry sound information to the brain. Once Swiller healed from the surgery, the receiver was connected to the external parts of the device: a microphone and processor that resemble a traditional hearing aid.
Swiller says when a doctor switched on his device for the first time, he wasn't prepared for the way people would sound. "I remember sitting in a room and thinking everyone was a digital version of themselves," he says.
The voices seemed artificial, high pitched and harsh. And he couldn't figure out what people were saying. "Your brain is understanding, 'OK, this is a language, but I need to figure out how to interpret this language,' " he says.
Over the next few months, Swiller's brain learned to decode words and sentences. And he started listening to music again, which is when he realized that his tastes had changed.
"All of our senses give us the ability to experience very different worlds," Swiller says.
Lots Of Beat, Not Much Tune
A cochlear implant is designed to do one thing really well — allow users to understand speech.
"Where it is somewhat lacking is more in relating information about pitch and timbre," says Phillips-Silver. Current implants simply leave out much of the information needed to tell the difference between notes that are close together on a keyboard or instruments that sound similar, she says.
To show how this affects listening to music, Phillips-Silver did an experiment that involved people with cochlear implants listening to "Suavamente," a merengue song with a strong dance beat. "There is a lot going on," she says. "A lot of different instrument sounds, there's a vocal line, there's a great range of frequencies. It's fairly intricate music."
Even so, people with cochlear implants had no trouble moving in time with the music, Phillips-Silver says. They also kept the beat when they heard a version of the song that consisted entirely of drum tones.
But when they heard a version played on a solo piano, participants had no success keeping the beat, Phillips-Silver says, because they couldn't rely on the bass or drums.
Better Implants For Music
It should be possible to make cochlear implants more music-friendly, says Les Atlas, a professor of electrical engineering at the University of Washington. But there's a reason current models are so limited, he says.
"There is no easy way to encode pitch as an electrical stimulation pattern," Atlas says. Also, it takes a lot of computing power to encode something like pitch information in real time.
As a result, even the newest cochlear implants provide very little information about pitch, Atlas says. To make his point, he plays a recording of a few piano notes, followed by a simulation of how those notes might sound through a cochlear implant. The simulated version sounds like the same note being played again and again.
So Atlas and other researchers are working on software that allows an implant to convey more information about pitch and timbre. "It explicitly looks for the tonality in the music and uses it in how things are encoded," he says.
When he plays a simulation of the piano music as encoded by his software, changes in pitch are much more obvious. The result may not replicate music exactly, Atlas says, but it offers implant users a lot more information about musical sounds.
"And lo and behold the results we get now on the few people we've tested is that they do get better music cues," he says. "They can hear the difference between musical instruments. The richness of their experience when they listen to music has increased."
The extra information isn't just useful for music, Atlas says. It also should help people with implants who need to understand highly tonal languages like Chinese and Vietnamese. And it could help people with cochlear implants pick out one voice in a room filled with voices.
Even with technical improvements, the experience of hearing the world through a cochlear implant will still be different. But that's a given, says Sam Swiller, who has now spent a decade with his implant.
"All of our senses give us the ability to experience very different worlds," Swiller says. "Even though we're walking side by side we are experiencing a very different street, a very different day, very different colors."
And that's a good thing, Swiller says. Someday we all may be listening to new and different sounds created by musicians with cochlear implants, he says. "When we truly engage each other we get to experience a little bit of each other's world and I think that's where real creativity happens."
9(MDE1MTIxMDg0MDE0MDQ3NTY3MzkzMzY1NA001))
Transcript
STEVE INSKEEP, HOST:
Today in Your Health, we're going to explore what it's like to experience music through a cochlear implant. To people who depend on these surgically implanted hearing devices, a song like this...
(SOUNDBITE OF CREAM SONG, "SUNSHINE OF YOUR LOVE")
INSKEEP: ...May sound more like this.
(SOUNDBITE OF COCHLEAR IMPLANT SIMULATION)
INSKEEP: NPR's Jon Hamilton reports on a man whose love of music has persisted through hearing loss and cochlear implantation.
JON HAMILTON, BYLINE: By the time Sam Swiller turned 1, even loud noises had become faint. Hearing aids helped, but Swiller was living in a different world when it came to perceiving sounds.
SAM SWILLER: The earliest memory I have of that awareness is a family picnic around the time I was 4 or 5, really feeling isolated and separated from everyone even though it was a picnic filled with family friends, young kids running around.
HAMILTON: Swiller says even with hearing aids, he was understanding maybe 1 word in 3. He relied on lip reading and creativity to get by.
(SOUNDBITE OF NIRVANA SONG, "SMELLS LIKE TEEN SPIRIT")
HAMILTON: And he took refuge in music, loud music that cranked up the sounds he heard best - drums and bass.
SWILLER: Isolation was kind of a common theme in my childhood, and so Nirvana kind of spoke to me in a way.
(SOUNDBITE OF SONG, "SMELLS LIKE TEEN SPIRIT")
NIRVANA: (Singing) Load up on guns. Bring your friends. It's fun to lose...
HAMILTON: But Swiller had no idea what the lyrics were saying until MTV started closed captioning its music videos. He says it didn't matter.
SWILLER: I just loved music, not just the sound of music but the whole theory of music, the energy that's created, the connection between a band and the audience and just the whole idea of just rocking out.
HAMILTON: Swiller kept rocking through high school and college. Then, in his late 20s, the hearing he had left pretty much vanished. So in 2005, he swapped his hearing aids for a cochlear implant. One part of the device is implanted under the skin behind the ear. It receives information from a microphone and processor that are worn like a traditional hearing aid. When a doctor switched it on for the first time, Swiller wasn't prepared for the way people would sound.
SWILLER: I remember sitting in a room and thinking everyone was a digital version of themselves.
HAMILTON: The voices seemed artificial, high-pitched and harsh, and he couldn't figure out what people were saying.
SWILLER: You're basically remapping the audio world. And so your brain is understanding, OK, I understand this is a language, but I need to figure out how to interpret this language.
HAMILTON: Which he did over the next few months. And he started listening to music again. But through the implant, Nirvana was less appealing.
SWILLER: So I was kind of getting pushed away from sounds that I used to love but also being more attracted to sounds that I never really appreciated before.
(SOUNDBITE OF SONG, "STONEMILKER")
BJORK: (Singing) A juxtaposition in fate.
SWILLER: I started to like a lot more folk music and a lot more vocals. So, like, Bjork is a good example.
(SOUNDBITE OF SONG, "STONEMILKER")
BJORK: (Singing) coordinates.
HAMILTON: A cochlear implant isn't just a fancy hearing aid. Jessica Phillips-Silver, a neuroscience researcher at Georgetown University, says the devices work in very different ways.
JESSICA PHILLIPS-SILVER: The hearing aid is really just an amplifier. The cochlear implant is actually bypassing the damaged part of the ear and delivering electrical impulses directly to the auditory nerve.
HAMILTON: The brain has to learn how to interpret these impulses, and every brain does that a bit differently. Phillips-Silver says another factor is that implants simply ignore some of the sound information in music. She showed how this affects listening in an experiment that involved people with cochlear implants responding to some dance music.
(SOUNDBITE OF SONG, "SUAVEMENTE")
ELVIS CRESPO: (Singing in Spanish).
PHILLIPS-SILVER: It was called "Suavemente" by Elvis Crespo (laugher). It's one that's heard commonly in the clubs, and it gets people going.
HAMILTON: At first, people with implants responded just like other people. They began moving in time with the music. But which sounds were they responding to?
PHILLIPS-SILVER: There is a lot going on. There's a lot of different instrument sounds. There's a vocal line. There's a great range of frequencies. It's fairly intricate music.
HAMILTON: So Phillips-Silver had participants listen to several stripped-down versions of "Sauvemente."
(SOUNDBITE OF STRIPPED-DOWN SONG, "SUAVEMENTE," #1)
HAMILTON: When the volunteers heard this drum-tone version, they had no trouble keeping the beat. But when they heard this piano version, they had a lot of trouble.
(SOUNDBITE OF STRIPPED-DOWN SONG, "SUAVEMENTE," #2)
HAMILTON: Phillips-Silver says that's because what a cochlear implant does really well is transmit the information needed to understand speech.
PHILLIPS-SILVER: Where it is somewhat lacking is more in relating information about pitch and timbre - so, for example, being able to tell the difference between notes that are close together on a keyboard or being able to tell the difference between two similar instruments.
HAMILTON: Other scientists are trying to make those things possible. Les Atlas is a professor of electrical engineering at the University of Washington.
LES ATLAS: There is no easy way to encode pitch as an electrical stimulation pattern. That's problem number one. Problem number two is to be able to do that in real-time - that means as the music is coming out - is a difficult problem.
HAMILTON: As a result, even the newest cochlear implants provide very little information about pitch. Take a simple tune on a piano.
(SOUNDBITE OF MUSIC)
HAMILTON: Through an implant, it may sound more like this simulation.
(SOUNDBITE OF COCHLEAR IMPLANT SIMULATION)
HAMILTON: So Atlas and other researchers are working on software that allows implants to convey more information.
ATLAS: It explicitly looks for the tonality in the music and uses it in how things are encoded.
HAMILTON: Instead of this...
(SOUNDBITE OF COCHLEAR IMPLANT SIMULATION)
HAMILTON: ...The implant sends a more complicated signal that allows the brain to decode information about pitch.
(SOUNDBITE OF COCHLEAR IMPLANT SIMULATION)
ATLAS: And lo and behold, the results we get now on the few people we've tested is that they do get better music cues. They can hear, not perfectly but much better, the difference between musical instruments. The richness of their experience when they listen to music has increased.
HAMILTON: Atlas says the extra information also should help people with implants who need to understand highly tonal languages like Chinese and Vietnamese. Even with technical improvements, the experience of hearing the world through a cochlear implant will still be different, but Sam Swiller, who's had his implant for a decade now, says he's OK with that.
SWILLER: All of our senses give us the ability to experience very different worlds, and even though we're walking side-by-side, we're experiencing a very different street, a very different day, very different colors. And so when we truly engage each other, we get to experience a little bit of each other's world, and that's where I think real creativity happens.
HAMILTON: Jon Hamilton, NPR News. Transcript provided by NPR, Copyright NPR.
300x250 Ad
300x250 Ad