March 12, 2015
The squirrels are wary at first.
They carefully sniff at the traps set on the chilly ground of Alaska’s north slope, suspicious of their sudden arrival. But soon, unable to resist the temptation of the small bits of carrot set as bait, they venture inside and snap! They find themselves behind bars.
The traps were set by Michael Sheriff, an assistant professor of mammalogy and ecology at Penn State. He routinely travels to northern Alaska to study the body temperatures of arctic ground squirrels and how they fluctuate throughout the year as the squirrels go about their daily lives.
The temperature changes, which are monitored with internal sensors that Sheriff and his colleagues Brian Barnes and Loren Buck from the University of Alaska surgically implant in the squirrels, show exactly when they hit milestones like the beginning or end of hibernation or mating season.
After a year passes, Sheriff recaptures the squirrels and removes their sensors so he can collect the data. This past year, he shared the collected information with Mark Ballora, associate professor of music technology, back at Penn State.
Ballora is interested in sonification: the process of taking large data sets — like the results of monitoring a group of squirrels’ body temperatures for a year — and translating them into musical audio files. The resulting sonification illustrates the pattern of the data while being pleasing to the ear. A sonification of an earthquake, for example, might get louder as the earthquake gets stronger.
“There’s something about music; everyone reacts to it. People are so responsive to sound,” says Ballora. “So if you can listen to a data set instead of just look at it, maybe that can tap into or create an experience or understanding you wouldn’t otherwise get.”
Sheriff hopes the sonifications will help him reach new audiences for his data, like younger students who may respond better to sound than looking at graphs. The research will help scientists — both established and emerging — better understand how animals adapt to shifting environmental conditions, like those from climate change.
“By studying squirrels in Alaska, I’m trying to better understand how animals respond to changes in snow cover. When they end hibernation and breed, for example,” says Sheriff. “Climate change affects and alters when snow falls in winter and melts in spring, so I’d like to learn more about animals’ ability to respond and alter the timing of these life events.”
The sonifications of Sheriff’s data follow the rise and fall of the squirrels’ body temperatures. The data controls the behavior of the sound (whether the audio gets louder or softer, or deeper or higher, for example), but Ballora has some creative license when it comes to what type of sound will represent which types of data.
When facing a new sonification project, Ballora says he begins by making sure he understands what the data means and what it is he’s sonifying. He does additional research for context, if necessary. Then, he moves on to deciding what the sonification will actually sound like. Ballora says he chooses sounds based on which best represent the data’s behavior and melody.
“If I’m sonifying data having to do with hurricanes, then making the sonification sound like swirling wind might be an easy decision,” says Ballora. “But with Michael Sheriff’s squirrel research it was a little more difficult. What does a body temperature sound like?”
Ballora says he eventually went with rich, active and warm tones.
“Sometimes the first few tries will sound awful,” he says with a laugh. “But you just need to go in and tweak until it sounds the way you’d like it to. You need to make gut musical decisions while keeping the data accurate.”
Further fusing music with science and technology, Ballora creates the sonifications with the help of an audio synthesis program and programming language called SuperCollider that can be used to filter and modulate sound.
Once the sonifications are complete, Ballora hands them off to the original researchers, who use them for various purposes.
In December 2014, Jenni Evans, a professor of meteorology at Penn State, took sonifications Ballora made using hurricane data to the International Workshop on Tropical Cyclones in Korea. People from every country affected by cyclones attended the conference, and Evans says forecasters in particular were excited by the sonifications, which were played in conjunction with satellite animations.
“The addition of audio really grabs the attention of people who are used to seeing just the animations,” says Evans. “Some attendees said the sonifications helped them understand the changes in the cyclones’ intensity better than if they were just viewing the data.”
Also wanting to share their project with the public, Ballora and Sheriff will present and display their squirrel data sonifications on March 27 at this year’s Polar Day — an annual outreach event for The Polar Center, Penn State University.
Ballora’s no newcomer to Polar Day, though. Last year he presented the work he did sonifying data showing changes in the Antarctic ice throughout the past 400,000 years. He created six sonifications, with each one covering a set of data such as sea level, ground ice, floating ice and more. Ballora compiled them in a seventh track that played all six simultaneously.
Mark Ballora worked with Jenni Evans to sonify data from cyclones like Hurricane Sandy.
Helping Ballora with the sonifications was Matthew Kenney, a Penn State graduate student pursuing his master’s in fine arts degree in new media. Kenney and Ballora took turns composing and programming the instruments that made up each sonification before combining them into the final product and presenting them at Polar Day.
“I was extremely excited to work with data that spanned such a long period of time and could tell such a long history,” says Kenney. “It was also great to create a sonification that could speak toward current problems surrounding Antarctic ice melt and climate change.”
While the sonifications are pleasant to listen to, they also serve a valuable purpose. Sheriff says that for him, sonifications provide an opportunity to get his research to a new audience.
“A nice-sounding sonification can engage the public — perhaps younger students, for example — in ways a visual graph can’t,” says Sheriff. “It engages different senses and makes it a unique experience.”
For Kenney, sonification is also about the opportunity to collaborate and tell a story.
“Sonifying the data allowed us to really dig into the history of this region and tell its story from thousands of years ago to the present," says Kenney. "It was also a great opportunity to collaborate with world-class scientists at the Polar Center and to explore territory, both sonically and conceptually, that I would not have waded into otherwise.”
So, whether it’s telling the story of Antarctic ice throughout hundreds of thousands of years or the story of squirrels over just one, sonification is helping record our world’s history one note at a time.