So Loud That it Hurts - Why You Need to Try Data Sonification

Over the past couple of years, I've been getting deeper and deeper into sonification - the art and science of turning data into sound and music. I've released a sonification podcast, I've hosted a sonification festival, I've worked on sonification projects for multinational commercial clients, I've created sonification art pieces, and I've spoken on the subject of sonification for countless events, meet-ups and podcasts.

In the process, I've become convinced that sonification is the most valuable underused tool in an information designer's repertoire and that it has vast potential for improving our work.

There are three main reasons why I think this. The first is the emotional impact that sound can deliver. I summed this up in a quote for Alli's Data Viz Today podcast:

The point I'm making there is not that your work should be painful, but that there's only so much experience that can be delivered with a traditional chart. Some people have done great work with breaking conventions in visualization to show extremes - Jon Schwabish mentions a bar chart that showed the number of gun deaths in the United States relative to other countries. "You just had to keep scrolling," he said. "That hurt (visually) pretty bad." But to get to that experience, you had to spend time and attention on reading the chart and understanding what it's showing. You had to be invested. Sound and music, on the other hand, require no such investment. You hear it and it generates an immediate, visceral emotional reaction.

What's more, music has a much wider emotional range than traditional data visualization. Think of a "sad song", or an "angry song", or a "joyful song" and examples immediately flood your brain, no matter what culture you come from.

You can even appreciate sad, angry, or joyful music in other languages.

Sound offers a far richer emotional palette than visualization - which is largely limited to colour (the meaning of which varies dramatically between cultures) and photography (which is also underused in visualization work).

The second reason to use sonification is accessibility. There are plenty of people in the world who can't clearly see the thing you want them to look at. For some of them, that problem is permanent - they might be blind or partially sighted. For some it's temporary - they might be recovering from an eye infection, or outdoors on a sunny day with a phone screen that doesn't cope well in daylight. For some it's momentary - they might be distracted by something, or just not looking in the right direction. All of these people would benefit from an experience that engages more of your senses than sight alone.

Even for those who are paying full attention and can see your work clearly, adding an extra auditory layer of data encoding deepens the likelihood that they'll understand what you want them to understand.

As a data communicator, your very first responsibility is for the maximum number of people in your target audience to get what it is that you want them to get. Maximise your opportunities to make that connection by using sonification. More opportunities mean a higher success rate. It's that simple.

The third reason for taking advantage of sonification is the newfound ubiquity of technology that enables it. Unless you exclusively make graphics for print media, it's likely that most of your visualization work is seen on devices equipped with high-fidelity speakers. Why not use them? It seems ridiculous that we're surrounded by supercomputers all day long and we're only taking in information from them through our eyes.

Most people haven't really noticed the incredible advances from the last couple of decades in miniaturised audio reproduction - but think back to the brick-like phone that you had in the mid-2000s, and how awful it sounded when it played music. Then play some music through your current paper-thin smartphone. It's night and day.

Adding sound to your work lets you take full advantage of the capabiliities that modern audio technology affords us, and it also gives you an advantage over your competition, who haven't noticed this yet.

Get out in front by integrating sonification into your work.

One final note, before we end, is that with great sonification power comes great sonification responsibility.

It's so much easier to reach people (emotionally or otherwise) by using sound than with visuals alone, but this has a flip side - it's easier to annoy or upset people with sound than it is with visuals. No-one wants to be that person whose phone is ringing in the cinema, do they?

You have a responsibility to use sound in a way that's appropriate to the circumstances.

In practice, this usually means keeping the sound elements of anything that you do relatively subtle - especially when dealing with subjects that are particularly emotive. For example, the New York Times' 2017 interactive on the fire rate of weapons equipped with "bump stocks" uses simple click sounds, rather than the sounds of guns firing, which would be extremely unpleasant for the many people who've had a loved one die from gun violence. Sometimes emotional associations can do more harm than good, and it's your job as a sonification designer to walk that fine line with care.

This post is very much about the "why" of sonification, and not about the "how". The "how" is more complicated, and there's a whole spectrum of tools from Twotone.io, to the Web Audio API, to Sonic Pi, to "punk" sonification techniques that just use voice and other ambient or physical sounds. I love talking about the "how" - so if you want to dive in and are looking for some tips, then drop me a line any time.