Experimenting with audio in digital experience design

Experimenting with audio in digital experience design


This is the story of a MEX experiment in audio design, where we blindfolded 100 conference participants and played them three original compositions – each a minute in length – by Peter ‘pdx’ Drescher.

Participants were allowed to briefly remove their blindfolds between each composition to write or sketch their instant reaction.

It was part of a larger (and ongoing) effort – MEX Pathway #9 – to encourage digital designers to strive beyond purely visual interactions and tap into additional dimensions such as sound and tactility.

Here’s what happened and some reflections on what it might mean for designers…

Sound station for the audio experiment at MEX/9

The responses were collated into three separate ‘walls’, which remained present during both days of the MEX/9 conference in May 2011. Participants were encouraged to listen to the compositions again at sound stations placed beside each of the walls, while reviewing what others had written about their experiences.

Each of the compositions may be listened to below and is followed by a selection of the participants’ original reactions.

Selection of reactions to MEX Audio Collage 1

Selection of reactions to MEX Audio Collage 2

Selection of reactions to MEX Audio Collage 3

The experiment prompted several observations:

  1. Certain themes emerged consistently with each track, suggesting elements of the audio were universally understood. Sometimes these were sounds associated with familiar reference points (such as the ‘helicopter’ in Collage 1), but also strayed into the conceptual (e.g. the ‘forest’ feel identified in Collage 1) or specific sound characteristics (e.g. the tempo of Collage 3).
  2. However, each composition always prompted a unique element to the participants’ responses, often related to how the sound reminded them of a personal experience (e.g. ‘finishing half of Brussels Marathon), a specific artist (e.g. Manu Chao) or time period (e.g. 1980s).
  3. The range of interpretations was astounding, perhaps helped by the knowledge that responses would be used anonymously. From sitting in the same conference room at the same time, the audio prompted these 100 participants to encounter different feelings ranging across: temperature, colour, stress levels, physical environments, social context, memories, sports, music and many more. It is hard to imagine a visual experience resulting in the same variety.
  4. We might, therefore, conclude audio has the potential to reach users at a deeper emotional level, but presents higher risk of varied interpretation. This would suggest it would be most effectively applied to digital experience elements where ambiguous interpretation would not cause critical errors, but a level of playful engagement could significantly enhance the users’ perception of the overall experience.

MEX has revisited the Pathway #9 theme of audible and multi-sensory design on numerous occasions since this experiment and will do so again at the MEX/16 conference on 12th/13th October. Peter Law and Tom Pursey of Flying Object will deliver a creative session in which participants learn how they combine multiple visual, audible, tactile and even taste and smell sensations into digital experiences for the likes of the Tate Modern gallery.

For further MEX research on this theme, try:

With special thanks to Peter ‘pdx’ Drescher for his original compositions and contributions at MEX/9.

3 Comments

Add yours
  1. 1
    Peter "pdx" Drescher

    Even after hearing these collages for the first time in 5 years, I can still easily identify what MY intentions were when creating them:

    #1: “Twittering Machine” — the initial “spark of life”, followed by a mysterious machine spinning up, that at full speed, produces bird songs. A sonic impression of the Paul Klee painting I named my production studio for (http://twittering.com/twittering/html/about.html)

    #2: “The History of Synthesis” — starting with the most basic (sine) wave possible, the waveforms become increasingly complex, following the development of 8-bit “video game” sound, DX7 “FM synthesis”, and finally modern digital audio using sampling, sequencing, granular, and other synthesis techniques.

    #3: “The World of Rhythm” — I like to say that “humans are the animals that keep time”, starting with your heart beat, (complimented by / related to) your breath and your gait. The constant marking of the passage of time (hands clapping in syncopation, clock ticking) generates musical rhythm, expressed using many kinds of drums playing a wide range of characteristic patterns. Indeed, since rhythm is completely intrinsic to human nature, all humans in every culture participate in it, from entire villages playing drums and dancing in Burundi, to samba batteries in Brazil (plus many many more, but I only had a minute! 🙂

    It’s interesting that none of the audience responses came close to deciphering my themes/motifs, but this wasn’t really surprising. Sound and music can produce incredibly strong emotional responses without conveying specific information — a fact I use extensively in my Audio UI design work. When a computer responds to a user request, the information is conveyed using visual/text/color/whatever means; but how we want the user to FEEL about the interaction can be most viscerally communicated using sound …

  2. 2
    Marek Pawlowski

    Thank you for posting this detailed insight into your original motivations Peter. Wonderful to get a sense of what was going through your mind as you composed these pieces. The session remains one of my fondest memories from our MEX adventures over the years, not least how it motivated the participants to think more expansively about sound throughout the rest of their challenges at the conference. I was reminded to revisit this story by an upcoming podcast episode (MEX Design Talk #19, due to be published on 31st August), in which @alexguest and I talk about inspirations – one of which relates strongly to audio design by bridging the worlds of fashion, textiles and sound.

+ Leave a Comment