With communal experiences, sharing the same information is paramount. Reece Webb speaks with Tim Devine to discover how spatial audio can be used to revolutionise the sports experience.
When it comes to sharing human experiences, there are unfortunately still barriers for some in the form of impairments or disabilities that can hinder a shared experience between people.
Fortunately, thanks to a new spatial audio development, tennis fans who have visual impairments or blindness can experience the same amount of tension, excitement and thrills offered to other viewers.
Enter design studio AKQA, a company that worked closely with the Australian Open to develop a new way to bring the partially sighted and the blind into the action of a tennis game with the same level of interaction as other viewers.
The approach harnesses a new spatial audio technique that uses sounds to represent the location of the ball on the court in real time.
The approach harnesses a new spatial audio technique that uses sounds to represent the location of the ball on the court in real time.
Tim Devine works as an executive innovation director with AKQA. His expertise in computer-generated audio and interest in the physics of sound has served Devine well during his 11 years with AKQA and its innovation team.
Devine explains: “I see myself and the role of the innovation team as a conduit for emerging research, understanding what’s happening in academia and translating it in a way that it can be embedded in the real world.”
The technology, dubbed Action Audio, debuted at the 2022 Australian Open tennis tournament and provided a new way for the blind and partially sighted to engage with the game and engage socially with full-sighted people in a way that was previously impossible.
The system is designed in a way that is ‘filterable’, providing an experience that is comprehensive but not distracting. The Action Audio system works via the Australian Open’s radio broadcasts, streamed to accompany the video broadcast.
Devine says: “During a tennis match, they have electronic line calling technology to detect whether the ball lands in the line or out. That also tracks the players and where the ball is in relation to the court, so that’s a huge amount of highly detailed spatial data.
Devine adds: “The sounds are similar to the rattle [to those used in blind tennis] and allows for real tension. Sport is all about tension and performance, so we give an indication of how close the ball is to the line to help build that tension and create entertainment.”
The development of Action Audio, though designed with tennis matches in mind, offers a new way to engage with other participants, but is certainly not exclusive to that singular field. Action Audio’s design ties into the trend of increasing immersion and a larger focus on the expansion of digital technologies across many sectors.
"What we want to do with Action Audio is give some agency back to people who are blind or have low vision." - Tim Devine, Action Audio
Devine: “This system is being built on technologies that are being developed anyway. Sport will continue to be more quantified using digital technologies and spatialised audio will continue to be developed because augmented reality and extended reality experiences rely on the same idea: immersion and persistence of objects in space. Sound is a massive part of that.
“We’re looking at sports and video games, particularly e-sports . We think there’s a really good opportunity to create a seamless accessibility standard across all kinds of virtual and real sports.
“What Action Audio does is create an abstract commentary of what’s happening in a game. What we want to do with Action Audio is give some agency back to people who are blind or have low vision. They can start to make their own appraisals about what is happening in the game because, after all, sport is about discussion.”
The development of Action Audio is not just limited to the niche of sports, with the possibility of branching out to be used in applications beyond its initial intentions.
Devine adds: “At the core of Action Audio is a need to provide more information where there isn’t enough to create a more inclusive environment. Where this technology goes is to be determined. Currently, our focus is on sports, games and entertainments and I think if we build on that, we have an opportunity to grow into a commercial market but how that translates is yet to be seen.”
The next event to take place with Action Audio will happen place with a visual feed, providing more information in addition to the visual content.
Devine: “The next step is to take it to a more interactive level. Right now, it is linear and happens in response to what is happening in the broadcast, but we like the idea of giving people the ability to curate the sounds and it’s something we’re really aware of. There’s a limit to how much information you can add to a soundscape, but we know that we are good at selective attention so that we can tune in to somebody’s conversation and focus on different things. The future of Action Audio is a more curated, personal experience.”
Action Audio’s spatial audio development has received positive feedback from users and aims to take the concept forward as the technology progresses.
Devine closes: “Feedback has been phenomenal. Of all the projects I’ve ever worked on, this is the project that has had the most positive response. People appreciate the effort and consideration as this is something that has been missing.
“Even people who can’t see in the centre of their eye, who have to sit in front of the screen, are presented with challenges in a social environment [when sharing the experience with others]. Action Audio reduces a part of that friction associated with those challenges and it’s really meaningful to people who use it.”