Art is often viewed as a visual medium, but there’s growing awareness that all senses can be engaged to reach and move audiences. Anna Mitchell looks at the space where audio meets art.
With an ever widening body of multisensory experiences on offer; sound, touch and smell are increasingly being explored by today’s artists as ways to evoke responses. With new audio products and approaches helping drive some of the most ambitious projects today, the space where art and audio combine is fascinating and well worth a look for even the most practically-minded of audio professionals.
Installed in dark rooms, some audio artworks deliberately dull other senses to bring the audio to the fore. Others combine a visual element. But the common factor in the most interesting projects is that audio is never a soundtrack to support the visual. It doesn’t take a secondary role, it is a valid and vital part of the resulting work.
An extraordinary example of this can be found in the work of Newark, New Jersey based artist Wolfgang Gil who designs “sound emitting sculptures”.
“My approach to sound is very much driven by visual culture,” says Gil. “So the way I'm treating sound is as if it was a plastic medium, a malleable material that can be physically stretched in space.”
Gil has a background in systems engineering and a passion for music. Before he started exploring art installations, he was experimenting with writing algorithms for beat making. It is algorithms written by Gil that are delivering the audio for his artworks today.
“I started working with stereo speaker systems,” he explains. “My idea was to fill up the space between the speakers with these granular sounds in order to create a field that would stretch between the two speakers.”
Developing more sophisticated algorithms allowed to Gil to start exploring more complicated speaker configurations.
“Adding another speaker all of a sudden provides you with elevation and when you have elevation, there’s the opportunity to create a shape, a surface where sound can exist. That evolved into different types of speaker configurations and shapes.
“With a bigger setup I started to explore creating objects that have a physical geometry and also the idea that a geometric object could generate sound through vibration. That was the start of a series of works called Resonant Bodies where there is no speaker, it’s a metal object made of aluminium or steel that has transducers attached to it. The transducers make the whole surface of the object vibrate. The next step was to see if the geometry of that object would cast a shape on the sound. And, yeah, at times it does.”
"Every time I make a new shape, and a new speaker arrangement, my algorithm needs to be extended to work for that specific configuration." - Wolfgang Gil
Gil’s description of sound as something that fills physical space directly ties into what is arguably the most exciting area of audio development currently: immersive audio and 3D and spatial sound systems.
Richard Foss, co-founder of ImmersiveDSP, believes it’s no accident that artists are adopting these technologies. “Art projects create visual objects, often within three-dimensional space,” he says. “Sometimes these objects are dynamic, sometimes static. Immersive audio technologies complement the visual since they have the capability to place and move audio objects in three-dimensional space.
“Immersive sound is quite a new and experimental technology. Art installations themselves are often explorations into new realms of creative expression,” adds Foss.
Tim Sherratt, strategic planning manager, professional audio, Sennheiser UK, adds more weight to the point: “From an artistic point of view, 3D audio installations are probably the best way to emotionally touch people and get them involved in whatever a project is about by creating an unprecedented as-if-there experience. Such an impactful immersive audio installation involves remixed or upmixed 3D audio content including virtualisation of sound sources and high-quality reproduction equipment from amplifiers to complex loudspeakers set-ups.”
Art and spatial audio’s perfect pairing is evidenced in an installation currently on show (until February 14) in New York’s bitforms gallery. Alchemical is a collaborative exhibition by artists Casey Reas and Jan St. Werner with Reas' projected film stills, prints, and videos supported by electroacoustic sounds created by Werner.
The pair bring formidable experience to the project between them. Reas is a prolific artist who has exhibited all over the world and is a professor at the University of California, Los Angeles. He’s had a significant impact on the arts, helping to develop Processing, an open-source programming language and environment for the visual arts. Werner is an experimental musician and one half of electronic duo Mouse on Mars. He’s created music in solo projects as well as multiple collaborations and is a professor at the Academy of Fine Arts, Nuremberg.
On the topic of spatial audio, Werner says: “The concept of stereo feels romantic since it’s mostly used to reconstitute the notion of ‘stereo equals two ears’. In fact, we orientate in space with ever changing focus and we shift between foreground and background, detail and overview in milliseconds, the brain does the operation, the ears provide clues. Spatial audio is addressing this fact more profoundly.”
Werner has used many techniques to deliver spatial audio in his work including multichannel sound diffusion and wave field synthesis. He’s also used Meyer Sound’s Spacemap Go spatial sound design and mixing tool, experimenting with a system installed in a Berlin studio he works in. Reas and Werner’s interest in bringing the effects Werner had experimented with using Spacemap Go in the studio into a gallery setting was a key part of bringing Alchemical to life.
Reas says he’s “a lapsed drummer” and although he was involved in making music for over a decade, audio didn’t find its way into his visual art until 2015. “Now that I've started, it's changed everything,” he says. “[Alchemical] is my first collaboration with Jan and my first time completely sharing authorship.”
Early on in the project Reas shared a sketch of the long, skinny gallery space at bitforms to show the simple arrangement of two projection screens with Steve Ellison, Meyer Sound’s director, Spatial Sound. Adding stereo sound to each screen was easily accomplished by positioning four speakers, left and right of each screen.
However, content alternates between the screens and the addition of a fifth speaker added an extra dimension to the stereo channels to create the ability for quad distribution and cohesively bring the displays and content together.
“It’s so simple and so effective,” says Ellison who worked with Reas and Richard Bugg, digital products solutions architect at Meyer Sound, to program and test the system at Meyer Sound’s Burbank facility. “The two video streams as well as the stereo audio is sourced from a Mac mini running QLab software. The audio is transmitted to a single Galaxy-816 using Milan/AVB. The AVB routing, set up in Spacemap Go, sends the left channel from the Mac to Galaxy-816 inputs 1, 3, 5, and 7 while the right channel is sent to inputs 2, 4, 6, and 8.
Casey Reas and Jan St. Werner’s Untitled 3 (I withdraw) for Alchemical, bitforms gallery in New York
“We used the processor’s U-Shaping filters to allow us to individually place the lows, mids and highs from the left and right channels into the sound field. Spacemap Go’s Mix Snapshots synchronously transition the positions between screens A and B over three seconds when the video changes screens.”
Back in the gallery in New York, Meyer Sound worked with rental company Sound Associates to install, calibrate and program a Meyer Sound system consisting of five UP-4XP loudspeakers, one MM-10XP miniature subwoofer and the Galaxy network platform. The spatial mix was finalised on site using Spacemap Go.
Ellison describes the resulting effect: "The perspective of the video and audio are consistent when the content changes from screen A to screen B and back again. The video and audio move seamlessly in the gallery.”
Reas adds: “When images are too concrete (representational), they can dominate audio. For these videos, they oscillate. At times the audio is resolved and the images aren't and vice versa. We're compressing the two sensory domains into one stream, these works are not visual media with sound. In the installation, the 2D video images transform into volumetric works through the spatial audio and the way the light from the projectors fills the room and scatters.
“This work breathes through the relationships between the image and sound, each video is a duet. I'm always more interested in ideas than specific technologies, but ideas that I'm interested in, like artificial life and simulation, often require working with emerging technology to explore.”
While Alchemical is currently open for socially distanced viewing, unfortunately an exhibition of Gil’s at the Newark Museum of Art in New Jersey, USA, remains out of sight from the public at the time of writing due to the ongoing pandemic.
In the meantime, Gil has created a virtual space where 12 sculptures from his Resonant Bodies and Aural Fields series are exhibited. He calls it Vision 2020 and it includes works that have been physically created, as well as others he’s designed but not constructed. He describes it as “a five year long vision for my career”.
Materials are carefully selected for their sonic and aesthetic properties and each sculpture is a work of engineering where physics are considered and approaches tested to achieve or avoid certain effects, whether that’s “stretching audio” or making sure sound doesn’t bleed. Proximity sensors are also used in some to introduce interactivity and involve the audience
Some sculptures are designed to “contain the audio”, so it almost exists within the form of the sculpture, moving around within the shape. In later works, Gil frees the sound from the form so it “jumps between objects”.
All are driven by Gil’s algorithm. “The algorithm has a core synthesiser but every time I make a new shape, and a new speaker arrangement, this algorithm needs to be extended to work for that specific configuration,” Gil explains.
“The way I’m using spatialisation is different from an approach that would use a mono signal that you're basically panning. My approach combines a particle system we spacialised at a grain level. I can generate, for example 1,000 microsonic events, like simple waveforms, and then place them in space independently. Using the time separation between them and the duration of the sounds, you can almost weave them together spatially. And that's what creates these different temporal shapes.”
When looking at the use of audio in artistic projects it would be easy to assume that artists convey objectives to technical audio teams to deliver them. The reality is far more blurred with technical ability and creativity shared between artists and audio professionals.
Both Gil’s hands-on approach, and the way Meyer Sound worked with Reas and Werner, demonstrate this. A look back to Inavate’s March 2020 edition also sees Sennheiser’s Sherratt heavily involved in bringing the work of Theaster Gates to life at the Tate Liverpool.
Another example comes from Sean Davenport, who founded ImmersiveDSP with Foss and has been hands-on in working with artists who want to deploy audio technologies in creative ways.
In 2019 he collaborated with Cameron Lowry to deliver The Earth Space Program at indie games and immersive arts festival Playtopia 2019 in Cape Town, South Africa. The pair created an immersive audio “chill tent” which offered a reactive lighting and immersive audio experience using a turntable, Microsoft Kinect and ImmersiveDSP’s ImmerGo 13.1 sound system.
“By using MaxMSP with ImmerGo, we were able to easily process a complex max/MSP patch, which used the Microsoft Kinect as input to control lighting, with heavy spatial audio processing on an analogue audio input from a turntable and sample triggers,” explains Davenport.
“ImmerGo gave us the ability to feed separate audio processing chains into different audio playback objects on the loudspeaker system. These could be controlled using various modulator logic within Max/MSP. This allowed for interesting effects such as virtual point source, particle-system style delays, super spreaders and stereo movement on the turntable audio - we found this was really great for vinyl scratching!
“As well as this, using ImmerGo’s Ambisonic decoder, we were also able to play back a variety of Ambisonic files to create a rich background rain scene which provided context for the vinyl records being played back. It was truly a very fun and engaging piece that mixed analogue and digital interactivity in a unique way.”
The majority of artists aren’t wealthy and arts funding is often in short supply. When it comes to audio however you can be dealing with some expensive kit.
Gil’s sculptures cost thousands to create, which is partly the driver behind Vision 2020, allowing him to virtually design and test works before getting the funding to realise them. Sometimes the funds will come from partnerships, commissions or purchases. He’s also set up The Honk-Tweet, an experiential design agency that focuses on mixed reality experiences. Profits from project work there are often diverted to building his sonic sculptures.
Wolfgang Gil’s Resonant Bodies #1 and Aural Fields #3 in Vision 2020
Meanwhile Meyer Sound says that Alchemical demonstrates that you don’t need an overly complex and expensive system to deliver a powerful effect. Meyer Sound and Sound Associates also provided technology and expertise for this work, which underlines another point about art projects; they evoke reactions that drum up support. The ROI is often to be a part of something that’s considered important.
Over at ImmersiveDSP they have a proposition that is likely to bring immersive sound to a wider audience. “Affordable immersive sound is a particular concern of ours at ImmersiveDSP,” says Foss. “Our current development efforts are geared towards using the processing capabilities of widely available audio interfaces and mixing consoles to enable immersive sound. The controlling devices are similarly easily available laptops and mobile devices. This makes it feasible to incorporate immersive sound into low budget art installations, exhibits and live sound events.”
Top image shows Wolfgang Gil’s Resonant Bodies #6 in Vision 2020