Dr Hyunkook Lee is on a mission to advance the knowledge of the human auditory experience. Paul Milligan sat down with him to explore the methods behind this goal.
Before joining Huddersfield University in the UK, Dr Hyunkook Lee was a senior research engineer at LG in South Korea, where he led a project to develop audio algorithms for mobile phones. Lee moved to the UK more than 20 years ago and graduated from the music and sound recording (Tonmeister) course at the University of Surrey, where he also gained his PhD degree later in 2006, on the topic of the objective measurements of inter-channel crosstalk in multi-channel microphone techniques. He joined Huddersfield in 2010 and is now a professor of Audio and Psychoacoustic Engineering.
In 2014 he established (and is now the director of) the Applied Psychoacoustics Lab (APL), an experimental hub created to advance the knowledge of the mechanism of the human auditory experience, which provides solutions for practical applications. How would Lee describe what happens at the APL? “We try to identify what things are missing, what do we need to know to make sound better? Then we do all the fundamental research, listening tests, statistical analysis and modelling and then feed the output of our research back into practical engineering.” This approach has already seen the APL develop software tools, plugins, and microphone arrays in its first decade. The APL is currently involved in a variety of research projects, “anything to do with the psychoacoustics, spatial audio perception and rendering,” explains Lee.
One current project is studying the potential impact from the low noise levels emitted by electric vehicles on safety. “That was a new adventure for us, but we found some interesting stuff about psychoacoustic annoyance, which is an objective way of measuring how annoying sound is, which is important for mental health and wellbeing, with all the noise around us
in modern life. The objective measurement method is very old and doesn’t work properly, because it’s an overly simplified model, and nobody knows how the model was derived. We tried to measure annoyance with different types of sounds for electric vehicles, because there is a European legislation in which [16] mandatory and specific tones [like frequencies] have to be combined to create the sounds of electric vehicles. We used a machine learning method to classify them into different categories and, by capturing sounds from the street, we superimposed those vehicle sounds using the method we developed. We then created a VR environment where participants come into the lab and see this virtual space with electric vehicles moving and the sound playing, and then we measure how the perceived annoyance is rated. It’s the first of its kind for this research.”
The APL calls itself an applied psychoacoustics lab says Lee, “because pure psychoacoustics is all about understanding perceptual threshold, how loud the sound should be, pure lab-based studies using controlled sources, like noise, tones, they are really important in order to understand human behaviour. If it goes into a real-world scenario, a lot of these theories don’t work because the sound of the real world is much more complex, we need to conduct experiments in an ecologically valid way, as well as a controlled way.”
What are the methods the APL employs in its varied research topics? It has an ITU-R BS.111 compliant listening room, used for immersive mixing and for music production. But the APL will also venture outside a controlled environment if it helps the research, as this recent example, on urban soundscapes illustrates. It is using a special microphone array, designed by the APL on a previous research project, which then became the basis for German mic manufacturer Schoeps to develop a 3D audio version of its ORTF model. “Our array provides more realistic and spacious representation of real sound. We’ve been recording lots of city soundscapes (the sound of a park or street), and then we reproduce those sounds over loudspeakers, and it sounds like you’re there, the speakers disappear, it’s even better with VR glasses, because we capture the video in 360 degrees, and with head tracking, you can get the place and hear the sound as if you’re there,” says Lee. The microphone the APL helped to develop has since been used at Abbey Road Studios, the football World Cup and is also being used to capture 3D sound for VR.
Another area of interest to the APL is spatial audio, how big an influence does Lee think it can have in the wider world? The opportunity is huge he says. “The major advantage of spatial audio is that you can experience sounds in a more realistic way. This realism can be really important for military applications, who have been doing research for 40 years, because fighter pilots have to actually identify what direction the enemy jets are coming from. We can’t see things behind us, so we have to rely on the auditory cues and that has implications on car audio too.” There is evidence that immersive sound can help with reducing stress and anxiety too says Lee, and it doesn’t end there. “Most people understand spatial audio as Dolby Atmos, but it’s just one of the formats, it can be used for museums and galleries, AV installations, immersive cinemas and far more. I think there is a great opportunity for spatial audio to be used as an intersection to connect different areas.”
The phrase immersive audio is also misleading explains Lee. “I’ve written a paper on what do we mean by immersive? How do we measure immersive? There are immersive systems available with speakers and VR etc, but they don’t necessarily give you a nice experience. If the content is really immersive then you may not need all the speakers you think you do. Spatial audio done well can boost the experience even further. You also have to understand the importance of the context of the environment you’re in. When you listen to binaural audio a lot of people get disappointed because the environment you hear with virtual acoustics doesn’t match the environment you’re physically in, and it gives a very poor sense of realism. It’s very difficult to actually create immersive experiences, it’s not just about the (AV) system.”
Another interesting project the APL is working on involves a cross-section of audio and VR and is another with great commercial potential. “We captured a string quartet with a motion capture system, they were playing music, and we captured their emotions in 3D, and we turned them into virtual humans. Using VR you can see them playing in all their different positions so when you walk around you can interact with them, the sounds come perfectly from each position, not within your head,” says Lee. “You can imagine it being used in art galleries or museums, it has a lot of opportunities.”
What would Lee say are the proudest achievements of the APL? Its research which has advanced the understanding of height in 3D audio recording and a plug-in it has released, called Virtuoso, which is a commercially available binaural rendering plugin, he answers. “You can superimpose holographic audio images in the real world, and you can explore the space, and it sounds like you’re listening to real stuff as you walk around,” explains Lee. “We’re super proud of it because it’s been recognised as the best sounding binaural renderer by a lot of top sound engineers and magazines. We are still in the very early stages of commercialisation, but it’s a new adventure for us.”
What does the future hold for the APL? Continued development of Virtuoso, and also getting to grips with its brand new upgraded audio kit. The APL has recently updated its existing Genelec kit with brand new replacements, in the form of the 8341 coaxial monitors from Genelec’s ‘The Ones’ Smart Active Monitoring series.
“When we hosted a recent AES conference on Spatial and Immersive Audio with 21 people in the room, wherever they sat, they had an excellent experience. The tonal balance was very consistent across the room, which was very important for this kind of demonstration situation,” explains Lee. The new setup also allows APL to create a Dolby Atmos 9.1.6 space, while the remaining 8040s ensure that this can be expanded to cover higher channel count formats such as NHK’s 22.2 standard. “Recently, we’ve been focusing on binaural audio for virtual monitoring and XR applications,” explains Lee. “And my current research focuses on what kind of roles audio plays in providing an immersive experience. And for that, it’s all about understanding what content producers really think about immersive audio, and what kind of experience users expect from these immersive systems. We need to understand each other and try to narrow the gap and work together in a collaborative environment to discuss what really makes spatial audio truly immersive.”