Real-time visuals and musical talent combine in this one-of-a-kind installation. Reece Webb reports.
Architectural LED is becoming an ever more present part of modern city living, and London is no exception to this trend. But when a concept becomes commonplace, how can a screen differentiate itself in a meaningful way?
For Sysco Productions, the answer to this question came in the form of a novel approach that blends musical prowess with ever-evolving visuals, putting audiences at the heart of real-time content generation in a unique way.
Located near the beating heart of London’s financial district, visitors passing through Hogarth Court, a modern alleyway located within a large building, can look to the heavens to see a 180 sq m LED screen above their heads, embedded in the ceiling.
But this is no ordinary LED installation. The screen works in conjunction with a piano, located near the entrance to the court’s lift system, and works in conjunction with the LED screen to blend real-time content with musical input.
The venue, called Skyframe, changes the intensity of weather, lightning and snow in real time, mirroring the style and speed of a live piano player, no matter their skill. Blizzards of varying intensity, thunderclaps timed to live music, and sunsets streaked with falling leaves unfolded above the players, with LED content shifting in pace to match the score.
This installation marks the first time that a public piano has been used as an input to generate on-screen visuals, bringing real-time generative content into the public sphere to people of all musical abilities.
“It’s not an alien concept for pianos to generate visuals,” explains Alex Wilcox, visualisation engineer, Sysco Productions. “It hasn’t been seen in public spaces but has been used in concerts. Typically, everything is curated, you know what music it’s going to be, and we were excited about how generative you can make that, how you can respond to a key press uniquely and use that tried and tested theory in a positive user experience for anybody to turn up and play.”
This was no small feat. Sysco Productions set itself the challenge of delivering concert-grade visuals for unpredictable levels of playing ability, designing a system that not only looks visually enticing for professional players, but also delivers the same level of visually engaging content for the most casual player in a matter of seconds.
“We wanted this to be something for anybody that comes along, and a big challenge is that this kind of setup is usually built for a concert pianist performing amazing compositions. We asked how could we build something that’s responsive to somebody that wants to press a few keys and leave, while also rewarding that person who is a trained pianist and knows how to play. This was the challenge we set ourselves: How could we make something reactive and exciting that mixes a public piano with concert visuals.”
Sysco Productions decided early on that it needed to create a system that responded quickly with striking visuals, with an evolving selection of content that becomes more subtle and intricate as players continue to play. Starting in a rainy, dark environment with lightning striking at the press of a key, the content transforms into an autumnal scene with a fading sunset and leaves falling gently from the trees.
Wilcox explains: “Those two scenes were what we were anticipating that most people would end up seeing, and the effects in those scenes are always responsive to a button press. There are wider scale effects that last for those whole scenes, but if you push a button, you see something happen. That was a big aim for those early scenes, which we allowed to become ‘looser’ as time goes on, but if you’re playing for seven or eight minutes, you see more subtle shifts in how your playing affects the content.
“This design philosophy addressed both kinds of people that would play the piano, but we didn’t anticipate groups of people crowding around it. Instead of one person performing a long piece, 20 people would take turns, each playing a few keys at a time. It was incredible that we got this reaction. We hadn’t anticipated that a session would be shared across multiple people, seeing all the media on offer, which was really nice for us.”
"We haven’t seen anybody do something like this that gives this level of control over to the public." - Alex Wilcox, Sysco Productions
This technology upgrade saw real-time rendering infrastructure introduced so that the piano can interface with the courtyard’s screen. Sysco Productions had installed the Absen LED screen above the space seven years ago, creating a new lease on life for the setup at Hogarth Court that brings new infrastructure and opportunities. Sysco updated the LED processor with a full real-time render PC, offering the capability to expand into new use cases and concepts for the screen that allows the space to continue to captivate audiences without replacing all aspects of the AV.
Sysco used Unreal Engine to create the content for this piece, transitioning from the integrator’s usual simulation approach to a hands-on approach of content creation that pushed Sysco Productions’ skillset further.
“We use Unreal Engine for simulation work that we do,” says Wilcox. “When we engage with a project, we’ll use Unreal Engine to bring clients’ 3D modelling through into Unreal and give it life, and we’ve created some unique plugins to demo LED, projection and LCD screens that includes gauze technology that can be difficult to program around.
“We realised that we weren’t far off content production with this process, so we started to think that we have the capability to get familiar with Unreal, and that encouraged us to think about content and how we can utilise that for ourselves.”
With no barrier for interaction, the public were free to interact confidently in the knowledge that their playtime would generate a meaningful interaction.
“We haven’t seen anybody do something like this that gives this level of control over to the public. It’s not the most complex thing in the world to implement, but it was an exciting moment when we put the piano down there for the first time, hooked up the connectors and pushed a key, then it reacted. We were so worried about latency, whether we could responsively give that feeling of pushing a key without a second or two delay. Because of the scale, you still feel that if you push a key, something happens immediately after, and we didn’t have to completely upgrade the network infrastructure, we ran some cabling down and that was it.
“There were operational concerns about the piano, but the installation went surprisingly smoothly. We are already thinking about ways in which we can do seasonable reruns as well as up-end the system and do something completely different that may be more abstract. We’ve also talked about things we can do with sensors in the space, how we can ‘gamify’ peoples’ positions, creating a game by moving through the space. There are a lot of different ideas, and we are trying to be as new and playful as we can with this space.”