Paul Milligan speaks with Joel Krieger from Second Story, the creators of a unique algorithm used to create content for a giant LED screen which reflects the city around it.
When asked to describe what he does, the answer from Joel Krieger, the executive creative director from experiential design studio Second Story, reveals his personality but also his approach to work; “People are really interesting creatures, I think that our job is just to be very curious about human behaviour and psychology, and to come up with interesting moments for a new experience.”
READ OUR INTERVEW WITH NANOLUMENS ABOUT THE UNIFY PROJECT HERE
The company is 25 years old this year and has offices in Atlanta and New York, as well as its HQ in Portland. The company’s credo is that it ‘builds stories you can step inside of’, so is the story the be all and end all, or is getting the technology right the key to engagement? “Setting the story and then fine tuning the execution are equally important,” he says. “Ideas are really important, but that's the easy part, in a way execution is everything. A metaphor for our process is a sculptor forming clay, you start out with raw clay and you shape it on the wheel, over time it goes from this very lo-fi thing to a hi-fi thing. We're not necessarily doing the story and then technology or execution in a linear order, often it evolves together.”
Second Story’s latest project, working with LED manufacturer Nanolumens and systems integrator Cenaro, involves a unique installation called Unify in the city of Charlotte in North Carolina for Legacy Union, a new building combining offices and residential space. The centrepiece of Unify is a huge 64x36 feet 4K display, claimed to be the world’s largest 4K LED wall display. Despite the massive size of the display, the most interesting aspect belongs to Second Story, who has developed an algorithm that never repeats the same visual. The behaviour of each pixel influences the other pixels around it, using colour, movement and sound to create a swirling visual display.
The challenge posed to Second Story in this job says Krieger was finding a way to make it interesting for people who would walk past the building every day. “How do we create something that isn't going to get boring as they're going to see it all the time? How do we make something they look forward to as part of their daily ritual? How do we create something that enhances the ambiance and the vibe of the building, and is an extension of the brand of the owners of the building? What does it say about them as a company?”
The key to the whole project was creating the algorithm. Krieger explains how it works; “The artwork we did was done using a generative method. We are at a paradigm shift in terms of how content can be created. The old method is we'll work in CG or 3D and will output a video, and this video is a constraint artefact, a set thing that runs for a certain length of time. Often content is priced as it is in the advertising world i.e. a 30-second slot. Its time based, so you get X amount of CGI visual effects for X amount of dollars. Now we can create software for an environment that has rules, you're creating a universal with laws and rules. It’s a beast with a mind of its own, and then you let it go and it does its thing and makes visuals. The design process to get there is radically different, because you have an idea of how you want it to look and behave and feel but you're constantly tweaking the rules of this universe to get the desired outcome.”
The algorithm has a deeper meaning says Krieger; “There are infinite possibilities, this one was designed to be a metaphor for community. If you think about community it's really just a collection of people, and all these people have relationships in between them. The actions of one person affects all the people around them, even with people who are new. It's a really beautiful network of relationships and impact. That was the guide to how wrote the algorithm, to create that interaction and behaviour, when you have that happening it's a very complex interplay. There's infinite possibilities, while the world, the aesthetic we're in, is going to stay more or less the same for the entire year, what happens is constantly unfolding and always changing and always different, just like a real community.”
Krieger says the testing period was long and hard, but fun nonetheless. “We were there for two nights and I didn't get tired of looking at it. The test for me was 16 hours staring at this screen.”
The ingenuity didn’t stop with the video adds Krieger, Second Story crated unique audio content too. “There's an immersive soundscape in the lobby that is also fed by the same algorithm. It uses a technique called granular synthesis. It's the same general principle, you have these little grains of sound, some are extrapolated from city sounds, some of them are extracted from tonal instruments, and these grains, just like the visual pixels behave in the same way, and for that reason you end up with very organic patterns, like rain or the rustling of leaves, which is never the same twice, but you know what it is when you hear it.”
Krieger feels the approach for large scale public projects is changing, because it simply has to. “As more and more of these large format media canvases come online in the public and built environments, we are all asking what do we put on these things? We're in this weird transition where I think the advertising models that we use to monetise them doesn’t work in this new world. The internet 15-20 years ago was a lot more awesome of a place than it is now, it's become completely saturated with advertising. Back in the day it felt a lot more pleasant to swim through and surf the internet and explore things. Now it feels like this constant barrage of attention, you're swatting these ads away everywhere. I think this era of interruptions is over, I don't think it can continue into the built environment. The saturation of advertising has completely filled up mobile and web channels. As public canvases and built environments come online the tendency is going to be to fill it with advertising. But unlike your phone or your laptop, you can’t shut it off, if you have the willpower you can extricate yourself from your phone, but you can't do that in the built environment.”
Digital content is at a crossroads says Krieger, “We're at this really interesting point of talking through what are the ethics of design for spaces when it comes to digital? What can you create that is in the service of the brands or the clients, but also in service of the people? It's a real pivotal moment in terms of trying to get this right. We're looking to work with partners who want to explore how we do something different because this old model of being interrupted and shoving messages in people's faces just can't continue.”