The reality of virtual innovation
As Covid-19 caused widespread lockdowns, virtual events enjoyed a rapid rise but at the cost of true engagement. Michel Buchner explores how VR could be used to get back some of that connection and value during, and after, the pandemic.
Our live industry has been hit disproportionately hard by Covid with unknown long-term consequences. Theatres are closed, physical events are cancelled, or go online. Innovation now becomes vital and may come from an unexpected direction.
At the end of July, there was a local industry streaming broadcast with the theme “The New Live”. The Covid crisis was discussed with two corporate event clients, a creative lead from a future centre and a neuroscientist. Each of them had their vision on the situation, but all agreed that the current online streaming of events is an emergency solution and falls short in true engagement and satisfaction. The neuroscientist put it down strongly: Watching a video is a "leaning back" experience, where an event is a "leaning in" experience, something that was backed-up by the data of MRI Brain scans.
Alternatives or future developments were not discussed, not even by the creative lead of the future centre…
Covid caught the live industry on the back foot and it took a lot of effort to switch to streaming for those who could. Understandably, we all moved over to the streaming of events. But even when you make an event interactive with polls and viewer feedback, you still have (interactive) television, a 70-year-old medium. You take in the content from behind your laptop without a feeling of connection with your fellow viewers, without being present. As event makers, we want to engage the audience with experience design and not being obstructed by any form of framed vision.
Virtual reality
The framed image is disappearing with virtual reality (VR), an emerging medium but still invisible to many. It is an alternative that we miss between a streamed event and a physical event and offers new opportunities for our industry. It is already closer than we might realise because it has been in development for the last years.
Our association with VR is mainly about games and wearing bulky viewing glasses. Everyone knows about the demise of 3D stereo glasses because 3D television has never lived up to its promise. Why would VR prove differently? We should first be aware that 3D glasses were still around in cinemas before the pandemic. 3D provides an experience with a large screen where you have the feeling that you are immersed, something difficult to achieve at home in the living room. So, for a temporary immersive experience, glasses are apparently not an obstacle.
Covid has accelerated the development of VR. Meeting platforms in VR suddenly became relevant and experienced a peak in growth. For those who already own VR glasses, there are about 100 platforms today where you can meet for business meetings or larger events.
Companies like the airline KLM are experimenting with the Engage platform and the Netherlands police are doing so in the Glue platform. HTC Vive held a global event for 700 people in Engage and in WaveXR, DJ sets and performances are being held with John Legend acting as an avatar.
A screenshot of the HTC Vive event in EngageVR
In a VR platform, you walk around as an avatar (a representation of yourself) where you have the free choice to go wherever you want in the space and converse with fellow visitors. Forbes journalist and university teacher Charlie Fink describes it as having presence and agency, which leads to greater engagement.
There are two obstacles to overcome here. First and foremost, VR glasses have not yet been adopted by consumers and are not professionally available in large numbers via traditional AV rental companies.
The second issue is the avatar. Even though we have been gaming in 3D worlds since the 1990s, as a working professional it’s still a barrier to appear digitally as an avatar. This is a matter of time and acceptance, but here also we can see developments.
From 3D to VR
The innovative event industry has been working for years in digital 3D for set design and scenographic content. They have the skills and creativity to create experiences with the latest technology. Despite the drive to innovate, the industry didn’t have VR as a mass medium in its sight. But the adoption of the medium could be easier to make than we might think, as long as we want to see the future.
Recently I noticed a press release from a European TV studio provider offering “Extended Reality possibilities”. Extended Reality (XR) is the collective name for augmented reality, mixed reality, and virtual reality. After enquiring I learned that there will be a high-end studio in which a beautiful 3D setup can be made with digital scenography. The output will eventually be streamed to a 2D image frame. Unfortunately, you can no longer speak of an Extended Reality here. XR then becomes nothing more than a buzzword. With a fixed frame we are still talking about a 3D TV studio set, in which the viewer views the offered image from a distance as TV.
However, there is an enormous opportunity here. When this studio provider makes it accessible to VR on a large scale, they will offer their audience an experience that goes beyond streaming and is true extended reality. This studio provider has an AV hardware rental sister company. And when the sister company provides a rental stock of 2,000 VR glasses, they are a venue for 2,000 visitors, broadcaster, studio, AV-supplier, content provider, and community manager. All in one...
The Belgian Tomorrowland festival could also make a similar move to VR. Digital scenography was raised to new heights by Tomorrowland around the world 2020, the digital version of last year's real-life event. A complete dance festival was virtually streamed into the living room in a never seen way. Well done. The only need I had myself was to really be immersed in it, away from that screen.
Tomorrowland
The Tomorrowland setup is a mix of Unreal Engine, a game engine, and Depence a platform for light, fireworks, and laser visualisations. But especially the Unreal Engine as a game engine is interesting, which makes the experience scalable. Via Unreal you are already halfway into VR to receive a large audience.
Media Server manufacturers Disguise and AV Stumpfl both received development grants from Epic, the maker of Unreal. It would make sense for them to aim directly for a free reining platform-agnostic approach (Smartphone, PC, and VR) where VR has the highest immersive experience. After all, it’s always possible to generate a 2D stream for those choosing the static option of streaming.
The popular game Fortnite also runs on Unreal Engine and had 12.3m live viewers on the Sony Playstation platform during a live avatar performance by rapper Travis Scott. For Sony’s Kenichiro Yoshida, this was a wake-up call, especially with the upcoming release of the Playstation 5 console, with optional VR glasses. According to the Financial Times, Yoshida sent out a message to its employees, pointing at the possible new opportunity for the Sony Group.
A VR platform for music performances such as WaveXR has already raised $42m (approximately €35m) in investment without paying visitors. Tomorrowland welcomed 1.2m visitors over the last three years and was sold out every edition. The Tomorrowland investment in such a complex digital production is one for the longer term, but one with massive potential.
The Burning Man Festival in Nevada recently announced that it will offer some festival zones in VR when the festival takes place online, at the end of August. This "Dusty multiverse" will be developed entirely pro bono by the festival's community.
So entering a festival via VR will happen and is here to stay. Soon you will buy your new VR glasses together with your new phone and a 5G subscription.
Developments
Small news but with global impact appeared in May. Chip manufacturer Qualcomm had agreed on a standard for Extended Reality with OEM’s and global telecom corporations. The smartphone will be the XR device to process and pass it on to a pair of glasses that will be offered by the telcos. With the 5G promise of bandwidth and low latency, a Tomorrowland and a Burning Man can be offered as a high-end visual experience via the cloud, in VR.
Another significant development is the display of "real" people in VR and MR. Not as an avatar, but like we are used to from photo and video. This is called volumetric video where you can move around a person virtually and spatially. The technology has been in development for television for a decade and has been given a boost by VR/MR. Today there are more than 51 volumetric studios worldwide with different capturing techniques where the majority needs post-processing after the capture.
3D volumetric live streaming of persons is on its way. The Canon Volumetric Video Studio in Kawasaki Japan, streamed a live performance to YouTube, late August with more than 100 cameras. The Canon VVS is a spin-off of the Canon Free Viewpoint System for sports stadiums where a complete field game can be captured like the Rugby World Cup which they did in 2019.
Intel volumetric studio
The mother of all volumetric studios is built by Intel in Los Angeles with 100 cameras placed in a geodesic dome. With a capture volume of 10,000 cubic feet, it is 35% larger than the Canon volumetric Studio in Japan, which is already of considerable size.
The interesting thing here is the content they make. An interesting case is a cooperation between Intel and Paramount Pictures to remake the movie Grease, the music blockbuster from the sevetnies. And here is the interesting bit: A 124 year old industry is stepping out of the frame to experiment with a new medium.
Intel operators
And here you see that conventions dissolve. How would you call the experience when you are watching this Grease remake in VR?
Is it a movie? or television? or a new reality?
And when we have a live version of Grease the Musical in this same dome, would you call it a theatrical performance with digital and real props? To be honest, I’m still undecided.
As live professionals, the only frame we had was the theatrical proscenium, if it was present. More often than not we use the complete space to bring a performance and submerge the audience into a different world. With Covid we are forced to move into the frame and enter the realm of television production. Maybe it’s time to look beyond the horizon and step out of this frame again.
Connecting the dots
The reality of today is that we fled into the streaming of events and we experience less engagement with our visitors.
The described technologies were for years in the making and were coming our way anyway. Covid will hopefully vanish and when it does, these technologies will still be around. Any investment in time and money into the adoption of these technologies is one for the future. And mind you, there is a lot to learn and discover.
Expect a big wave of devices on the user side. Big consumer tech companies like Apple and Facebook are gearing up for this next wave, bringing their timelines forward spotting an opportunity in a time of urgency.
I don’t have a dystopian vision where the family sits on the couch with their VR headsets on. VR will become an additional medium which we will consciously choose for the desired activity or interaction.
How to proceed?
In the coming months, there are still plenty of Covid obstacles to overcome. Everyone will try to stay afloat in their own way. And if that works with streaming events, keep it up, but know there are alternatives present.
Highly recommended is the book "Remote Collaboration and Virtual Conferencing" by
Charlie Fink. Fink is editor of Forbes and a teacher of XR at Chapman University. Together with eight students, he researched the full spectrum of remote collaboration during the Covid lockdown in the US. From Zoom to VR platforms, all is discussed. There is an extensive reference work with 86 communication platforms.
If you should experience Zoom fatigue with your colleagues again in September, buy or rent some VR glasses and try out the existing various platforms internally. Put together an avatar and get to work with your colleagues. Organize a meeting on a tropical island… just because it's possible.
About the author
Michel Buchner (1976) is an independent consultant in Xtended Reality (XR) and immersive technologies from the Netherlands. Originating from theatre and events he has worked for an experience design bureau and a VR startup. Fields of focus are video scenography, video projected mapped environments and the Xtended Reality space. Since the rise of Covid-19, Buchner is a strong advocate to think outside the video frame and is persuading the industry to start experimenting with Virtual Reality as an alternative for meetings, events and education.
www.linkedin.com/in/michel-buchner/