Throughout the Covid-19 pandemic, event organisers, production companies and more have turned to new and exciting ways to produce content with cutting edge technology. Reece Webb explores the world of XR studios and how they could revolutionise AV workflows.
But what is ‘XR’ anyway? There are plenty of buzzwords flying around when it comes to any kind of extended reality technology, and it is easy for terms over time to blur together, either through a lack of understanding or to sound just that little bit sexier.
Daniel Gray, director, Graymatter Video, works alongside sister event production company iMAG. Gray explained: “XR stands for extended reality. Rather than it being just virtual reality, an XR stage encompasses many different mediums of virtual. The XR stages that have been developed in the last two years use LED screen technology to create the environment around a person or subject.
“Unlike previous virtual stages, which are traditionally green screen, the new LED XR stages allow a person to stand within the environment and to see the environment around them, projected onto LED screens by the media servers. It helps actors, presenters or anybody within that environment to interact with the environment rather than having to imagine it, as is the case with green screen. You can also bring augmented reality into that environment adding 3D objects generated within the servers that can be virtually inserted into the set.
Andy Hook, technical solutions director, White Light, clarified: “Extended reality (XR) is an umbrella term that was coined at VR Days [an immersive technology event] to cover everything augmented reality (AR), virtual reality (VR) and mixed reality (MR), but has taken on a life of its own as a term for immersive spaces.
“To us, XR is an immersive space where the presenter can physically see or interact with their surroundings rather than a virtual studio or set where they wouldn’t be able to see their surroundings.”
XR studios can offer new ways of presenting content and putting on shows across a wide variety of verticals that feel involved and interactive, be it to beam in a live presenter for a keynote speech from a continent away or to enhance the participant and audience experience over the course of a game show.
But it’s not as easy as just turning on a screen and rolling the camera, many technologies have to work in tandem to achieve an effect that feels natural and professional.
Hook said: “LED panels are one of the biggest considerations, you can’t use cheap LED panels. There’s a ‘race to the bottom’ in terms of LED for traditional use cases, but with XR, the camera is a lot less forgiving than the eye, so we need to use high quality panels. Everyone was using ROE at the beginning, and now there are other manufacturers who have tried to catch up and release their own product, particularly over the last year.
“You need the image on camera to look real and believable, it needs to look real from wherever the camera is and it needs to match the virtual world. Processing for the LED is also important as well as the media server/real time content layer.”
Gray added: “It’s all about the R&D. The higher the quality equipment you use, the better end product you will get. For example, we use Brompton HDR-enabled ROE visual LED panels coupled with Brompton processing. We have also tested lots of cameras with different lenses, alongside different pitch LEDs to determine the best combination.
Moiré effect [a visual interference pattern visible on camera] continues to be a problem, so understanding what the limits are and how to get around any issues is vital. A key factor in creating full 360 degree set extensions has been the floor. It requires a very specific matte finish as we can’t deal with reflections when calibrating the system to get the tracking to line up with the cameras and the servers.
“There have been various successes with various tracking system brands such as Trackmen, Stype and Mosis which seem to be the three main ones that people use. We use Stype because we had a better experience with it.”
While LED displays sit as the centrepiece of most XR studios, it’s what happens behind the scenes that really holds the key to a successful facility. LED processors, media servers and render engines are the lifeblood of any successful XR studio and knowing how to adapt your workflow effectively could make the difference between success and failure.
Hook: “It’s not easy to work with XR. The software and setup is complicated. Overall, it’s the expertise that matters. We are at the cutting edge of broadcast technology, so having that expertise to set the cameras and LED processors up is key. Most of us use Brompton [for processing] and disguise [for media servers], it’s their XR workflow that powers most of these XR stages. Everything else like the cameras and lighting is supplementary.”
Gray said: “As with everything, content is key. We work with content houses through the production process, making sure the content is as good as it can be before we even get to site. That’s one advantage of the XR workflow because it’s all done in real-time, the environment is being generated on the server in real time, rather than pre-rendered. If there are problems or something needs tweaking, we can do it there and then.
“Another key link is the media servers, disguise in particular is at the forefront of being the digital ‘glue’ that holds everything together. Real time engines like Notch, Unreal Engine and Unity which are excellent at what they do, have been developed over the years to make content, but they don’t get content out onto screens; that’s where the servers bring all that together, integrating [these engines] further into their systems.”
For those with a background in the gaming world, render engines like Notch, Unreal Engine and Unity will be familiar titles, having pioneered the rendering capabilities of leading AAA games in recent years. With a push towards generation of real time content for commercial AV purposes, these gaming technologies could become increasingly relevant in the industry and lead to an unexpected form of AV/gaming convergence.
Hook explained: “We’ve been using real-time engines in our AV installations for quite a long time. I’m an advocate of the fact that real-time content will completely revolutionise everything we’re doing in every market. Everything we build at the moment is centred around digital and immersive spaces, reactive and interactive content etc. whether that’s retail, theatre, training, education and more. If you’re going to interact with the content, it can’t be pre-rendered.”
The pandemic has offered a boom for XR studios and virtual/real-time content production as a whole, but what is its long-term future? While there is no definite answer in this period of uncertainty, the gut feeling of many is that the XR market will shrink from its pandemic boom in some areas such as the events industry but will remain strong in film and TV as well an essential tool to create engaging and exciting experiences post pandemic.
Hook closed: “On the live event side, which saw a massive explosion during the pandemic, XR will be another tool that people use. Over the next six to 12months, I think we will see a wave of people pushing back to live events because they’ve missed it. We’ll also see people turn to the XR volume to do more events per year as an additional tool.
“Everyone’s got used to producing more content. Every institution has been producing more content than ever before and it’s here to stay. Organisations are adapting to do that, as well as with hybrid working. XR will be a tool that allows us to continue producing content in new ways.”
XR studios: best and brightest
Inavate has collated some of the ‘best and brightest’ XR studios throughout EMEA, highlighting stand-out examples of these cutting-edge facilities, as well as the highlight events and technologies which put them on the map.
Studio name: AED Studios 4 & 7
Location: Antwerp, Belgium
Located at the heart of AED Studios’ headquarters ,studio 7 near Antwerp, Belgium, clocks in at a size of 1,500 square metres with an LED screen size of 29mx4m.As the largest studio facility in Belgium, AED Studios hosted the world premiere of the first live tv show in XR in January 2021 and is currently being expanded to accommodate large scale film productions. The studio features an additional hybrid studio area for full XR productions with an additional greenkey area and LED area for digital livestreams, this studio offers a flexible experience for the ever-changing needs of its customers.
Tech highlights: ROE Visual Diamond 2 and BlackMarble 4 LED screens; LEX Media servers; Stype tracking; Grass Valley LDX; Panasonic cameras; Barco processing.
Studio name: SmartStudioXR
Location: London, UK
Providing a unique backdrop for corporate clients, the SmartStudioXR at the luxury Royal Lancaster London hotel in the UK, comes equipped with a large event space with an LED backdrop and floor and an array of XR equipment. As host for the 2020 entrepreneur of the year awards, Vorwekr’s annual conference and Hospa’s virtual conference, the studio boasts an on-site production team including Unreal editors, disguise operators and camera operators, as well as multiple studio suites to accommodate XR, green screen and virtual events.
Introducing SmartStudioXR at Royal Lancaster London from 4Wall Europe on Vimeo.
Tech highlights: Absen 2.5mm LED; Aximmetry, Notch & Unreal render engines; Barco switcher; Blackmagic Atem 2; Brompton Tessera S8; disguise media servers; ROE LED floor; Shure mic.
Studio name: Prox & Reverie: The Forge
Location: Doncaster, UK
A new studio offering a multi-sensory immersive experience at the heart of Doncaster, UK, this experimental facility, titled ‘The Forge’, pioneers a new kind of XR-powered ‘Metaverse studio’ to access synthetic, physical and virtual realities simultaneously. Using an array of advanced haptics, capture suits and head mounted displays, this studio offers a new take in the world of XR.
Tech highlights: Astera lighting; Epson projector; Fujifilm camera system; HTC Vive headset; Lightform AR projectors; Varjo HMD; Manus XSENS capture gloves; Microsoft Azure Kinect DK volumetric cameras; Hololens MR HMDs; Neumann speakers; NVIDIA GPU; Oculus VR kit, Scatter Depth kit studio volumetric system.
Studio name: White Light’s SmartStage
Location: London, UK
Designed for the workflows of the ‘new normal’, White Light’s SmartStage at the Mermaid London, UK, boasts a 5mx5m LED volume with a fully automated LED moving-light grid. Located within a 600-seat auditorium, the studio features an ROE M2.6 LED wall with an ultrawide viewing angle and 1500 nits brightness, supported by a ROE BM4 matt LED panel (4.8mm pixel pitch) as the floor LED. White Light’s SmartStage has been host to the British and Irish Lions 2021 tour squad announcement, BBC’s ‘Bamous’ transmission pilot and Fusion Comms’ BDO Rethink Festival, a global virtual conference streamed to more than 2,500 delegates.
Tech highlights: Brompton SX40 LED; Mo-Sys Star Trackers; disguise VX2; Martin Mac Aura XB; Arri SkyPanels; Shure mics and in-ear monitors; Riedel Bolero.
Studio name: iMAG mobile virtual production facility
Location: Bournemouth, UK
A production facility that can be deployed as a temporary installation or a long-term solution, the ‘tour ready’ iMAG virtual production studio is currently set up in a large configuration at the Bournemouth International Centre (BIC) in Bournemouth, UK. With a 20mx5m curved LED backdrop with 1,500 nits max brightness, this space excels at adapting and reconfiguring to the required dimensions of its users, calibrating the ROE Visual screens with Brompton’s Hydra measurement camera to make them HDR ready. Harnessing Tessera 3.2 software, this studio enables frame remapping, HFR+ and ShutterSync processing to automatically adjust the timing of the LED refresh to the preferred camera setup.
Check out iMAG's studio here!
Tech highlights: ROE Black Diamond 2.8mm LED screen and MC7 7.5mm ceiling for ambient lighting; Brompton Tessera SX40 4K LED video processor; disguise gx2c media servers.
Studio name: milkit xR lab
Location: London, UK
Located at the Old Dairy Court near central London, UK, the milkit xR lab is a small, permanent XR studio that works with all popular XR worflows (Unreal, Notch and Unity), having worked closely with UK broadcaster ITV to shoot a new ident to mark Mental Health Awareness Week. Fully equipped with a production space and green room for virtual productions, with an array of lighting options with an overhead rig with GLP fixtures such as LED wash and spots with high CRI.
ITV_IDENT_BTS_02 from bluman associates on Vimeo.
Tech highlights: Aluvision 3mx3m 2.5m floor LED, artnet patch system, d&b audio playback system, disguise gx2c servers, RX, Sony FS7 c/w, 18-100 Cine lens, Jib arm + track and dolly, Mo-SYS Startracker, Novastar 4K processor, , Unilumin 3mx3m 2.6mm LED screens.
Studio name: Deep Space
Location: London, UK
Located at Creative Technology’s UK headquarters, this multipurpose studio space boasts a 11mx3m ROE 2.8mm LED screen with Brompton processing at 3872x1056 pixels. Featuring re-useable environments, an in-house content team, HDR capabilities and film-grade LED/processing, this state-of-the-art studio has played host to the AIM Independent Music Awards, Panasonic Business Showcase and Ford Horizon demo event. Ready for any eventuality, the Deep Space studio even comes equipped with fully ISP redundant internet connections to counteract any potential interruptions.
Our Global XR Studios from Creative Technology on Vimeo.
Tech highlights: Astra LED lighting, Brompton processing, Canon camera lenses, Chamsys lighting control, disguise gx2c server, Mac Aura XB lighting, Mark Roberts motion control Robert camera arm, MoSys Startracker camera tracking hardware, Panasonic UC4000 camera channels, ROE BO 2.8mm LED screen, BM4 matt finish LED Riedel Bolero comms system, Ross Carbonite ultra mixer 2 M/E frame, black 2S 2 M/E panel, RTS Odin comms system, VYV S15 media server.
Studio name: MADO XR – PRG Studio
Location: Paris, France
Launched in January 2021, MADO Xr’s PRG Studio in Paris stands out as one of the largest XR studios in France, located within an 800 metres squared cinema studio in Epinay. The studio Is set up as an LED cave, with a 22 metre curve and ceiling, 100 metres of floor and a movable front screen. The XR studio outputs 4x4k native resolution natively at 50fps. The studio has been host to a number of high profile events, including the debut of Peugot’s new logo, the Balmain fashion show and a conference for Assault Systems.
Tech highlights: Brompton SX40 processors, disguise media server, MADO XR Nodes, RTX A6000 Nvidia cards, ROE BlackOnyx LED displays, Unreal render engine.
Studio name: PRG/Cinegate Virtual Production Studio XR
Location: Hamburg, Germany
Located in Hamburg, Germany, the PRG/Cinegate Virtual Production Studio-XR offers a space for digital production as well as pre-recorded content or live production. With a studio height of 5.50 metres and 360 square metres of ground floor, this studio provides control over lighting and recorded scenes, a 90 metres squared LED back wall and 25 metres squared LED ceiling as well as full compliance with Covid-19 safety regulations. With socially distanced workspaces and breakout work zones, this flexible space can be reconfigured for the size and requirements of any production.
Tech highlights: disguise VX4 media server; LDX 80 Worldcam; ROE Black Onyx LED screens; Stype tracking.
Main image photo credit: Andrew Derr, Shutterstock