XR studios and virtual production have created a lot of buzz, but also confusion too. Paul Milligan asks the experts.
Few technologies received the hype that XR studios did last year, and it has become one of the hottest topics in the AV world. But with this newfound attention came confusion as to what the term actually means. This was exacerbated by the fact that virtual production is often lumped in with XR studios by the media and advertisers. XR studios and virtual production are linked, and can use some of the same technologies, but are ultimately separate entities. We spoke to those providing and installing the technology, to find out how widespread the confusion was in an effort to foster a greater understanding of the two technologies, and to find out where the opportunity lies.
First of all, is this confusion widely apparent out there in the industry? It appears so. “There definitely is [confusion], the industry discusses both these terms and quite often they are interchangeable, and sometimes not appropriately so,” says Brian Macauto, director of virtual production, Absen. Does it actually matter if we get the terms mixed up though? Yes, it does says Tony Van Moorleghem, technology and product director, XR division from InfiLED. “We have to split up the two markets completely. From a technology point of view as an LED manufacturer it makes no difference. But the markets are completely different, and the requirements are completely different.”
Where has this confusion come from? The issue is two-fold says Mark Pilborough-Skinner, head of virtual production, Garden Studios. “Virtual production as a term is quite nebulous. It started back in the early 1900s with people painting over glass and putting it in front of a camera, everything from green screen matte paintings, to rear view projection, to LED volumes. The second part of it is just marketing and branding.”
Levels of confusion also differ depending on what market you are talking about says Hans van Houdt, director of business development, Leyard Europe. “If you look at the cinema market there’s no confusion because everybody knows exactly what it’s about. If you go to the event industry there’s already a lot more buzzwords where people don’t really understand the concept, but just want to be a part of it. if you go to the corporate and broadcast markets there’s some confusion. Everybody’s talking about the concept, but not everybody really understands.”
This from Will Newman, business development and DMPC manager from Sony Professional Europe is as good as an attempt you’ll see at two clear definitions. “XR stands for Extended Reality, and is an umbrella term encompassing AR, MR (Mixed Reality) and VP (Virtual Production). The confusion often is that people believe anything involving an LED wall is VP, however an LED wall can just be used for 2D plate playback much in the same way rear projection has been done for years. Virtual Production however involves precise camera tracking data being fed through a game engine that is then displayed on an LED wall.”
When there is confusion between the two, if enquires are made at a tradeshow or an event, do those interested parties typically want to talk about XR or VP? “Many customers know they want to get involved in this new technology. So we explain the different levels of VP, what can be accomplished with that, and what XR is, and what can be accomplished with that. Usually, my very first question is to ask what are they trying to accomplish in this space? And then try to understand their needs,” says Macauto.
When clients or agencies ask about XR or VP, does the decision come down to the budget available or the size of the room available for filming, or is it driven by the content aims? “I would say it absolutely starts with the end result of the content and then moves to the size of the room and the budget,” says Macauto. If the goal is to create a lifelike scene that the audience has no idea whether it’s being filmed in the real world or on a soundstage in Hollywood, but the audience has no concept of that idea whatsoever, then the choice should be ICVFX (In-Camera Visual Effects), where you have a very large LED wall and the LED wall is so big that the camera shot never leaves the LED wall, and the background is always handled by ICVFX. “If a client is trying to reproduce green screen and they’re trying to do something on a small stage but they’re trying to create a large world with it, like a music video, that’s definitely something that XR has been used for,” adds Macauto.
There is no doubt there is a craving for both technologies, even when confusion reigns. Once confusion is addressed, and it seems as though this can be done with a simple conversation with your chosen LED supplier, is there an opportunity here for the AV world to make some real money? Yes, but not in every aspect of XR/VP however. “Everybody believed that virtual production was going to be a massive investment, and a lot of companies have heavily invested in it, but a lot of companies have failed, and a lot of these studios are closing down. We do see there’s going to be a massive decrease in the amount of big VP studios,” says Van Moorleghem. A boom, caused by the pandemic, may be over for VP, but there is still opportunity there, if plans are scaled back, and AV tech can help in this regard. “Large scale VP walls and projects are expensive endeavours, however small pixel pitch screens (such as our Crystal LED 1.2mm/1.5mm) allow for smaller LED volumes, which you can then also have your talent closer to. This means that you no longer need to have a large studio to be able to entertain the idea of a VP setup. Also, the smaller volumes will be a cheaper overall setup so much more accessible,” says Sony’s Newman.
Scaled down is where the opportunity lies agrees Prak, “If you look at the larger film volumes, we’ve done a lot of those in the past two years, but there is a stretch in that market, and I think we have reached that point. In smaller applications there’s a large market segment for commercial shoots. There’s huge possibility there. But also with companies that want their own XR stage to be able to have keynotes for their international employees, there’s a lot of possibilities there too.” Leyard’s van Houdt reports ‘massive interest’ from discussions he’s been having, citing two markets in particular; broadcast and corporate. “Of those two, the corporate market has huge potential for XR. After Covid we are used to hybrid comms. Companies have to communicate via the internet to their own people and to the market. People expect a much higher quality of videos coming out than was done two to three years ago. Video has to be up to a professional level. There’s huge potential for XR technology for that market.”
The corporate market is one that everyone we spoke to mentioned as the biggest opportunity for the AV world. “I think that’s a huge potential market, if we get it right,” says van Houdt. “One of the big challenges will be how to provide studio technology that should be in-house for a company, because you will not get a CEO to a studio. It should be in your building. The setup needs to be in such a way that the people who normally look after the IT can do this. They will definitely need remote support from industry professionals, and we need to develop services around it to help people get used to it.” Because of the cost involved, even in a small XR setup, it will probably be larger, international companies that plump for it initially for a variety of reasons says ROE’s Prak.
“They have the pressure on being sustainable and travelling less, but it’s also a way to reach out in a very engaging way.” Could we even get to the point where every medium to large corporate organisation has an XR studio? From a logistical point of view, they could be fitted without too much adaption in a 10-20 person meeting room for example. And there are more of those free now with the increase in hybrid working. “I absolutely think it could be standard practice,” says Macauto. The example he gives is that if you look at the cost of doing high-end productions ad-hoc, because there are no in-house resources to produce this, the per training cost of that video is high. “If you look at producing a very simple in-house studio, that company can produce quite a bit of content with it. That simple in-house studio is probably just a room with a very simple backdrop and some camera systems,” he adds. Another sector that offers potential for the AV world is education, “A lot of universities are adapting XR stages as a means of either communication or training. I see a definite development there,” says Prak. To demonstrate this, Prak has recently worked with a young company developing classes for young children, who were brought into a cubicle and taught about dinosaurs in an immersive environment which drew them in to a whole other world.
If there is potential for the corporate market to embrace XR studios, and it seems anecdotally there very much is, could the management of such spaces be passed off to an IT department eventually? Or would they always need specialist AV or production skills? The answer to these questions could well be very different in five years’ time, to what it is now. The answer for now is probably not, in the future the answer is almost certainly.
This example from Garden Studio’s Pilborough-Skinner highlights why it’s probably not going to happen right away. “When we first opened we had someone from the live events industry handling all of our power LED image processing, we had a traditional film background person who handled cameras and learned about the tracking systems, I’m from a games programming background, so I was dealing with the games engine and getting the images to render. Between the three of us we have the skills to run the stage. I don’t think anytime soon you’re going to be having general IT practitioners being able to run and customise a full stage.”
This need for a highly trained team may not last however, especially with the inevitable advancements in hardware and software that will come. “A very specialised integration company could really put together an iPad-type system and hand it to somebody. It could be set up as a cloud-based system, the end user could log in and pick one of the ten environments that are available. All I need to do is tell it the size of my XR stage, hit ‘calibrate system’ and it’s even going to tell the camera jib to all do all these movements for us, and it’s going to look amazing,” says Macauto.
Even though virtual production has been around for quite a while, the same can’t be said for XR studios, so even though there is some confusion between the two terms, that won’t last as it’s just a symptom of XR’s youth. Especially if the current demand from the corporate and education sectors is followed up on with real investment. This demand presents a great opportunity for the AV world, not just in product sales and fees for design and installation, but also for the service and support of technology-rich spaces once they have been installed.
