Realistic training is translating to better working outcomes. Reece Webb explores the changing world of simulation and immersive training.
Measure twice, cut once,' a famous adage used by tailors for generations when crafting the most timeless and elegant garments without wastage of material and time. This is a phrase that has served many well in crafting solutions that are watertight in their preparation, but what if we don’t need to cut at all? What if we can measure and cut virtually, feel the weight and movement of the fabric without even having it to hand?
This is the reality that has been enabled by advancements in simulation and immersive training technology, breaking beyond the barriers of a traditional projection CAVE environment to enable industries of all stripes, from healthcare to military and corporates, to embrace a new level of realism, sophistication and accessibility in their simulated systems.
John Mould, commercial development manager, ST Engineering Antycip, explains: “The trend for immersive applications, from what we have seen, is an increased interest in linking participants collaboratively between display devices of all levels and types -from linking VR Caves to HMD displays and connecting remote third parties, who may only have a simple laptop or desktop to join, over a network connection. This means they can all collaborate and interact in the virtual world together, seeing each other as a virtual avatar. The ability to share conversation as if present in this virtual environment is also key so you can converse with your colleagues, partners or trainees.
“There also seems to be a trend to mix the real with the virtual, for example a medical resuscitation training dummy being used whilst the student is immersed within the virtual setting that may be appropriate, such as a hospital ward or incident scene at the roadside.
The evolution of haptic gloves has given users the ability to feel in VR with a realism level not experienced before. Some of the latest tech has over 130 microfluidic air pockets that inflate and deflate, precisely integrated within the fingers/palms of the gloves to provide a sense of interaction that is uncanny.
So whilst visual senses are always advancing, so are the other senses such as touch and even bringing the smells that matter into the virtual experience. For some military requirements, the smells associated with the battlefield can test soldiers' natural gag reflexes and condition them to be more effective in the worst of simulations.”
“We’ve been around for 27 years, and some of the first systems that we built were simulation environments for the oil and gas industry," explains Peter Stewart, EVP global alliances, strategy and corporate development, Cyviz, “It was incredibly expensive and dangerous to fly people out to oil rigs and train them. Instead, we’d have on-the-ground simulation in a 360 environment where you are operating a simulated oil rig. This required very sophisticated blended projection systems which were very cumbersome, but new display technologies and our platform have made it a lot easier and cheaper to build these simulation environments.”
Real world results
Simulation technology not only needs to look convincing but must feel convincing, too. This not just from the hardware, but to the software and physics engines that must be as realistic as possible without unrealistic physics exploits or glitches that can have an impact on training outcomes. Mould explains: “A lot of what we see in training is bringing in repeatable physics. Having a physics engine that can replicate real world forces and constraints means that when you are doing a lift exercise or digging with an excavator, it won’t simulate something that isn’t possible in the real world. If the crane begins to exhibit a pendulum type swing, you’re going to get the reality of what would happen if you tried to exceed the parameters of the crane. It’s important to have realism, not a gaming system that looks realistic but doesn’t deliver physics that are achievable in the real world. We have to be delivering a more professional grade level of physics with simulators, as you can’t do things on there that are impossible in the real world, otherwise it’s a negative training value for the operator.“
IBM Cyber Campus by Cyviz
Creating an environment that delivers real-world training results in a safe yet realistic environment can be tough. A standout example of this can be found in Cyviz’s collaboration with IBM to deliver a global cyber campus programme, preparing students to face real world cybersecurity threats in a realistic working environment. The technology installed by Cyviz allows students to have a realistic, high quality experience in real-world scenarios that blend role play with industry-standard AV technology to combat computer viruses, malware and ransomware in high pressure environments without potentially losing millions to hackers.
Peter Stewart, EVP global alliances, strategy and corporate development, Cyviz, explains: “IBM had a business training their corporate customers on how to run cybersecurity centres, running the executives through a hacking simulation where everybody has a role as CEO, CTO, etc. and they would go through an intense, immersive exercise. They even built film studios to conduct mock interviews for the CEOs during a crisis. Universities recognised that there was a skills gap between what they are doing and what the market needs. Up to 10 trillion USD can be chalked up to the impact of cybersecurity breaches, and the speed, volume and intelligence of attacks are all increasing.
“IBM was part of a White House key strategic initiative to train university students in a realistic environment where they can be fully prepared for a career in cybersecurity centres. IBM brought their experience, and we came in with all the AV technology and we’ve built these campuses all over the US, Canada and Italy.”
Cyviz delivered a simulation system that works as a fully functioning control centre, where every student plays a role. It is outfitted with display technology and videoconferencing capability to showcase live feeds with other participants to enhance the real world experience. Cyviz uses display technology to simulate news broadcasts, cybersecurity monitoring software displaying threats, and a data centre where trainers can safely unleash a virus for students to eliminate. Cyviz even brought in elements of irrelevant ‘noise’ aimed at providing sound for the students to filter through to improve their situational awareness and hone their skills in realistic environments.
Stewart continues: “On our system, you can choose what content is where through the control system and our dynamic video processing, but it’s still really easy to use. We have a configurable control system that is purpose-built with our video processor to give a bespoke user experience. We use giant, ultra-wide LED walls with dynamic content and side displays, with all of these spaces being hybrid enabled. The trainers will pull in external people to the meeting to participate, which is a reality in post-Covid workspaces. We even simulate news broadcasts which the students have to react to, and the phone could ring with an ‘FBI handler’ on the line asking about a status update. The student could be challenged on how they know it’s the FBI and not a hacker, so it’s little things like that they have to run through, because in the moment, you could be arming a nefarious actor with additional information. There aren’t many universities where students get to do this.
“We also use visual indicators to highlight the state of the room. LEDs can be used to show a ‘red’ state for example to see the condition of the room, displaying different levels of threat, much like a real command centre.” Cyviz has rolled out this technology to multiple campuses in North America and Italy, scaling the setup up and down based on the size of the room to create a ‘one size fits all’ approach to realistic training, wherever they are in the world.
Broader horizons
Simulation usage and technology is experiencing a renaissance as new technologies, particularly LED, make an impact alongside motion simulation and immersive audio to fully enhance the experience.
Mould clarifies: “There is a broader use of simulation, and at the same time it has become more niche. We have seen a huge surge in driving simulators especially,
because simulators now have a motion platform, physics and hardware capability that reflects real life. We are partnering with a company that makes these platforms and the fidelity of that is akin to the real vehicle, so if you are making a car and commissioning a vehicle, you can get all the ride quality information and understand how the suspension works before you even make the vehicle, saving hundreds of thousands. You also get a realistic drive out of that simulation technology, which is why it’s used in autosports. They’re the guys who need to shave off milliseconds on laps and they need hardware that reflects the real
world. Reality is key in simulators, and that’s something that has grown with the hardware that is delivering against real world parameter.
“On the technology side, we’ve seen a lot of manufacturers who are doing very sophisticated head mounted displays (HMDs) which is changing a lot of training requirements. We’re seeing the weight of HMDs go down to 300 grams, and that’s a big difference to how it used to be. “We even have projectors for simulation that are 4,096 x 2,176 native resolution, and projectors that can deliver up to 360Hz putting frame rates through the roof. This gives us lots of capabilities with projectors now, so if we are doing multi-view stereo in training, we can use projectors with 360 frames per second between three individual people who are tracked dynamically through their viewpoints, looking at the same screen but seeing something totally different. “We see this as quite a game changer for display technology, and for training, having three people in the same world, each with a unique dynamic perspective. This changes what VR CAVEs can do as it emulates reality where we are all independently able to choose what we see.
“Today we’re starting to deliver LED based displays, using DirectView LED tiles. We now have technology that can do Multiview stereo with LED tiles, something we are delivering for a university. We’re seeing the frame rates on those tiles coming up and the brightness is giving us more abilities for the 3D support that projectors sometimes struggle with. You don’t need to have the cinematic conditions of the past, today we can have these very vivid displays that can hold their own in the ambient conditions of a room space. We’ve even done a VR CAVE with LED, and it’s very different to doing it with projects.”
Get real
With a widening scope and greater demand for simulation technologies in a variety of industries, what will the future hold for this ever more sophisticated and efficient learning tool?
Stewart says: “It’s so much easier and cheaper to create and run simulations with new AI cloud technologies. When we think back on the supercomputers that we used to have to create and modify
content, you can now have a general purpose PC with cloud computing, which is a game changer for simulations that makes it cheaper, easier and faster to create a simulated environment. We’ve been talking with a beverage company that want to simulate different stores that a person can walk through as part of virtual planning for building physical stores. They had one simulated environment that cost millions, but now they can have more of those centres at a lower cost, closer to their customers.
“For engaging university students, you have Gen Z and Gen Alpha that need to learn in a different way. They consume so much content, so everything must be more engaging. If you have an environment where you have to actively participate, they will react to it in real time. It’s not like listening to a lecture and taking a test, you have to contribute, and those students have a competitive test when they apply for roles as they have real world experience in executing programs and dealing with real-world scenarios in real time.”
For Mould, the wider adoption of more sophisticated simulation technology is a question of when, not if: “Pilots are mandated to train simulated hours, and that’s accepted as a training medium. As simulation technology becomes more realistic, it will feel natural for more people to train and build competence in virtual environments - especially when those experiences translate into
real-world benefits. This will encourage more people to adopt this technology, for example it is not mandatory in construction, but the industry is seeing the advantage of having simulations. It’s producing better candidates, better outcomes when workers get on the real machines. It’s better for costs as workers aren’t damaging equipment and making mistakes because they’ve been on simulators, applying base skills and retaining useful muscle memory. We’ve seen more companies gravitate towards simulation technology because it’s becoming more cost effective and accessible, so it’s just a matter of
time before it becomes less of a want and more of a necessity.”