Surgeons see 3D models in eyepiece via AR

The boundaries of modern medicine continue to be pushed by technology with the latest use of an augmented virtual reality mapping tool by Mount Sinai in New York. The hospital has announced the first practical use of the CaptiView AR system, a heads-up-display that overlays data and 3D models to the eyepiece in a surgical microscope.

For medical professionals AR and VR can offer critical information on blocked or hidden parts of the body through body mapping. A system such as ‘CaptiView’ from Leica Microsystems has been designed to take advantage of this by sending "neurovascular and fibre-track information in 2D or 3D as well as on-screen video overlays visible" through to the eyepiece of a microscope. 

The microscope with CaptiView uses an image injection system that overlays VR imaging onto the brain when viewed through an eyepiece during surgery. The microscope gives surgeons the ability to change from live and pre-operative anatomical images via handle controls or a footswitch. 

The system is also aware where the operator is looking and autofocuses by following the gaze.

Check out a video of the first neurosurgeon to use CaptiView, Joshua Bederson, MD, talking about the experience above.