Aerospace and defence company, Raytheon, has been selected to develop MAGIC: An AI powered AR device that guides medics through medical procedures.
The MAGIC (Medical Assistance, Guidance, Instruction and Correction) device will be designed to provide real-time, hands-free guidance for 50 different medical procedures, with the aim of increasing medic proficiency and reducing treatment errors in challenging environments such as on battlefields.
Though designed with a military use case in mind, such technology could easily be transferable for use by first responders, surgeons and more.
The device is being developed under a DARPA award, with Raytheon-BBN working with Valkyries Austere Medical Solutions for the project. To train the AI, the team will collate more than 2,500 videos of combat medical data sets recorded in stereo to provide 3D data.
50 million images are used to identify the correct sequence of procedures, using DARPA’s task guidance program’s AR goggles which will use AI to use visual and audible sensors that track tasks and guide medics in performing correct medical procedures.
The built-in cameras of the AR goggles observe the user’s actions, using AI to follow the steps correctly either on request or when intervention is needed through auditory suggestions or visual overlays in the headset, such as follow-on actions or measurements such as elapsed time or dosing.
The first demonstration of the MAGIC device is expected in November 2023, with training data collected for the program shared with the research community.
Brian VanVoorst, scientist, Raytheon BBN, explained: “Our goal for the Raytheon BBN MAGIC AI tool is to help support personnel to provide guidance as needed without disrupting their concentration.”