A research team from Japan and China is working on a new technique that can address the issues found previously with augmented reality (AR) glasses, where the technology has struggled in providing realistic virtual images while maintaining the brightess of digital information from the real world.
A research team including Xiaodan Hu, an assistant professor at Shibaura Institute of Technology in Japan and a key contributor to the project, collaborated with Dr. Yan Zhang and Professor Xubo Yang from Shanghai Jiao Tong University in China and Professor Kiyoshi Kiyokawa from the Nara Institute of Science and Technology in Japan to develop a perception-driven display strategy that jointly considers human visual perception and real-world lighting conditions.
Instead of relying solely on hardware improvements, the researchers introduced a mask-balancing method that dynamically adjusts the visibility of real and virtual scenes according to what users can actually perceive. The study was published in IEEE Transactions on Visualization and Computer Graphics.
The proposed method combines a polarised component that supports pixel-level occlusion with another polarised component that bypasses the optical system and preserves the natural brightness of the real world. By adjusting the cross-angle between a polarising beam splitter and a linear polariser, the system dynamically blends these two views. Real-time eye-tracking and scene analysis estimate the visibility of both the environment and virtual objects, allowing the display to continuously optimise the balance between them.
To establish perceptual thresholds, the researchers conducted a series of user studies. Experiments involving 12 participants quantified how much contrast was required for users to recognise textures in virtual objects, while another study with 24 participants evaluated the perception of lighting effects. The team then integrated these findings into a dynamic balancing strategy and validated it with a benchtop prototype. A final user study with 12 participants demonstrated that the system improved real-world visibility while maintaining a convincing appearance for virtual content across different illumination conditions.
"Our research demonstrates that human visual perception can be used to dynamically balance the visibility of the real world and virtual content," said Prof. Hu. "By adapting the display according to what users can actually perceive, our method improves real-world visibility while preserving the appearance of virtual objects under different lighting conditions."
The study also highlights a broader shift in AR display design. Rather than optimising optical hardware alone, the researchers show that understanding how people perceive visual information can lead to more effective display control strategies. This perception-driven approach could help future AR glasses present virtual objects that blend more naturally into the real world while maintaining user safety and comfort.
Potential applications for optical see-through AR systems could include industrial maintenance, medical assistance, education, navigation, and remote collaboration, where users must simultaneously monitor their surroundings and interact with digital information. By preserving both real-world awareness and virtual image quality, the technology could make AR devices more practical for demanding real-world environments.
image credit: Credit: Assistant Professor Xiaodan Hu from Shibaura Institute of Technology, Japan