The Microsoft Research team are making use of everyone’s favourite interactivity solution, Kinect, in a research project that should have fans of future technology truly excited. The HoloDesk allows users to pick up and manipulate virtual 3D objects, whilst also recognising the presence of other non-virtual objects in the environment. The research team hasn't revealed too much about how its new natural user interface system works, but read one to see what we do know, and a video of it in action.
It's about the size of a filing cabinet and is made up of an overhead screen that projects a 2D image through a half-silvered beam splitter into a viewing area beneath. A Kinect camera keeps tabs on a user's hand position within the 3D virtual environment, a webcam tracks the user's face to help with placement accuracy, and custom algorithms bring everything together in (something very close to) real time.
The user looks down through a transparent display into the viewing area where holographic objects can be picked up and stacked on top of real-world ones, and real hands can juggle virtual balls or shoot them at targets, or play with a non-existent smart phone.