An advanced, 3D, gesture recognition system has been unveiled by Fraunhofer FIT. The German institute explores human interaction with IT and says its latest development detects multiple hand and finger positions in real time for translation into action commands. The sophisticated interaction follows similar command gestures propelled into the mainstream by products such as Apple’s iPhone but, in this case, does not require touch.
The prototype device tracks users’ hands in front of a 3D camera. This camera uses the time of flight principle, in this approach each pixel is tracked and the length of time it takes light to be filmed travelling to and from the tracked object is determined. This allows for the calculation of the distance between the camera and the tracked object.
"A special image analysis algorithm was developed which filters out the positions of the hands and fingers. This is achieved in real-time through the use of intelligent filtering of the incoming data. The raw data can be viewed as a kind of 3D mountain landscape, with the peak regions representing the hands or fingers." said Georg Hackenberg, who developed the system as part of his Master's thesis. In addition plausibility criteria are used, these are based around: the size of a hand, finger length and the potential co-ordinates.
The institute does note that work remains to be done on removing elements which confuse the system, for example reflections caused by wristwatches and palms which are positioned orthogonal to the camera.
"With Microsoft announcing Project Natal, it is likely that similar techniques will very soon become standard across the gaming industry. This technology also opens up the potential for new solutions in the range of other application domains, such as the exploration of complex simulation data and for new forms of learning," predicts Prof. Dr. Wolfgang Broll of the Fraunhofer Institute for Applied Information Technology FIT.