21.01.11

Fraunhofer lets shoppers buy with gestures

Window shopping takes on a literal meaning as Fraunhofer researchers unveil a digital signage concept that allows shoppers to purchase goods from the shop window. The system uses four cameras to record the 3D positions of people’s hands, face and eyes allowing them to control the window display to look at and buy goods even after the shop has closed.

The camera system was developed at the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (HHI) in Berlin, Germany. Paul Chojecki, a scientist at the Institute, said: “Interactive shopping has been standard operating procedure in the web for a long time. Now, we’re putting this technology into pedestrian passageways and shopping centres with the entire unit behind the window.”

Four small cameras continually record the 3-D positions of the hands, faces and eyes of people passing by. Then, image-processing software calculates the co-ordinates and transforms them into the corresponding inputs for selecting goods, viewing them in detail and immediately purchasing them. Further product information such as colour, material, price, availability and information on the manufacturer can be called-up.

Chojecki notes: “There’s nothing comparable in Germany yet and today shops only use touchscreens in shop windows, if at all. But, you can interact with our interactive shop window without any physical contact.“

Two stereo cameras record the face and eyes while the other two record the motion of the hands. Image processing recognises both gestures such as when you turn your hand and when you point to a button with your finger that you can see on a monitor. The system doesn’t store any personal data and only the co-ordinates of the body parts it recognises are passed onto the visualisation.

Interactive shop window is compatible with all displays. Furthermore, shop owners can link the system with existing software such as content management or merchandise information system. The way the payment process is taken care of is also left up to the shop owner.

The interactive shop window identifies how many people are in front of the shop window and can suggest on the basis of the gathered data what products and information the people passing by are interested in. It also has customised greeting texts on the display.

Although the interactive shop window was developed for use in shopping centres and the retail trade, Chojecki thinks it would be possible to install it in museums or at trade fairs. The 3D recording system currently a prototype and researchers will be demonstrating it at the CeBit Fair in Hannover, Germany Fraunhofer’s joint stand (Hall 9, Stand B36) March 1 to 5, 2011.