fujitsu's augmented reality touchscreen interface

fujitsu’s mindblowing augmented reality touchscreen interfaceimage © diginfo

fujitsu laboratories has developed an augmented reality user interface which can accurately detect the users finger and what it is selecting, creating an interactive touchscreen-like system using off-the-shelf cameras and projectors. from the technology, real-world information such as text from books or printed images can be imported from a document. the function enables a user to trace a finger across a document on a table, copy it as digital data, and display it virtually.

‘we think paper and many other objects could be manipulated by touching them, as with a touchscreen. this system doesn’t use any special hardware; it consists of just a device like an ordinary webcam, plus a commercial projector. its capabilities are achieved by image processing technology.’

touchscreen interface for seamless data transfer between the real and virtual worldvideo courtesy diginfonews

augmented camera captureimage © diginfo

recognizing and converting coordinates of physical objects and ICT equipmentimage courtesy fujistu

the technology that uses a camera to automatically measure irregularly shaped objects on a table and then automatically adjust the coordinate systems for the camera, projector and the actual objects. this makes it possible for the software to take the finger movements and touching of objects made by the user, and then match them with the digital display projected onto physical objects.

stabilization of finger recognition technologyimage courtesy fujistu

the system recognizes the shape of the user’s fingers by extracting the fingers’ colors and contour features. other controls the colorand lighting of the camera image, depending on the surrounding ambient light, along with technology that corrects for differences among individual fingers. a stable extraction of the form of the user’s fingers, one that is minimally impacted by the environment and individual variations, can be obtained.

accurate and rapid fingertip recognition technologyimage courtesy fujistu

by enhancing images of the user’s fingertips, the technology is able to obtain a level of precision that is sufficient for touch detection, even using low-resolution images that can be captured on a regular webcam. moreover, the technology is capable of fingertip tracking speeds of 300 mm per second, thereby enabling it to follow natural finger movements.