Hand detection and gesture recognition are two of the most studied topics in Human-Computer Interaction (HCI). The increasing availability of sensors able to provide real-time depth measurements, such as Time-of-Flight (ToF) cameras or the more recent Kinect, has helped researchers to find more and more efficient solutions for these issues. With the main aim to implement effective gesture-based interaction systems, this paper presents an approach to hand detection and tracking that exploits two different video streams: the depth one and the colour one. Both hand and gesture recognition are based only on geometrical and colour constraints, and no learning phase is needed. The use of a Kalman filter to track hands guarantees system robustness also in presence of many persons in the scene. The entire procedure is designed to maintain a low computational cost and is optimised to efficiently execute HCI tasks. As use cases two common applications are described: a virtual keyboard and a 3D object manipulation virtual environment. These applications have been tested with a representative sample of non-trained users to assess the usability and flexibility of the system.

Development of gesture-based HCI applications by fusion of depth and colour video streams

DONDI, PIERCARLO;LOMBARDI, LUCA;PORTA, MARCO
2014-01-01

Abstract

Hand detection and gesture recognition are two of the most studied topics in Human-Computer Interaction (HCI). The increasing availability of sensors able to provide real-time depth measurements, such as Time-of-Flight (ToF) cameras or the more recent Kinect, has helped researchers to find more and more efficient solutions for these issues. With the main aim to implement effective gesture-based interaction systems, this paper presents an approach to hand detection and tracking that exploits two different video streams: the depth one and the colour one. Both hand and gesture recognition are based only on geometrical and colour constraints, and no learning phase is needed. The use of a Kalman filter to track hands guarantees system robustness also in presence of many persons in the scene. The entire procedure is designed to maintain a low computational cost and is optimised to efficiently execute HCI tasks. As use cases two common applications are described: a virtual keyboard and a 3D object manipulation virtual environment. These applications have been tested with a representative sample of non-trained users to assess the usability and flexibility of the system.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11571/804469
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 15
  • ???jsp.display-item.citation.isi??? 11
social impact