This web site is no longer maintained and the content may be outdated.
Please visit www.cmpe.boun.edu.tr for up-to-date information.
 
CmpE RSS
No upcoming events...

Home / Graduate / M.S. Theses Completed
 
 
 
 
  Cem Keskin, 2003  [download thesis]    

Thesis Title

VISION BASED REAL–TIME CONTINUOUS 3D HAND GESTURE RECOGNITION INTERFACE FOR GENERIC APPLICATIONS BASED ON INPUT–OUTPUT HIDDEN MARKOV MODELS


Abstract

This study focuses on the application of Input–Output Hidden Markov Models (IOHMM) to the recognition of 3D hand gestures that involve both hand motion and hand posture in a stereo vision–based approach. The proposed system is designed as a real time gestural interface that allows both communicative and manipulative gestures to control target PC applications. The system allows training of new communicative gestures and automatically distinguishes these from manipulative gestures in continuous streams. Uniquely colored gloves with no preset colors are used as markers, which increases the recognition rate and simplifies the hand localization problem. Camera calibration and gesture training tools are provided with the system.

The hand shape is modeled with Hu moments and the angle of the hand. As a novel approach the inclusion of normalized time information as an input to the IOHMMs is proposed, which has the effect of a continuous state variable and is shown to handle temporal information better. Another novel method is proposed for gesture spotting in IOHMM–based frameworks, which distinguishes the meaningful gestures in a continuous input stream in real time using a threshold model with a single hidden state.

The system is tested on two datasets that have 10 gestures with distinct trajectories, and 20 gestures in 10 pairs that share the same trajectories. The proposed system is able to attain a recognition rate of 97.6% on the former dataset in 2885 trials, and a recognition rate of 94.1% on the latter dataset in 8675 trials. The gesture segmentation test results for the same datasets with the inclusion of the proposed threshold model are 97.1% for the former and 93.8% for the latter dataset.
 
 
Bođaziçi University Department of Computer Engineering
Address: 34342 Bebek, Istanbul, TURKEY
Phone: +90 212 359 4523-24 Fax: +90 212 287 2461
general information: infocmpe.boun.edu.tr   webmaster: webmastercmpe.boun.edu.tr