A real-time American Sign Language word recognition system based on neural networks and a probabilistic model

Abstract

The development of an American Sign Language (ASL) word recognition system based on neural networks and a probabilistic model is presented. We use a CyberGlove and a Flock of Birds motion tracker to extract the gesture data. The finger joint angle data obtained from the sensory glove defines the handshape while the data from the motion tracker describes the trajectory of the hand movement. The four gesture features, namely handshape, hand position, hand orientation, and hand movement, are recognized using different functions that include backpropagation neural networks. The sequence of these features is used to generate a specific sign or word in ASL based on a probabilistic model. The system can recognize the ASL signs in real time and update its database based interactively. The system has an accuracy of 95.4% over a vocabulary of 40 ASL words.