Kramer's Talking Glove Project


James Kramer and his supervisor, Larry Leifer, have been working on a method for communication between Deaf, Deaf-Blind and Non-vocal individuals. It is a complete system, which attempts to integrate a number of technologies together, in such a way that all parties can communicate. This is shown in thefigure below.



Figure: The complete Talking Glove system.

Of interest and relevance to us is the glove Kramer developed (the CyberGlove) and the technology used for recognizing American finger-spelling.

At the moment, the system has acceptable finger-spelling performance, and James Kramer has set up a company, called Virtual Technologies, to market the system. In September 1995 VirTex released a commercial finger-spelling recognition package called GesturePlus. However, the system costs US$3500, and this price does not include the CyberGlove.

Initially, he used a prototyping algorithm. There were prototypes for each letter, which Kramer termed ``beacons''. These beacons which are points in the hand state vector space, are then surrounded by hyperspheres. When a hand enters the the hypersphere, a letter is produced. This hypersphere, which Kramer calls a ``recognition ball'', lies within another hypersphere, which Kramer terms the ``hysteresis ball''. To repeat a sign, the hand must move outside the hysteresis ball, before another handshape is recognised (this intuitively makes sense, since when finger-spelling and a sign is to be repeated, it is typically made distinct in a similar manner -- ie the gesture is repeated twice). See figure below. This shows the hand-state moving through a vector space, which in this case is simplified to three dimensions, but there are 16 dimensions in the actual model.


Figure: The approach adopted by Kramer to recognise finger spelling.

The above system has been road-tested and has been shown to be practical.

Currently, Kramer is working on moving the above model to a neural-network implementation, and also broadening the scope to more than finger-spelling -- that is to full signs.