Imagine you’re a signing Deaf person and you find out about a spoken lecture. You need a human interpreter, but there’s no time. What do you do?
In the UK, soon you’ll be able to use IBM UK’s Say It Sign It (SiSi) translation system.
SiSi converts spoken English to British Sign Language (BSL) via speech recognition, translation, and a computer-animated avatar character.
Demonstration video on Youtube (14 seconds)
This is a wonderful invention for simple communication and last-minute occasions. I hope, however, that people don’t try to replace human interpreters with speech-recognition avatars. The nuances of the face, which can carry important grammatical information in sign languages, can’t be perfectly rendered in animation in all their complexity and subtlety.
I also wonder how good the speech-recognition and translation programs are. I suspect the device is really translating into a pidgin sign language that includes elements of BSL and English. The way information is presented in natural sign languages and spoken languages can be quite different. Sign languages use a lot of simultaneity/overlapping of information in space and what are called size and shape specifiers, or classifiers (such as for flat objects or round objects or vehicles or people). Translation is not easy.
Still, I hope IBM can make Say It Sign It better and better.