BSL Translation in Your Hand

Machine translation has proven a difficult nut to crack, even for written and spoken languages. That’s even more true for sign languages, though we’ve written about some previous innovations in the field like the Fingual translation glove.

However, that may be about to change. Researchers at Aberdeen University are working on providing sign language users with translation capabilities on their smartphones and laptops, and they’re almost there. In the Scotsman, Dr Ernesto Compatangelo, the director of the project, explained the ultimate goal of the project, called the Portable Sign Language Translator (PSLT):

“The user signs into a standard camera integrated into a laptop, netbook, Smartphone or other portable device such as a tablet. Their signs are immediately translated into text which can be read by the person they are conversing with. The intent is to develop an application – an “app” in Smartphone terms – that is easily accessible and could be used on different devices.”

The technology is being developed with British Sign Language (BSL), but can easily be made to work with other sign languages as well.  Additionally, users can program the software to recognize gestures of their own, allowing them to get around limitations in sign language vocabulary.

In the Sydney Morning Herald, Dr Compatangelo gave the example of a student who wishes to learn a trade such as joinery, and needs an easy way to express words like “dovetail joint,” which are so specialized that BSL does not have gestures for them.

Meanwhile, on MIT’s Technology Review blog, David Zax gave another example:

“Plenty of new jargon and terminology is constantly emerging in computer science, but American Sign Language is unlikely to keep up with all that jargon. You could use the app to invent signs to express these bits of jargon, saving you the trouble of having to spell out every word letter by letter.”

Another potential use for an app like this would be to provide people who are learning sign language an easy way to practice. In the Scotsman, Dr Compatangelo explained:

“As a learning tool, the PSLT can be easily and effectively used by those who are learning to sign. So far, these learners needed a sign language expert in front of them to check that they were able to sign correctly. This is a problem, due to the scarce availability of sign language experts and to the consequent cost of such training.”

Another possible use? The ability to control your household appliances with gestures, which would be especially useful to people with mobility issues.