These Sign Language Translation Gloves Won a Prize, But Can They Really Translate?

2 US students just won a prize for inventing the SignAloud gloves,  a pair of gloves that translates American Sign Language to speech. But can these sign language translation gloves translate in the real world?

The idea of a wearable device that can translate sign language is certainly not new. Over the years, we’ve seen prototypes for everything from jewelry to gloves that promise to make it easier for deaf and hard of hearing people to communicate with the hearing world.

Every so often, someone will announce an amazing new invention to make sign language translation easy, automatic and convenient. The press gives them great coverage. Then, the excitement dies down, and the products go nowhere.  So where are the sign language translator wearables? What are the obstacles that have kept them from mass production, and will these new sign language gloves make it out to the real world any time in the near future?

3 Obstacles That Keep Sign Language Translation Devices Stuck in Science Fiction

Sign Language Is Complex

One common misconception is that “sign language” is a single language. In fact, there are many different sign languages around the world. For example, ASL and BSL are quite different to each other, much more so that American English and British English. In fact, the two sign languages only have 33% of signs in common.

But even within a single sign language, there are countless regional and individual variations. As one ASL interpreter commented on a Daily Mail article about the SignAloud gloves:

ASL is a fluid language, always changing by new signs and regional signs. Signing style is based on family history, what type of education a person has had, etc. On any given day I see 5 signs for Wal-Mart. Could it handle those variations? I get it, just not so sure about it.

One potential solution to this problem is to make the device “trainable,” so that individuals can program it to recognize their own particular signs, much like “voice-to-text” programs can be trained to be more accurate. This is the way the wrist sensor translator prototype under development at Texas A & M University works. But this takes time…about 30 minutes to learn just 40 signs, though researchers hope to make it faster.

Additionally, true ASL cannot be translated “word-to-word” into English. There are differences in word order and syntax. In the video demonstration, the student is basically using “signed English,” but that might not come naturally to a deaf person who grew up speaking ASL or BSL.

Sign Language Is More Than Gestures  

But when it comes to sign language translator gloves, there’s another real-world obstacle: Sign language is more than gestures. No matter how accurately a machine can read muscle movements in the hands and arms, it would be difficult to provide a truly accurate translation without also taking facial expressions and body language into account.

As Sign Peach, a deaf commenter in the Daily Mail, explained:

“Sign language is so much more than a few hand movements picked up my a computer & translated into words. Body placement, factual expressions, interaction, cultural influences & level of education……. This is nothing more than an intellectual exercise with no real use in the world of sign language.”

Technology Can Be Cumbersome

Another key challenge: creating a device that is both effective and practical.  While A&M researchers hope to one day shrink their device into something the size of a watch, for now it a cumbersome construct of sensors and wires.

As SignAloud inventor Thomas Pryor told the Daily Mail, this is one of the key advantages of their gloves over other wearable sign language translators:

“Many of the sign language translation devices already out there are not practical for everyday use. Some use video input, while others have sensors that cover the user’s entire arm or body. Our gloves are lightweight, compact and worn on the hands, but ergonomic enough to use as an everyday accessory, similar to hearing aids or contact lenses.'”

So, can these gloves work in the real world? It will be interesting to see where this technology goes, but it’s probably not going to be ready for real-world translation use anytime in the immediate future.

That said, the UNI, a sign language device that uses a tablet for real-time sign language translation, is expected to be released this summer. It faces many of the same obstacles as the SignAloud gloves. It will have a small vocabulary of signs to start with, and it does require the user to carry around a tablet. That said, it relies on cameras, so it is expected to include body and expression recognition, and it was built by members of the deaf community.  According to developer’s website, it will not replace interpreters, but is intended to be “another option that allows the deaf and hard of hearing communities to communicate when interpreters simply aren’t available.”

That’s probably the most we can expect from any sign language translation device in the near future.