Translation in Your Own Voice

At a presentation in China, Microsoft recently demonstrated an improved machine translation technology that allows for real-time translation in your own voice. Using the system, Microsoft’s Chief Research Officer Rick Rashid was able to give a presentation in Chinese in his own voice, even though he doesn’t speak the language.

How does it work? Prompted by the attention his presentation generated, Rashid wrote a blog post to explain the technology behind the system:

“In my presentation, I showed how we take the text that represents my speech and run it through translation- in this case, turning my English into Chinese in two steps. The first takes my words and finds the Chinese equivalents, and while non-trivial, this is the easy part. The second reorders the words to be appropriate for Chinese, an important step for correct translation between languages.”

To more accurately perform the first step of the process, Microsoft is using a technique called Deep Neural Networks, which it says mimics the patterns of the human brain to make speech recognition more accurate. One caveat: the Deep Neural Networks technology may be better at recognizing words, but it’s still no substitute for the brain of a skilled interpreter. As Rashid wrote:

“We have been able to reduce the word error rate for speech by over 30% compared to previous methods. This means that rather than having one word in 4 or 5 incorrect, now the error rate is one word in 7 or 8…Of course, there are still likely to be errors in both the English text and the translation into Chinese, and the results can sometimes be humorous. Still, the technology has developed to be quite useful.”

While this technology is amazing and will definitely have its uses in the future, I doubt it will replace the knowledge and understanding that a trained translator brings to the job any time soon. What do you think?

Microsoft to Translate Windows 7 into 10 African Languages

Microsoft has joined the cause of linguistic diversity. The software giant just announced that it will be releasing its new product, Windows 7, in 10 different African languages by 2011.

The software will be available in languages including Sesotho sa Leboa, Setswana, isiXhosa, isiZulu, Afrikaans, Hausa, Igbo, Yoruba, kiSwahili and Amharic.

Previously, according to an article in The Industry Standard, important technology was mostly available in English and French, to the point that Africans who could not read and write in one of these two languages are considered illiterate, even though they may be quite capable of reading and writing in their native tongues.

The lack of inclusion has also encouraged software piracy, as legal software that supports these languages is not available. Microsoft mentioned “fighting piracy” as one reason for expanding Windows 7 into different languages. However, it may be too late for that, as the Industry Standard notes that pirated software is so readily available in Africa that the native language support may simply become another selling point for pirates.

Still, nobody should be considered illiterate if they can read and write in the language that they grew up speaking. Microsoft’s introduction of local language support for African languages is a big step forward. As  Francis Hook, manager at IDC East Africa, states in the Industry Standard article:

“The localization will most certainly increase content from Africa by allowing expression in local languages, it will help with the survival and continued relevant of African languages amidst globalization.”

In addition to benefiting African computer users, this move will likely also benefit Microsoft, even if it does little to curb piracy. As Hezron Mogambi, a linguistics lecturer at the University of Nairobi, explains:

“Language has the power to draw more people into a product and internet use more than advertising can do. People want to see and feel a product that represents their community and settings.”

Microsoft Releases Online Hmong Translator

In February, Microsoft announced that they had added Hmong to the list of languages translated by their search engine, Bing.  Last week, Microsoft announced the release of a new, free online Hmong/English translator for smart phones, chats and websites. What’s with the focus on Hmong? It’s actually part of an initiative to help preserve the language within the Hmong community in the US.

The Hmong currently living in the US are mainly refugees from the Indochina Wars, as many Hmong sided with America in both Vietnam and in the “Secret War” in Laos. In the US, they are safe from the persecution they faced in their homeland, but their culture is at risk as their children often neglect Hmong in favor of speaking only English.

As Microsoft program manager Will Lewis explained to Business Week,

“All these years, the language has been preserved, despite efforts to eradicate it. Now, the irony is that in the United States, a country where they’re free to speak it, the thing that never happened in Hmong history is happening; some children are not learning Hmong.”

So, Hmong community leaders decided to partner with Microsoft to bring the language to the place kids spend most of their time these days: the Internet. Though the online translator will also help older Hmong refugees who can’t speak English, the big hope is that it will also help keep the language relevant for the younger generation. As Fresno State University outreach counselor Phong Yang told a local news affiliate, “Without language, a culture will disappear.”

Also, the technology used to create the translator holds the promise of being able to do the same thing for the thousands of other threatened minority languages scattered across the globe. Using dictionary entries and documents in both the original language and English, the program is able to “learn” what words are likely to mean by using context clues.

Learning the language of their grandparents along with English allows kids to take pride in their heritage. Need proof? The Business Week article quotes US-born Joshua Lor. As a young boy, Lor told his mother that he didn’t want to be Hmong any more. Lor said that learning the language was key to changing that perspective:

“My grandpa told me stories about the Hmong, about how he served in the war, and how they moved from Laos to Thailand to America. The language opened my eyes to the history of Hmong culture. It’s exciting that the translator can help kids do that.”

Hmong is actually a macrolanguage with numerous closely-related dialects. Currently, the translator only works with Hmong White, though the team is working on one for Hmong Green, the other major dialect spoken by Hmong in the United States.

Photo Credit: Attribution Some rights reserved by Arian Zwegers

Translation Technology Replicates Your Voice

Microsoft has been experimenting with some very interesting translation technology. For example, the company’s chief research and strategy officer, Craig Mundie, recently appeared on a computer screen in Beijing, speaking perfectly fluent Mandarin. Nothing unusual about that, except for the fact that the real Craig Mundie doesn’t speak Mandarin at all.

Instead, Microsoft has created a virtual clone of him that speak whatever language they want it to. Plus, as he explained to PCWorld, his avatar’s similarity to the real Craig Mundie is more than skin deep:

“What was spoken in Mandarin today I never recorded. But it is my voice. They have a computer model of my voice box.”  Read more

Microsoft App Adds Translation Subtitles

Microsoft just released an updated version of its translation app for Windows phones, and it comes equipped with nifty new features.  As described on the Bing blog, the augmented reality option sounds especially cool:

“With the Translator App for Windows Phone you can now translate printed language by simply pointing the camera. From street signs and posters to transit schedules and restaurant menus translating is now a snap. Well, easier than a snap – all you do is point and scan. Think of this as automatic subtitles for everyday life.”

Think of how much confusion that could save you the next time you’re somewhere where nobody speaks your language.

Of course, as with all machine translation, there’s always the risk that you’ll get back something awkward, misleading, or flat-out wrong, but it’s definitely better than having to bumble along with no help at all.

Another cool feature is the ability to use this app without a data connection. This is an exceptionally important capability because not only is data often unavailable in remote regions, even if it is available you’re likely to pay through the nose for using it.

The app also has voice recognition, so you can speak into it in your language and then have your phone play back the translated version of what you just said.

Travel is a great learning experience, and hopefully most people will use this app to teach themselves at least some key words and phrases in the language of the country they are traveling in, instead of using it as a crutch.  Either way, though, if it helps motivate people to get out of their comfort zones and experience another culture, it’s a good thing.

As Vikram Dendi, the Director of Product Management for Microsoft/Bing Translator, put it in a post on the Microsoft company blog:

“If we are able to provide you that little bit of extra confidence that makes the difference between going somewhere and not – then we would have succeeded.”

Microsoft Translation System Lets You Speak 26 Languages

Attempts at creating a “universal translator” are getting ever-closer to the science fiction that inspired them, as Microsoft’s latest translation demonstration makes clear. Earlier this month, the tech giant took the opportunity provided by TechFest 2012 to show off its newest invention: translation software that lets you “speak” any one of 26 different languages, all in your own voice.

Although it goes by the unassuming name of Monolingual TTS, there’s nothing unassuming about its capabilities. After about an hour of “training,” the software learns your voice. Once that’s done, it can translate anything you say into one of the 26 languages it knows, using a digitally animated version of your own head and a digitally simulated version of your voice. The result is about two parts awesome, one part creepy, as you can see in the video demonstration below:


Get Microsoft Silverlight


Still, it’s undeniably useful. For example, if you’re travelling and you need to communicate with someone in another country, all you have to do is speak what you want to say, and the software will repeat it for you in the correct language.

In an article on, Microsoft researcher Frank Soong said that other applications could include translating GPS directions and helping language learners work on their pronunciation.

Why read back the translations in your own voice? University of Southern California professor Shrikant Narayanan told Technology Review that words are just one part of how we communicate through speech:

“The word is just one part of what a person is saying. Preserving voice, preserving intonation, those things matter, and this project clearly knows that. Our systems need to capture the expression a person is trying to convey, who they are, and how they’re saying it.”

Of course, this system doesn’t do that entirely. It can capture someone’s pitch and intonation fairly well, with just a slight digital edge. But it doesn’t seem like it would be able to capture the emotions behind the words, yet. The truth is, there’s usually more to translating than meets the ear, so to speak. Machine translation can be helpful if there’s no other way to communicate, but the best way to preserve the meaning that you’re trying to get across is still to use a human translator who is familiar with both cultures.