Last week, Google released a new version of Google Translate that uses “Deep Learning” to reduce translation errors. The company claims the new process results in translations that are almost as good as a human translator.
But is it the real deal? Are human translators about to be replaced by machines? Can you now “just use Google Translate” for your business? Let’s look at what makes the new and improved Google Translate so groundbreaking, and whether or not it’s actually an acceptable substitute for a human translator.
How Is the New Google Translate Different?
The new-and-improved Google Translate takes a different approach from the current version. Let’s get to know our new robot overlords, shall we?
Previously, Google Translate worked by analysing texts one word or phrase at a time. However, the new Google Translate breaks them up into sentences to better determine their meanings. It’s also capable of understanding and analysing the relationships between words, to determine which possible translation is more likely to be correct.
Most importantly, the new Google Translate does all of this using a “deep neural network” of processors set up to mimic the human brain. This network is even capable of training itself. In fact, it “learns” better if left to its own devices, without human programmers mucking things up.
How Does It Compare to Humans, Really?
That’s all well and good, but here’s the real question: Is Google Translate as good as a human translator?
You could be easily forgiven for thinking that’s the case. After all, the headlines are saying just that.
But the whole truth rarely fits in a headline. And in this case, the truth is that the new Google Translate is a major improvement on the old … but the reports of it being “almost as good” as a human are greatly exaggerated.
Google researchers claim that by using deep neural networks, they’ve been able to reduce error rates by 55-85%.
But how accurate were those tests?
To measure the quality of the translations, Google used bilingual human “raters.” The raters rated translations from three sources: old Google Translate, new Google Translate, and human translators. The passages used in the tests came from Wikipedia.
Google Translate: 3 Possible Problems With the Tests
Over at eMpTy Pages, Kirti Vashee notes that this approach is not without issues:
“Humans are unlikely to rate 3 different translations of the same thing on a scale of 0 to 6 (crap to perfect) accurately and objectively. Ask them to do it 500 times and they are quite likely to give you pretty strange and illogical results.”
He also noted that the since the passages came from Wikipedia, the sentences were overwhelmingly simple and well-crafted. So, they were perhaps not the best measure of how the technology will perform in the real world.
Meanwhile, in a comment on the same post, Shai Navé made this excellent observation:
A “human translator” in and of itself is not a benchmark for the very simple reason that not all humans, and not even all those who claim to be translators, are competent at translation and therefore the quality they produce covers the entire quality spectrum … The fact some piece of text was translated by some random human doesn’t mean that particular piece of translation is automatically considered top-notch (or even fit for purpose), nor a representative demonstration of translation skills.
It’s easy enough to compare MT to low quality/mediocre translation, which is generally the quality most of those dealing with MT have access to, and paint a favorable picture.
The New Google Translate: Other Opinions
As you can see, it’s not so easy to answer the question of whether or not Google Translate is “as good as a human.” Consider these 3 variables:
- The difficulty level of the text itself
- What the translation is for
- The skill of the translator competing against Google Translate
With that in mind, Quartz tested the new Google Translate Chinese-to-English translation in a variety of situations. Google Translate churned out acceptable translations for easy, conversational chats between friends.
But when they used academic writing with more complex sentences, Google Translate stuttered and stammered. And their attempt to translate a poem was just sad.
Meanwhile, Slator.com interviewed 13 experts in translation and machine translation to see what they thought of Google’s announcement. The consensus? This is an exciting announcement that illustrates the potential of neural machine translation. That said, most experts believed the results had been somewhat overstated and overhyped, by the media if not by the Google researchers themselves.
NOW Can I Just Use Google Translate?
Still, after comparing the “old” translations with the “new” translations in Google’s tests, it’s clear that they have made significant improvements. Note, however, that the “new” Google Translate is only currently available for Chinese-to-English translations.
But I’m sure it will roll out to other languages soon enough. So, is Google Translate now good enough that you can use it?
In some cases, yes. If you only need to get the “gist” of a passage in another language, fine. If a translation error wouldn’t cost you time, money or cause legal issues, fine. On holiday with no interpreter? Fine.
But in business communications, the devil is often in the details. And the details are what Google Translate is likely to get wrong.
In a blog post, even the team that built the system acknowledges that it’s not perfect:
“Machine translation is by no means solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page. There is still a lot of work we can do to serve our users better.”
So in a situation where errors matter, you’d still want to use a human. In a situation where the details matter, you’d still want to use a human. Google Translate may be able to approximate the meaning of your content in another language. But it might get it wrong. And it can’t translate the tone, the rhythm or the emotional impact.
In other words, what Rich wrote 3 years ago in Can’t I Just Use Google Translate? still holds true today:
The process of taking your message, breaking it down to its constitute parts, understanding the external influences in region, applying the relevant legislation and putting it all back together in another language remains a complex one. Don’t rely on a free service to do this (you wouldn’t for almost anything else in your business) and if your LSP is using it – talk to us.