Researched and Compiled by: ATA Language Technology Division Leadership Council – Percy Balemans, Viktoryia Baum, Andy Benzo, Claudio Cambon, Blaise Hylak, Bridget Hylak, Karen Leube, Bob Makovei, Daniel Sebesta, Robert Sette, 7 October 2022 – 23 May 2023
Introduction
Put 100 trained linguists in a room to discuss MT and MTPE, and you will likely hear 200 opinions on the good, the bad and the ugly. At this point, things are changing so quickly that you may even hear a whole lot more than you want to–generative AI’s influence on MT, GPT as a tool behind and/or with MT, and the ever-evolving concept of “the loop” involved in any given translation or localization project, and where (or even if) you fit in.
After all opinions, accolades and frustrations have been aired by the credentialed linguists in that imaginary room, bluntly ask them (and yourself, since you are reading this), “So, all theory aside, would you fly in a newly assembled airplane built using only the instructions provided in a machine-translated and post-edited assembly manual?”
What answers would you expect to hear? The majority would likely respond, “No way!” or perhaps might offer the more informed answer, “It depends on who did the PE… and with what tools…!”
If you were to ask 100 individuals with no language industry background that very same question, the answers would likely be strikingly different. For example, one might expect to hear, “Sure, why not?” coming from an unsuspecting domestic traveler, or even a CFO or two charged with cutting translation budget costs saying, “Ya’ gotta’ die of something, right…?”
Why such a disparity of opinions, and who to believe? What’s happening with all this “new” technology in our industry, and how do we react, adapt, adopt or even opine?
First and foremost, Machine Translation (MT) is far from new, and in fact, is older than most active linguists. Forget ChatGPT—for some of us, recognizing the power and the legacy of MT is the first morsel we need to bite, chew and swallow. To understand the progress of Machine Translation over the years, let’s look at a watered-down history.
“Education is key, but it’s haphazard, […] and clients and MT post-editors with minimal language background simply don’t know what they don’t know.”
MT was introduced in the 1950s with Rule-Based Machine Translation (RBMT). This rigid form of MT relied on rules developed by human language experts. Statistical Machine Translation (SMT) came into its infancy in the late 80s as computational power increased. SMT creates statistical models by analyzing large sets of bilingual data to create the desired MT output. SMT would learn from each data set and was an early example of machine learning. For many years SMT remained at the forefront of industry standards regarding MT.
However, in 2016, Google announced the development of their Neural Machine Translation (NMT) system. Since then, the development of NMT exploded and became the MT model of choice across the industry. SMT broke the input sentence into short strings of words, translated them independently and tried to reorder the pieces. NMT, on the other hand, attempts to build a representation of the meaning of the whole input and uses that to produce the output. This helps it produce fluent output that works well between languages with different grammatical structures. In summary? It was a game changer!
Most recently AMT, or Adaptive Machine Translation, joined the family of MT offspring being used by linguists, adding another layer of accuracy to MT output. As a live linguist translates and even uses standard MT engines in crafting their output, AMT learns from the linguist’s work and can provide better suggestions in the future. Like any good partnership, AMT holding hands with a linguist only gets better as the two work together and grow their relationship over time.
Concerns about MT and MTPE
According to an April 2020 study conducted by CSA Research, only 37% of freelance translators believed raw MT output quality was “good.” This opinion was likely influenced by the fact that 81% of those same linguists noted that raw MT output varied significantly from client to client, making even the prospect of working with it unpredictably painstaking.
However, by October 2022, another comprehensive industry study released by Smartcat garnered a response where 55% of the linguists said their experience with AI translation (which includes MT) was “good, very good, or excellent.”
A notable caveat is that linguists who work with lower-resource languages or rare language combinations readily assert that for many of their purposes, MT is still not ready for prime time, and this is simply because robust, reliable engines in many languages and combinations are still being trained. Professional translators, then—those who know and understand the nuances of a “good translation” best—still have valid concerns and varying opinions about what the use of MT and post-editing means for them and their clients, especially with MT being increasingly implemented across industries. These findings emphasize the need for more education and training in order to dispel any myths and to explain the advantages of MT, MTPE and other emerging technologies..
In particular, large LSPs and language technology platforms that include a freelancer “marketplace” seem to be leading the charge in this regard, or at least the conversation. Likewise, then, a “new standard” for correctly handling MT and producing proper MTPE output (and eventually, other AI-driven output) is vital to keeping talented linguists relevant and gainfully employed.
The increase in the use of MT has been driven by a number of factors. There are just not enough human translators to handle the astounding amount of textual data that is being produced every day, driven by increasing globalization and an avalanche of information of all types being published online. That’s good news! MT has become an indispensable tool to help expedite the delivery of translated content and has gained increasing acceptance thanks to marked advancements made in the output quality of NMT, AMT and AI.
“We must train ourselves to think, talk, and educate others about MT, not just in conjunction with PE, but in new ways that include MT engine training, asset management, language learning, artificial intelligence, and many other possibilities yet to be discovered/proven.”
– Bridget Hylak, LTD Administrator
It’s important to keep in mind that MT does a whole lot more than PE. Who knew last year that the cryptic acronym “GPT” would be on everyone’s tongue, or how useful it could become when properly used for many personal and professional purposes? The takeaway is that, even as we specifically discuss MTPE today, future flexibility and continued career development are part of the new normal.
Should human linguists cower in fear? The reviews are mixed. Some say pack it up and go home, but the overwhelming opinion as of this final writing (May 2023) is, of course, “No!” In fact, during a recent (April 2023) TAUS online seminar, leading language industry panelists addressed the burning question, “Is generative AI making MT obsolete?” to which the unanimous panel opinion was a resounding “No!”
At the same time, MT and other AI-based tools are already changing the industry as we know it. “The world most of us trained for is gone,” said language industry veteran and researcher Jonathan Downie at ATA’s May 2023 Virtual Conference “Translating and Interpreting the Future.” Downie paused with a degree of sympathy, then quickly added, “Our future depends on how we react to the present.”
The hot concept of “HITL” (Human in the Loop) linguistic efforts, a possible misnomer for what should be dubbed “EITL” (Expert in the Loop) or even “MOTL” (Master of the Loop) (Claudio Cambon, LTD internal communication, May 2023) is still considered the gold standard by industry experts who know the promise and the challenge of working with these tools. Change, progress and adaptation are essential in every industry, with the impact of tech being particularly acute as trained professionals from surgeons to auto repair mechanics must become proficient in new ways of accomplishing their duties.
Which brings us to MTPE as a service. Where’s a motivated linguist to begin? Currently, minimal to no linguistic training is required to become “certified” by certain organizations (RWS, Transperfect, etc.) as an MT post-editor. The result is an increasing dependence upon, and/or false trust in, non-linguist MT post-editors whose skills are derived mainly from on-the-job training (i.e., “sink or swim”), confidence in their own sense of language, and (often uninformed) reliance on the assistance of improved technology shortcuts. This chaotic landscape was the driving impetus behind this research, and will hopefully provide our members with some guidance, ideas and foresight.
Pricing must also be thoroughly and thoughtfully considered. While clients assume that technology advancements make the work of linguists faster and therefore cheaper, that is not always the case. The addition of well-implemented technology and the rigors of proper training (language, tech and “on the bridge”) increase not only efficiency and consistency, but cost as well. Client demands, knowledge and budgets vary greatly. Education is key, but it’s haphazard, such that the biggest risk becomes the fact that clients and MT post-editors with minimal language background simply don’t know what they don’t know.
ATA recognizes that the facts mentioned above represent a challenge as well as an opportunity. The demand for linguists who specialize is increasing. At the same time, linguists who differentiate themselves by adopting new tools and embracing MT, MTPE, AI and more can offer additional services to clients who request them.
To that end, and recognizing that MTPE is already an indispensable skill in a flourishing industry, the ATA Language Technology Division has researched and compiled the following chart of MTPE guidelines from various industry leaders such as RWS Trados, Phrase, memoQ, Smartling, ISO and TAUS. This list simply reviews and compares these guidelines while offering no opinion on their correctness, longevity or worth. In fact, in a global tech environment experiencing so much flux and influencing so many career paths, it’s almost impossible to predict what new tools, rules or technologies may pop up on any given day!
ATA is uniquely positioned to establish its own language industry–based guidelines that become vital to MT post-editors and beyond, and even to lead the way toward a new standard, such as a Training Verification (“TV”) that will respect language nuance and incorporate language, technology and business requirements. This is something that ATA is taking very seriously by conducting research, discussing the topic at board, committee and division meetings, and scrutinizing training standards for linguists using MTPE and other technologies. Interested association members should keep an eye out for further information, or even better, get involved as an ATA volunteer to help influence, or even moderate, the change! Contact ATA HQ or the Language Technology Division for more information.
Disclaimer:
The guidelines presented in this article were taken directly from each publicly accessible resource between October 2022 and April 2023, and are presented for discourse, convenience and potential action. Updated “opt-in” references may exist for additional consideration. Keep in mind that some of these resources may represent entities with varying goals or interests (e.g., selling a course, data mining, or even data brokering). Various entities, companies and opinion-makers are also releasing additional, though unofficial, opinions and guidelines on the topic, urging all of us to proceed with caution.
For its part, ATA asserts in its 2018 “Position Paper on Machine Translation” that, “For reliable and secure translation, machine translation should not be used without the ongoing involvement of professional translators.” While the ATA’s legal status as a nonprofit organization prevents it from commenting on pricing matters and certain other details, it does assert that MT choices “are designed to allow for generic, ad-hoc communication between users of different languages on the internet where neither strict accuracy nor confidentiality is of central importance.” It’s important to note, however, that these assertions were published in 2018—light years ago by modern tech standards. Our organization continues to review and research the landscape and may at some future point provide an updated guidance. This discourse may be considered one step in that direction.
The ATA LTD did not write, nor do we intend to express, any opinion on the information presented here. Moreover, linguists should also be mindful that the definitive “guidelines” that influence output on any project, whether MTPE or otherwise, are ultimately determined by their clients. (But perhaps they shouldn’t be…).
Overview-of-MTPE-Guidelines
(click to enlarge)
References:
- RWS. (n.d.). About Machine Translation Post-Editing. RWS Moravia. https://moravia.rws.com/hubfs/M_Files/RWS_Moravia_Machine_Translation_Post-editing_Services.pdf
- Dalibor, P. (2022, September 22). Best Practices for Machine Translation Post-Editing. Phrase. https://phrase.com/blog/posts/machine-translation-post-editing-best-practices/
- Smartling. (n.d.). Machine Translation Post Editing (MTPE). Smartling. https://www.smartling.com/resources/101/a-hybrid-translation-approach-machine-translation-post-editing-mtpe/
- Lelner, Z. (2022, February 15). Machine Translation vs. Machine Translation Post-Editing: Which One to Use and When? MemoQ Blog. https://blog.memoq.com/machine-translation-vs.-machine-translation-post-editing-which-one-to-use-and-when
- Loney, N. (2020, December 30). Machine translation adoption is on the rise – what does this mean for the freelance translator? Trados Blog. https://www.trados.com/blog/machine-translation-adoption-is-on-the-rise-what-does-this-mean-for-the-freelance-translator.html
- Pielmeier, H., O’Mara, P. D. (2020, April 22). The State of the Linguist Supply Chain. CSA Research. https://cdn2.hubspot.net/hubfs/4041721/Newsletter/The%20State%20of%20the%20Linguist%20Supply%20Chain%202020.pdf
- ATA. (2018, August 14). ATA Position Paper on Machine Translation: A Clear Approach to a Complex Topic. American Translators Association (ATA). https://www.atanet.org/advocacy-outreach/ata-position-paper-machine-translation-a-clear-approach-to-a-complex-topic/
- TAUS. (2020, September). MT Post-Editing Guidelines. TAUS Business Intelligence Bulletin. https://info.taus.net/mt-post-editing-guidelines
- ISO. (2017). ISO 18587:2017 Translation Services — Post-Editing of Machine Translation Output — Requirements. Inicio. https://dgn.isolutions.iso.org/obp/ui#!iso:std:iso:18587:ed-1:v1:en
- Lilt. (n.d.). What is Interactive, Adaptive MT? Lilt Support. https://support.lilt.com/kb/what-is-interactive-adaptive-mt
- Hylak, B. (2023, March/April). MTPE? nQsf! (MT Post-Editing? Not QUITE So Fast!). ATA Chronicle, 52(2).
- TAUS. (2023, April 19). Generative AI & Translation [Webinar]. TAUS. https://www.taus.net/resources/webinar/generative-ai-and-translation
- Businesswire. (2022, October 13). Groundbreaking Translation Industry Study Reveals Major Gaps in Linguistic Supply Chain Efficiency. https://www.businesswire.com/news/home/20221013005471/en/Groundbreaking-Translation-Industry-Study-from-Smartcat-Reveals-Major-Gaps-in-Linguistic-Supply-Chain-Efficiency
- Dalibor, P. (2022, September 22). Best Practices for Machine Translation Post-Editing. Phrase. https://phrase.com/blog/posts/machine-translation-post-editing-best-practices