A note from the DLD Administrator:
Welcome to the DLD’s first blog post of 2023. Let me start by wishing you all a very Happy New Year, on behalf of the entire DLD Leadership Council! May your year be filled with good health, fulfilling relationships and professional success. We hope that the DLD will be part of your 2023 journey.
In December you had a chance to get to know our Distinguished Speaker Tia Nutters. You can now enjoy the first part of a two-part series about her presentations at ATA’s 63rd Conference in Los Angeles last October.
–Irene Stoel
ATA63 Recap: MACHINE TRANSLATION AND LITERATURE: A marriage made in hell?
A presentation by DLD Distinguished Speaker, Tia Nutters
By Leo van Zanten
This year’s ATA Annual Conference took place at the Westin Bonaventure in Downtown Los Angeles. The event counted over 1,100 attendees who could choose from 168 sessions, featuring a variety of languages and topics. Two of those were presented by the Distinguished Speaker for the Dutch Language Division, Tia Nutters. For a personal introduction I kindly refer to the earlier blog post about the interview with Tia. She gave two very interesting presentations: one on Thursday October 13 at 2 pm and another one on Friday October 14 at 11 am.
Below you’ll find my synopsis of her first presentation about Machine Translation and Literature: A marriage made in hell?
We never know how many people will attend each session during the conference, but I was happy to see that this one had a great turnout. The room was filled with more than 60 attendees, including several translators who work in literary translation. This was an interactive session covering a combination of translation theory and practice.
Tia presented a research project that she participated in as a volunteer researcher, together with Antonio Toral and Andreas van Cranenburgh. The total project duration was 2.5 years, in which they built a Translation Machine (TM) for literature. Tia’s role was to assess the quality of the machine translations.
The motivation for this project came from a previous project in which the researchers had built a Translation Machine for literary translation from English into Catalan, and where their machine did better than a generic translation machine. If it worked well for this language combination, would it also do a good job for English into Dutch?
So, what is the advantage of using a TM for the translation from English into Dutch? And how can we make it perform better? Do we need to feed the machine with more data? Or maybe feed it with larger chunks of text (entire paragraphs)?
When you build a Translation Machine, it’s usually based on bilingual text corpora, where you feed sentences into a TM. For example, Google Translate™ uses text in many languages independently, and DeepL™ uses originals plus the translations to feed the machine.
The latter two are examples of Neural Translation Machines, where the machine has a self-learning capacity. We can train a TM with segments (usually sentences) that can be either monolingual, bilingual or originals in the target language.
In this study, the researchers built 5 different versions of a literary translation machine. The first version was fed with a large number of fiction texts, and each next version would get fed with more texts. In the fifth and last version they fed the TM with entire passages of text, such as paragraphs (approx. 800 characters). You can imagine that it takes days to train a machine, and this requires considerable computing resources. So don’t try this at home on your laptop.
In the evaluation stage of this project the Quality Assurance, or QA, was done automatically with metrics like BLEU (BiLingual Evaluation Understudy) and COMET (a neural framework for training multilingual machine translation evaluation models). These methods provide a score for comparing machine translation with human translation.
This sounds all very mathematical, and is great for the research, but we’re talking about literature, and literature is meant to be enjoyed by the reader! That is why the researchers included a survey to add the readers’ perspective. Also, several professional ATA translators were asked to analyze the text. Some of them were present in the room to comment on their experience.
After this thorough overview of the research objectives and evaluation, everyone in the room was waiting in suspense for the outcome.
Does this literary translation machine perform better than a generic one for English into Dutch?
The answer is yes!
Compared to DeepL, this TM produced on average 4% (COMET) better results. The results were even slightly better for the genre fiction (note: Non-fiction was excluded from the feed). However, after confirming through human evaluation, the advantage was lost for highly literary works. Too bad for those translating Shakespeare or Chaucer.
Does the machine version with larger building blocks perform better than the version based on isolated sentences?
Again, the answer is yes, but only slightly better.
Using larger blocks works better to connect the pieces, but this means it’s also less precise, and more often leads to mistranslations.
The researchers found several obstacles to the use of machine translation (MT) in literary translation.
In general, Translation Machines are gradually getting better, and in this study the researchers wanted their literary TM to be better than a generic one.
However, with an average score between 3.5 out of 5, the quality of the literary machine translations does not suffice.
Another obstacle is that Literary translators are unfamiliar with the use of MT. Currently only 18% of Dutch literary translators use language technology, consisting mainly of translation memories and term bases (footnote: See J. Daems (2021). https://www.tijdschrift-filter.nl/webfilter/dossier/literair-vertalen-en-technologie/januari-2021/wat-denken-literaire-vertalers-echt-over-technologie/ )
Lastly, there is the matter of copyright. Literary translators own their work, which means that the translation machine cannot be made publicly available, as all its content is protected by copyright.
One of the potential applications could be that a publisher asks the specific translator to provide material and use this to develop a translation machine that will only be used internally, for example for one specific author. Can this benefit the translator and the publisher?
At the end of the session there was time for more interaction with the audience to discuss some ideas and answer a few brief questions.
Q: Has machine translation ever been used for literary translations in real-life?
A: Yes, in China a publisher uses this for English into Chinese.
Q: Can machine translation take style into consideration?
A: Yes, it can to a certain extent.
Q: What happens to the use of machine translation when a publisher wants to adapt the translation of a novel to another type of target audience?
A: You will need a human translator for this!
This presentation answered some questions, but also generated more food for thought on a very dynamic topic in our Translation & Interpretation world.
We thank Tia Nutters for her passionate contribution to this conference and our profession, and for engaging with the ATA community during her presentations and throughout the conference.
