{"id":719,"date":"2019-08-13T14:06:06","date_gmt":"2019-08-13T14:06:06","guid":{"rendered":"https:\/\/www.ata-divisions.org\/AVD\/?p=719"},"modified":"2019-08-13T18:29:38","modified_gmt":"2019-08-13T18:29:38","slug":"automatic-speech-recognition-softwares-will-they-replace-audiovisual-translators-in-the-near-future-part-1","status":"publish","type":"post","link":"https:\/\/www.ata-divisions.org\/AVD\/automatic-speech-recognition-softwares-will-they-replace-audiovisual-translators-in-the-near-future-part-1\/","title":{"rendered":"Automatic Speech Recognition Softwares: Will they replace audiovisual translators in the near future? (Part 1)"},"content":{"rendered":"
\n

By Mara Campbell<\/h4>\n

Let me start with a straight answer to the titular question so that you can continue reading with a light heart: No! Now that we can all breathe easily, I will elaborate.<\/p>\n

You might be familiar with the concept of Speech Recognition Software (SRS) if you use Siri, Alexa, or Google Now. The software \u201clistens\u201d to you, \u201cunderstands\u201d you, and takes action. As translators, many of us have used Dragon Naturally Speaking or the Speech Recognition apps that come with Windows and Apple operative systems, which listen and transcribe what we dictate. These technologies also produce live closed captions with a technique called \u201cRespeaking.\u201d Nowadays, automatic speech recognition (ASR) software transcribes audio or video files automatically, and some even time or spot the transcription to produce closed captions (mainly live, but off-line, too) or subtitles for a show. You might be familiar with Google\u2019s ASR tool which produces closed captions in many YouTube videos. These SRS and ASR tools fall under the umbrella of Artificial Intelligence.<\/p>\n

In 2016, I attended the NAB (National Association of Broadcasters) Show in Las Vegas and the IBC (International Broadcasting Convention) show in Amsterdam, both broadcasting technologies trade fairs. In those events, I met with different companies that develop ASR solutions and decided to try them out to find a tool we could use in our company to cut costs and turnaround times. My company only provides off-line subtitling and captioning services, so I wanted to see how the various software options, developed mostly with live captioning in mind, would respond in this environment.<\/p>\n

These companies work mainly on applications of speech recognition technologies that already exist. For example, Amazon developed software for their Alexa system, which they have already \u201ctaught\u201d to recognize speech and have loaded with a huge knowledge-base and vocabulary. Companies like the ones I tested can use that technology to inform their software development. What mostly sets companies apart is how they apply and adapt these existing technologies, and which user interface they provide.<\/p>\n

I tested three tools, two English and one Dutch. I also am aware of a Swiss tool which I have not yet used; hopefully, in the second part of this article I will be able to present an analysis on that software, too. I prepared 10-minute clips of shows and movies with different challenges, to see how they responded and compared. Since most of these software tools offered multilingual resources, I set up a small sampling of the typical shows and films we work with.<\/p>\n

In English<\/strong>: Brooklyn Nine-Nine<\/em>, a fast-paced police sitcom; Drugs in Hollywood<\/em>, a documentary series about drugs, featuring a scripted narrator and non-scripted dialog delivered by drunk and\/or drugged people; Uncle Grandpa<\/em>, an animation show for children with very fast dialog and multiple characters with cartoonish voices.<\/p>\n

In Latin American Spanish<\/strong> (note that LAS presents a multitude of different accents and variants, with differing vocabulary, grammatical constructions, etc.): El Show de Jaime Maussan<\/em>, a Mexican journalistic show about UFOs and aliens, mostly scripted, but with some interviews; El Puntero<\/em>, an Argentine drama about a political broker who mediates between underclass poverty and politicians; El Se\u00f1or de los Cielos<\/em>, a Mexican show about drug lords and trafficking; Soy Luna<\/em>, an Argentine teenage telenovela<\/em> with many characters\/actors from all over Latin America; Odisea Argentina<\/em>, an un-scripted Argentine political program with three presenters who debate current events and politics.<\/p>\n

In Brazilian Portuguese<\/strong>: O Caminho das Nuvens<\/em>, a dramatic film with many adult and child characters.<\/p>\n

I encountered some difficulties when testing the software, so I could not test all clips on all ASRs. The Dutch tool does not offer Portuguese as a language. One of the English companies does not have a user interface yet so I could only test one clip with them so far (via email), but they tested it on four different speech recognition services (Amazon, IBM, Google, and their proprietary one), and provided an accuracy percentage for each. The second English software supports all languages and has a very good user interface, but offered a limited trial, so I could not process each clip.<\/p>\n

Next is the question of whether the tools provide a text transcription only, or a timed subtitling file. Both English companies only provide text transcription, which is useful only to some extent or for a small part of the job. The Dutch company generates a very well timed transcription, and has a commendable grasp of line and subtitle breaks. The Dutch and one of the English tools showed a sound management of punctuation, although they definitively do not punctuate like real linguists\u2026 but who does, right?<\/p>\n

To date, I have the Brooklyn Nine-Nine<\/em> clip processed by each tool, which allows me to do a good comparison, and then at least one version of each of the other clips, which means I cannot compare results to determine which is more accurate, but I definitely can tell if they are useful for a typical audiovisual translator\u2019s workflow.<\/p>\n

So far, my conclusion is that these tools are years away from becoming useful for audiovisual translators and AVT companies. First of all, the files produced by some of the ASR tools need to be segmented and\/or timed, which could be quite time-consuming. The Dutch software is definitively the most useful tool, because their user interface is very good and easy to use, and they even have a built-in editing tool for timing and text. But the actual transcription requires heavy editing.<\/p>\n

Here there are three examples from the Brooklyn Nine-Nine<\/em> show for comparison. For the English company (\u201cEnglish company 1\u201d) that provided four different transcriptions from different services, I will use the one generated by Amazon, which had an overall accuracy scoring of 91%.<\/p>\n

 <\/p>\n

EXAMPLE 1 <\/strong>(Quotes added by me; line breaks, capital letters, and punctuation \u2014 or lack thereof \u2014 are original):<\/p>\n

English company 1: <\/strong><\/p>\n

\u201cit\u2019s my craft anyways grandson\u2019s coming in they reunite and I throw another case on the old solved it hey micros aren\u2019t you want to see me captain\u201d<\/p>\n

Dutch company: <\/strong><\/p>\n

17
\n00:00:53,120 \u2013> 00:00:59,760
\nIt\u2019s my craft anyways grandsons
\ncoming in they reunite and I threw
\nanother case on the old salted pile.<\/p>\n

18
\n00:01:03,640 \u2013> 00:01:11,840
\nHey Microsoft. You wanted to see me
\ncaptain.<\/p>\n

(Note that in subtitle 2 there are actually two different speakers; in real-life subtitling, we would need to adjust the line break and add a dialog line or split the subtitle in two.)<\/p>\n

English company 2: <\/strong><\/p>\n

\u201cIt\u2019s my craft. Anyways grandson\u2019s coming in. They reunite and I throw another case on the old solved pile. Microsoft. Went to see me Captain.\u201d (Note: this tool identifies different speakers relatively accurately, even marking them as a female or male voice, and, in this case, it did recognize a change of speaker in \u201cWent to see me Captain.\u201d)<\/p>\n

The correct human-made subtitling: <\/strong><\/p>\n

31
\n00:00:53,178 \u2013> 00:00:54,847
\nIt\u2019s my craft.<\/p>\n

32
\n00:00:54,847 \u2013> 00:00:56,598
\nAnyways,
\ngrandson\u2019s coming in.<\/p>\n

33
\n00:00:56,640 \u2013> 00:00:58,100
\nThey reunite,
\nand I throw another case<\/p>\n

34
\n00:00:58,100 \u2013> 00:00:59,810
\non the old \u201csolved it\u201d pile.<\/p>\n

35
\n00:01:03,272 \u2013> 00:01:04,606
\nHey, my croissant.<\/p>\n

36
\n00:01:10,153 \u2013> 00:01:11,613
\nYou wanted to see me,
\nCaptain?<\/p>\n

 <\/p>\n

EXAMPLE 2 <\/strong>(Quotes added by me; line breaks, capital letters, and punctuation\u2014or lack thereof\u2014are original):<\/p>\n

English company 1: <\/strong><\/p>\n

\u201cabsolutely sir I won\u2019t just head it up I will head and shoulders it up I will dive into women rounded just be just be altogether good with it be more articulate when you speak to the children\u201d (Note: \u201cBe more articulate\u2026\u201d is spoken by another character.)<\/p>\n

Dutch company:<\/strong><\/p>\n

22
\n00:01:25,040 \u2013> 00:01:35,760
\nAbsolutely, sir, I won\u2019t just headed
\nup I will head and shoulders it up I
\nwill die women around it just be
\naltogether good with it<\/p>\n

23
\n00:01:35,960 \u2013> 00:01:38,120
\n, the more articulate when you speak
\nto the children.<\/p>\n

English company 2:<\/strong><\/p>\n

\u201cAbsolutely sir. I won\u2019t just headed up I will head and shoulders it up. I will dive in swim around it. Just be all together good with it.<\/p>\n

Be more articulate when you speak to the children.\u201d (The software identified the change in speaker and separated the text in another line.)<\/p>\n

The correct human-made subtitling:<\/strong><\/p>\n

45
\n00:01:24,168 \u2013> 00:01:26,670
\nAbsolutely, sir.<\/p>\n

46
\n00:01:26,670 \u2013> 00:01:28,046
\nI won\u2019t just head it up,<\/p>\n

47
\n00:01:28,046 \u2013> 00:01:29,798
\nI will
\nhead and shoulders it up.<\/p>\n

48
\n00:01:29,798 \u2013> 00:01:32,050
\nI will dive in,
\nswim around it,<\/p>\n

49
\n00:01:32,050 \u2013> 00:01:35,762
\nand just be altogether
\ngood with it.<\/p>\n

50
\n00:01:35,762 \u2013> 00:01:38,390
\nBe more articulate
\nwhen you speak to the children.<\/p>\n

 <\/p>\n

EXAMPLE 3 <\/strong><\/p>\n

English company 1:<\/strong><\/p>\n

\u201cuh mystery missus torino I\u2019m glad you\u2019re here Man present to you Oh my darlings Thank god Found you uh look at those beautiful cheeks\u201d (Note that sentences starting with \u201cOh, my darlings\u201d are spoken by a second speaker.)<\/p>\n

Dutch company: <\/strong><\/p>\n

47
\n00:02:50,640 \u2013> 00:02:55,440
\nMr-Mrs to Reno glad you\u2019re here
\nman present to you properly.<\/p>\n

48
\n00:02:55,640 \u2013> 00:03:01,680
\nMy darlings then God I found you oh
\nlook at those who in full she.<\/p>\n

English company 2:<\/strong><\/p>\n

\u201cThis you\u2019re Mrs. Torino. I\u2019m glad you\u2019re here. May I present to you.<\/p>\n

My darlings. Thank God I found you. Oh look at those beautiful cheeks.\u201d (The software identified the speaker change and separated text into another line.)<\/p>\n

The correct human-made subtitling:<\/strong><\/p>\n

89
\n00:02:50,128 \u2013> 00:02:51,755
\nAh, Mr. and Mrs. Terrino.<\/p>\n

90
\n00:02:51,755 \u2013> 00:02:53,173
\nI\u2019m glad you\u2019re here.<\/p>\n

91
\n00:02:53,173 \u2013> 00:02:54,842
\nMay I present to you\u2026<\/p>\n

92
\n00:02:54,842 \u2013> 00:02:58,637
\nOh, my darlings.
\nThank God I found you.<\/p>\n

93
\n00:02:58,637 \u2013> 00:03:01,598
\nOh, look at
\nthose beautiful cheeks.<\/p>\n

English company 2 consistently presents more accurate transcriptions and includes better punctuation and capital letters. But the Dutch company, although not quite as accurate in the transcription, creates timed subtitles (if you compare the timecodes with the final subtitles, you see that they are very well timed), which might compensate for some of the misunderstandings.<\/p>\n

Regarding the other clips in English that I got transcribed, the results were not very good. The file for Drugs in Hollywood<\/em> was not useful at all, but I actually expected that, being a non-scripted show full of slurred dialog and slang. Even a human transcriber would have trouble tackling that show. And the one for Uncle Grandpa<\/em>, although better, still presented errors probably caused by the cartoonish voices and the outlandish topics covered, including strange names and invented words, so the software had serious problems with it, too. Clearly, this technology is still not perfected to be used on all kinds of material, and it probably performs acceptably on scripted material and single-speaker shows like newscasts, documentaries, lectures, sermons, etc.<\/p>\n

As for the tests I performed on the Spanish clips, the results were very poor. The different accents and variants were too much for the ASR software options to compute, and the transcriptions were utterly unintelligible. Editing would have been more time-consuming than transcribing from scratch. The Portuguese was also quite bad, although better, compared to the Spanish.<\/p>\n

I knew quite well that these tools would provide minimal assistance and extensive post-edition would be needed before resulting ASR files could be used as translation templates or for delivery to client. My next step will be to edit the English transcriptions into final deliverables and compare the time and effort each takes. I will share my findings in part 2 of this article, coming soon in the next issue of Deep Focus<\/em>.<\/p>\n

 <\/p>\n<\/div>\n

\n
Mara Campbell is an ATA-certified translator from Buenos Aires, Argentina, and has been subtitling, translating subtitles and scripts for dubbing for the past 20 years. Mara is currently COO of True Subtitles, the company she founded in 2005, that has clients in three continents. Her work has been seen on Netflix, Hulu, HBO, BBC, Amazon, and more. She teaches courses on subtitling, closed captioning, and Latin American Neutral Spanish, and has spoken in conferences in Argentina, Uruguay, and Germany.<\/h5>\n
\n
Published in Deep Focus<\/em>, Issue 2, March 2019<\/h6>\n","protected":false},"excerpt":{"rendered":"

By Mara Campbell Let me start with a straight answer to the titular question so that you can continue reading with a light heart: No! Now that we can all breathe easily, I will elaborate. You might be familiar with the concept of Speech Recognition Software (SRS) if you use Siri, Alexa, or Google Now. […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[58,12],"tags":[22,20,62,33,30,38],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/posts\/719"}],"collection":[{"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/comments?post=719"}],"version-history":[{"count":1,"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/posts\/719\/revisions"}],"predecessor-version":[{"id":720,"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/posts\/719\/revisions\/720"}],"wp:attachment":[{"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/media?parent=719"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/categories?post=719"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ata-divisions.org\/AVD\/wp-json\/wp\/v2\/tags?post=719"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}