At SlatorCon London 2024, a panel moderated by Slator’s Managing Director Florian Faes brought together experts from AppTek, Deluxe, and Brazil’s Public Broadcasting Company (EBC, Empresa Brasil de Comunicação) to discuss ways in which language and localization AI technology can support new use cases, such as EBC’s 24/7 online international channel and similar broadcasting initiatives.
The discussion began with Heitor Luiz Gomes de Castro, Lead Global Localization Architect at EBC, who described the company’s current work on its international channel. In partnership with AppTek, EBC plans on delivering AI-dubbed content using a staged approach.
Discussing AI implementation, EBC’s Gomes de Castro told attendees that the company is conducting some automated dubbing tests using content that includes documentaries and that the results with AppTek’s technology have been very good.
Gomes de Castro further explained that due to low budgets and restricted resources, human dubbing was not a viable option for the vast volumes EBC expects to dub, and that the AI dubbing option was planned for from the start. He also spoke about relying on vendors like AppTek to understand the technologies and ensure the ultimate success of the dubbed content.
Regarding how AI dubbing has been increasing in popularity, Parnia Bahar, Lead Scientist for Speech Translation at AppTek, explained that fully automated AI dubbing (or automatic revoicing) involves more than text-to-speech translation and the steps in between.
Enabling the transfer of a speech to another language in the speaker’s voice and style, and with automated lip syncing, Bahar explained, “is a very complex task … it combines a number of challenges that all need to be addressed at the same time, which makes dubbing even harder than the sum of its parts.”
AppTek’s key breakthrough in the field, continued Bahar, is the advancement of deep learning methods, including the transformer architecture for machine translation (MT). Bahar observed that clients are looking for good emotional and adaptive text-to-speech (TTS) technology that ensures the integrity of the viewer’s experience, a process that at this time requires a human-in-the-loop to ensure good quality.
Regarding client requirements for AI dubbing, Bahar offered that AppTek looks first at the level of localization expected, the type of content, and the audiences for that content. Technology-wise, the main questions revolve around training data for large language models (LLMs), whether the client has data they can provide, and whether a pivot language is required, among others.
Bahar also explained that natural, human-like intonation is a frequent request from dubbing clients. She added that although major progress has been made in this area, achieving it remains a challenge that AppTek continues to work on solving.
Speaking about applications at Deluxe and how the partnership with AppTek comes into focus for AI automation, Rich Welsh, Senior Vice President of Innovation, explained that the main issue on everyone’s mind at Deluxe is efficiency in the face of very high expectations of quality (and deadlines).
Deluxe is looking at AppTek to help with AI for what Welsh called “off-the-shelf efficiencies” as well as customized models for specific customers, with data protection and other assurances in place. Welsh emphasized data security as a major concern for Deluxe and its clients, highlighting the vulnerabilities of translation engines to attacks.
Specifically, Welsh commented on how Deluxe looks at the security layer and translation tool protection. “It’s things like that we’re thinking about that go beyond just the mechanics of translation and quality,” he said.