Large Language Models Struggle to Evaluate Long AI Translations, Amazon Finds – slator.com

A new study from Amazon has revealed a limitation in using large language models (LLMs) to evaluate AI translation quality: performance drops as input length increases. While LLMs are increasingly used for high-quality sentence-level AI translation evaluation, the study finds that these models become “less reliable when evaluating long-form translation outputs.”  Amazon researchers Tobias Domhan […]

Google Calls for Rethink of Single-Metric AI Translation Evaluation – slator.com

A new study by researchers from Google and Imperial College London challenges a core assumption in AI translation evaluation: that a single metric can capture both semantic accuracy and naturalness of translations. “Single-score summaries do not and cannot give the complete picture of a system’s true performance,” the researchers said. In the latest WMT general […]

Research Pits Traditional Machine Translation Against LLM-Powered AI Translation – slator.com

As large language models (LLMs) continue to transform translation workflows, a new study underscores the ongoing importance of conventional, domain-specific machine translation (MT) models. While recognizing the impact of LLMs on translation processes, the researchers emphasize the need for careful evaluation of workflows to ensure optimal outcomes. Previous research has shown that MT systems often […]