Google Calls for Rethink of Single-Metric AI Translation Evaluation – slator.com

A new study by researchers from Google and Imperial College London challenges a core assumption in AI translation evaluation: that a single metric can capture both semantic accuracy and naturalness of translations. “Single-score summaries do not and cannot give the complete picture of a system’s true performance,” the researchers said. In the latest WMT general […]

Unbabel Tackles Metric Bias in AI Translation – slator.com

In a March 11, 2025 paper, Unbabel introduced MINTADJUST, a method for more accurate and reliable machine translation (MT) evaluation.  MINTADJUST addresses metric interference (MINT), a phenomenon where using the same or related metrics for both model optimization and evaluation leads to over-optimistic performance estimates. The researchers identified two scenarios where MINT commonly occurs and […]

Google Expands Low-Resource AI Translation with SMOL Dataset – slator.com

On February 17, 2025, Google released SMOL (Set of Maximal Overall Leverage), a dataset translated by professional translators aimed at improving machine translation (MT) for 115 low-resource languages (LRLs). SMOL consists of two components: SMOLSENT, a collection of 863 English sentences translated into 81 languages, and SMOLDOC, a dataset of 584 English documents translated into […]