In a June 22, 2024 paper, Zhaopeng Feng, Ruizhe Chen, Zijie Meng, and Zuozhu Liu from Zhejiang University, along with Yan Zhang from Tencent, presented “Ladder,” a model-agnostic and cost-effective tool designed to boost the performance of large language models (LLMs) in machine translation (MT). 

Unlike conventional methods that require extensive computing resources, significant data, and human annotations, Ladder leverages so-called “pseudo-refinement triplets” created from LLMs, reducing the need for additional human effort.

Pseudo-refinement triplets are a source sentence, an intermediate translation generated by an LLM, and a reference translation (the refined translation).

This approach enables the creation of training data for MT refinement in an automated manner and “without extra labor costs.”

The researchers noted that these triplets share a format similar to the automatic post-editing (APE) triplets, which typically consist of a source sentence, a translation with errors, and post-edits. However, the APE annotation process requires substantial human resources for tasks such as evaluation, error identification, and post-editing.

The triplets are then categorized into three hierarchies: easy, medium, and hard. Easy translations have significant differences from the reference, allowing for more improvements. On the other hand, hard translations are almost perfect, making them difficult to refine, and medium translations fall in between these two categories. This categorization is based on the COMET scores assigned to each triplet.

A hierarchical fine-tuning follows, which involves training the Ladder model in a progressive manner, starting with easy examples and gradually moving to medium and hard examples. This hierarchical fine-tuning approach allows the model to learn and improve its refining performance incrementally.

“Instead of directly fine-tuning a translation-target LLM, we train an LLM to refine translations using refinement datasets without human evaluation or post-edits, employing an instruction-following refinement task,” said the researchers.

Elevate Translations

Ladder can integrate with any general-purpose LLM to improve translation performance without requiring significant changes to the existing model structure. This flexibility makes Ladder a versatile tool for various LLMs.

The researchers tested Ladder in two ways. Firstly, they checked how well Ladder can improve different types of language models, including those designed specifically for translation tasks, including BigTranslate, NLLB, and ALMA, and more general language models, including GPT-3.5 and GPT-4. Secondly, we compared Ladder to the best-known methods for refining translations or post-editing, including Unbabel’s TowerInstruct.

They found that Ladder can significantly improve the overall translation quality across most translation-specific and general-purpose LLMs. Ladder can “elevate raw translations to the level of top-tier open-source models,” they said.

The paper and the code are available on GitHub.