For a given source sentence \(x\), whether translated from or to English, we utilize both GPT-4 and ALMA-13B-LoRA to generate respective translations, denoted as \(y_{\text{grt-4}}\) and \(y_{\text{alma}}\). Together with the original target reference \(y_{\text{ref}}\), this forms a triplet \(\mathbf{y}=(y_{\text{ref}},y_{\text{grt-4}},y_{\text{alma}})\), representing three different translation outputs for the input \(x\). The reference-free evaluation models KIWI-XXL and XCOMET are then employed to score these translations, with the average scores represented as \(\mathbf{s}=(s_{\text{ref}},s_{\text{grt-4}},s_{\text{alma}})\).3 The highest-scoring translation is labeled as the preferred translation \(y_{w}\), and the lowest-scoring as the dis-preferred translation \(y_{l}\), i.e., \(y_{w}=\mathbf{y}_{\arg\max_{i}(\mathbf{s})},y_{l}=\mathbf{y}_{\arg\min_{i}( \mathbf{s})}\), where \(i\) represents the index in the triplet. Translations with intermediate scores are not considered. An illustrative example of this selection process is depicted in Figure 3. It is important to note that even the dis-preferred translations may be of high-quality. The designation 'dis-preferred' indicates that there is still room for improvement, perhaps through the addition of minor details. This approach of using high-quality but not flawless translations as dis-preferred data aids in training the model to refine details and achieve perfection in generated translations.