bbc-to-ind-nmt-v8

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the nusatranslation_mt dataset. It achieves the following results on the evaluation set:

• Loss: 1.1397
• Sacrebleu: 37.8271
• Gen Len: 37.329

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Sacrebleu	Gen Len
5.9033	1.0	207	3.5144	25.8748	39.2155
2.4501	2.0	414	1.4664	31.5752	38.098
1.2896	3.0	621	1.1951	35.3334	37.154
1.0669	4.0	828	1.1503	36.5808	36.977
0.969	5.0	1035	1.1384	37.1725	37.2425
0.9036	6.0	1242	1.1310	37.6427	37.141
0.8572	7.0	1449	1.1333	37.6234	37.264
0.821	8.0	1656	1.1333	37.7638	37.222
0.7971	9.0	1863	1.1385	37.8469	37.378
0.7811	10.0	2070	1.1397	37.8271	37.329

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.14.6
• Tokenizers 0.19.1