t5-small-finetuned-v2-chinese-to-hausa

This model is a fine-tuned version of google-t5/t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7757
• Bleu: 1.9523
• Gen Len: 18.6997

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0006
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 3000
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
3.5237	1.0	1103	2.5564	0.9857	18.8055
2.5377	2.0	2206	2.1815	1.6392	18.7259
2.2418	3.0	3309	2.0216	1.8732	18.5659
2.0818	4.0	4412	1.9331	1.9183	18.1569
1.9874	5.0	5515	1.8834	1.6409	18.0559
1.9213	6.0	6618	1.8511	1.9679	18.6534
1.8712	7.0	7721	1.8295	1.8632	18.4841
1.8292	8.0	8824	1.8101	2.5462	18.5024
1.7949	9.0	9927	1.7990	1.847	18.3106
1.7666	10.0	11030	1.7867	1.849	18.4893
1.7428	11.0	12133	1.7826	1.7849	18.6368
1.7256	12.0	13236	1.7757	1.9587	18.7077
1.7124	13.0	14339	1.7746	2.2943	18.5367
1.7051	14.0	15442	1.7757	1.9676	18.7081
1.7001	15.0	16545	1.7757	1.9523	18.6997

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.1