bart-base-finetuned-w-data-augm-4e-5

This model is a fine-tuned version of facebook/bart-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3874
• Sacrebleu: 89.8161
• Rouge1: 95.6774
• Rouge2: 91.8937
• Rougel: 94.6649
• Rougelsum: 94.6595
• Bertscore Precision: 0.9414
• Bertscore Recall: 0.9376
• Bertscore F1: 0.9395

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.4252514647201465e-05
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Sacrebleu	Rouge1	Rouge2	Rougel	Rougelsum	Bertscore Precision	Bertscore Recall	Bertscore F1
0.1504	1.0	761	0.2797	90.9313	96.2421	92.8783	95.4262	95.4043	0.9496	0.9444	0.9469
0.0348	2.0	1522	0.2473	91.7583	96.3865	93.2655	95.6899	95.6811	0.9532	0.9504	0.9517
0.0587	3.0	2283	0.2413	91.828	96.4392	93.4124	95.7079	95.6976	0.9517	0.9508	0.9512
0.0269	4.0	3044	0.2588	91.9835	96.578	93.6221	95.8992	95.8798	0.9524	0.9527	0.9525
0.0439	5.0	3805	0.2678	92.1033	96.6815	93.6391	95.9677	95.9469	0.9544	0.9536	0.954

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1