t5-base-neg2pos

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1739
• Rouge1: 62.4361
• Rouge2: 46.2313
• Rougel: 60.8217
• Rougelsum: 60.8436
• Gen Len: 14.2644

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	63	0.6190	25.7819	13.3116	24.5077	24.7927	12.0511
No log	2.0	126	0.2495	3.5073	2.2341	3.3849	3.4491	1.6422
No log	3.0	189	0.1894	23.3895	16.6499	22.4422	22.4241	5.9222
No log	4.0	252	0.1762	59.7842	43.6137	57.9791	57.9826	13.7778
No log	5.0	315	0.1739	62.4361	46.2313	60.8217	60.8436	14.2644

Framework versions

• Transformers 4.40.1
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.19.1