mt5-small-finetuned-billsum

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1086
• Rouge1: 72.606
• Rouge2: 64.3705
• Rougel: 71.2857
• Rougelsum: 71.3956

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
1.4879	1.0	124	1.3033	71.4229	61.8019	70.0374	70.1665
1.4816	2.0	248	1.2268	73.2518	64.6915	71.8575	71.9894
1.3856	3.0	372	1.1861	73.1259	64.7309	71.7198	71.812
1.2986	4.0	496	1.1576	71.8392	63.7365	70.6684	70.7165
1.2637	5.0	620	1.1425	72.5544	64.3521	71.1831	71.3131
1.2161	6.0	744	1.1173	71.9681	63.8108	70.749	70.8831
1.1976	7.0	868	1.1116	72.6794	64.6099	71.4453	71.5871
1.2013	8.0	992	1.1086	72.606	64.3705	71.2857	71.3956

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0