prophetnet-large-uncased-samsum

This model is a fine-tuned version of microsoft/prophetnet-large-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.8806
• Rouge1: 42.7998
• Rouge2: 20.6028
• Rougel: 32.7447
• Rougelsum: 38.6683
• Gen Len: 46.5763

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.2102	1.0	1842	1.9800	46.2865	22.7077	36.6175	42.7446	37.8791
1.6648	2.0	3684	1.8943	43.8689	21.5646	34.0359	39.9618	44.8193
1.2354	3.0	5526	1.8806	42.7998	20.6028	32.7447	38.6683	46.5763
0.9176	4.0	7368	1.9474	43.5931	20.9008	33.236	39.1182	46.1087
0.6827	5.0	9210	2.0220	42.9591	20.2499	32.94	38.5586	46.9658

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.0
• Tokenizers 0.15.0