distill-pegasus-cnn-arxiv-pubmed-v3-e16

This model is a fine-tuned version of theojolliffe/distill-pegasus-cnn-arxiv-pubmed on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4922
• Rouge1: 53.3238
• Rouge2: 36.6165
• Rougel: 38.9255
• Rougelsum: 50.4853
• Gen Len: 125.7407

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.7655	1.0	795	2.1110	49.0541	29.7039	33.8403	44.2825	126.1296
2.2882	2.0	1590	1.9469	48.4651	30.1425	33.9702	44.3518	125.7778
2.1958	3.0	2385	1.8079	49.2302	31.0952	34.4448	45.5764	125.7778
2.0221	4.0	3180	1.7501	48.1928	29.9098	33.0587	44.6023	125.3148
1.9078	5.0	3975	1.6677	49.697	31.671	34.3162	46.5108	125.5185
1.8624	6.0	4770	1.6393	49.6517	31.7371	35.2019	46.2846	125.6852
1.7853	7.0	5565	1.6038	50.3151	33.0952	36.0028	47.3894	125.6852
1.7513	8.0	6360	1.5717	50.299	33.038	35.6841	47.4086	124.5556
1.7026	9.0	7155	1.5570	51.6216	34.7609	37.5598	48.5247	124.7037
1.6999	10.0	7950	1.5365	51.0888	34.2642	37.0603	48.5712	125.3519
1.6832	11.0	8745	1.5249	51.3422	34.2941	37.7111	48.556	124.9259
1.6093	12.0	9540	1.5092	51.4622	34.6397	38.1768	48.6346	124.8889
1.6049	13.0	10335	1.5002	52.2463	35.4629	38.2049	49.4066	124.7963
1.5904	14.0	11130	1.4957	51.6498	34.9739	38.4215	48.9704	125.0185
1.5963	15.0	11925	1.4920	52.769	35.9563	38.4861	49.9185	125.6481
1.5742	16.0	12720	1.4922	53.3238	36.6165	38.9255	50.4853	125.7407

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0+cu113
• Datasets 2.1.0
• Tokenizers 0.12.1