metadata
license: apache-2.0
base_model: google/flan-t5-small
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-summarization-headers-50-epochs
results: []
t5-summarization-headers-50-epochs
This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.2125
- Rouge: {'rouge1': 0.4117, 'rouge2': 0.2163, 'rougeL': 0.2158, 'rougeLsum': 0.2158}
- Bert Score: 0.8818
- Bleurt 20: -0.8026
- Gen Len: 14.46
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 7
- eval_batch_size: 7
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge | Bert Score | Bleurt 20 | Gen Len |
|---|---|---|---|---|---|---|---|
| 3.0256 | 1.0 | 186 | 2.6300 | {'rouge1': 0.4643, 'rouge2': 0.1902, 'rougeL': 0.1973, 'rougeLsum': 0.1973} | 0.8664 | -0.8801 | 15.55 |
| 2.734 | 2.0 | 372 | 2.4218 | {'rouge1': 0.4489, 'rouge2': 0.2037, 'rougeL': 0.209, 'rougeLsum': 0.209} | 0.8737 | -0.8686 | 14.995 |
| 2.5147 | 3.0 | 558 | 2.3219 | {'rouge1': 0.4363, 'rouge2': 0.1984, 'rougeL': 0.2067, 'rougeLsum': 0.2067} | 0.8742 | -0.8762 | 14.69 |
| 2.3007 | 4.0 | 744 | 2.2752 | {'rouge1': 0.4465, 'rouge2': 0.2043, 'rougeL': 0.2022, 'rougeLsum': 0.2022} | 0.8761 | -0.8603 | 14.625 |
| 2.1922 | 5.0 | 930 | 2.2331 | {'rouge1': 0.425, 'rouge2': 0.2033, 'rougeL': 0.2042, 'rougeLsum': 0.2042} | 0.8779 | -0.829 | 14.87 |
| 2.1185 | 6.0 | 1116 | 2.2092 | {'rouge1': 0.4231, 'rouge2': 0.2096, 'rougeL': 0.2073, 'rougeLsum': 0.2073} | 0.8783 | -0.8359 | 14.68 |
| 2.0584 | 7.0 | 1302 | 2.1993 | {'rouge1': 0.4302, 'rouge2': 0.2114, 'rougeL': 0.2126, 'rougeLsum': 0.2126} | 0.8793 | -0.8202 | 15.015 |
| 2.0189 | 8.0 | 1488 | 2.1872 | {'rouge1': 0.4255, 'rouge2': 0.2086, 'rougeL': 0.2106, 'rougeLsum': 0.2106} | 0.879 | -0.8359 | 14.485 |
| 1.8933 | 9.0 | 1674 | 2.1967 | {'rouge1': 0.4307, 'rouge2': 0.2175, 'rougeL': 0.2165, 'rougeLsum': 0.2165} | 0.8821 | -0.7803 | 14.865 |
| 1.8859 | 10.0 | 1860 | 2.1905 | {'rouge1': 0.4342, 'rouge2': 0.2139, 'rougeL': 0.2193, 'rougeLsum': 0.2193} | 0.8828 | -0.7683 | 14.93 |
| 1.8395 | 11.0 | 2046 | 2.2006 | {'rouge1': 0.42, 'rouge2': 0.2135, 'rougeL': 0.2175, 'rougeLsum': 0.2175} | 0.8815 | -0.7958 | 14.485 |
| 1.7848 | 12.0 | 2232 | 2.1970 | {'rouge1': 0.4309, 'rouge2': 0.2096, 'rougeL': 0.2171, 'rougeLsum': 0.2171} | 0.8826 | -0.8131 | 14.51 |
| 1.7855 | 13.0 | 2418 | 2.2026 | {'rouge1': 0.4218, 'rouge2': 0.2099, 'rougeL': 0.2182, 'rougeLsum': 0.2182} | 0.8812 | -0.8068 | 14.555 |
| 1.6971 | 14.0 | 2604 | 2.2006 | {'rouge1': 0.4035, 'rouge2': 0.2056, 'rougeL': 0.2109, 'rougeLsum': 0.2109} | 0.8816 | -0.817 | 14.145 |
| 1.7226 | 15.0 | 2790 | 2.2000 | {'rouge1': 0.413, 'rouge2': 0.2072, 'rougeL': 0.2145, 'rougeLsum': 0.2145} | 0.8818 | -0.8106 | 14.415 |
| 1.7164 | 16.0 | 2976 | 2.2067 | {'rouge1': 0.4117, 'rouge2': 0.212, 'rougeL': 0.215, 'rougeLsum': 0.215} | 0.8815 | -0.8198 | 14.235 |
| 1.6908 | 17.0 | 3162 | 2.2061 | {'rouge1': 0.4125, 'rouge2': 0.2193, 'rougeL': 0.2154, 'rougeLsum': 0.2154} | 0.8814 | -0.8089 | 14.37 |
| 1.6865 | 18.0 | 3348 | 2.2088 | {'rouge1': 0.4125, 'rouge2': 0.2173, 'rougeL': 0.217, 'rougeLsum': 0.217} | 0.8819 | -0.807 | 14.46 |
| 1.6225 | 19.0 | 3534 | 2.2127 | {'rouge1': 0.4111, 'rouge2': 0.2161, 'rougeL': 0.2123, 'rougeLsum': 0.2123} | 0.8815 | -0.8039 | 14.425 |
| 1.6304 | 20.0 | 3720 | 2.2125 | {'rouge1': 0.4117, 'rouge2': 0.2163, 'rougeL': 0.2158, 'rougeLsum': 0.2158} | 0.8818 | -0.8026 | 14.46 |
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0