|
--- |
|
license: apache-2.0 |
|
base_model: google/flan-t5-small |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- rouge |
|
model-index: |
|
- name: t5-summarization-headers-50-epochs |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# t5-summarization-headers-50-epochs |
|
|
|
This model is a fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 2.2125 |
|
- Rouge: {'rouge1': 0.4117, 'rouge2': 0.2163, 'rougeL': 0.2158, 'rougeLsum': 0.2158} |
|
- Bert Score: 0.8818 |
|
- Bleurt 20: -0.8026 |
|
- Gen Len: 14.46 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0001 |
|
- train_batch_size: 7 |
|
- eval_batch_size: 7 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_ratio: 0.1 |
|
- num_epochs: 20 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Rouge | Bert Score | Bleurt 20 | Gen Len | |
|
|:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------:|:----------:|:---------:|:-------:| |
|
| 3.0256 | 1.0 | 186 | 2.6300 | {'rouge1': 0.4643, 'rouge2': 0.1902, 'rougeL': 0.1973, 'rougeLsum': 0.1973} | 0.8664 | -0.8801 | 15.55 | |
|
| 2.734 | 2.0 | 372 | 2.4218 | {'rouge1': 0.4489, 'rouge2': 0.2037, 'rougeL': 0.209, 'rougeLsum': 0.209} | 0.8737 | -0.8686 | 14.995 | |
|
| 2.5147 | 3.0 | 558 | 2.3219 | {'rouge1': 0.4363, 'rouge2': 0.1984, 'rougeL': 0.2067, 'rougeLsum': 0.2067} | 0.8742 | -0.8762 | 14.69 | |
|
| 2.3007 | 4.0 | 744 | 2.2752 | {'rouge1': 0.4465, 'rouge2': 0.2043, 'rougeL': 0.2022, 'rougeLsum': 0.2022} | 0.8761 | -0.8603 | 14.625 | |
|
| 2.1922 | 5.0 | 930 | 2.2331 | {'rouge1': 0.425, 'rouge2': 0.2033, 'rougeL': 0.2042, 'rougeLsum': 0.2042} | 0.8779 | -0.829 | 14.87 | |
|
| 2.1185 | 6.0 | 1116 | 2.2092 | {'rouge1': 0.4231, 'rouge2': 0.2096, 'rougeL': 0.2073, 'rougeLsum': 0.2073} | 0.8783 | -0.8359 | 14.68 | |
|
| 2.0584 | 7.0 | 1302 | 2.1993 | {'rouge1': 0.4302, 'rouge2': 0.2114, 'rougeL': 0.2126, 'rougeLsum': 0.2126} | 0.8793 | -0.8202 | 15.015 | |
|
| 2.0189 | 8.0 | 1488 | 2.1872 | {'rouge1': 0.4255, 'rouge2': 0.2086, 'rougeL': 0.2106, 'rougeLsum': 0.2106} | 0.879 | -0.8359 | 14.485 | |
|
| 1.8933 | 9.0 | 1674 | 2.1967 | {'rouge1': 0.4307, 'rouge2': 0.2175, 'rougeL': 0.2165, 'rougeLsum': 0.2165} | 0.8821 | -0.7803 | 14.865 | |
|
| 1.8859 | 10.0 | 1860 | 2.1905 | {'rouge1': 0.4342, 'rouge2': 0.2139, 'rougeL': 0.2193, 'rougeLsum': 0.2193} | 0.8828 | -0.7683 | 14.93 | |
|
| 1.8395 | 11.0 | 2046 | 2.2006 | {'rouge1': 0.42, 'rouge2': 0.2135, 'rougeL': 0.2175, 'rougeLsum': 0.2175} | 0.8815 | -0.7958 | 14.485 | |
|
| 1.7848 | 12.0 | 2232 | 2.1970 | {'rouge1': 0.4309, 'rouge2': 0.2096, 'rougeL': 0.2171, 'rougeLsum': 0.2171} | 0.8826 | -0.8131 | 14.51 | |
|
| 1.7855 | 13.0 | 2418 | 2.2026 | {'rouge1': 0.4218, 'rouge2': 0.2099, 'rougeL': 0.2182, 'rougeLsum': 0.2182} | 0.8812 | -0.8068 | 14.555 | |
|
| 1.6971 | 14.0 | 2604 | 2.2006 | {'rouge1': 0.4035, 'rouge2': 0.2056, 'rougeL': 0.2109, 'rougeLsum': 0.2109} | 0.8816 | -0.817 | 14.145 | |
|
| 1.7226 | 15.0 | 2790 | 2.2000 | {'rouge1': 0.413, 'rouge2': 0.2072, 'rougeL': 0.2145, 'rougeLsum': 0.2145} | 0.8818 | -0.8106 | 14.415 | |
|
| 1.7164 | 16.0 | 2976 | 2.2067 | {'rouge1': 0.4117, 'rouge2': 0.212, 'rougeL': 0.215, 'rougeLsum': 0.215} | 0.8815 | -0.8198 | 14.235 | |
|
| 1.6908 | 17.0 | 3162 | 2.2061 | {'rouge1': 0.4125, 'rouge2': 0.2193, 'rougeL': 0.2154, 'rougeLsum': 0.2154} | 0.8814 | -0.8089 | 14.37 | |
|
| 1.6865 | 18.0 | 3348 | 2.2088 | {'rouge1': 0.4125, 'rouge2': 0.2173, 'rougeL': 0.217, 'rougeLsum': 0.217} | 0.8819 | -0.807 | 14.46 | |
|
| 1.6225 | 19.0 | 3534 | 2.2127 | {'rouge1': 0.4111, 'rouge2': 0.2161, 'rougeL': 0.2123, 'rougeLsum': 0.2123} | 0.8815 | -0.8039 | 14.425 | |
|
| 1.6304 | 20.0 | 3720 | 2.2125 | {'rouge1': 0.4117, 'rouge2': 0.2163, 'rougeL': 0.2158, 'rougeLsum': 0.2158} | 0.8818 | -0.8026 | 14.46 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.2 |
|
- Pytorch 2.1.0+cu121 |
|
- Datasets 2.16.1 |
|
- Tokenizers 0.15.0 |
|
|
