|
--- |
|
language: |
|
- es |
|
- maz |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- bleu |
|
model-index: |
|
- name: byt5-base-es_maz |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# byt5-base-es_maz |
|
|
|
This model is a fine-tuned version of [google/byt5-base](https://huggingface.co/google/byt5-base) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.0917 |
|
- Bleu: 14.8412 |
|
- Gen Len: 98.6675 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 65 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 100.0 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len | |
|
|:-------------:|:-----:|:-----:|:---------------:|:------:|:-------:| |
|
| No log | 1.0 | 393 | 1.0346 | 0.0473 | 19.0 | |
|
| 1.5209 | 2.0 | 786 | 0.8939 | 0.1413 | 19.0 | |
|
| 1.0258 | 3.0 | 1179 | 0.8334 | 0.1641 | 19.0 | |
|
| 0.9177 | 4.0 | 1572 | 0.7867 | 0.1729 | 19.0 | |
|
| 0.9177 | 5.0 | 1965 | 0.7543 | 0.1742 | 19.0 | |
|
| 0.8482 | 6.0 | 2358 | 0.7317 | 0.1692 | 19.0 | |
|
| 0.7957 | 7.0 | 2751 | 0.7106 | 0.1742 | 19.0 | |
|
| 0.7557 | 8.0 | 3144 | 0.6849 | 0.216 | 19.0 | |
|
| 0.7204 | 9.0 | 3537 | 0.6731 | 0.189 | 19.0 | |
|
| 0.7204 | 10.0 | 3930 | 0.6562 | 0.2063 | 19.0 | |
|
| 0.6901 | 11.0 | 4323 | 0.6510 | 0.2025 | 19.0 | |
|
| 0.6635 | 12.0 | 4716 | 0.6423 | 0.2266 | 19.0 | |
|
| 0.6346 | 13.0 | 5109 | 0.6330 | 0.2229 | 19.0 | |
|
| 0.6132 | 14.0 | 5502 | 0.6257 | 0.2195 | 19.0 | |
|
| 0.6132 | 15.0 | 5895 | 0.6192 | 0.2344 | 19.0 | |
|
| 0.5885 | 16.0 | 6288 | 0.6104 | 0.2424 | 19.0 | |
|
| 0.5682 | 17.0 | 6681 | 0.6048 | 0.2536 | 19.0 | |
|
| 0.5452 | 18.0 | 7074 | 0.6057 | 0.2541 | 19.0 | |
|
| 0.5452 | 19.0 | 7467 | 0.6047 | 0.2526 | 19.0 | |
|
| 0.5294 | 20.0 | 7860 | 0.6066 | 0.2644 | 19.0 | |
|
| 0.5072 | 21.0 | 8253 | 0.6080 | 0.2666 | 19.0 | |
|
| 0.4921 | 22.0 | 8646 | 0.6092 | 0.2499 | 19.0 | |
|
| 0.4753 | 23.0 | 9039 | 0.6132 | 0.2719 | 19.0 | |
|
| 0.4753 | 24.0 | 9432 | 0.6088 | 0.2724 | 19.0 | |
|
| 0.4597 | 25.0 | 9825 | 0.6128 | 0.2683 | 19.0 | |
|
| 0.4443 | 26.0 | 10218 | 0.6183 | 0.2856 | 19.0 | |
|
| 0.4301 | 27.0 | 10611 | 0.6246 | 0.3006 | 19.0 | |
|
| 0.418 | 28.0 | 11004 | 0.6312 | 0.2788 | 19.0 | |
|
| 0.418 | 29.0 | 11397 | 0.6295 | 0.2843 | 19.0 | |
|
| 0.4002 | 30.0 | 11790 | 0.6350 | 0.2982 | 19.0 | |
|
| 0.3913 | 31.0 | 12183 | 0.6441 | 0.2822 | 19.0 | |
|
| 0.3755 | 32.0 | 12576 | 0.6430 | 0.3215 | 19.0 | |
|
| 0.3755 | 33.0 | 12969 | 0.6486 | 0.3024 | 19.0 | |
|
| 0.3673 | 34.0 | 13362 | 0.6527 | 0.2985 | 19.0 | |
|
| 0.352 | 35.0 | 13755 | 0.6660 | 0.31 | 19.0 | |
|
| 0.3408 | 36.0 | 14148 | 0.6737 | 0.288 | 19.0 | |
|
| 0.3307 | 37.0 | 14541 | 0.6773 | 0.2995 | 19.0 | |
|
| 0.3307 | 38.0 | 14934 | 0.6903 | 0.29 | 19.0 | |
|
| 0.3182 | 39.0 | 15327 | 0.7059 | 0.2848 | 19.0 | |
|
| 0.3077 | 40.0 | 15720 | 0.6986 | 0.2878 | 19.0 | |
|
| 0.298 | 41.0 | 16113 | 0.7053 | 0.2859 | 19.0 | |
|
| 0.29 | 42.0 | 16506 | 0.7198 | 0.2871 | 19.0 | |
|
| 0.29 | 43.0 | 16899 | 0.7275 | 0.2813 | 19.0 | |
|
| 0.2787 | 44.0 | 17292 | 0.7370 | 0.2972 | 19.0 | |
|
| 0.268 | 45.0 | 17685 | 0.7426 | 0.26 | 19.0 | |
|
| 0.2638 | 46.0 | 18078 | 0.7529 | 0.2846 | 19.0 | |
|
| 0.2638 | 47.0 | 18471 | 0.7603 | 0.2898 | 19.0 | |
|
| 0.253 | 48.0 | 18864 | 0.7711 | 0.277 | 19.0 | |
|
| 0.244 | 49.0 | 19257 | 0.7779 | 0.3005 | 19.0 | |
|
| 0.2368 | 50.0 | 19650 | 0.7815 | 0.2931 | 19.0 | |
|
| 0.2301 | 51.0 | 20043 | 0.8020 | 0.2998 | 19.0 | |
|
| 0.2301 | 52.0 | 20436 | 0.8051 | 0.2806 | 19.0 | |
|
| 0.2217 | 53.0 | 20829 | 0.8119 | 0.294 | 19.0 | |
|
| 0.2158 | 54.0 | 21222 | 0.8288 | 0.2921 | 19.0 | |
|
| 0.2079 | 55.0 | 21615 | 0.8341 | 0.2954 | 19.0 | |
|
| 0.2027 | 56.0 | 22008 | 0.8365 | 0.2884 | 19.0 | |
|
| 0.2027 | 57.0 | 22401 | 0.8441 | 0.2995 | 19.0 | |
|
| 0.1954 | 58.0 | 22794 | 0.8488 | 0.3115 | 19.0 | |
|
| 0.1918 | 59.0 | 23187 | 0.8710 | 0.3085 | 19.0 | |
|
| 0.1857 | 60.0 | 23580 | 0.8718 | 0.2932 | 19.0 | |
|
| 0.1857 | 61.0 | 23973 | 0.8777 | 0.2923 | 19.0 | |
|
| 0.1796 | 62.0 | 24366 | 0.8832 | 0.3038 | 19.0 | |
|
| 0.1753 | 63.0 | 24759 | 0.8997 | 0.3063 | 19.0 | |
|
| 0.1703 | 64.0 | 25152 | 0.9198 | 0.3047 | 19.0 | |
|
| 0.1661 | 65.0 | 25545 | 0.9194 | 0.3159 | 19.0 | |
|
| 0.1661 | 66.0 | 25938 | 0.9243 | 0.2962 | 19.0 | |
|
| 0.1606 | 67.0 | 26331 | 0.9376 | 0.3065 | 19.0 | |
|
| 0.1582 | 68.0 | 26724 | 0.9339 | 0.3002 | 19.0 | |
|
| 0.1533 | 69.0 | 27117 | 0.9420 | 0.3096 | 19.0 | |
|
| 0.1503 | 70.0 | 27510 | 0.9522 | 0.2919 | 19.0 | |
|
| 0.1503 | 71.0 | 27903 | 0.9620 | 0.3085 | 19.0 | |
|
| 0.1469 | 72.0 | 28296 | 0.9673 | 0.2946 | 19.0 | |
|
| 0.1416 | 73.0 | 28689 | 0.9706 | 0.3019 | 19.0 | |
|
| 0.1401 | 74.0 | 29082 | 0.9877 | 0.3103 | 19.0 | |
|
| 0.1401 | 75.0 | 29475 | 0.9860 | 0.2903 | 19.0 | |
|
| 0.1376 | 76.0 | 29868 | 1.0073 | 0.2855 | 19.0 | |
|
| 0.1341 | 77.0 | 30261 | 1.0067 | 0.2927 | 19.0 | |
|
| 0.1307 | 78.0 | 30654 | 1.0064 | 0.3 | 19.0 | |
|
| 0.1296 | 79.0 | 31047 | 1.0221 | 0.2886 | 19.0 | |
|
| 0.1296 | 80.0 | 31440 | 1.0217 | 0.297 | 19.0 | |
|
| 0.126 | 81.0 | 31833 | 1.0278 | 0.2919 | 19.0 | |
|
| 0.1238 | 82.0 | 32226 | 1.0329 | 0.2951 | 19.0 | |
|
| 0.1214 | 83.0 | 32619 | 1.0351 | 0.3043 | 19.0 | |
|
| 0.1206 | 84.0 | 33012 | 1.0498 | 0.2964 | 19.0 | |
|
| 0.1206 | 85.0 | 33405 | 1.0433 | 0.2971 | 19.0 | |
|
| 0.1186 | 86.0 | 33798 | 1.0525 | 0.2964 | 19.0 | |
|
| 0.116 | 87.0 | 34191 | 1.0547 | 0.2943 | 19.0 | |
|
| 0.116 | 88.0 | 34584 | 1.0585 | 0.2876 | 19.0 | |
|
| 0.116 | 89.0 | 34977 | 1.0631 | 0.2904 | 19.0 | |
|
| 0.1131 | 90.0 | 35370 | 1.0678 | 0.2859 | 19.0 | |
|
| 0.1124 | 91.0 | 35763 | 1.0764 | 0.3027 | 19.0 | |
|
| 0.1109 | 92.0 | 36156 | 1.0759 | 0.3037 | 19.0 | |
|
| 0.1097 | 93.0 | 36549 | 1.0738 | 0.2962 | 19.0 | |
|
| 0.1097 | 94.0 | 36942 | 1.0855 | 0.2966 | 19.0 | |
|
| 0.1093 | 95.0 | 37335 | 1.0902 | 0.2968 | 19.0 | |
|
| 0.1082 | 96.0 | 37728 | 1.0859 | 0.2958 | 19.0 | |
|
| 0.1073 | 97.0 | 38121 | 1.0867 | 0.3023 | 19.0 | |
|
| 0.1063 | 98.0 | 38514 | 1.0902 | 0.3004 | 19.0 | |
|
| 0.1063 | 99.0 | 38907 | 1.0910 | 0.3018 | 19.0 | |
|
| 0.1065 | 100.0 | 39300 | 1.0917 | 0.3021 | 19.0 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.29.2 |
|
- Pytorch 2.0.1+cu118 |
|
- Datasets 2.12.0 |
|
- Tokenizers 0.13.3 |
|
|
