---
license: apache-2.0
base_model: google/mt5-small
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: mt5-summarize-te
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mt5-summarize-te

This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.1257
- Rouge1: 0.5211
- Rouge2: 0.4338
- Rougel: 0.4813
- Rougelsum: 0.4819

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 2
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 16
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 90
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|
| 3.8667        | 0.2   | 100  | 2.5990          | 0.4450 | 0.3695 | 0.4151 | 0.4154    |
| 3.2326        | 0.39  | 200  | 2.5107          | 0.5023 | 0.4156 | 0.4567 | 0.4571    |
| 3.1169        | 0.59  | 300  | 2.4503          | 0.5092 | 0.4204 | 0.4775 | 0.4762    |
| 2.9083        | 0.79  | 400  | 2.4005          | 0.5053 | 0.4179 | 0.4699 | 0.4709    |
| 2.9652        | 0.98  | 500  | 2.3218          | 0.4833 | 0.4003 | 0.4528 | 0.4555    |
| 2.8848        | 1.18  | 600  | 2.3262          | 0.5309 | 0.4415 | 0.4868 | 0.4879    |
| 2.6585        | 1.37  | 700  | 2.3118          | 0.5168 | 0.4273 | 0.4780 | 0.4773    |
| 2.6662        | 1.57  | 800  | 2.2823          | 0.5112 | 0.4233 | 0.4713 | 0.4727    |
| 2.7628        | 1.77  | 900  | 2.2381          | 0.5158 | 0.4269 | 0.4798 | 0.4798    |
| 2.7156        | 1.96  | 1000 | 2.2466          | 0.5280 | 0.4452 | 0.4836 | 0.4844    |
| 2.5683        | 2.16  | 1100 | 2.2495          | 0.5184 | 0.4300 | 0.4779 | 0.4773    |
| 2.5248        | 2.36  | 1200 | 2.2498          | 0.5179 | 0.4282 | 0.4790 | 0.4803    |
| 2.5809        | 2.55  | 1300 | 2.2336          | 0.5233 | 0.4385 | 0.4895 | 0.4920    |
| 2.7113        | 2.75  | 1400 | 2.2368          | 0.5079 | 0.4207 | 0.4707 | 0.4716    |
| 2.6151        | 2.95  | 1500 | 2.1993          | 0.5108 | 0.4236 | 0.4681 | 0.4679    |
| 2.5172        | 3.14  | 1600 | 2.2197          | 0.5138 | 0.4257 | 0.4778 | 0.4781    |
| 2.5873        | 3.34  | 1700 | 2.1900          | 0.5185 | 0.4312 | 0.4823 | 0.4821    |
| 2.4245        | 3.53  | 1800 | 2.1982          | 0.5222 | 0.4332 | 0.4837 | 0.4853    |
| 2.4983        | 3.73  | 1900 | 2.1756          | 0.5125 | 0.4247 | 0.4809 | 0.4810    |
| 2.3963        | 3.93  | 2000 | 2.1900          | 0.5259 | 0.4400 | 0.4870 | 0.4884    |
| 2.3465        | 4.12  | 2100 | 2.1963          | 0.5300 | 0.4412 | 0.4900 | 0.4915    |
| 2.4625        | 4.32  | 2200 | 2.1818          | 0.5277 | 0.4384 | 0.4868 | 0.4882    |
| 2.4257        | 4.52  | 2300 | 2.1504          | 0.5212 | 0.4342 | 0.4833 | 0.4842    |
| 2.368         | 4.71  | 2400 | 2.1463          | 0.5252 | 0.4418 | 0.4856 | 0.4869    |
| 2.427         | 4.91  | 2500 | 2.1581          | 0.5161 | 0.4267 | 0.4766 | 0.4771    |
| 2.3443        | 5.11  | 2600 | 2.1551          | 0.5167 | 0.4281 | 0.4794 | 0.4794    |
| 2.2923        | 5.3   | 2700 | 2.1596          | 0.5183 | 0.4255 | 0.4668 | 0.4686    |
| 2.2956        | 5.5   | 2800 | 2.1438          | 0.5125 | 0.4268 | 0.4747 | 0.4754    |
| 2.2973        | 5.69  | 2900 | 2.1523          | 0.5139 | 0.4259 | 0.4712 | 0.4722    |
| 2.3013        | 5.89  | 3000 | 2.1514          | 0.5138 | 0.4236 | 0.4741 | 0.4742    |
| 2.2222        | 6.09  | 3100 | 2.1558          | 0.5172 | 0.4300 | 0.4773 | 0.4784    |
| 2.3957        | 6.28  | 3200 | 2.1451          | 0.5203 | 0.4326 | 0.4815 | 0.4817    |
| 2.1995        | 6.48  | 3300 | 2.1476          | 0.5146 | 0.4264 | 0.4747 | 0.4752    |
| 2.2931        | 6.68  | 3400 | 2.1252          | 0.5120 | 0.4252 | 0.4683 | 0.4683    |
| 2.3062        | 6.87  | 3500 | 2.1313          | 0.5197 | 0.4339 | 0.4803 | 0.4807    |
| 2.2844        | 7.07  | 3600 | 2.1281          | 0.5197 | 0.4339 | 0.4868 | 0.4876    |
| 2.1158        | 7.27  | 3700 | 2.1438          | 0.5208 | 0.4333 | 0.4818 | 0.4823    |
| 2.2523        | 7.46  | 3800 | 2.1221          | 0.5197 | 0.4324 | 0.4783 | 0.4788    |
| 2.2389        | 7.66  | 3900 | 2.1336          | 0.5144 | 0.4262 | 0.4769 | 0.4771    |
| 2.2209        | 7.85  | 4000 | 2.1317          | 0.5211 | 0.4338 | 0.4813 | 0.4819    |
| 2.1828        | 8.05  | 4100 | 2.1366          | 0.5208 | 0.4336 | 0.4814 | 0.4816    |
| 2.2746        | 8.25  | 4200 | 2.1325          | 0.5219 | 0.4342 | 0.4819 | 0.4823    |
| 2.229         | 8.44  | 4300 | 2.1334          | 0.5214 | 0.4329 | 0.4809 | 0.4812    |
| 2.2762        | 8.64  | 4400 | 2.1223          | 0.5161 | 0.4288 | 0.4761 | 0.4769    |
| 2.2005        | 8.84  | 4500 | 2.1322          | 0.5197 | 0.4320 | 0.4793 | 0.4799    |
| 2.1975        | 9.03  | 4600 | 2.1294          | 0.5211 | 0.4338 | 0.4813 | 0.4819    |
| 2.3219        | 9.23  | 4700 | 2.1251          | 0.5148 | 0.4260 | 0.4768 | 0.4772    |
| 2.252         | 9.43  | 4800 | 2.1261          | 0.5211 | 0.4338 | 0.4813 | 0.4819    |
| 2.2594        | 9.62  | 4900 | 2.1236          | 0.5200 | 0.4331 | 0.4808 | 0.4814    |
| 2.1675        | 9.82  | 5000 | 2.1257          | 0.5211 | 0.4338 | 0.4813 | 0.4819    |


### Framework versions

- Transformers 4.35.0
- Pytorch 2.1.0+cu118
- Datasets 2.14.6
- Tokenizers 0.14.1