
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mt5-large-fce-e8-b16

This model is a fine-tuned version of [google/mt5-large](https://huggingface.co/google/mt5-large) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3526
- Rouge1: 84.5329
- Rouge2: 76.3656
- Rougel: 83.9027
- Rougelsum: 83.9238
- Gen Len: 15.4614

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adafactor
- lr_scheduler_type: linear
- num_epochs: 8

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 1.2105        | 0.23  | 400   | 0.4344          | 84.6268 | 76.3447 | 84.0402 | 84.0182   | 15.4564 |
| 0.4664        | 0.45  | 800   | 0.4256          | 84.3821 | 75.6104 | 83.8113 | 83.8303   | 15.4404 |
| 0.434         | 0.68  | 1200  | 0.3839          | 84.0212 | 75.7319 | 83.4232 | 83.431    | 15.4952 |
| 0.406         | 0.9   | 1600  | 0.3713          | 84.7743 | 76.7805 | 84.2379 | 84.2352   | 15.4514 |
| 0.3193        | 1.13  | 2000  | 0.3665          | 84.634  | 76.5132 | 84.0604 | 84.0755   | 15.4774 |
| 0.2693        | 1.35  | 2400  | 0.3718          | 84.6587 | 76.7057 | 84.099  | 84.1045   | 15.4619 |
| 0.2815        | 1.58  | 2800  | 0.3617          | 84.5181 | 76.6792 | 83.9922 | 83.9976   | 15.4820 |
| 0.2776        | 1.81  | 3200  | 0.3526          | 84.5329 | 76.3656 | 83.9027 | 83.9238   | 15.4614 |
| 0.2551        | 2.03  | 3600  | 0.3720          | 84.504  | 76.6676 | 83.9957 | 84.0108   | 15.4801 |
| 0.1617        | 2.26  | 4000  | 0.3648          | 84.4385 | 76.3684 | 83.8585 | 83.8657   | 15.4897 |
| 0.1711        | 2.48  | 4400  | 0.3671          | 84.5241 | 76.6518 | 83.9862 | 83.9987   | 15.4902 |
| 0.1771        | 2.71  | 4800  | 0.3607          | 84.6437 | 76.6682 | 84.103  | 84.1174   | 15.4683 |
| 0.1803        | 2.93  | 5200  | 0.3582          | 84.479  | 76.6205 | 83.9509 | 83.9504   | 15.4715 |
| 0.1199        | 3.16  | 5600  | 0.3971          | 84.6367 | 76.7872 | 84.0191 | 84.0534   | 15.4715 |
| 0.1005        | 3.39  | 6000  | 0.4085          | 84.5153 | 76.6564 | 83.9365 | 83.9506   | 15.4820 |
| 0.1033        | 3.61  | 6400  | 0.4007          | 84.3191 | 76.399  | 83.8183 | 83.8142   | 15.4728 |
| 0.1067        | 3.84  | 6800  | 0.4014          | 84.5289 | 76.5335 | 83.9706 | 83.9967   | 15.4674 |
| 0.09          | 4.06  | 7200  | 0.4328          | 84.3978 | 76.6231 | 83.8654 | 83.8728   | 15.4783 |
| 0.0574        | 4.29  | 7600  | 0.4305          | 84.4476 | 76.7198 | 83.8943 | 83.9      | 15.4820 |
| 0.0579        | 4.51  | 8000  | 0.4510          | 84.5536 | 76.7635 | 83.977  | 83.9745   | 15.4719 |
| 0.061         | 4.74  | 8400  | 0.4447          | 84.5632 | 76.9892 | 84.0419 | 84.0501   | 15.4815 |
| 0.0608        | 4.97  | 8800  | 0.4353          | 84.6004 | 76.8883 | 84.0518 | 84.0596   | 15.4788 |
| 0.0362        | 5.19  | 9200  | 0.4853          | 84.7169 | 77.1321 | 84.1485 | 84.1486   | 15.4760 |
| 0.0333        | 5.42  | 9600  | 0.5053          | 84.851  | 77.4661 | 84.307  | 84.3106   | 15.4829 |
| 0.0325        | 5.64  | 10000 | 0.5066          | 84.7412 | 77.3031 | 84.2107 | 84.2006   | 15.4948 |
| 0.0335        | 5.87  | 10400 | 0.4947          | 84.7596 | 77.2636 | 84.2156 | 84.224    | 15.4906 |
| 0.0269        | 6.09  | 10800 | 0.5306          | 84.7484 | 77.2693 | 84.1824 | 84.1962   | 15.4811 |
| 0.0184        | 6.32  | 11200 | 0.5535          | 84.8066 | 77.3749 | 84.2765 | 84.2989   | 15.4756 |
| 0.0177        | 6.55  | 11600 | 0.5555          | 84.7335 | 77.2108 | 84.1917 | 84.2084   | 15.4865 |
| 0.0168        | 6.77  | 12000 | 0.5538          | 84.7053 | 77.2902 | 84.184  | 84.1929   | 15.4792 |
| 0.0165        | 7.0   | 12400 | 0.5614          | 84.7332 | 77.3098 | 84.2055 | 84.2055   | 15.4879 |
| 0.0092        | 7.22  | 12800 | 0.6222          | 84.7668 | 77.3059 | 84.2235 | 84.2397   | 15.4724 |
| 0.0086        | 7.45  | 13200 | 0.6485          | 84.8211 | 77.4247 | 84.2857 | 84.2996   | 15.4751 |
| 0.0098        | 7.67  | 13600 | 0.6417          | 84.7854 | 77.4226 | 84.2457 | 84.2652   | 15.4865 |
| 0.0088        | 7.9   | 14000 | 0.6445          | 84.7809 | 77.4171 | 84.2396 | 84.2591   | 15.4852 |


### Framework versions

- Transformers 4.28.1
- Pytorch 1.11.0a0+b6df043
- Datasets 2.12.0
- Tokenizers 0.13.3

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "metrics": ["rouge"], "model-index": [{"name": "mt5-large-fce-e8-b16", "results": []}]}
transformers pytorch tensorboard mt5 text2text-generation generated_from_trainer license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #tensorboard #mt5 #text2text-generation #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

mt5-large-fce-e8-b16
====================


This model is a fine-tuned version of google/mt5-large on the None dataset.
It achieves the following results on the evaluation set:


* Loss: 0.3526
* Rouge1: 84.5329
* Rouge2: 76.3656
* Rougel: 83.9027
* Rougelsum: 83.9238
* Gen Len: 15.4614


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.001
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adafactor
* lr\_scheduler\_type: linear
* num\_epochs: 8


### Training results



### Framework versions


* Transformers 4.28.1
* Pytorch 1.11.0a0+b6df043
* Datasets 2.12.0
* Tokenizers 0.13.3



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.001
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adafactor
* lr\_scheduler\_type: linear
* num\_epochs: 8 ### Training results ### Framework versions


* Transformers 4.28.1
* Pytorch 1.11.0a0+b6df043
* Datasets 2.12.0
* Tokenizers 0.13.3
TAGS
#transformers #pytorch #tensorboard #mt5 #text2text-generation #generated_from_trainer #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 0.001
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adafactor
* lr\_scheduler\_type: linear
* num\_epochs: 8 ### Training results ### Framework versions


* Transformers 4.28.1
* Pytorch 1.11.0a0+b6df043
* Datasets 2.12.0
* Tokenizers 0.13.3