# OPT-125M finetuned Portuguese 

Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.

###### Hyper-parameters


- learning_rate = 5e-5
- batch_size = 32
- warmup = 500
- seq_length = 512
- num_train_epochs = 2.0

With an A100 with 40GB of RAM, the training took around 3 hours

**Perplexity:** 9.4

## Sample Use

```python
from transformers import pipeline
generator = pipeline('text-generation', model='Mirelle/opt-125M-pt-br-finetuned', max_length=100, do_sample=True)
generator("Em uma bela manhã de")
```
{"language": ["pt"], "license": "mit", "library_name": "transformers", "datasets": ["mc4"], "metrics": ["perplexity"], "widget": [{"text": "Em uma bela manh\u00e3 de"}, {"text": "Em uma cidade t\u00e3o grande como"}, {"text": "Maria e Joana s\u00e3o"}], "pipeline_tag": "text-generation"}
transformers pytorch opt text-generation pt dataset:mc4 license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #opt #text-generation #pt #dataset-mc4 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

# OPT-125M finetuned Portuguese 

Fine-tuning the OPT-125M model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.

###### Hyper-parameters


- learning_rate = 5e-5
- batch_size = 32
- warmup = 500
- seq_length = 512
- num_train_epochs = 2.0

With an A100 with 40GB of RAM, the training took around 3 hours

Perplexity: 9.4

## Sample Use


# OPT-125M finetuned Portuguese 

Fine-tuning the OPT-125M model on a reduced corpus of mc4-Portuguese with approximately 300M tokens. ###### Hyper-parameters


- learning_rate = 5e-5
- batch_size = 32
- warmup = 500
- seq_length = 512
- num_train_epochs = 2.0

With an A100 with 40GB of RAM, the training took around 3 hours

Perplexity: 9.4 ## Sample Use
TAGS
#transformers #pytorch #opt #text-generation #pt #dataset-mc4 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # OPT-125M finetuned Portuguese 

Fine-tuning the OPT-125M model on a reduced corpus of mc4-Portuguese with approximately 300M tokens. ###### Hyper-parameters


- learning_rate = 5e-5
- batch_size = 32
- warmup = 500
- seq_length = 512
- num_train_epochs = 2.0

With an A100 with 40GB of RAM, the training took around 3 hours

Perplexity: 9.4 ## Sample Use