# Model Card

An Arabic LLM derived from Google's mT5 multi-lingual model 

## Model Details

### Model Description

This is a smaller version of the google/mt5-base model with only Arabic and some English embeddings left.

The original model has 582M parameters, with 384M of them being input and output embeddings.
After shrinking the sentencepiece vocabulary from 250K to 30K (top 10K English and top 20K Arabic tokens) the number of model parameters reduced to 244M parameters, and model size reduced from 2.2GB to 0.9GB - 42% of the original one.

The creation of this model was inspired from David Dales'article "<a href="https://towardsdatascience.com/how-to-adapt-a-multilingual-t5-model-for-a-single-language-b9f94f3d9c90">How to adapt a multilingual T5 model for a single language</a>" in which mT5 was compressed to support Russian and English languages along with the source code.



- **Developed by:** Moustafa Banbouk
- **Model type:** Unsupervised LLM
- **Language(s) (NLP):** Arabic, English
- **License:** MIT


### Downstream Uses

Quesion Answering, Summarization, Classification ...

{"language": ["ar", "en"], "license": "mit", "tags": ["T5", "mT5", "Transformers"]}
transformers pytorch t5 text2text-generation T5 mT5 Transformers ar en license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #t5 #text2text-generation #T5 #mT5 #Transformers #ar #en #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

# Model Card

An Arabic LLM derived from Google's mT5 multi-lingual model 

## Model Details

### Model Description

This is a smaller version of the google/mt5-base model with only Arabic and some English embeddings left.

The original model has 582M parameters, with 384M of them being input and output embeddings.
After shrinking the sentencepiece vocabulary from 250K to 30K (top 10K English and top 20K Arabic tokens) the number of model parameters reduced to 244M parameters, and model size reduced from 2.2GB to 0.9GB - 42% of the original one.

The creation of this model was inspired from David Dales'article "<a href="URL to adapt a multilingual T5 model for a single language</a>" in which mT5 was compressed to support Russian and English languages along with the source code.



- Developed by: Moustafa Banbouk
- Model type: Unsupervised LLM
- Language(s) (NLP): Arabic, English
- License: MIT


### Downstream Uses

Quesion Answering, Summarization, Classification ...

# Model Card

An Arabic LLM derived from Google's mT5 multi-lingual model ## Model Details ### Model Description

This is a smaller version of the google/mt5-base model with only Arabic and some English embeddings left.

The original model has 582M parameters, with 384M of them being input and output embeddings.
After shrinking the sentencepiece vocabulary from 250K to 30K (top 10K English and top 20K Arabic tokens) the number of model parameters reduced to 244M parameters, and model size reduced from 2.2GB to 0.9GB - 42% of the original one.

The creation of this model was inspired from David Dales'article "<a href="URL to adapt a multilingual T5 model for a single language</a>" in which mT5 was compressed to support Russian and English languages along with the source code.



- Developed by: Moustafa Banbouk
- Model type: Unsupervised LLM
- Language(s) (NLP): Arabic, English
- License: MIT ### Downstream Uses

Quesion Answering, Summarization, Classification ...
TAGS
#transformers #pytorch #t5 #text2text-generation #T5 #mT5 #Transformers #ar #en #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # Model Card

An Arabic LLM derived from Google's mT5 multi-lingual model ## Model Details ### Model Description

This is a smaller version of the google/mt5-base model with only Arabic and some English embeddings left.

The original model has 582M parameters, with 384M of them being input and output embeddings.
After shrinking the sentencepiece vocabulary from 250K to 30K (top 10K English and top 20K Arabic tokens) the number of model parameters reduced to 244M parameters, and model size reduced from 2.2GB to 0.9GB - 42% of the original one.

The creation of this model was inspired from David Dales'article "<a href="URL to adapt a multilingual T5 model for a single language</a>" in which mT5 was compressed to support Russian and English languages along with the source code.



- Developed by: Moustafa Banbouk
- Model type: Unsupervised LLM
- Language(s) (NLP): Arabic, English
- License: MIT ### Downstream Uses

Quesion Answering, Summarization, Classification ...