Hugging Face's logo

castorini
/
afriteva_small

Text2Text Generation
Transformers
PyTorch
t5
text-generation-inference
Inference Endpoints
Model card Files Community
2
afriteva_small / readme.md
ToluClassics's picture
ToluClassics
update readme
c8540f5
preview code
|
raw
history blame
1.65 kB

Hugging Face's logo

language:

  • om
  • am
  • rw
  • rn
  • ha
  • ig
  • pcm
  • so
  • sw
  • ti
  • yo
  • multilingual
  • T5

afriteva_small

Model desription

AfriTeVa small is a sequence to sequence model pretrained on 10 African languages

Languages

Afaan Oromoo(orm), Amharic(amh), Gahuza(gah), Hausa(hau), Igbo(igb), Nigerian Pidgin(pcm), Somali(som), Swahili(swa), Tigrinya(tig), Yoruba(yor)

More information on the model, dataset:

The model

  • 64M parameters encoder-decoder architecture (T5-like)
  • 6 layers, 8 attention heads and 512 token sequence length

The dataset

  • Multilingual: 10 African languages listed above
  • 143 Million Tokens (1GB of text data)
  • Tokenizer Vocabulary Size: 70,000 tokens

Intended uses & limitations

afriteva_small is pre-trained model and primarily aimed at being fine-tuned on multilingual sequence-to-sequence tasks. 

>>> from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

>>> tokenizer = AutoTokenizer.from_pretrained("castorini/afriteva_small")
>>> model = AutoModelForSeq2SeqLM.from_pretrained("castorini/afriteva_small")

>>> src_text = "Ó hùn ọ́ láti di ara wa bí?"
>>> tgt_text =  "Would you like to be?"

>>> model_inputs = tokenizer(src_text, return_tensors="pt")
>>> with tokenizer.as_target_tokenizer():
        labels = tokenizer(tgt_text, return_tensors="pt").input_ids

>>> model(**model_inputs, labels=labels) # forward pass

Training Procedure

For information on training procedures, please refer to the AfriTeVa paper or repository

BibTex entry and Citation info

coming soon ...