|
--- |
|
language: es |
|
license: cc-by-4.0 |
|
tags: |
|
- multilingual |
|
- bert |
|
pipeline_tag: fill-mask |
|
widget: |
|
- text: ¿Qué es la vida? Un [MASK]. |
|
--- |
|
|
|
# ALBERTI |
|
|
|
ALBERTI is a set of two BERT-based multilingual model for poetry. One for verses and another one for stanzas. This model has been further trained with the PULPO corpus for verses using [Flax](https://github.com/google/flax), including training scripts. |
|
|
|
This is part of the |
|
[Flax/Jax Community Week](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104), organised by [HuggingFace](https://huggingface.co/) and TPU usage sponsored by Google. |
|
|
|
## PULPO |
|
|
|
PULPO, the Prodigious Unannotated Literary Poetry Corpus, is a set of multilingual corpora of verses and stanzas with over 95M words. |
|
|
|
The following corpora has been downloaded using the [Averell](https://github.com/linhd-postdata/averell/) tool, developed by the [POSTDATA](https://postdata.linhd.uned.es/) team: |
|
|
|
### Spanish |
|
- [Disco v3](https://github.com/pruizf/disco) |
|
- [Corpus of Spanish Golden-Age Sonnets](https://github.com/bncolorado/CorpusSonetosSigloDeOro) |
|
- [Corpus general de poesía lírica castellana del Siglo de Oro](https://github.com/bncolorado/CorpusGeneralPoesiaLiricaCastellanaDelSigloDeOro) |
|
- [Gongocorpus](https://github.com/linhd-postdata/gongocorpus) - [source](http://obvil.sorbonne-universite.site/corpus/gongora/gongora_obra-poetica) |
|
### English |
|
- [Eighteenth-Century Poetry Archive (ECPA)](https://github.com/alhuber1502/ECPA) |
|
- [For better for verse](https://github.com/waynegraham/for_better_for_verse) |
|
### French |
|
- [Métrique en Ligne](https://crisco2.unicaen.fr/verlaine/index.php?navigation=accueil) - [source](https://github.com/linhd-postdata/metrique-en-ligne) |
|
### Italian |
|
- [Biblioteca italiana](https://github.com/linhd-postdata/biblioteca_italiana) - [source](http://www.bibliotecaitaliana.it/) |
|
### Czech |
|
- [Corpus of Czech Verse](https://github.com/versotym/corpusCzechVerse) |
|
### Portuguese |
|
- [Stichotheque](https://gitlab.com/stichotheque/stichotheque-pt) |
|
|
|
Also, we obtained the following corpora from these sources: |
|
### Spanish |
|
- [Poesi.as](https://github.com/linhd-postdata/poesi.as) - [source](http://www.poesi.as/) |
|
### English |
|
- [A Gutenberg Poetry Corpus](https://github.com/aparrish/gutenberg-poetry-corpus) |
|
### Arabic |
|
- [Arabic Poetry dataset](https://www.kaggle.com/ahmedabelal/arabic-poetry) |
|
### Chinese |
|
- [THU Chinese Classical Poetry Corpus](https://github.com/THUNLP-AIPoet/Datasets/tree/master/CCPC) |
|
### Finnish |
|
- [SKVR](https://github.com/sks190/SKVR) |
|
### German |
|
- [TextGrid Poetry Corpus](https://github.com/linhd-postdata/textgrid-poetry) - [source](https://textgrid.de/en/digitale-bibliothek) |
|
- [German Rhyme Corpus](https://github.com/tnhaider/german-rhyme-corpus) |
|
### Hungarian |
|
- [verskorpusz](https://github.com/ELTE-DH/verskorpusz) |
|
### Portuguese |
|
- [Poems in Portuguese](https://www.kaggle.com/oliveirasp6/poems-in-portuguese) |
|
### Russian |
|
- [19 000 Russian poems](https://www.kaggle.com/grafstor/19-000-russian-poems) |
|
|
|
## Team members |
|
|
|
- Álvaro Pérez ([alvp](https://huggingface.co/alvp)) |
|
- Javier de la Rosa ([versae](https://huggingface.co/versae)) |
|
- Aitor Díaz ([aitordiaz](https://huggingface.co/aitordiaz)) |
|
- Elena González-Blanco |
|
- Salvador Ros ([salva](https://huggingface.co/salva)) |
|
|
|
## Useful links |
|
|
|
- [Community Week timeline](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104#summary-timeline-calendar-6) |
|
- [Community Week README](https://github.com/huggingface/transformers/blob/master/examples/research_projects/jax-projects/README.md) |
|
- [Community Week thread](https://discuss.huggingface.co/t/bertin-pretrain-roberta-large-from-scratch-in-spanish/7125) |
|
- [Community Week channel](https://discord.com/channels/858019234139602994/859113060068229190) |
|
- [Masked Language Modelling example scripts](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling) |
|
- [Model Repository](https://huggingface.co/flax-community/alberti-bert-base-multilingual-cased/) |
|
|
|
## Acknowledgments |
|
|
|
This project would not have been possible without the infrastructure and resources provided by HuggingFace and Google Cloud. Moreover, we want to thank POSTDATA Project (ERC-StG-679528) and the Computational Literary Studies Infrastructure (CLS INFRA No. 101004984) of the European Union's Horizon 2020 research and innovation programme for their support and time allowance. |
