# GrεTa

The paper [Exploring Language Models for Classical Philology](https://todo.com) is the first effort to systematically provide state-of-the-art language models for Classical Philology. GrεTa is a T5-base sized, monolingual, encoder-decoder variant.  

This model was trained in two stages. Initially, it was pre-trained on a recently acquired corpus that leverages OCR scans obtained from the [Internet Archive](https://archive.org/).  Subsequently, the model was further trained using data from the [Open Greek & Latin Project](https://opengreekandlatin.org/), the CLARIN corpus [Greek Medieval Texts](https://inventory.clarin.gr/corpus/890), and the [Patrologia Graeca](https://patristica.net/graeca/).


Further information can be found in our paper or in our [GitHub repository](https://github.com/Heidelberg-NLP/ancient-language-models).

## Usage
```python
from transformers import AutoTokenizer, AutoModelForConditionalGeneration

tokenizer = AutoTokenizer.from_pretrained('bowphs/GreTa')
model = AutoModelForConditionalGeneration.from_pretrained('bowphs/GreTa')
```
Please check out the awesome Hugging Face tutorials on how to fine-tune our models.

## Evaluation Results
When fine-tuned on data from [Universal Dependencies 2.10](https://universaldependencies.org/), GrεTa achieves the following results on the Ancient Greek Perseus dataset:

| Task | XPoS | UPoS  | UAS | LAS | Lemma |
|:--:|:--:|:--:|:--:|:--:|:--:|
|      |94.44|89.03|87.32|83.06|91.14|

Please note that the PoS tagging and dependency parsing results are obtained using _only_ the encoder component of the model.

## Contact
If you have any questions or problems, feel free to [reach out](mailto:riemenschneider@cl.uni-heidelberg.de).

## Citation
```bibtex
@incollection{riemenschneiderfrank:2023,
    address = "Toronto, Canada",
    author = "Riemenschneider, Frederick and Frank, Anette",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23)",
    note = "to appear",
    pubType = "incollection",
    publisher = "Association for Computational Linguistics",
    title = "Exploring Large Language Models for Classical Philology",
    url = "https://arxiv.org/abs/2305.13698",
    year = "2023",
    key = "riemenschneiderfrank:2023"
}
```
{"language": "grc", "license": "apache-2.0", "inference": false}
transformers pytorch tf jax t5 text2text-generation grc arxiv:2305.13698 license:apache-2.0 autotrain_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #tf #jax #t5 #text2text-generation #grc #arxiv-2305.13698 #license-apache-2.0 #autotrain_compatible #text-generation-inference #region-us 

GrεTa
=====


The paper Exploring Language Models for Classical Philology is the first effort to systematically provide state-of-the-art language models for Classical Philology. GrεTa is a T5-base sized, monolingual, encoder-decoder variant.


This model was trained in two stages. Initially, it was pre-trained on a recently acquired corpus that leverages OCR scans obtained from the Internet Archive. Subsequently, the model was further trained using data from the Open Greek & Latin Project, the CLARIN corpus Greek Medieval Texts, and the Patrologia Graeca.


Further information can be found in our paper or in our GitHub repository.


Usage
-----


Please check out the awesome Hugging Face tutorials on how to fine-tune our models.


Evaluation Results
------------------


When fine-tuned on data from Universal Dependencies 2.10, GrεTa achieves the following results on the Ancient Greek Perseus dataset:



Please note that the PoS tagging and dependency parsing results are obtained using *only* the encoder component of the model.


Contact
-------


If you have any questions or problems, feel free to reach out.



TAGS
#transformers #pytorch #tf #jax #t5 #text2text-generation #grc #arxiv-2305.13698 #license-apache-2.0 #autotrain_compatible #text-generation-inference #region-us 
