Update README.md

README.md

@@ -1,3 +1,45 @@
----
-license: cc-by-nc-4.0
----
+# Logion: Machine Learning for Greek Philology
+
+The most advanced Ancient Greek BERT model trained to date! Read the paper on [arxiv](https://arxiv.org/abs/2305.01099) by Charlie Cowen-Breen, Creston Brooks, Johannes Haubold, and Barbara Graziosi. 
+
+Originally based on the pre-trained weights and tokenizer made available by Pranaydeep Singh's [Ancient Greek BERT](https://huggingface.co/pranaydeeps/Ancient-Greek-BERT), we train on a corpus of over 70 million words of premodern Greek.
+
+Further information on this project and code for beam-searching over multiple masked tokens can be found on [GitHub](https://github.com/charliecb/Logion).
+
+We're adding more models trained with cleaner data and different tokenizations - keep an eye out! 
+
+## How to use
+
+Requirements:
+
+```python
+pip install transformers
+```
+
+Load the model and tokenizer directly from the HuggingFace Model Hub:
+
+
+```python
+from transformers import BertTokenizer, BertForMaskedLM
+tokenizer = BertTokenizer.from_pretrained("cabrooks/LOGION-base")
+model = BertForMaskedLM.from_pretrained("cabrooks/LOGION-base")  
+```
+
+## Model pre-training and tokenizer
+
+The model was initialized from Pranaydeep Singh's [Ancient Greek BERT](https://huggingface.co/pranaydeeps/Ancient-Greek-BERT), which itself used a [Modern Greek BERT](https://huggingface.co/nlpaueb/bert-base-greek-uncased-v1) as pre-training.
+Singh's Ancient Greek BERT was trained on data pulled from First1KGreek Project, Perseus Digital Library, PROIEL Treebank, and Gorman's Treebank. We train futher on over 70 million words of premodern Greek, which we are happy to make available upon request. For more information, please see footnote 2 on the [arxiv paper](https://arxiv.org/abs/2305.01099). Please also refer to this paper for details on training and evaluation. 
+
+
+## Cite
+
+If you use this model in your research, please cite the paper:
+
+```
+@inproceedings{logion-base,
+author = {Cowen-Breen, Charlie and Brooks, Creston and Haubold, Johannes and Graziosi, Barbara},
+title = {Logion: Machine Learning for Greek Philology},
+year = {2023},
+url = {https://arxiv.org/abs/2305.01099}
+}
+```