Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ widget:
|
|
10 |
---
|
11 |
|
12 |
# gaELECTRA
|
13 |
-
[gaELECTRA](https://
|
14 |
|
15 |
### Limitations and bias
|
16 |
Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.
|
@@ -20,24 +20,23 @@ Some data used to pretrain gaBERT was scraped from the web which potentially con
|
|
20 |
If you use this model in your research, please consider citing our paper:
|
21 |
|
22 |
```
|
23 |
-
@
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
bibsource = {dblp computer science bibliography, https://dblp.org}
|
42 |
}
|
43 |
```
|
|
|
10 |
---
|
11 |
|
12 |
# gaELECTRA
|
13 |
+
[gaELECTRA](https://aclanthology.org/2022.lrec-1.511/) is an ELECTRA model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper. For fine-tuning this model on a token classification task, e.g. Named Entity Recognition, use the discriminator model.
|
14 |
|
15 |
### Limitations and bias
|
16 |
Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.
|
|
|
20 |
If you use this model in your research, please consider citing our paper:
|
21 |
|
22 |
```
|
23 |
+
@inproceedings{barry-etal-2022-gabert,
|
24 |
+
title = "ga{BERT} {---} an {I}rish Language Model",
|
25 |
+
author = "Barry, James and
|
26 |
+
Wagner, Joachim and
|
27 |
+
Cassidy, Lauren and
|
28 |
+
Cowap, Alan and
|
29 |
+
Lynn, Teresa and
|
30 |
+
Walsh, Abigail and
|
31 |
+
{\'O} Meachair, M{\'\i}che{\'a}l J. and
|
32 |
+
Foster, Jennifer",
|
33 |
+
booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference",
|
34 |
+
month = jun,
|
35 |
+
year = "2022",
|
36 |
+
address = "Marseille, France",
|
37 |
+
publisher = "European Language Resources Association",
|
38 |
+
url = "https://aclanthology.org/2022.lrec-1.511",
|
39 |
+
pages = "4774--4788",
|
40 |
+
abstract = "The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provides better representations for a downstream parsing task. We also show how different filtering criteria, vocabulary size and the choice of subword tokenisation model affect downstream performance. We compare the results of fine-tuning a gaBERT model with an mBERT model for the task of identifying verbal multiword expressions, and show that the fine-tuned gaBERT model also performs better at this task. We release gaBERT and related code to the community.",
|
|
|
41 |
}
|
42 |
```
|