MichelBartelsDeepset
commited on
Commit
•
18c01a6
1
Parent(s):
930271a
Update README.md
Browse files
README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1 |
---
|
2 |
language: en
|
3 |
datasets:
|
4 |
-
-
|
5 |
license: mit
|
6 |
thumbnail: https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg
|
7 |
tags:
|
@@ -19,12 +19,7 @@ tags:
|
|
19 |
**Published**: Apr 21st, 2021
|
20 |
|
21 |
## Details
|
22 |
-
-
|
23 |
-
- The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published [online](https://deepset.ai/germanquad).
|
24 |
-
- The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536answers, because we removed 76 wrong answers.
|
25 |
-
- In addition to the annotations in GermanQuAD, haystack's distillation feature was used for training. deepset/roberta-large-squad2 was used as the teacher model.
|
26 |
-
|
27 |
-
See https://deepset.ai/germanquad for more details and dataset download in SQuAD format.
|
28 |
|
29 |
## Hyperparameters
|
30 |
```
|
@@ -38,11 +33,6 @@ temperature = 1.5
|
|
38 |
distillation_loss_weight = 0.75
|
39 |
```
|
40 |
## Performance
|
41 |
-
We evaluated the extractive question answering performance on the SQuAD v2 dev set.
|
42 |
-
Model types and training data are included in the model name.
|
43 |
-
For finetuning XLM-Roberta, we use the English SQuAD v2.0 dataset.
|
44 |
-
The GELECTRA models are warm started on the German translation of SQuAD v1.1 and finetuned on \\\\germanquad.
|
45 |
-
The human baseline was computed for the 3-way test set by taking one answer as prediction and the other two as ground truth.
|
46 |
```
|
47 |
"exact": 79.8366040596311
|
48 |
"f1": 83.916407079888
|
|
|
1 |
---
|
2 |
language: en
|
3 |
datasets:
|
4 |
+
- squad_v2
|
5 |
license: mit
|
6 |
thumbnail: https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg
|
7 |
tags:
|
|
|
19 |
**Published**: Apr 21st, 2021
|
20 |
|
21 |
## Details
|
22 |
+
- haystack's distillation feature was used for training. deepset/roberta-large-squad2 was used as the teacher model.
|
|
|
|
|
|
|
|
|
|
|
23 |
|
24 |
## Hyperparameters
|
25 |
```
|
|
|
33 |
distillation_loss_weight = 0.75
|
34 |
```
|
35 |
## Performance
|
|
|
|
|
|
|
|
|
|
|
36 |
```
|
37 |
"exact": 79.8366040596311
|
38 |
"f1": 83.916407079888
|