Update README.md
Browse files
README.md
CHANGED
@@ -61,7 +61,7 @@ Also note that Japanese Wikipedia was duplicated 10 times to make the total size
|
|
61 |
## Training procedure
|
62 |
|
63 |
We first segmented texts in the corpora into words using [Juman++ 2.0.0-rc3](https://github.com/ku-nlp/jumanpp/releases/tag/v2.0.0-rc3) for whole word masking.
|
64 |
-
Then, we built a sentencepiece model with
|
65 |
|
66 |
We tokenized raw corpora into character-level subwords using the sentencepiece model and trained the Japanese DeBERTa model using [transformers](https://github.com/huggingface/transformers) library.
|
67 |
The training took one day using 8 NVIDIA A100-SXM4-40GB GPUs.
|
|
|
61 |
## Training procedure
|
62 |
|
63 |
We first segmented texts in the corpora into words using [Juman++ 2.0.0-rc3](https://github.com/ku-nlp/jumanpp/releases/tag/v2.0.0-rc3) for whole word masking.
|
64 |
+
Then, we built a sentencepiece model with 22,012 tokens including all characters that appear in the training corpus.
|
65 |
|
66 |
We tokenized raw corpora into character-level subwords using the sentencepiece model and trained the Japanese DeBERTa model using [transformers](https://github.com/huggingface/transformers) library.
|
67 |
The training took one day using 8 NVIDIA A100-SXM4-40GB GPUs.
|