Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,8 @@ widget:
|
|
| 9 |
- text: "今天[MASK]很好，我想去公园玩！"
|
| 10 |
---
|
| 11 |
## 介绍
|
| 12 |
-
|
|
|
|
| 13 |
|
| 14 |
训练逻辑参考了这里。https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/language_model/ernie-1.0
|
| 15 |
|
|
|
|
| 9 |
- text: "今天[MASK]很好，我想去公园玩！"
|
| 10 |
---
|
| 11 |
## 介绍
|
| 12 |
+
Pretrained model on 13G Chinese corpus(clue corpus small). Masked language modeling(MLM) and sentence order prediction(SOP) are used as training task.
|
| 13 |
+
在13g的clue corpus small数据集上进行的预训练，使用了`Whole Mask LM` 和 `SOP` 任务
|
| 14 |
|
| 15 |
训练逻辑参考了这里。https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/language_model/ernie-1.0
|
| 16 |
|