Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,9 @@ pipeline_tag: text-generation
|
|
| 19 |
- 1024 max_seq_len
|
| 20 |
- 파라미터 수: 210M
|
| 21 |
|
|
|
|
|
|
|
|
|
|
| 22 |
## 학습 환경 및 하이퍼파라미터
|
| 23 |
- TPU V2-8
|
| 24 |
- Learning Rate: 5e-4, Batch Size: 512(=64 accum x 8 devices), Scheduler: Linear, WarmUp: 1000 step
|
|
|
|
| 19 |
- 1024 max_seq_len
|
| 20 |
- 파라미터 수: 210M
|
| 21 |
|
| 22 |
+
### 성능 벤치마크
|
| 23 |
+
<img src="https://github.com/HeegyuKim/language-model/blob/63d8bd7cd39f25e87e0e376cdd18df3f8b460dee/image/benchmark0304.png?raw=true" />
|
| 24 |
+
|
| 25 |
## 학습 환경 및 하이퍼파라미터
|
| 26 |
- TPU V2-8
|
| 27 |
- Learning Rate: 5e-4, Batch Size: 512(=64 accum x 8 devices), Scheduler: Linear, WarmUp: 1000 step
|