Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ pipeline_tag: text-generation
|
|
10 |
<h2 style='text-align: center '><em>Open-access Multilingual Language Model based on BLOOM</em> </h2>
|
11 |
<h3 style='text-align: center '>Model Card</h3>
|
12 |
|
13 |
-
Version 1.0 /
|
14 |
|
15 |
This model is a joint collaboration between CKIP lab at Acedemia Sinica, MediaTek Research, and National Academy for Educational Research.
|
16 |
|
@@ -26,7 +26,7 @@ This model is a joint collaboration between CKIP lab at Acedemia Sinica, MediaTe
|
|
26 |
9. [Model Card Authors](#model-card-authors)
|
27 |
|
28 |
## Model Details
|
29 |
-
BLOOM-zh is a modification from [
|
30 |
BLOOM-zh is trained extendedly on larger amounts of Traditional Chinese text data while it still maintains its pretrained English ability.
|
31 |
|
32 |
|
@@ -88,7 +88,7 @@ BLOOM-zh is trained extendedly on larger amounts of Traditional Chinese text dat
|
|
88 |
|
89 |
**Compute infrastructure:**
|
90 |
|
91 |
-
* Hardware:
|
92 |
|
93 |
|
94 |
* Software:
|
@@ -99,7 +99,7 @@ BLOOM-zh is trained extendedly on larger amounts of Traditional Chinese text dat
|
|
99 |
|
100 |
* DeepSpeed ([Github link](https://github.com/microsoft/DeepSpeed))
|
101 |
|
102 |
-
* PyTorch (pytorch-1.
|
103 |
|
104 |
* apex ([Github link](https://github.com/NVIDIA/apex))
|
105 |
|
@@ -108,8 +108,6 @@ BLOOM-zh is trained extendedly on larger amounts of Traditional Chinese text dat
|
|
108 |
|
109 |
Details are provided in the [paper](https://arxiv.org/).
|
110 |
|
111 |
-
- Number of epochs: 1
|
112 |
-
|
113 |
- Dates: Feb. 2023
|
114 |
|
115 |
#### **Tokenization**
|
|
|
10 |
<h2 style='text-align: center '><em>Open-access Multilingual Language Model based on BLOOM</em> </h2>
|
11 |
<h3 style='text-align: center '>Model Card</h3>
|
12 |
|
13 |
+
Version 1.0 / 20.Feb.2023
|
14 |
|
15 |
This model is a joint collaboration between CKIP lab at Acedemia Sinica, MediaTek Research, and National Academy for Educational Research.
|
16 |
|
|
|
26 |
9. [Model Card Authors](#model-card-authors)
|
27 |
|
28 |
## Model Details
|
29 |
+
BLOOM-zh is a modification from [BLOOMZ](https://huggingface.co/bigscience/bloomz).
|
30 |
BLOOM-zh is trained extendedly on larger amounts of Traditional Chinese text data while it still maintains its pretrained English ability.
|
31 |
|
32 |
|
|
|
88 |
|
89 |
**Compute infrastructure:**
|
90 |
|
91 |
+
* Hardware: 2 A6000 48GB GPUs (1 node):
|
92 |
|
93 |
|
94 |
* Software:
|
|
|
99 |
|
100 |
* DeepSpeed ([Github link](https://github.com/microsoft/DeepSpeed))
|
101 |
|
102 |
+
* PyTorch (pytorch-1.12 w/ CUDA-11.3; see [Github link](https://github.com/pytorch/pytorch))
|
103 |
|
104 |
* apex ([Github link](https://github.com/NVIDIA/apex))
|
105 |
|
|
|
108 |
|
109 |
Details are provided in the [paper](https://arxiv.org/).
|
110 |
|
|
|
|
|
111 |
- Dates: Feb. 2023
|
112 |
|
113 |
#### **Tokenization**
|