Update README.md
Browse files
README.md
CHANGED
@@ -44,7 +44,8 @@ Mistral-Small-Instruct-2409 ๋ฒ ์ด์ค๋ชจ๋ธ์ ์ฌ์ฉํด์ H100-80G 8๊ฐ๋ฅผ ํต
|
|
44 |
9์ฒ๋ง๊ฑด์ ํ๊ธ ๋ด์ค ์ฝํผ์ค๋ฅผ ๊ธฐ์ค์ผ๋ก ๋ค์ํ ํ
์คํฌ๋ณ ํ๊ตญ์ด-์ค๊ตญ์ด-์์ด-์ผ๋ณธ์ด ๊ต์ฐจ ํ์ต ๋ฐ์ดํฐ์ ์ํ ๋ฐ ๋
ผ๋ฆฌํ๋จ ๋ฐ์ดํฐ๋ฅผ ํตํ์ฌ ํ์ค์ผ์ ์ธ์ด ๊ต์ฐจ ์ฆ๊ฐ ์ฒ๋ฆฌ์ ๋ณต์กํ ๋
ผ๋ฆฌ ๋ฌธ์ ์ญ์ ๋์ ๊ฐ๋ฅํ๋๋ก ํ๋ จํ ๋ชจ๋ธ์ด๋ค.<br>
|
45 |
-ํ ํฌ๋์ด์ ๋ ๋จ์ด ํ์ฅ ์์ด ๋ฒ ์ด์ค ๋ชจ๋ธ ๊ทธ๋๋ก ์ฌ์ฉ<br>
|
46 |
-๊ณ ๊ฐ ๋ฆฌ๋ทฐ๋ ์์
ํฌ์คํ
๊ณ ์ฐจ์ ๋ถ์ ๋ฐ ์ฝ๋ฉ๊ณผ ์๋ฌธ, ์ํ, ๋
ผ๋ฆฌํ๋จ ๋ฑ์ด ๊ฐํ๋ ๋ชจ๋ธ<br>
|
47 |
-
-
|
|
|
48 |
-Deepspeed Stage=3, rslora ๋ฐ BAdam Layer Mode ์ฌ์ฉ <br>
|
49 |
<br><br>
|
50 |
|
@@ -53,7 +54,8 @@ about 20% of total parameters Korean CPT(Continued-Pretraining)->SFT->DPO traini
|
|
53 |
It is a model that has been trained to handle Korean-Chinese-English-Japanese cross-training data and 90M korean news corpus and logic judgment data for various tasks to enable cross-fertilization processing and complex Korean logic & math problems. <br>
|
54 |
-Tokenizer uses the base model without word expansion<br>
|
55 |
-Models enhanced with high-dimensional analysis of customer reviews and social posts, as well as coding, writing, math and decision making<br>
|
56 |
-
-
|
|
|
57 |
-Deepspeed Stage=3, use rslora and BAdam Layer Mode<br>
|
58 |
<br><br>
|
59 |
|
|
|
44 |
9์ฒ๋ง๊ฑด์ ํ๊ธ ๋ด์ค ์ฝํผ์ค๋ฅผ ๊ธฐ์ค์ผ๋ก ๋ค์ํ ํ
์คํฌ๋ณ ํ๊ตญ์ด-์ค๊ตญ์ด-์์ด-์ผ๋ณธ์ด ๊ต์ฐจ ํ์ต ๋ฐ์ดํฐ์ ์ํ ๋ฐ ๋
ผ๋ฆฌํ๋จ ๋ฐ์ดํฐ๋ฅผ ํตํ์ฌ ํ์ค์ผ์ ์ธ์ด ๊ต์ฐจ ์ฆ๊ฐ ์ฒ๋ฆฌ์ ๋ณต์กํ ๋
ผ๋ฆฌ ๋ฌธ์ ์ญ์ ๋์ ๊ฐ๋ฅํ๋๋ก ํ๋ จํ ๋ชจ๋ธ์ด๋ค.<br>
|
45 |
-ํ ํฌ๋์ด์ ๋ ๋จ์ด ํ์ฅ ์์ด ๋ฒ ์ด์ค ๋ชจ๋ธ ๊ทธ๋๋ก ์ฌ์ฉ<br>
|
46 |
-๊ณ ๊ฐ ๋ฆฌ๋ทฐ๋ ์์
ํฌ์คํ
๊ณ ์ฐจ์ ๋ถ์ ๋ฐ ์ฝ๋ฉ๊ณผ ์๋ฌธ, ์ํ, ๋
ผ๋ฆฌํ๋จ ๋ฑ์ด ๊ฐํ๋ ๋ชจ๋ธ<br>
|
47 |
+
-32k ์ํ์ค ๊ธธ์ด<br>
|
48 |
+
-ํ์
์ฝ ์ง์<br>
|
49 |
-Deepspeed Stage=3, rslora ๋ฐ BAdam Layer Mode ์ฌ์ฉ <br>
|
50 |
<br><br>
|
51 |
|
|
|
54 |
It is a model that has been trained to handle Korean-Chinese-English-Japanese cross-training data and 90M korean news corpus and logic judgment data for various tasks to enable cross-fertilization processing and complex Korean logic & math problems. <br>
|
55 |
-Tokenizer uses the base model without word expansion<br>
|
56 |
-Models enhanced with high-dimensional analysis of customer reviews and social posts, as well as coding, writing, math and decision making<br>
|
57 |
+
-32k sequence length<br>
|
58 |
+
-Function calling<br>
|
59 |
-Deepspeed Stage=3, use rslora and BAdam Layer Mode<br>
|
60 |
<br><br>
|
61 |
|