instruction-pretrain
commited on
Commit
•
21cd91a
1
Parent(s):
bbcfb36
Update README.md
Browse files
README.md
CHANGED
@@ -17,8 +17,9 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
|
|
17 |
</p>
|
18 |
|
19 |
**************************** **Updates** ****************************
|
|
|
20 |
* 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
|
21 |
-
* 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M
|
22 |
<p align='left'>
|
23 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
|
24 |
</p>
|
|
|
17 |
</p>
|
18 |
|
19 |
**************************** **Updates** ****************************
|
20 |
+
* 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/medicine-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
|
21 |
* 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
|
22 |
+
* 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
|
23 |
<p align='left'>
|
24 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/66711d2ee12fa6cc5f5dfc89/0okCfRkC6uALTfuNxt0Fa.png" width="500">
|
25 |
</p>
|