Update README.md

README.md

@@ -9,7 +9,7 @@ datasets:
 - GAIR/lima
 - WizardLM/WizardLM_evol_instruct_V2_196k
 ---
-# Instruction Pre-Training: Language Models are Supervised Multitask Learners
+# Instruction Pre-Training: Language Models are Supervised Multitask Learners (EMNLP 2024)
 This repo contains the **finance model developed from Llama3-8B** in our paper [Instruction Pre-Training: Language Models are Supervised Multitask Learners](https://huggingface.co/papers/2406.14491).
 
 We explore supervised multitask pre-training by proposing ***Instruction Pre-Training***, a framework that scalably augments massive raw corpora with instruction-response pairs to pre-train language models. The instruction-response pairs are generated by an efficient instruction synthesizer built on open-source models. ***Instruction Pre-Training* outperforms *Vanilla Pre-training* in both general pre-training from scratch and domain-adaptive continual pre-training.** In pre-training from scratch, *Instruction Pre-Training* not only improves pre-trained base models but also benefits more from further instruction tuning. **In continual pre-training, *Instruction Pre-Training* enables Llama3-8B to be comparable to or even outperform Llama3-70B.**
@@ -19,7 +19,9 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
 </p>
 
 **************************** **Updates** ****************************
-* 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/finance-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
+* 2024/9/20: Our paper has been accepted by EMNLP 2024 main conference🎉
+* 2024/9/11: Updated [FAQ on continual pre-training from Llama3](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
+* 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/medicine-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
 * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
 * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
 <p align='left'>
@@ -140,7 +142,7 @@ text_ids = tokenizer(text, add_special_tokens=False, **kwargs).input_ids
 ## Citation
 If you find our work helpful, please cite us:
 
-Instruction Pre-Training
+[Instruction Pre-Training](https://huggingface.co/papers/2406.14491) (EMNLP 2024)
 ```bibtex
 @article{cheng2024instruction,
   title={Instruction Pre-Training: Language Models are Supervised Multitask Learners},
@@ -150,7 +152,7 @@ Instruction Pre-Training
 }
 ```
 
-[Adapt LLM to Domains](https://huggingface.co/papers/2309.09530)
+[Adapt LLM to Domains](https://huggingface.co/papers/2309.09530)（ICLR 2024）
 ```bibtex
 @inproceedings{
 cheng2024adapting,