Update README.md

README.md

@@ -8,9 +8,9 @@ tags:
 ---
 # Chinese-CodeLlama-7B-PT
 
-We have further expanded the vocabulary based on Chinese-LLaMA-2-7B which from 599296 to 75548, and we pre-trained the model based on LoRA, including `embed_tokens` and `lm_head` layers.
+We have further expanded the vocabulary based on Chinese-LLaMA-2-7B which from 599296 to 75548, it is worth noting that the most of them are code tokens. And we pre-trained the model based on LoRA, including `embed_tokens` and `lm_head` layers.
 
-The training data contains approximately 400 million tokens, which from high-quality code dataset on HuggingFace.
+The training data contains approximately 400 million tokens which from high-quality code dataset on HuggingFace.
 
 In addition, we applied `memory_efficient_attention` to the pre-training, which saves us a lot of GPU memory space.