\(\bullet\)_Coding_. To improve the coding ability of LLMs, CodeLLAMA [235] is developed based on LLaMA 2 [99] (2T general tokens \(\rightarrow\) 500B code-heavy tokens), aiming to improve the code generation ability and retain natural language understanding skills. CodeLLaMA also provides a version that is further specialized to a certain programming language, namely CodeLLaMA-Python (2T general tokens \(\rightarrow\) 500B code-heavy tokens \(\rightarrow\) 100B Python-heavy tokens).