Language Model Training DynamicsInvestigating the training dynamics of language models is essential for understanding their behavior throughout the training process. This research includes studying internal representations (Saphra and Lopez, 2018), the acquisition of linguistic knowledge (Choshen et al., 2021; Liu et al., 2021), and the phenomenon of grokking (Power et al., 2022).