**Stage 2 (new output embeddings):** Our goal is to enhance the model's proficiency in accurately generating new tokens across various contexts by solely adjusting the output embeddings (lm_head). The decision to freeze all other parameters stems from the model's current unstable state. Allowing both input and output embeddings to be trained simultaneously would complicate achieving convergence, thus hindering the model's progress toward optimal performance. By freezing most of the parameters, we achieve more stable convergence. Moreover, this approach significantly reduces the training time, as it eliminates the necessity for back-propagation through the other layers.