We replicated the Amber model utilizing the Llama2-7B configuration. The tokenized data from Amber, originally processed with a sequence length of 2048, was further detokenized and re-encoded to a sequence length of 4096 as per the preprocessing strategy of Megatron-LM (Shoeybi et al., 2020). In aligning with the Llama2 model's architecture and hyperparameters, we adopted a sequence length of 4096 and a batch size of 4 million tokens. In order to increase the training efficiency, we incorporate GQA technologies, utilizing a configuration with 32 attention heads and a group size of 4.