Specifically, we train our models with a setup of 8 x NVIDIA H100 GPUs with 80GB memory each, utilizing 64 CPU cores. For EEVE-Korean-10.8B-v1.0, under bf16 precision, the training process is configured with a sequence of length 4096, gradient accumulation steps set to 4, and a micro-batch size of 8, whereas EEVE-Korean-2.8B-v1.0 adopts a sequence length of 2048, gradient accumulation of 16, and a micro-batch size of 16. We employ the AdamW (Loshchilov and Hutter, 2018) optimizer, paired with a cosine learning rate scheduler that includes a warmup phase of 10 steps. The learning rate for the 10.8B variant is set to 4e-5, while we used 2e-4 for the small model. We continued training at each stage until the loss converged, observing the loss converged before reaching 400 global steps, which signifies the efficiency of our training strategy. Though our training strategy involves 7 different stages, it is noteworthy that, for our 2.8B variant, the overall pre-training can be done in less than two days as optimizing only the output embeddings doesn't incur much computation.