sh2orc

/

Llama-3.1-Korean-8B-Instruct

like 10

Thank you so much for sharing your model! I’m currently working on a similar task, fine-tuning an LLM on Korean datasets using a continued fine-tuning approach. However, I’m running into some issues with catastrophic forgetting—particularly, the HumanEval score drops significantly after fine-tuning. I noticed that your fine-tuning datasets didn’t seem to include any programming data. Would you mind sharing any tips or suggestions you might have for tackling this problem?

Thanks a lot!