How did you finetuned it for translation ?
Hello, I want to make a model translate and understand good my language (romanian) ...what is the recipe for that ? I tryed with a bi lingual translation dataset BUT it become dumber after fine tune ... not smarter :( . Thank you if you can help me with a guidance or an idea . Catalin
Hello, I want to make a model translate and understand good my language (romanian) ...what is the recipe for that ? I tryed with a bi lingual translation dataset BUT it become dumber after fine tune ... not smarter :( . Thank you if you can help me with a guidance or an idea . Catalin
Try to finetune it with a lot of text in your language, in your example of Romanian, you can use this awesome dataset:
https://huggingface.co/datasets/CohereForAI/aya_collection_language_split/viewer/romanian
In addition I'd suggest reading different papers about the subject, here are a few I found interesting:
https://arxiv.org/pdf/2404.10922v1
https://www.researchgate.net/publication/369924463_Multilingual_Machine_Translation_with_Large_Language_Models_Empirical_Results_and_Analysis
It might takes a few shots of trial and error, there's no a single formula so far.