Pre-train phi-1 on another programming language
#6
by
Kiki-sen
- opened
Hi,
According to your technical report, you only pre-train Phi-1 on Python corpus. So If I want to pre-train it for another programming language, what should I do?
Thanks
Hello @Kiki-sen !
To pre-train on another programming language, you can follow the same model architecture as depicted here and use transformers to conduct the training.
This should serve as a reference: https://huggingface.co/learn/nlp-course/chapter7/6?fw=pt.
Regards,
Gustavo.
gugarosa
changed discussion status to
closed