instruction-pretrain

/

instruction-synthesizer

like 67

Thanks for your interest. We plan to share the training code soon. In the meantime, could you specify the problems you encountered when training the synthesizer yourself? We may be able to help you quickly before the code release.

Once the dataset in generated can you share the code for pretraining

Okay, we will try to release the code as soon as possible.

hello, any progress?

Sorry for the late response. We provide guidance on data formulation and loss calculation in ft-instruction-synthesizer-collection. Beyond that, our fine-tuning process is the same as regular fine-tuning of language models. Therefore, we recommend using popular open-source code like Llama-factory for fine-tuning, which might be even easier to use than our own code.

If you still encounter implementation difficulties, please don't hesitate to contact us.