Learning Rate during pretraining
#58
by
shuyuej
- opened
May I know the initial learning rate
and scheduler
settings during the pre-training process?
Thank you very much, and have a nice day!
Best regards,
Shuyue
July 18th, 2024
May I also know the max_grad_norm
during the pre-training process?
Thank you very much, and have a nice day!
Best regards,
Shuyue
July 18th, 2024