prompt format in readme does not match tokenizer_config.json
the prompt format in the readme appears to be chatml, while the format in tokenizer_config.json looks like mistral instruct. just a heads up.
one other thought - if you chose to use chatml, you should update the vocabulary to have a special token for <|im_start|> and <|im_end|>. if you just follow the chatml format without adding the new tokens it's a little confusing, and it breaks llm hosts that expect to see a single token to decide when to stop streaming. this would require retraining the model if it doesn't already have those, though, so maybe for a future release
Thanks for the heads up, v3 will definitely include the tokenizer fix!
@francislabounty on a related subject, anywhere we can watch your progress? the serpai twitter seems dead but want to make sure i'm up to date on latest developments and will catch V3 when it drops :)