Gemma 2B model finetuning not working with <s> and </s>

#99
by nicyscaria - opened

Gemma 2B model finetuning is not working with s and /s, within <> (not displayed on the page directly) as bos and eos tokens and with [INST] and [/INST] for separating instruction from output. Is it mandatory to use the template in Gemma-it models?

nicyscaria changed discussion title from Gemma 2B model finetuning not working with s and /s, within <> to Gemma 2B model finetuning not working with <s> and </s>
Google org

Hi @nicyscaria , It's not mandatory to use the specified template in Gemma-it models. Alternative tokenization and separation strategies may be more suitable for your specific task. Thank you.

Sign up or log in to comment