EOS should be 32000

#4
by TheBloke - opened

EOS should be 32000, otherwise generation doesn't terminate on the first <|endofturn|> token, making it seem like generation has frozen (in fact it's generating endless <|endofturn|> tokens until it reaches the max token limit)

Reference for the issue: https://huggingface.co/TheBloke/openchat_3.5-GGUF/discussions/1#6544063ae5c235427a3abde9

imone changed pull request status to merged

Sign up or log in to comment