EOS should be 32000
#4
by
TheBloke
- opened
EOS should be 32000, otherwise generation doesn't terminate on the first <|endofturn|> token, making it seem like generation has frozen (in fact it's generating endless <|endofturn|> tokens until it reaches the max token limit)
Reference for the issue: https://huggingface.co/TheBloke/openchat_3.5-GGUF/discussions/1#6544063ae5c235427a3abde9
imone
changed pull request status to
merged