Gibberish output

#1
by Adriato - opened

I only get gibberish output in Oobabooga Text-gen from all Qwen-2 models. No other. Am I missing something?

if you're offloading to cuda, make sure to enable flash attention

Still gibberish with flash attention. Enabling no_offload_kqv did the trick, though. Using the Q6-K-quant. Thanks.

Sign up or log in to comment