Weird outputs
#4
by
lazyDataScientist
- opened
Getting weird outputs from the model.
Input:
Hello!
Output:
})
})
})
Using llama.cpp with alpaca template (and tested with a few others, no success).
Other Params used
{ 'temperature': 0.7,
'top_p': 0.9,
'top_k': 20,
'repetition_penalty': 1.15}
You're using the gguf? I mean at least in ooba with llama cpp, I had to do this for soliloquy on which this finetune is based:
Since the context is stretched by 3, I had to put a 3 there (if I put 1, I get ;;;;; or something similar). I just tried 3SOME and it worked fine for me (although slower than other similar llama-3 quants, probably due to the context thing)