sayhan

/

gemma-7b-it-GGUF-quantized

like 19

Hi.
THX for this GGUF modells !
Settings on my RTX-3060 (12GB)
Model tab- Download model
sayhan/gemma-7b-it-GGUF-quantized
gemma-7b-it.Q5_K_M.gguf
Model loader
ctransformers
n-gpu-layers
100

Parameters tab - max_new_tokens
4096
temperature
0.3

Chat tab
Instruct

And seems give very good answers at 30 token/s,
snake.py not fully correct for first attempts.