LLAma 3.1 8B f16 looks better then Q3_KM

pinned

by gopi87 - opened Jul 24

Jul 24

hi guys i just tested some coding task with llama 3.1 8B looks better then some 70B quant. i will check other quant

gopi87 changed discussion title from LLAma 3.1 8B f16 looks better the Q3_KM to LLAma 3.1 8B f16 looks better then Q3_KM Jul 24

MaziyarPanahi pinned discussion Jul 24

Owner Jul 24

Thank you for testing these quants! This is a very nice direct comparison to know when to choose 8B over 70B within the same VRAM requirements.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment