FrankenMonarch-7B

Description

This repo contains GGUF format model files for mlabonne's FrankenMonarch-7B .

Name	Quant method	Bits	Size	Max RAM required	Use case
frankenmonarch-7b.Q2_K.gguf	Q2_K	2	2.72 GB	5.22 GB	significant quality loss - not recommended for most purposes
frankenmonarch-7b.Q3_K_M.gguf	Q3_K_M	3	3.52 GB	6.02 GB	very small, high quality loss
FrankenMonarch-7B.Q4_K_S.gguf	Q4_K_S	4	4.14 GB	6.64 GB	small, greater quality loss
frankenmonarch-7b.Q4_K_M.gguf	Q4_K_M	4	4.37 GB	6.87 GB	medium, balanced quality - recommended
frankenmonarch-7b.Q5_K_M.gguf	Q5_K_M	5	5.13 GB	7.63 GB	large, very low quality loss - recommended
frankenmonarch-7b.Q6_K.gguf	Q6_K	6	5.94 GB	8.44 GB	very large, extremely low quality loss
frankenmonarch-7b.Q8_0.gguf	Q8_0	8	7.70 GB	10.20 GB	very large, extremely low quality loss - not recommended