GGUF conversion
#1
by
WillRanger
- opened
Hey ! Could you share the codes you used for merging the adapter to the model, and to convert it to GGUF.
I used the merged model (not adapter) as source.
Would recommend Maxime Labonne's content to learn about the conversion process: https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html
This Colab notebook is a good place to start: https://colab.research.google.com/drive/1P646NEg33BZy4BfLDNpTz0V0lwIU3CHu?usp=sharing
Oh ok, do you think it is possible to take a merged model in GPTQ format and convert it to GGUF ?