Quantizing this by following your article
#4
by
ritvikshandilya
- opened
Hey, I was trying to quantize it by following your article https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html but the tokenizer.model file is missing, can you help us how we can use your quantization tutorial (gguf) on colab fine tuning files?
This error happens because the model you want to quantize (llama-2-7b-meditext) doesn't have a tokenizer in its repo. You can simply download the tokenizer files from Llama-2 (https://huggingface.co/meta-llama/Llama-2-7b-hf/tree/main) and place them in your model's folder.