Update README.md

README.md

@@ -35,11 +35,16 @@ Can I ask a question?<|im_end|>
 
 ## Support
 
-In order to inference this model you will have to use Aphrodite or vLLM as llama.cpp has not yet merged the required pull request to fix llama3.1 rope_freqs not respecting custom head_dim - You can however get around this by quanting the model yourself with the following fixes for a working GGUF. However, it will be stuck at 8k context until [this PR](https://github.com/ggerganov/llama.cpp/pull/9141) is merged.
+To run inference on this model, you'll need to use Aphrodite or vLLM, as llama.cpp hasn't yet merged the required pull request to fix the llama3.1 rope_freqs issue with custom head dimensions. 
 
-1. Remove `"rope_scaling": {}` from `config.json`
+However, you can work around this by quantizing the model yourself to create a functional GGUF file. Note that until [this PR](https://github.com/ggerganov/llama.cpp/pull/9141) is merged, the context will be limited to 8k tokens.
+
+To create a working GGUF file, make the following adjustments:
+
+1. Remove the `"rope_scaling": {}` entry from `config.json`
 2. Change `"max_position_embeddings"` to `8192` in `config.json`
 
+These modifications should allow you to use the model with llama.cpp, albeit with the mentioned context limitation.
 
 ## Credits