darkproger commited on
Commit
0011f8d
1 Parent(s): c97ed7e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -1
README.md CHANGED
@@ -95,7 +95,17 @@ You can download the [Mistral-7B-v0.1 base model in the GGUF format](https://hug
95
  and use `ggml-adapter-model.bin` from this repository like this:
96
 
97
  ```
98
- ./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood [/INST]" --lora ./ggml-adapter-model.bin
 
 
 
 
 
 
 
 
 
 
99
  ```
100
 
101
  ### Training Dataset and Resources
 
95
  and use `ggml-adapter-model.bin` from this repository like this:
96
 
97
  ```
98
+ ./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood? [/INST]" --lora ./ggml-adapter-model.bin
99
+ ```
100
+
101
+ ### Running the model with mlx-lm
102
+
103
+ We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
104
+
105
+ You can run the model using [mlx-lm](https://pypi.org/project/mlx-lm/):
106
+
107
+ ```
108
+ python -m mlx_lm.generate --model lang-uk/dragoman-4bit --prompt '[INST] who holds this neighborhood? [/INST]' --temp 0 --max-tokens 100
109
  ```
110
 
111
  ### Training Dataset and Resources