darkproger
commited on
Commit
•
0011f8d
1
Parent(s):
c97ed7e
Update README.md
Browse files
README.md
CHANGED
@@ -95,7 +95,17 @@ You can download the [Mistral-7B-v0.1 base model in the GGUF format](https://hug
|
|
95 |
and use `ggml-adapter-model.bin` from this repository like this:
|
96 |
|
97 |
```
|
98 |
-
./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood [/INST]" --lora ./ggml-adapter-model.bin
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
99 |
```
|
100 |
|
101 |
### Training Dataset and Resources
|
|
|
95 |
and use `ggml-adapter-model.bin` from this repository like this:
|
96 |
|
97 |
```
|
98 |
+
./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood? [/INST]" --lora ./ggml-adapter-model.bin
|
99 |
+
```
|
100 |
+
|
101 |
+
### Running the model with mlx-lm
|
102 |
+
|
103 |
+
We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
|
104 |
+
|
105 |
+
You can run the model using [mlx-lm](https://pypi.org/project/mlx-lm/):
|
106 |
+
|
107 |
+
```
|
108 |
+
python -m mlx_lm.generate --model lang-uk/dragoman-4bit --prompt '[INST] who holds this neighborhood? [/INST]' --temp 0 --max-tokens 100
|
109 |
```
|
110 |
|
111 |
### Training Dataset and Resources
|