darkproger
commited on
Commit
•
a2f7ac1
1
Parent(s):
0011f8d
Update README.md
Browse files
README.md
CHANGED
@@ -87,25 +87,30 @@ outputs = model.generate(**input_ids)
|
|
87 |
print(tokenizer.decode(outputs[0]))
|
88 |
```
|
89 |
|
90 |
-
### Running the model with
|
91 |
|
92 |
-
We converted Dragoman PT adapter into the [GGUF format](https://huggingface.co/lang-uk/dragoman/blob/main/ggml-adapter-model.bin).
|
93 |
|
94 |
-
|
95 |
-
|
|
|
|
|
96 |
|
97 |
```
|
98 |
-
|
99 |
```
|
100 |
|
101 |
-
|
102 |
|
103 |
-
We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
|
104 |
|
105 |
-
|
|
|
|
|
|
|
|
|
|
|
106 |
|
107 |
```
|
108 |
-
|
109 |
```
|
110 |
|
111 |
### Training Dataset and Resources
|
|
|
87 |
print(tokenizer.decode(outputs[0]))
|
88 |
```
|
89 |
|
90 |
+
### Running the model with mlx-lm on an Apple computer
|
91 |
|
|
|
92 |
|
93 |
+
We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
|
94 |
+
|
95 |
+
You can run the model using [mlx-lm](https://pypi.org/project/mlx-lm/).
|
96 |
+
|
97 |
|
98 |
```
|
99 |
+
python -m mlx_lm.generate --model lang-uk/dragoman-4bit --prompt '[INST] who holds this neighborhood? [/INST]' --temp 0 --max-tokens 100
|
100 |
```
|
101 |
|
102 |
+
MLX is a recommended way of using the language model on an Apple computer with an M1 chip and newer.
|
103 |
|
|
|
104 |
|
105 |
+
### Running the model with llama.cpp
|
106 |
+
|
107 |
+
We converted Dragoman PT adapter into the [GGUF format](https://huggingface.co/lang-uk/dragoman/blob/main/ggml-adapter-model.bin).
|
108 |
+
|
109 |
+
You can download the [Mistral-7B-v0.1 base model in the GGUF format](https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF) (e.g. mistral-7b-v0.1.Q4_K_M.gguf)
|
110 |
+
and use `ggml-adapter-model.bin` from this repository like this:
|
111 |
|
112 |
```
|
113 |
+
./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood? [/INST]" --lora ./ggml-adapter-model.bin
|
114 |
```
|
115 |
|
116 |
### Training Dataset and Resources
|