Text Generation
PEFT
Safetensors
Ukrainian
English
translation
Eval Results
darkproger commited on
Commit
a2f7ac1
1 Parent(s): 0011f8d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -9
README.md CHANGED
@@ -87,25 +87,30 @@ outputs = model.generate(**input_ids)
87
  print(tokenizer.decode(outputs[0]))
88
  ```
89
 
90
- ### Running the model with llama.cpp
91
 
92
- We converted Dragoman PT adapter into the [GGUF format](https://huggingface.co/lang-uk/dragoman/blob/main/ggml-adapter-model.bin).
93
 
94
- You can download the [Mistral-7B-v0.1 base model in the GGUF format](https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF) (e.g. mistral-7b-v0.1.Q4_K_M.gguf)
95
- and use `ggml-adapter-model.bin` from this repository like this:
 
 
96
 
97
  ```
98
- ./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood? [/INST]" --lora ./ggml-adapter-model.bin
99
  ```
100
 
101
- ### Running the model with mlx-lm
102
 
103
- We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
104
 
105
- You can run the model using [mlx-lm](https://pypi.org/project/mlx-lm/):
 
 
 
 
 
106
 
107
  ```
108
- python -m mlx_lm.generate --model lang-uk/dragoman-4bit --prompt '[INST] who holds this neighborhood? [/INST]' --temp 0 --max-tokens 100
109
  ```
110
 
111
  ### Training Dataset and Resources
 
87
  print(tokenizer.decode(outputs[0]))
88
  ```
89
 
90
+ ### Running the model with mlx-lm on an Apple computer
91
 
 
92
 
93
+ We merged Dragoman PT adapter into the base model and uploaded the quantized version of the model into https://huggingface.co/lang-uk/dragoman-4bit.
94
+
95
+ You can run the model using [mlx-lm](https://pypi.org/project/mlx-lm/).
96
+
97
 
98
  ```
99
+ python -m mlx_lm.generate --model lang-uk/dragoman-4bit --prompt '[INST] who holds this neighborhood? [/INST]' --temp 0 --max-tokens 100
100
  ```
101
 
102
+ MLX is a recommended way of using the language model on an Apple computer with an M1 chip and newer.
103
 
 
104
 
105
+ ### Running the model with llama.cpp
106
+
107
+ We converted Dragoman PT adapter into the [GGUF format](https://huggingface.co/lang-uk/dragoman/blob/main/ggml-adapter-model.bin).
108
+
109
+ You can download the [Mistral-7B-v0.1 base model in the GGUF format](https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF) (e.g. mistral-7b-v0.1.Q4_K_M.gguf)
110
+ and use `ggml-adapter-model.bin` from this repository like this:
111
 
112
  ```
113
+ ./main -ngl 32 -m mistral-7b-v0.1.Q4_K_M.gguf --color -c 4096 --temp 0 --repeat_penalty 1.1 -n -1 -p "[INST] who holds this neighborhood? [/INST]" --lora ./ggml-adapter-model.bin
114
  ```
115
 
116
  ### Training Dataset and Resources