michaelfeil
commited on
Commit
•
e76f5e7
1
Parent(s):
9fbacf4
Upload VMware/open-llama-7b-open-instruct ctranslate fp16 weights
Browse files- README.md +3 -3
- model.bin +2 -2
- vocabulary.json +0 -0
README.md
CHANGED
@@ -17,14 +17,14 @@ Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on
|
|
17 |
|
18 |
quantized version of [VMware/open-llama-7b-open-instruct](https://huggingface.co/VMware/open-llama-7b-open-instruct)
|
19 |
```bash
|
20 |
-
pip install hf-hub-ctranslate2>=2.0.8 ctranslate2>=3.
|
21 |
```
|
22 |
-
Converted on 2023-06-
|
23 |
```
|
24 |
ct2-transformers-converter --model VMware/open-llama-7b-open-instruct --output_dir /home/michael/tmp-ct2fast-open-llama-7b-open-instruct --force --copy_files README.md tokenizer_config.json generation_config.json special_tokens_map.json .gitattributes --quantization int8_float16 --trust_remote_code
|
25 |
```
|
26 |
|
27 |
-
Checkpoint compatible to [ctranslate2>=3.
|
28 |
and [hf-hub-ctranslate2>=2.0.8](https://github.com/michaelfeil/hf-hub-ctranslate2)
|
29 |
- `compute_type=int8_float16` for `device="cuda"`
|
30 |
- `compute_type=int8` for `device="cpu"`
|
|
|
17 |
|
18 |
quantized version of [VMware/open-llama-7b-open-instruct](https://huggingface.co/VMware/open-llama-7b-open-instruct)
|
19 |
```bash
|
20 |
+
pip install hf-hub-ctranslate2>=2.0.8 ctranslate2>=3.16.0
|
21 |
```
|
22 |
+
Converted on 2023-06-15 using
|
23 |
```
|
24 |
ct2-transformers-converter --model VMware/open-llama-7b-open-instruct --output_dir /home/michael/tmp-ct2fast-open-llama-7b-open-instruct --force --copy_files README.md tokenizer_config.json generation_config.json special_tokens_map.json .gitattributes --quantization int8_float16 --trust_remote_code
|
25 |
```
|
26 |
|
27 |
+
Checkpoint compatible to [ctranslate2>=3.16.0](https://github.com/OpenNMT/CTranslate2)
|
28 |
and [hf-hub-ctranslate2>=2.0.8](https://github.com/michaelfeil/hf-hub-ctranslate2)
|
29 |
- `compute_type=int8_float16` for `device="cuda"`
|
30 |
- `compute_type=int8` for `device="cpu"`
|
model.bin
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:6643ff563356272cafee853b207df6c353af9241c0fbd6e0fdff7102d3e3470d
|
3 |
+
size 6744404022
|
vocabulary.json
ADDED
The diff for this file is too large to render.
See raw diff
|
|