royleibov commited on
Commit
7033f8d
1 Parent(s): ddd9df0

Add ZipNN text

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -46,6 +46,20 @@ zipnn_hf()
46
  tokenizer = AutoTokenizer.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
47
  model = AutoModelForCausalLM.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
48
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
 
50
  # Model Card for Granite-7b-lab [Paper](https://arxiv.org/abs/2403.01081)
51
 
 
46
  tokenizer = AutoTokenizer.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
47
  model = AutoModelForCausalLM.from_pretrained("royleibov/granite-7b-instruct-ZipNN-Compressed")
48
  ```
49
+ ### ZipNN
50
+ ZipNN also allows you to seemlessly save local disk space in your cache after the model is downloaded.
51
+
52
+ To compress the cached model, simply run:
53
+ ```bash
54
+ python zipnn_compress_path.py safetensors --model royleibov/granite-7b-instruct-ZipNN-Compressed --hf_cache
55
+ ```
56
+
57
+ The model will be decompressed automatically and safely as long as `zipnn_hf()` is added at the top of the file like in the [example above](#use-this-model).
58
+
59
+ To decompress manualy, simply run:
60
+ ```bash
61
+ python zipnn_decompress_path.py --model royleibov/granite-7b-instruct-ZipNN-Compressed --hf_cache
62
+ ```
63
 
64
  # Model Card for Granite-7b-lab [Paper](https://arxiv.org/abs/2403.01081)
65