LoftQ
/

Llama-2-70b-hf-4bit-64rank

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

LoftQ commited on May 3

Commit

bdad119

•

1 Parent(s): 3c51d98

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -21,7 +21,7 @@ The backbone is under `LoftQ/Llama-2-70b-hf-4bit-64rank` and LoRA adapters are u
 - Stored format: `torch.bfloat16`
 - Size: ~ 140 GiB
 - Loaded format: bitsandbytes nf4
-- Size loaded on GPU: ~ 46 GiB
 ### LoRA adapters
 - rank: 64

 - Stored format: `torch.bfloat16`
 - Size: ~ 140 GiB
 - Loaded format: bitsandbytes nf4
+- Size loaded on GPU: ~ 43 GiB
 ### LoRA adapters
 - rank: 64