KnutJaegersberg commited on
Commit
ef26f36
1 Parent(s): a5231a1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -12,6 +12,16 @@ In this case the tokenizer is the yi_tokenizer, loading it requires trust_remote
12
 
13
  Have some fun with this fellow.
14
 
 
 
 
 
 
 
 
 
 
 
15
 
16
 
17
  License
 
12
 
13
  Have some fun with this fellow.
14
 
15
+ It can eat a lot of vram, depends on settings to make it useable on two 24 gb vram gpus:
16
+
17
+ Without fused attention, it's 27 gb vram, will need some if if yo do stuff.
18
+
19
+ ![image.png](https://cdn-uploads.huggingface.co/production/uploads/63732ebbbd81fae2b3aaf3fb/1cbqKp55WhN4BQD337E-n.png)
20
+
21
+ You can also let if have fused attention and just reduce the max_seq_length to something way smaller yet still useful
22
+
23
+ ![image.png](https://cdn-uploads.huggingface.co/production/uploads/63732ebbbd81fae2b3aaf3fb/JRi4sakPziGpmOFCBfcJS.png)
24
+
25
 
26
 
27
  License