KnutJaegersberg
commited on
Commit
•
ef26f36
1
Parent(s):
a5231a1
Update README.md
Browse files
README.md
CHANGED
@@ -12,6 +12,16 @@ In this case the tokenizer is the yi_tokenizer, loading it requires trust_remote
|
|
12 |
|
13 |
Have some fun with this fellow.
|
14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
|
17 |
License
|
|
|
12 |
|
13 |
Have some fun with this fellow.
|
14 |
|
15 |
+
It can eat a lot of vram, depends on settings to make it useable on two 24 gb vram gpus:
|
16 |
+
|
17 |
+
Without fused attention, it's 27 gb vram, will need some if if yo do stuff.
|
18 |
+
|
19 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/63732ebbbd81fae2b3aaf3fb/1cbqKp55WhN4BQD337E-n.png)
|
20 |
+
|
21 |
+
You can also let if have fused attention and just reduce the max_seq_length to something way smaller yet still useful
|
22 |
+
|
23 |
+
![image.png](https://cdn-uploads.huggingface.co/production/uploads/63732ebbbd81fae2b3aaf3fb/JRi4sakPziGpmOFCBfcJS.png)
|
24 |
+
|
25 |
|
26 |
|
27 |
License
|