macadeliccc commited on
Commit
dbddb2d
1 Parent(s): 8b881f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -5
README.md CHANGED
@@ -6,23 +6,34 @@ library_name: transformers
6
 
7
  ![laser_dolphin_image](./dolphin_moe.png)
8
 
9
- **New Version will be uploaded soon**
10
 
11
  Credit to Fernando Fernandes and Eric Hartford for their project [laserRMT](https://github.com/cognitivecomputations/laserRMT)
12
 
 
 
13
  This model is a medium-sized MoE implementation based on [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
14
 
15
- A 2x7b configuration offers better performance than a standard 7b model even if loaded in 4 bit. (9G VRAM)
16
 
17
- If this 2x7b model is loaded in 4 bit the hellaswag score is .8270 which is higher than the base model achieves on its own in full precision.
 
 
 
 
18
 
19
- The process is outlined in this [notebook](https://github.com/cognitivecomputations/laserRMT/blob/main/examples/laser-dolphin-mixtral-2x7b.ipynb)
 
 
 
 
20
 
21
  **These Quants will result in unpredicted behavior and I am working on new Quants as I have updated the model**
22
 
23
  Quatizations provided by [TheBloke](https://huggingface.co/TheBloke/laser-dolphin-mixtral-2x7b-dpo-GGUF)
24
 
25
-
 
 
26
 
27
  ## Code Example
28
  Switch the commented model definition to use in 4-bit. Should work with 9GB and still exceed the single 7B model by 5-6 points roughly
 
6
 
7
  ![laser_dolphin_image](./dolphin_moe.png)
8
 
9
+ **New Version out now!**
10
 
11
  Credit to Fernando Fernandes and Eric Hartford for their project [laserRMT](https://github.com/cognitivecomputations/laserRMT)
12
 
13
+ ## Overview
14
+
15
  This model is a medium-sized MoE implementation based on [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
16
 
 
17
 
18
+ ## Process
19
+
20
+ + The process is outlined in this [notebook](https://github.com/cognitivecomputations/laserRMT/blob/main/examples/laser-dolphin-mixtral-2x7b.ipynb)
21
+
22
+ + The mergekit_config is in the files.
23
 
24
+ + The models used in the configuration are not lasered, but the final product is. This is an update from the last version.
25
+
26
+ + This process is experimental. Your mileage may vary.
27
+
28
+ ## Quantizations
29
 
30
  **These Quants will result in unpredicted behavior and I am working on new Quants as I have updated the model**
31
 
32
  Quatizations provided by [TheBloke](https://huggingface.co/TheBloke/laser-dolphin-mixtral-2x7b-dpo-GGUF)
33
 
34
+ *Current [Quantizations](https://huggingface.co/macadeliccc/laser-dolphin-mixtral-2x7b-GGUF)*
35
+ - Q4_K_M
36
+ - Q5_K_M
37
 
38
  ## Code Example
39
  Switch the commented model definition to use in 4-bit. Should work with 9GB and still exceed the single 7B model by 5-6 points roughly