Update README.md
Browse files
README.md
CHANGED
|
@@ -32,6 +32,20 @@ For code LoRA module: we train the English code LoRA with the Magicoder dataset
|
|
| 32 |
## Gate Training
|
| 33 |
We use gates to fuse different LoRA modules. We employ few-shot training and have released our training data for further details please refer to our GitHub.
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
# Citation
|
| 36 |
if you find our repo is helpful, please cite the following
|
| 37 |
```bibtex
|
|
|
|
| 32 |
## Gate Training
|
| 33 |
We use gates to fuse different LoRA modules. We employ few-shot training and have released our training data for further details please refer to our GitHub.
|
| 34 |
|
| 35 |
+
# Result
|
| 36 |
+
We have released the result for LoRAs and LoRA-Flow
|
| 37 |
+
|
| 38 |
+
| **Method** | | **MGSM (Math)** | | | | **HumanEval (Code)** | | | |
|
| 39 |
+
|-----------------------|-------|-------------------------------|---------|---------|---------|----------------------------------|---------|---------|---------|
|
| 40 |
+
| | | **Zh** | **Ru** | **Es** | **Avg.**| **Zh** | **Ru** | **Es** | **Avg.**|
|
| 41 |
+
| **Base Model** | | 4.4 | 3.2 | 2.4 | 3.3 | 0.0 | 0.0 | 2.4 | 0.8 |
|
| 42 |
+
| **Single LoRA** | Lang | 5.2 | 3.6 | 3.6 | 4.1 | 12.2 | 14.0 | 10.4 | 12.2 |
|
| 43 |
+
| | Task | 26.8 | 32.8 | 41.2 | 33.6 | 18.3 | 23.2 | 21.9 | 21.1 |
|
| 44 |
+
| **LoRA Fusion** | Avg | 12.8 | 10.4 | 18.4 | 13.9 | 17.1 | 17.7 | 18.3 | 17.7 |
|
| 45 |
+
| | LoRA-Hub | 20.8 | 28.4 | 36.8 | 28.7 | 19.5 | 21.3 | 20.1 | 20.3 |
|
| 46 |
+
| | LoRA-Flow | **33.2** | **37.6**| **42.0**| **37.6**| **20.7** | **23.8**| **23.2**| **22.6**|
|
| 47 |
+
|
| 48 |
+
|
| 49 |
# Citation
|
| 50 |
if you find our repo is helpful, please cite the following
|
| 51 |
```bibtex
|