edwko commited on
Commit
c687f7c
1 Parent(s): 50b226f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +122 -3
README.md CHANGED
@@ -1,3 +1,122 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+ # Lite-Mistral-150M-v2-Instruct
5
+
6
+ This is a Lite series model based on the Mistral architecture, comprising approximately 157 million parameters.
7
+
8
+ ## Improvements from v1:
9
+ - Extended Training: Further refinement of the model, resulting in improved benchmark performance and overall text generation quality.
10
+ - Tokenizer changes.
11
+
12
+ ## How coherent is the 150M model?
13
+ Let's look at real-world examples:
14
+
15
+ ## Example 1
16
+ ```
17
+ Input:
18
+ <s>user
19
+ What is the capital of France?</s>
20
+ <s>assistant
21
+ The capital of France is Paris.</s>
22
+ <s>user
23
+ What is the capital of Spain?</s>
24
+
25
+ Response:
26
+ <s>assistant
27
+ The capital of Spain is Madrid, which is known for its beautiful architecture and rich history.</s>
28
+ ```
29
+
30
+ ## Example 2
31
+ ```
32
+ Input:
33
+ <s>user
34
+ Who wrote the play "Romeo and Juliet"?</s>
35
+
36
+ Response:
37
+ <s>assistant
38
+ The play "Romeo and Juliet" was written by William Shakespeare.</s>
39
+ ```
40
+
41
+ ## Example 3
42
+ ```
43
+ Input:
44
+ <s>user
45
+ Tell me a joke</s>
46
+
47
+ Response:
48
+ <s>assistant
49
+ Why did the chicken cross the playground? Because it wanted to see who could get the ball!</s>
50
+ ```
51
+
52
+ ## Example 4 (RP)
53
+ ```
54
+ Input:
55
+ <s>user
56
+ Let's roleplay. I'll be a customer at a restaurant, and you be the waiter. Ready?</s>
57
+ <s>assistant
58
+ Certainly! I'll play the role of a waiter. Please go ahead and start the roleplay when you're ready.</s>
59
+ <s>user
60
+ Hello, I'd like to see the menu please.</s>
61
+ <s>assistant
62
+ Good evening, sir! Welcome to our restaurant. Here's our menu for today. May I recommend our chef's special, the grilled salmon with lemon butter sauce? Let me know if you have any questions about our dishes.</s>
63
+ <s>user
64
+ I'll have the salmon. What wines do you recommend to go with it?</s>
65
+
66
+ Response:
67
+ <s> assistant
68
+ To go with your salmon, we suggest a pair of red-tailed or white-tailed jackfights. The grilled salmon is creamy and has a nice acidity. The lemon butter sauce is a great addition to this dish.</s>
69
+ ```
70
+
71
+ While the model performs well for straightforward tasks, it may lack depth in complex topics or sophisticated creative abilities.
72
+ It's optimized for scenarios where computational efficiency is paramount. It offers a compact solution for tasks that require a balance between resource constraints and acceptable performance.
73
+ As a lightweight model, it may exhibit reduced accuracy and coherence compared to larger language models.
74
+
75
+ ## Benchmarks:
76
+
77
+ <table style="text-align: left;">
78
+ <tr>
79
+ <th>Benchmark</th>
80
+ <th>5-shot</th>
81
+ <th>0-shot</th>
82
+ </tr>
83
+ <tr>
84
+ <td>ARC Easy</td>
85
+ <td>47.26</td>
86
+ <td>45.58</td>
87
+ </tr>
88
+ <tr>
89
+ <td>BoolQ</td>
90
+ <td>43.33</td>
91
+ <td>44.16</td>
92
+ </tr>
93
+ <tr>
94
+ <td>HellaSWAG</td>
95
+ <td>28.70</td>
96
+ <td>28.72</td>
97
+ </tr>
98
+ <tr>
99
+ <td>MMLU</td>
100
+ <td>26.09</td>
101
+ <td>25.28</td>
102
+ </tr>
103
+ <tr>
104
+ <td>OpenBookQA</td>
105
+ <td>16.00</td>
106
+ <td>18.20</td>
107
+ </tr>
108
+ <tr>
109
+ <td>PIQA</td>
110
+ <td>62.79</td>
111
+ <td>62.02</td>
112
+ </tr>
113
+ <tr>
114
+ <td>Winogrande</td>
115
+ <td>51.30</td>
116
+ <td>51.78</td>
117
+ </tr>
118
+ </table>
119
+
120
+ ## Risk Disclaimer
121
+
122
+ By using this model, you acknowledge that you understand and assume the risks associated with its use. You are solely responsible for ensuring compliance with all applicable laws and regulations. We disclaim any liability for problems arising from the use of this open-source model, including but not limited to direct, indirect, incidental, consequential, or punitive damages. We make no warranties, express or implied, regarding the model's performance, accuracy, or fitness for a particular purpose. Your use of this model is at your own risk, and you agree to hold harmless and indemnify us, our affiliates, and our contributors from any claims, damages, or expenses arising from your use of the model.