leaderboard-pt-pr-bot commited on
Commit
38eedb5
1 Parent(s): fb328f2

Adding the Open Portuguese LLM Leaderboard Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/eduagarcia-temp/portuguese-leaderboard-results-to-modelcard

The purpose of this PR is to add evaluation results from the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard) to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/eduagarcia-temp/portuguese-leaderboard-results-to-modelcard/discussions

Files changed (1) hide show
  1. README.md +170 -4
README.md CHANGED
@@ -1,16 +1,163 @@
1
  ---
2
- license: other
3
- license_name: glm-4
4
- license_link: https://huggingface.co/THUDM/glm-4-9b/main/LICENSE
5
  language:
6
  - zh
7
  - en
 
8
  tags:
9
  - glm
10
  - chatglm
11
  - thudm
 
 
12
  inference: false
13
  pipeline_tag: text-generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ---
15
 
16
  # GLM-4-9B
@@ -58,4 +205,23 @@ GLM-4 模型的权重的使用则需要遵循 [LICENSE](LICENSE)。
58
  archivePrefix={arXiv},
59
  primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
60
  }
61
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
2
  language:
3
  - zh
4
  - en
5
+ license: other
6
  tags:
7
  - glm
8
  - chatglm
9
  - thudm
10
+ license_name: glm-4
11
+ license_link: https://huggingface.co/THUDM/glm-4-9b/main/LICENSE
12
  inference: false
13
  pipeline_tag: text-generation
14
+ model-index:
15
+ - name: glm-4-9b
16
+ results:
17
+ - task:
18
+ type: text-generation
19
+ name: Text Generation
20
+ dataset:
21
+ name: ENEM Challenge (No Images)
22
+ type: eduagarcia/enem_challenge
23
+ split: train
24
+ args:
25
+ num_few_shot: 3
26
+ metrics:
27
+ - type: acc
28
+ value: 73.27
29
+ name: accuracy
30
+ source:
31
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
32
+ name: Open Portuguese LLM Leaderboard
33
+ - task:
34
+ type: text-generation
35
+ name: Text Generation
36
+ dataset:
37
+ name: BLUEX (No Images)
38
+ type: eduagarcia-temp/BLUEX_without_images
39
+ split: train
40
+ args:
41
+ num_few_shot: 3
42
+ metrics:
43
+ - type: acc
44
+ value: 64.53
45
+ name: accuracy
46
+ source:
47
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
48
+ name: Open Portuguese LLM Leaderboard
49
+ - task:
50
+ type: text-generation
51
+ name: Text Generation
52
+ dataset:
53
+ name: OAB Exams
54
+ type: eduagarcia/oab_exams
55
+ split: train
56
+ args:
57
+ num_few_shot: 3
58
+ metrics:
59
+ - type: acc
60
+ value: 55.95
61
+ name: accuracy
62
+ source:
63
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
64
+ name: Open Portuguese LLM Leaderboard
65
+ - task:
66
+ type: text-generation
67
+ name: Text Generation
68
+ dataset:
69
+ name: Assin2 RTE
70
+ type: assin2
71
+ split: test
72
+ args:
73
+ num_few_shot: 15
74
+ metrics:
75
+ - type: f1_macro
76
+ value: 93.34
77
+ name: f1-macro
78
+ source:
79
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
80
+ name: Open Portuguese LLM Leaderboard
81
+ - task:
82
+ type: text-generation
83
+ name: Text Generation
84
+ dataset:
85
+ name: Assin2 STS
86
+ type: eduagarcia/portuguese_benchmark
87
+ split: test
88
+ args:
89
+ num_few_shot: 15
90
+ metrics:
91
+ - type: pearson
92
+ value: 81.29
93
+ name: pearson
94
+ source:
95
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
96
+ name: Open Portuguese LLM Leaderboard
97
+ - task:
98
+ type: text-generation
99
+ name: Text Generation
100
+ dataset:
101
+ name: FaQuAD NLI
102
+ type: ruanchaves/faquad-nli
103
+ split: test
104
+ args:
105
+ num_few_shot: 15
106
+ metrics:
107
+ - type: f1_macro
108
+ value: 83.14
109
+ name: f1-macro
110
+ source:
111
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
112
+ name: Open Portuguese LLM Leaderboard
113
+ - task:
114
+ type: text-generation
115
+ name: Text Generation
116
+ dataset:
117
+ name: HateBR Binary
118
+ type: ruanchaves/hatebr
119
+ split: test
120
+ args:
121
+ num_few_shot: 25
122
+ metrics:
123
+ - type: f1_macro
124
+ value: 81.1
125
+ name: f1-macro
126
+ source:
127
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
128
+ name: Open Portuguese LLM Leaderboard
129
+ - task:
130
+ type: text-generation
131
+ name: Text Generation
132
+ dataset:
133
+ name: PT Hate Speech Binary
134
+ type: hate_speech_portuguese
135
+ split: test
136
+ args:
137
+ num_few_shot: 25
138
+ metrics:
139
+ - type: f1_macro
140
+ value: 70.87
141
+ name: f1-macro
142
+ source:
143
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
144
+ name: Open Portuguese LLM Leaderboard
145
+ - task:
146
+ type: text-generation
147
+ name: Text Generation
148
+ dataset:
149
+ name: tweetSentBR
150
+ type: eduagarcia/tweetsentbr_fewshot
151
+ split: test
152
+ args:
153
+ num_few_shot: 25
154
+ metrics:
155
+ - type: f1_macro
156
+ value: 73.03
157
+ name: f1-macro
158
+ source:
159
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=THUDM/glm-4-9b
160
+ name: Open Portuguese LLM Leaderboard
161
  ---
162
 
163
  # GLM-4-9B
 
205
  archivePrefix={arXiv},
206
  primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
207
  }
208
+ ```
209
+
210
+
211
+ # Open Portuguese LLM Leaderboard Evaluation Results
212
+
213
+ Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/THUDM/glm-4-9b) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
214
+
215
+ | Metric | Value |
216
+ |--------------------------|---------|
217
+ |Average |**75.17**|
218
+ |ENEM Challenge (No Images)| 73.27|
219
+ |BLUEX (No Images) | 64.53|
220
+ |OAB Exams | 55.95|
221
+ |Assin2 RTE | 93.34|
222
+ |Assin2 STS | 81.29|
223
+ |FaQuAD NLI | 83.14|
224
+ |HateBR Binary | 81.10|
225
+ |PT Hate Speech Binary | 70.87|
226
+ |tweetSentBR | 73.03|
227
+