AkimfromParis commited on
Commit
53cd244
1 Parent(s): 0cc7ac6

Test 2 update font and display About page v1.1

Browse files
Files changed (1) hide show
  1. src/about.py +46 -57
src/about.py CHANGED
@@ -91,84 +91,73 @@ LLM_BENCHMARKS_TEXT = f"""
91
  📈 We evaluate Japanese Large Language Models on 52 key benchmarks leveraging our evaluation tool [llm-jp-eval](https://github.com/llm-jp/llm-jp-eval), a unified framework to evaluate Japanese LLMs on various evaluation tasks.
92
 
93
  Benchmarks:
94
- **NLI (Natural Language Inference)**
95
 
96
- - `Jamp` JAMP, a Japanese NLI benchmark focused on temporal inference [Source](https://github.com/tomo-ut/temporalNLI_dataset) | License CC BY-SA 4.0
97
 
98
- ### JaNLI
99
 
100
- Sourcehttps://github.com/verypluming/JaNLI
101
- License:CC BY-SA 4.0
102
 
103
- #### JNLI
104
 
105
- Sourcehttps://github.com/yahoojapan/JGLUE
106
- License:CC BY-SA 4.0
107
 
108
- ###JSeM
109
 
110
- Source:https://github.com/DaisukeBekki/JSeM
111
- License:BSD 3-Clause
112
-
113
- ###JSICK
114
-
115
- Source:https://github.com/verypluming/JSICK
116
- License:CC BY-SA 4.0
117
-
118
- QA (Question Answering)
119
 
120
  ###JEMHopQA
121
 
122
- Sourcehttps://github.com/aiishii/JEMHopQA
123
- LicenseCC BY-SA 4.0
124
 
125
  ###NIILC
126
 
127
- Sourcehttps://github.com/mynlp/niilc-qa
128
- LicenseCC BY-SA 4.0
129
 
130
  ###JAQKET (AIO)
131
 
132
- Sourcehttps://www.nlp.ecei.tohoku.ac.jp/projects/jaqket/
133
- LicenseCC BY-SA 4.0(Other licenses are required for corporate usage)
134
 
135
- RC (Reading Comprehension)
136
 
137
  ###JSQuAD
138
 
139
- Sourcehttps://github.com/yahoojapan/JGLUE
140
- LicenseCC BY-SA 4.0
141
 
142
- MC (Multiple Choice question answering)
143
 
144
  ###JCommonsenseMorality
145
 
146
- Sourcehttps://github.com/Language-Media-Lab/commonsense-moral-ja
147
  License:MIT License
148
 
149
  ###JCommonsenseQA
150
 
151
- Sourcehttps://github.com/yahoojapan/JGLUE
152
- LicenseCC BY-SA 4.0
153
 
154
  ###Kyoto University Commonsense Inference dataset (KUCI)
155
 
156
- Sourcehttps://github.com/ku-nlp/KUCI
157
- LicenseCC BY-SA 4.0
158
 
159
- EL (Entity Linking)
160
 
161
  ###chABSA
162
 
163
- Sourcehttps://github.com/chakki-works/chABSA-dataset
164
- LicenseCC BY 4.0
165
 
166
- FA (Fundamental Analysis)
167
 
168
  ###Wikipedia Annotated Corpus
169
 
170
- Sourcehttps://github.com/ku-nlp/WikipediaAnnotatedCorpus
171
- LicenseCC BY-SA 4.0
172
  List of tasks:
173
 
174
  Reading Prediction
@@ -177,63 +166,63 @@ Dependency Parsing
177
  Predicate-argument structure analysis (PAS)
178
  Coreference Resolution
179
 
180
- MR (Mathematical Reasoning)
181
 
182
  ###MAWPS
183
 
184
- Sourcehttps://github.com/nlp-waseda/chain-of-thought-ja-dataset
185
  License:Apache-2.0
186
 
187
  ###MGSM
188
 
189
- Sourcehttps://huggingface.co/datasets/juletxara/mgsm
190
  License:MIT License
191
 
192
- MT (Machine Translation)
193
 
194
  ###Asian Language Treebank (ALT) - Parallel Corpus
195
 
196
- Sourcehttps://www2.nict.go.jp/astrec-att/member/mutiyama/ALT/index.html
197
- LicenseCC BY 4.0
198
 
199
  ###WikiCorpus (Japanese-English Bilingual Corpus of Wikipedia's articles about the city of Kyoto)
200
 
201
- Sourcehttps://alaginrc.nict.go.jp/WikiCorpus/
202
  License:CC BY-SA 3.0 deed
203
 
204
- STS (Semantic Textual Similarity)
205
 
206
  This task is supported by llm-jp-eval, but it is not included in the evaluation score average.
207
 
208
  ###JSTS
209
 
210
- Sourcehttps://github.com/yahoojapan/JGLUE
211
- LicenseCC BY-SA 4.0
212
 
213
- HE (Human Examination)
214
 
215
  ###MMLU
216
 
217
- Sourcehttps://github.com/hendrycks/test
218
  License:MIT License
219
 
220
  ###JMMLU
221
 
222
- Sourcehttps://github.com/nlp-waseda/JMMLU
223
  License:CC BY-SA 4.0(3 tasks under the CC BY-NC-ND 4.0 license)
224
 
225
- CG (Code Generation)
226
 
227
  ###MBPP
228
 
229
- Sourcehttps://huggingface.co/datasets/llm-jp/mbpp-ja
230
- LicenseCC-BY-4.0
231
 
232
- SUM (Summarization)
233
 
234
  ###XL-Sum
235
 
236
- Sourcehttps://github.com/csebuetnlp/xl-sum
237
  License:CC BY-NC-SA 4.0(Due to the non-commercial license, this dataset will not be used, unless you specifically agree to the license and terms of use)
238
 
239
 
 
91
  📈 We evaluate Japanese Large Language Models on 52 key benchmarks leveraging our evaluation tool [llm-jp-eval](https://github.com/llm-jp/llm-jp-eval), a unified framework to evaluate Japanese LLMs on various evaluation tasks.
92
 
93
  Benchmarks:
 
94
 
95
+ - **NLI (Natural Language Inference)**
96
 
97
+ * `Jamp`, a Japanese NLI benchmark focused on temporal inference [Source](https://github.com/tomo-ut/temporalNLI_dataset) (License CC BY-SA 4.0)
98
 
99
+ * `JaNLI`, Japanese Adversarial Natural Language Inference [Source](https://github.com/verypluming/JaNLI) (License CC BY-SA 4.0)
 
100
 
101
+ * `JNLI`, Japanese Natural Language Inference (part of JGLUE) [Source](https://github.com/yahoojapan/JGLUE) (License CC BY-SA 4.0)
102
 
103
+ * `JSeM`, Japanese semantic test suite [Source](https://github.com/DaisukeBekki/JSeM) (License BSD 3-Clause)
 
104
 
105
+ * `JSICK`, Japanese Sentences Involving Compositional Knowledge [Source](https://github.com/verypluming/JSICK) (License CC BY-SA 4.0)
106
 
107
+ - **NQA (Question Answering)**
 
 
 
 
 
 
 
 
108
 
109
  ###JEMHopQA
110
 
111
+ [Source](https://github.com/aiishii/JEMHopQA)
112
+ (License CC BY-SA 4.0)
113
 
114
  ###NIILC
115
 
116
+ [Source](https://github.com/mynlp/niilc-qa)
117
+ (License CC BY-SA 4.0)
118
 
119
  ###JAQKET (AIO)
120
 
121
+ [Source](https://www.nlp.ecei.tohoku.ac.jp/projects/jaqket/)
122
+ (License CC BY-SA 4.0)(Other licenses are required for corporate usage)
123
 
124
+ - **RC (Reading Comprehension)**
125
 
126
  ###JSQuAD
127
 
128
+ [Source](https://github.com/yahoojapan/JGLUE)
129
+ (License CC BY-SA 4.0)
130
 
131
+ - **MC (Multiple Choice question answering)**
132
 
133
  ###JCommonsenseMorality
134
 
135
+ [Source](https://github.com/Language-Media-Lab/commonsense-moral-ja)
136
  License:MIT License
137
 
138
  ###JCommonsenseQA
139
 
140
+ [Source](https://github.com/yahoojapan/JGLUE)
141
+ (License CC BY-SA 4.0)
142
 
143
  ###Kyoto University Commonsense Inference dataset (KUCI)
144
 
145
+ [Source](https://github.com/ku-nlp/KUCI
146
+ (License CC BY-SA 4.0)
147
 
148
+ - **EL (Entity Linking)**
149
 
150
  ###chABSA
151
 
152
+ [Source](https://github.com/chakki-works/chABSA-dataset)
153
+ (License CC BY-SA 4.0)
154
 
155
+ - **FA (Fundamental Analysis)**
156
 
157
  ###Wikipedia Annotated Corpus
158
 
159
+ [Source](https://github.com/ku-nlp/WikipediaAnnotatedCorpus)
160
+ (License CC BY-SA 4.0)
161
  List of tasks:
162
 
163
  Reading Prediction
 
166
  Predicate-argument structure analysis (PAS)
167
  Coreference Resolution
168
 
169
+ - **MR (Mathematical Reasoning)**
170
 
171
  ###MAWPS
172
 
173
+ [Source](https://github.com/nlp-waseda/chain-of-thought-ja-dataset)
174
  License:Apache-2.0
175
 
176
  ###MGSM
177
 
178
+ [Source](https://huggingface.co/datasets/juletxara/mgsm)
179
  License:MIT License
180
 
181
+ - **MT (Machine Translation)**
182
 
183
  ###Asian Language Treebank (ALT) - Parallel Corpus
184
 
185
+ [Source](https://www2.nict.go.jp/astrec-att/member/mutiyama/ALT/index.html)
186
+ (License CC BY-SA 4.0)
187
 
188
  ###WikiCorpus (Japanese-English Bilingual Corpus of Wikipedia's articles about the city of Kyoto)
189
 
190
+ [Source](https://alaginrc.nict.go.jp/WikiCorpus/)
191
  License:CC BY-SA 3.0 deed
192
 
193
+ - **STS (Semantic Textual Similarity)**
194
 
195
  This task is supported by llm-jp-eval, but it is not included in the evaluation score average.
196
 
197
  ###JSTS
198
 
199
+ [Source](https://github.com/yahoojapan/JGLUE)
200
+ (License CC BY-SA 4.0)
201
 
202
+ - **HE (Human Examination)**
203
 
204
  ###MMLU
205
 
206
+ [Source](https://github.com/hendrycks/test)
207
  License:MIT License
208
 
209
  ###JMMLU
210
 
211
+ [Source](https://github.com/nlp-waseda/JMMLU)
212
  License:CC BY-SA 4.0(3 tasks under the CC BY-NC-ND 4.0 license)
213
 
214
+ - **CG (Code Generation)**
215
 
216
  ###MBPP
217
 
218
+ [Source](https://huggingface.co/datasets/llm-jp/mbpp-ja)
219
+ (License CC BY-SA 4.0)
220
 
221
+ - **SUM (Summarization)**
222
 
223
  ###XL-Sum
224
 
225
+ [Source](https://github.com/csebuetnlp/xl-sum)
226
  License:CC BY-NC-SA 4.0(Due to the non-commercial license, this dataset will not be used, unless you specifically agree to the license and terms of use)
227
 
228