Hugging Face's logo

infgrad
/
stella-base-zh-v3-1792d

Sentence Similarity
sentence-transformers
PyTorch
bert
feature-extraction
mteb
Eval Results
Inference Endpoints
Model card Files Community
7
infgrad commited on
Commit
aaf6411
•
1 Parent(s): c632864

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1061 -0
README.md CHANGED
@@ -1,3 +1,1064 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # 1 开源清单
2
 
3
  本次开源2个通用向量编码模型和一个针对dialogue进行编码的向量模型，同时开源全量160万对话重写数据集和20万的难负例的检索数据集。
 
1
+ ---
2
+ pipeline_tag: sentence-similarity
3
+ tags:
4
+ - sentence-transformers
5
+ - feature-extraction
6
+ - sentence-similarity
7
+ - mteb
8
+ model-index:
9
+ - name: stella-base-zh-v3-1792d
10
+ results:
11
+ - task:
12
+ type: STS
13
+ dataset:
14
+ type: C-MTEB/AFQMC
15
+ name: MTEB AFQMC
16
+ config: default
17
+ split: validation
18
+ revision: None
19
+ metrics:
20
+ - type: cos_sim_pearson
21
+ value: 54.5145388936202
22
+ - type: cos_sim_spearman
23
+ value: 59.223125058197134
24
+ - type: euclidean_pearson
25
+ value: 57.819377838734695
26
+ - type: euclidean_spearman
27
+ value: 59.22310494948463
28
+ - type: manhattan_pearson
29
+ value: 57.44029759610327
30
+ - type: manhattan_spearman
31
+ value: 58.88336250854381
32
+ - task:
33
+ type: STS
34
+ dataset:
35
+ type: C-MTEB/ATEC
36
+ name: MTEB ATEC
37
+ config: default
38
+ split: test
39
+ revision: None
40
+ metrics:
41
+ - type: cos_sim_pearson
42
+ value: 54.544243591344866
43
+ - type: cos_sim_spearman
44
+ value: 58.43052988038229
45
+ - type: euclidean_pearson
46
+ value: 62.1608405146189
47
+ - type: euclidean_spearman
48
+ value: 58.43052762862396
49
+ - type: manhattan_pearson
50
+ value: 61.88443779892169
51
+ - type: manhattan_spearman
52
+ value: 58.26899143609596
53
+ - task:
54
+ type: Classification
55
+ dataset:
56
+ type: mteb/amazon_reviews_multi
57
+ name: MTEB AmazonReviewsClassification (zh)
58
+ config: zh
59
+ split: test
60
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
61
+ metrics:
62
+ - type: accuracy
63
+ value: 46.343999999999994
64
+ - type: f1
65
+ value: 44.46931958420461
66
+ - task:
67
+ type: STS
68
+ dataset:
69
+ type: C-MTEB/BQ
70
+ name: MTEB BQ
71
+ config: default
72
+ split: test
73
+ revision: None
74
+ metrics:
75
+ - type: cos_sim_pearson
76
+ value: 68.52081000538426
77
+ - type: cos_sim_spearman
78
+ value: 70.44089935351529
79
+ - type: euclidean_pearson
80
+ value: 69.24671010626395
81
+ - type: euclidean_spearman
82
+ value: 70.44090281761693
83
+ - type: manhattan_pearson
84
+ value: 69.00737718109357
85
+ - type: manhattan_spearman
86
+ value: 70.24344902456502
87
+ - task:
88
+ type: Clustering
89
+ dataset:
90
+ type: C-MTEB/CLSClusteringP2P
91
+ name: MTEB CLSClusteringP2P
92
+ config: default
93
+ split: test
94
+ revision: None
95
+ metrics:
96
+ - type: v_measure
97
+ value: 42.86119436460332
98
+ - task:
99
+ type: Clustering
100
+ dataset:
101
+ type: C-MTEB/CLSClusteringS2S
102
+ name: MTEB CLSClusteringS2S
103
+ config: default
104
+ split: test
105
+ revision: None
106
+ metrics:
107
+ - type: v_measure
108
+ value: 39.97521728440642
109
+ - task:
110
+ type: Reranking
111
+ dataset:
112
+ type: C-MTEB/CMedQAv1-reranking
113
+ name: MTEB CMedQAv1
114
+ config: default
115
+ split: test
116
+ revision: None
117
+ metrics:
118
+ - type: map
119
+ value: 88.34151862240452
120
+ - type: mrr
121
+ value: 90.40380952380953
122
+ - task:
123
+ type: Reranking
124
+ dataset:
125
+ type: C-MTEB/CMedQAv2-reranking
126
+ name: MTEB CMedQAv2
127
+ config: default
128
+ split: test
129
+ revision: None
130
+ metrics:
131
+ - type: map
132
+ value: 89.06288758814637
133
+ - type: mrr
134
+ value: 90.91285714285713
135
+ - task:
136
+ type: Retrieval
137
+ dataset:
138
+ type: C-MTEB/CmedqaRetrieval
139
+ name: MTEB CmedqaRetrieval
140
+ config: default
141
+ split: dev
142
+ revision: None
143
+ metrics:
144
+ - type: map_at_1
145
+ value: 25.651000000000003
146
+ - type: map_at_10
147
+ value: 38.576
148
+ - type: map_at_100
149
+ value: 40.534
150
+ - type: map_at_1000
151
+ value: 40.64
152
+ - type: map_at_3
153
+ value: 34.016000000000005
154
+ - type: map_at_5
155
+ value: 36.675999999999995
156
+ - type: mrr_at_1
157
+ value: 39.06
158
+ - type: mrr_at_10
159
+ value: 47.278
160
+ - type: mrr_at_100
161
+ value: 48.272999999999996
162
+ - type: mrr_at_1000
163
+ value: 48.314
164
+ - type: mrr_at_3
165
+ value: 44.461
166
+ - type: mrr_at_5
167
+ value: 46.107
168
+ - type: ndcg_at_1
169
+ value: 39.06
170
+ - type: ndcg_at_10
171
+ value: 45.384
172
+ - type: ndcg_at_100
173
+ value: 52.796
174
+ - type: ndcg_at_1000
175
+ value: 54.55
176
+ - type: ndcg_at_3
177
+ value: 39.497
178
+ - type: ndcg_at_5
179
+ value: 42.189
180
+ - type: precision_at_1
181
+ value: 39.06
182
+ - type: precision_at_10
183
+ value: 10.17
184
+ - type: precision_at_100
185
+ value: 1.6179999999999999
186
+ - type: precision_at_1000
187
+ value: 0.184
188
+ - type: precision_at_3
189
+ value: 22.247
190
+ - type: precision_at_5
191
+ value: 16.529
192
+ - type: recall_at_1
193
+ value: 25.651000000000003
194
+ - type: recall_at_10
195
+ value: 56.82899999999999
196
+ - type: recall_at_100
197
+ value: 87.134
198
+ - type: recall_at_1000
199
+ value: 98.709
200
+ - type: recall_at_3
201
+ value: 39.461
202
+ - type: recall_at_5
203
+ value: 47.329
204
+ - task:
205
+ type: PairClassification
206
+ dataset:
207
+ type: C-MTEB/CMNLI
208
+ name: MTEB Cmnli
209
+ config: default
210
+ split: validation
211
+ revision: None
212
+ metrics:
213
+ - type: cos_sim_accuracy
214
+ value: 83.1870114251353
215
+ - type: cos_sim_ap
216
+ value: 90.42393852164342
217
+ - type: cos_sim_f1
218
+ value: 84.10685985963323
219
+ - type: cos_sim_precision
220
+ value: 81.5229317533465
221
+ - type: cos_sim_recall
222
+ value: 86.85994856207621
223
+ - type: dot_accuracy
224
+ value: 83.1870114251353
225
+ - type: dot_ap
226
+ value: 90.41339758845682
227
+ - type: dot_f1
228
+ value: 84.10685985963323
229
+ - type: dot_precision
230
+ value: 81.5229317533465
231
+ - type: dot_recall
232
+ value: 86.85994856207621
233
+ - type: euclidean_accuracy
234
+ value: 83.1870114251353
235
+ - type: euclidean_ap
236
+ value: 90.42393581056393
237
+ - type: euclidean_f1
238
+ value: 84.10685985963323
239
+ - type: euclidean_precision
240
+ value: 81.5229317533465
241
+ - type: euclidean_recall
242
+ value: 86.85994856207621
243
+ - type: manhattan_accuracy
244
+ value: 82.77811184606134
245
+ - type: manhattan_ap
246
+ value: 90.18115714681704
247
+ - type: manhattan_f1
248
+ value: 83.75083130126357
249
+ - type: manhattan_precision
250
+ value: 79.62065331928345
251
+ - type: manhattan_recall
252
+ value: 88.33294365209258
253
+ - type: max_accuracy
254
+ value: 83.1870114251353
255
+ - type: max_ap
256
+ value: 90.42393852164342
257
+ - type: max_f1
258
+ value: 84.10685985963323
259
+ - task:
260
+ type: Retrieval
261
+ dataset:
262
+ type: C-MTEB/CovidRetrieval
263
+ name: MTEB CovidRetrieval
264
+ config: default
265
+ split: dev
266
+ revision: None
267
+ metrics:
268
+ - type: map_at_1
269
+ value: 68.388
270
+ - type: map_at_10
271
+ value: 76.819
272
+ - type: map_at_100
273
+ value: 77.153
274
+ - type: map_at_1000
275
+ value: 77.16
276
+ - type: map_at_3
277
+ value: 74.98700000000001
278
+ - type: map_at_5
279
+ value: 76.101
280
+ - type: mrr_at_1
281
+ value: 68.599
282
+ - type: mrr_at_10
283
+ value: 76.844
284
+ - type: mrr_at_100
285
+ value: 77.168
286
+ - type: mrr_at_1000
287
+ value: 77.17500000000001
288
+ - type: mrr_at_3
289
+ value: 75.044
290
+ - type: mrr_at_5
291
+ value: 76.208
292
+ - type: ndcg_at_1
293
+ value: 68.599
294
+ - type: ndcg_at_10
295
+ value: 80.613
296
+ - type: ndcg_at_100
297
+ value: 82.017
298
+ - type: ndcg_at_1000
299
+ value: 82.19300000000001
300
+ - type: ndcg_at_3
301
+ value: 76.956
302
+ - type: ndcg_at_5
303
+ value: 78.962
304
+ - type: precision_at_1
305
+ value: 68.599
306
+ - type: precision_at_10
307
+ value: 9.336
308
+ - type: precision_at_100
309
+ value: 0.996
310
+ - type: precision_at_1000
311
+ value: 0.101
312
+ - type: precision_at_3
313
+ value: 27.678000000000004
314
+ - type: precision_at_5
315
+ value: 17.619
316
+ - type: recall_at_1
317
+ value: 68.388
318
+ - type: recall_at_10
319
+ value: 92.36
320
+ - type: recall_at_100
321
+ value: 98.52499999999999
322
+ - type: recall_at_1000
323
+ value: 99.895
324
+ - type: recall_at_3
325
+ value: 82.53399999999999
326
+ - type: recall_at_5
327
+ value: 87.355
328
+ - task:
329
+ type: Retrieval
330
+ dataset:
331
+ type: C-MTEB/DuRetrieval
332
+ name: MTEB DuRetrieval
333
+ config: default
334
+ split: dev
335
+ revision: None
336
+ metrics:
337
+ - type: map_at_1
338
+ value: 25.1
339
+ - type: map_at_10
340
+ value: 77.71000000000001
341
+ - type: map_at_100
342
+ value: 80.638
343
+ - type: map_at_1000
344
+ value: 80.679
345
+ - type: map_at_3
346
+ value: 53.187
347
+ - type: map_at_5
348
+ value: 67.735
349
+ - type: mrr_at_1
350
+ value: 87.8
351
+ - type: mrr_at_10
352
+ value: 91.8
353
+ - type: mrr_at_100
354
+ value: 91.893
355
+ - type: mrr_at_1000
356
+ value: 91.89500000000001
357
+ - type: mrr_at_3
358
+ value: 91.51700000000001
359
+ - type: mrr_at_5
360
+ value: 91.704
361
+ - type: ndcg_at_1
362
+ value: 87.8
363
+ - type: ndcg_at_10
364
+ value: 85.55
365
+ - type: ndcg_at_100
366
+ value: 88.626
367
+ - type: ndcg_at_1000
368
+ value: 89.021
369
+ - type: ndcg_at_3
370
+ value: 83.94
371
+ - type: ndcg_at_5
372
+ value: 83.259
373
+ - type: precision_at_1
374
+ value: 87.8
375
+ - type: precision_at_10
376
+ value: 41.295
377
+ - type: precision_at_100
378
+ value: 4.781
379
+ - type: precision_at_1000
380
+ value: 0.488
381
+ - type: precision_at_3
382
+ value: 75.3
383
+ - type: precision_at_5
384
+ value: 64.13
385
+ - type: recall_at_1
386
+ value: 25.1
387
+ - type: recall_at_10
388
+ value: 87.076
389
+ - type: recall_at_100
390
+ value: 97.095
391
+ - type: recall_at_1000
392
+ value: 99.129
393
+ - type: recall_at_3
394
+ value: 56.013999999999996
395
+ - type: recall_at_5
396
+ value: 73.2
397
+ - task:
398
+ type: Retrieval
399
+ dataset:
400
+ type: C-MTEB/EcomRetrieval
401
+ name: MTEB EcomRetrieval
402
+ config: default
403
+ split: dev
404
+ revision: None
405
+ metrics:
406
+ - type: map_at_1
407
+ value: 53.300000000000004
408
+ - type: map_at_10
409
+ value: 63.01
410
+ - type: map_at_100
411
+ value: 63.574
412
+ - type: map_at_1000
413
+ value: 63.587
414
+ - type: map_at_3
415
+ value: 60.783
416
+ - type: map_at_5
417
+ value: 62.098
418
+ - type: mrr_at_1
419
+ value: 53.300000000000004
420
+ - type: mrr_at_10
421
+ value: 63.01
422
+ - type: mrr_at_100
423
+ value: 63.574
424
+ - type: mrr_at_1000
425
+ value: 63.587
426
+ - type: mrr_at_3
427
+ value: 60.783
428
+ - type: mrr_at_5
429
+ value: 62.098
430
+ - type: ndcg_at_1
431
+ value: 53.300000000000004
432
+ - type: ndcg_at_10
433
+ value: 67.876
434
+ - type: ndcg_at_100
435
+ value: 70.434
436
+ - type: ndcg_at_1000
437
+ value: 70.753
438
+ - type: ndcg_at_3
439
+ value: 63.275000000000006
440
+ - type: ndcg_at_5
441
+ value: 65.654
442
+ - type: precision_at_1
443
+ value: 53.300000000000004
444
+ - type: precision_at_10
445
+ value: 8.32
446
+ - type: precision_at_100
447
+ value: 0.9480000000000001
448
+ - type: precision_at_1000
449
+ value: 0.097
450
+ - type: precision_at_3
451
+ value: 23.5
452
+ - type: precision_at_5
453
+ value: 15.260000000000002
454
+ - type: recall_at_1
455
+ value: 53.300000000000004
456
+ - type: recall_at_10
457
+ value: 83.2
458
+ - type: recall_at_100
459
+ value: 94.8
460
+ - type: recall_at_1000
461
+ value: 97.3
462
+ - type: recall_at_3
463
+ value: 70.5
464
+ - type: recall_at_5
465
+ value: 76.3
466
+ - task:
467
+ type: Classification
468
+ dataset:
469
+ type: C-MTEB/IFlyTek-classification
470
+ name: MTEB IFlyTek
471
+ config: default
472
+ split: validation
473
+ revision: None
474
+ metrics:
475
+ - type: accuracy
476
+ value: 49.92689495959984
477
+ - type: f1
478
+ value: 37.784780470986625
479
+ - task:
480
+ type: Classification
481
+ dataset:
482
+ type: C-MTEB/JDReview-classification
483
+ name: MTEB JDReview
484
+ config: default
485
+ split: test
486
+ revision: None
487
+ metrics:
488
+ - type: accuracy
489
+ value: 86.26641651031895
490
+ - type: ap
491
+ value: 54.50750244841821
492
+ - type: f1
493
+ value: 80.94927946681523
494
+ - task:
495
+ type: STS
496
+ dataset:
497
+ type: C-MTEB/LCQMC
498
+ name: MTEB LCQMC
499
+ config: default
500
+ split: test
501
+ revision: None
502
+ metrics:
503
+ - type: cos_sim_pearson
504
+ value: 72.3980811478615
505
+ - type: cos_sim_spearman
506
+ value: 78.26906056425528
507
+ - type: euclidean_pearson
508
+ value: 77.87705501225068
509
+ - type: euclidean_spearman
510
+ value: 78.26905834518651
511
+ - type: manhattan_pearson
512
+ value: 77.77154630197
513
+ - type: manhattan_spearman
514
+ value: 78.1940918602169
515
+ - task:
516
+ type: Reranking
517
+ dataset:
518
+ type: C-MTEB/Mmarco-reranking
519
+ name: MTEB MMarcoReranking
520
+ config: default
521
+ split: dev
522
+ revision: None
523
+ metrics:
524
+ - type: map
525
+ value: 27.48003475319453
526
+ - type: mrr
527
+ value: 26.400793650793652
528
+ - task:
529
+ type: Retrieval
530
+ dataset:
531
+ type: C-MTEB/MMarcoRetrieval
532
+ name: MTEB MMarcoRetrieval
533
+ config: default
534
+ split: dev
535
+ revision: None
536
+ metrics:
537
+ - type: map_at_1
538
+ value: 64.373
539
+ - type: map_at_10
540
+ value: 73.604
541
+ - type: map_at_100
542
+ value: 73.953
543
+ - type: map_at_1000
544
+ value: 73.965
545
+ - type: map_at_3
546
+ value: 71.70100000000001
547
+ - type: map_at_5
548
+ value: 72.859
549
+ - type: mrr_at_1
550
+ value: 66.676
551
+ - type: mrr_at_10
552
+ value: 74.248
553
+ - type: mrr_at_100
554
+ value: 74.56099999999999
555
+ - type: mrr_at_1000
556
+ value: 74.572
557
+ - type: mrr_at_3
558
+ value: 72.59100000000001
559
+ - type: mrr_at_5
560
+ value: 73.592
561
+ - type: ndcg_at_1
562
+ value: 66.676
563
+ - type: ndcg_at_10
564
+ value: 77.417
565
+ - type: ndcg_at_100
566
+ value: 79.006
567
+ - type: ndcg_at_1000
568
+ value: 79.334
569
+ - type: ndcg_at_3
570
+ value: 73.787
571
+ - type: ndcg_at_5
572
+ value: 75.74
573
+ - type: precision_at_1
574
+ value: 66.676
575
+ - type: precision_at_10
576
+ value: 9.418
577
+ - type: precision_at_100
578
+ value: 1.0210000000000001
579
+ - type: precision_at_1000
580
+ value: 0.105
581
+ - type: precision_at_3
582
+ value: 27.832
583
+ - type: precision_at_5
584
+ value: 17.736
585
+ - type: recall_at_1
586
+ value: 64.373
587
+ - type: recall_at_10
588
+ value: 88.565
589
+ - type: recall_at_100
590
+ value: 95.789
591
+ - type: recall_at_1000
592
+ value: 98.355
593
+ - type: recall_at_3
594
+ value: 78.914
595
+ - type: recall_at_5
596
+ value: 83.56
597
+ - task:
598
+ type: Classification
599
+ dataset:
600
+ type: mteb/amazon_massive_intent
601
+ name: MTEB MassiveIntentClassification (zh-CN)
602
+ config: zh-CN
603
+ split: test
604
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
605
+ metrics:
606
+ - type: accuracy
607
+ value: 72.0544720914593
608
+ - type: f1
609
+ value: 69.61749470345791
610
+ - task:
611
+ type: Classification
612
+ dataset:
613
+ type: mteb/amazon_massive_scenario
614
+ name: MTEB MassiveScenarioClassification (zh-CN)
615
+ config: zh-CN
616
+ split: test
617
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
618
+ metrics:
619
+ - type: accuracy
620
+ value: 75.30262273032953
621
+ - type: f1
622
+ value: 75.05097671215634
623
+ - task:
624
+ type: Retrieval
625
+ dataset:
626
+ type: C-MTEB/MedicalRetrieval
627
+ name: MTEB MedicalRetrieval
628
+ config: default
629
+ split: dev
630
+ revision: None
631
+ metrics:
632
+ - type: map_at_1
633
+ value: 55.1
634
+ - type: map_at_10
635
+ value: 61.284000000000006
636
+ - type: map_at_100
637
+ value: 61.794000000000004
638
+ - type: map_at_1000
639
+ value: 61.838
640
+ - type: map_at_3
641
+ value: 59.75
642
+ - type: map_at_5
643
+ value: 60.64000000000001
644
+ - type: mrr_at_1
645
+ value: 55.300000000000004
646
+ - type: mrr_at_10
647
+ value: 61.38400000000001
648
+ - type: mrr_at_100
649
+ value: 61.894000000000005
650
+ - type: mrr_at_1000
651
+ value: 61.938
652
+ - type: mrr_at_3
653
+ value: 59.85
654
+ - type: mrr_at_5
655
+ value: 60.74
656
+ - type: ndcg_at_1
657
+ value: 55.1
658
+ - type: ndcg_at_10
659
+ value: 64.345
660
+ - type: ndcg_at_100
661
+ value: 67.148
662
+ - type: ndcg_at_1000
663
+ value: 68.36
664
+ - type: ndcg_at_3
665
+ value: 61.182
666
+ - type: ndcg_at_5
667
+ value: 62.808
668
+ - type: precision_at_1
669
+ value: 55.1
670
+ - type: precision_at_10
671
+ value: 7.3999999999999995
672
+ - type: precision_at_100
673
+ value: 0.8789999999999999
674
+ - type: precision_at_1000
675
+ value: 0.098
676
+ - type: precision_at_3
677
+ value: 21.767
678
+ - type: precision_at_5
679
+ value: 13.86
680
+ - type: recall_at_1
681
+ value: 55.1
682
+ - type: recall_at_10
683
+ value: 74.0
684
+ - type: recall_at_100
685
+ value: 87.9
686
+ - type: recall_at_1000
687
+ value: 97.5
688
+ - type: recall_at_3
689
+ value: 65.3
690
+ - type: recall_at_5
691
+ value: 69.3
692
+ - task:
693
+ type: Classification
694
+ dataset:
695
+ type: C-MTEB/MultilingualSentiment-classification
696
+ name: MTEB MultilingualSentiment
697
+ config: default
698
+ split: validation
699
+ revision: None
700
+ metrics:
701
+ - type: accuracy
702
+ value: 76.21666666666667
703
+ - type: f1
704
+ value: 76.03732395559548
705
+ - task:
706
+ type: PairClassification
707
+ dataset:
708
+ type: C-MTEB/OCNLI
709
+ name: MTEB Ocnli
710
+ config: default
711
+ split: validation
712
+ revision: None
713
+ metrics:
714
+ - type: cos_sim_accuracy
715
+ value: 81.8083378451543
716
+ - type: cos_sim_ap
717
+ value: 85.43050139514027
718
+ - type: cos_sim_f1
719
+ value: 83.25969563082965
720
+ - type: cos_sim_precision
721
+ value: 77.79816513761469
722
+ - type: cos_sim_recall
723
+ value: 89.54593453009504
724
+ - type: dot_accuracy
725
+ value: 81.8083378451543
726
+ - type: dot_ap
727
+ value: 85.43050139514027
728
+ - type: dot_f1
729
+ value: 83.25969563082965
730
+ - type: dot_precision
731
+ value: 77.79816513761469
732
+ - type: dot_recall
733
+ value: 89.54593453009504
734
+ - type: euclidean_accuracy
735
+ value: 81.8083378451543
736
+ - type: euclidean_ap
737
+ value: 85.43050139514027
738
+ - type: euclidean_f1
739
+ value: 83.25969563082965
740
+ - type: euclidean_precision
741
+ value: 77.79816513761469
742
+ - type: euclidean_recall
743
+ value: 89.54593453009504
744
+ - type: manhattan_accuracy
745
+ value: 81.53762858689767
746
+ - type: manhattan_ap
747
+ value: 84.90556637024838
748
+ - type: manhattan_f1
749
+ value: 82.90258449304174
750
+ - type: manhattan_precision
751
+ value: 78.30985915492957
752
+ - type: manhattan_recall
753
+ value: 88.0675818373812
754
+ - type: max_accuracy
755
+ value: 81.8083378451543
756
+ - type: max_ap
757
+ value: 85.43050139514027
758
+ - type: max_f1
759
+ value: 83.25969563082965
760
+ - task:
761
+ type: Classification
762
+ dataset:
763
+ type: C-MTEB/OnlineShopping-classification
764
+ name: MTEB OnlineShopping
765
+ config: default
766
+ split: test
767
+ revision: None
768
+ metrics:
769
+ - type: accuracy
770
+ value: 93.53
771
+ - type: ap
772
+ value: 91.62070655043128
773
+ - type: f1
774
+ value: 93.51908163199477
775
+ - task:
776
+ type: STS
777
+ dataset:
778
+ type: C-MTEB/PAWSX
779
+ name: MTEB PAWSX
780
+ config: default
781
+ split: test
782
+ revision: None
783
+ metrics:
784
+ - type: cos_sim_pearson
785
+ value: 38.451787103814375
786
+ - type: cos_sim_spearman
787
+ value: 43.97299462643919
788
+ - type: euclidean_pearson
789
+ value: 43.63298716626501
790
+ - type: euclidean_spearman
791
+ value: 43.973080252178576
792
+ - type: manhattan_pearson
793
+ value: 43.37465277323481
794
+ - type: manhattan_spearman
795
+ value: 43.71981281220414
796
+ - task:
797
+ type: STS
798
+ dataset:
799
+ type: C-MTEB/QBQTC
800
+ name: MTEB QBQTC
801
+ config: default
802
+ split: test
803
+ revision: None
804
+ metrics:
805
+ - type: cos_sim_pearson
806
+ value: 37.75882451277358
807
+ - type: cos_sim_spearman
808
+ value: 40.0244327844802
809
+ - type: euclidean_pearson
810
+ value: 38.11050875514246
811
+ - type: euclidean_spearman
812
+ value: 40.02440987254504
813
+ - type: manhattan_pearson
814
+ value: 38.03186803221696
815
+ - type: manhattan_spearman
816
+ value: 39.757452890246775
817
+ - task:
818
+ type: STS
819
+ dataset:
820
+ type: mteb/sts22-crosslingual-sts
821
+ name: MTEB STS22 (zh)
822
+ config: zh
823
+ split: test
824
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
825
+ metrics:
826
+ - type: cos_sim_pearson
827
+ value: 65.9133992390713
828
+ - type: cos_sim_spearman
829
+ value: 66.4894937647578
830
+ - type: euclidean_pearson
831
+ value: 66.19047142189935
832
+ - type: euclidean_spearman
833
+ value: 66.4894937647578
834
+ - type: manhattan_pearson
835
+ value: 66.6960935896136
836
+ - type: manhattan_spearman
837
+ value: 66.88179996508133
838
+ - task:
839
+ type: STS
840
+ dataset:
841
+ type: C-MTEB/STSB
842
+ name: MTEB STSB
843
+ config: default
844
+ split: test
845
+ revision: None
846
+ metrics:
847
+ - type: cos_sim_pearson
848
+ value: 80.55099417946924
849
+ - type: cos_sim_spearman
850
+ value: 83.05000687568048
851
+ - type: euclidean_pearson
852
+ value: 82.62744668792926
853
+ - type: euclidean_spearman
854
+ value: 83.05000687568048
855
+ - type: manhattan_pearson
856
+ value: 82.6543207325763
857
+ - type: manhattan_spearman
858
+ value: 83.06852715971705
859
+ - task:
860
+ type: Reranking
861
+ dataset:
862
+ type: C-MTEB/T2Reranking
863
+ name: MTEB T2Reranking
864
+ config: default
865
+ split: dev
866
+ revision: None
867
+ metrics:
868
+ - type: map
869
+ value: 66.48634798223672
870
+ - type: mrr
871
+ value: 76.30158461488861
872
+ - task:
873
+ type: Retrieval
874
+ dataset:
875
+ type: C-MTEB/T2Retrieval
876
+ name: MTEB T2Retrieval
877
+ config: default
878
+ split: dev
879
+ revision: None
880
+ metrics:
881
+ - type: map_at_1
882
+ value: 27.483999999999998
883
+ - type: map_at_10
884
+ value: 76.848
885
+ - type: map_at_100
886
+ value: 80.541
887
+ - type: map_at_1000
888
+ value: 80.607
889
+ - type: map_at_3
890
+ value: 54.111
891
+ - type: map_at_5
892
+ value: 66.46300000000001
893
+ - type: mrr_at_1
894
+ value: 90.045
895
+ - type: mrr_at_10
896
+ value: 92.552
897
+ - type: mrr_at_100
898
+ value: 92.642
899
+ - type: mrr_at_1000
900
+ value: 92.645
901
+ - type: mrr_at_3
902
+ value: 92.134
903
+ - type: mrr_at_5
904
+ value: 92.391
905
+ - type: ndcg_at_1
906
+ value: 90.045
907
+ - type: ndcg_at_10
908
+ value: 84.504
909
+ - type: ndcg_at_100
910
+ value: 88.23100000000001
911
+ - type: ndcg_at_1000
912
+ value: 88.85300000000001
913
+ - type: ndcg_at_3
914
+ value: 85.992
915
+ - type: ndcg_at_5
916
+ value: 84.548
917
+ - type: precision_at_1
918
+ value: 90.045
919
+ - type: precision_at_10
920
+ value: 41.91
921
+ - type: precision_at_100
922
+ value: 5.017
923
+ - type: precision_at_1000
924
+ value: 0.516
925
+ - type: precision_at_3
926
+ value: 75.15899999999999
927
+ - type: precision_at_5
928
+ value: 62.958000000000006
929
+ - type: recall_at_1
930
+ value: 27.483999999999998
931
+ - type: recall_at_10
932
+ value: 83.408
933
+ - type: recall_at_100
934
+ value: 95.514
935
+ - type: recall_at_1000
936
+ value: 98.65
937
+ - type: recall_at_3
938
+ value: 55.822
939
+ - type: recall_at_5
940
+ value: 69.868
941
+ - task:
942
+ type: Classification
943
+ dataset:
944
+ type: C-MTEB/TNews-classification
945
+ name: MTEB TNews
946
+ config: default
947
+ split: validation
948
+ revision: None
949
+ metrics:
950
+ - type: accuracy
951
+ value: 53.196
952
+ - type: f1
953
+ value: 51.51679244513836
954
+ - task:
955
+ type: Clustering
956
+ dataset:
957
+ type: C-MTEB/ThuNewsClusteringP2P
958
+ name: MTEB ThuNewsClusteringP2P
959
+ config: default
960
+ split: test
961
+ revision: None
962
+ metrics:
963
+ - type: v_measure
964
+ value: 67.87592101539063
965
+ - task:
966
+ type: Clustering
967
+ dataset:
968
+ type: C-MTEB/ThuNewsClusteringS2S
969
+ name: MTEB ThuNewsClusteringS2S
970
+ config: default
971
+ split: test
972
+ revision: None
973
+ metrics:
974
+ - type: v_measure
975
+ value: 62.4675464095125
976
+ - task:
977
+ type: Retrieval
978
+ dataset:
979
+ type: C-MTEB/VideoRetrieval
980
+ name: MTEB VideoRetrieval
981
+ config: default
982
+ split: dev
983
+ revision: None
984
+ metrics:
985
+ - type: map_at_1
986
+ value: 57.9
987
+ - type: map_at_10
988
+ value: 68.099
989
+ - type: map_at_100
990
+ value: 68.55499999999999
991
+ - type: map_at_1000
992
+ value: 68.566
993
+ - type: map_at_3
994
+ value: 66.4
995
+ - type: map_at_5
996
+ value: 67.46
997
+ - type: mrr_at_1
998
+ value: 57.9
999
+ - type: mrr_at_10
1000
+ value: 68.099
1001
+ - type: mrr_at_100
1002
+ value: 68.55499999999999
1003
+ - type: mrr_at_1000
1004
+ value: 68.566
1005
+ - type: mrr_at_3
1006
+ value: 66.4
1007
+ - type: mrr_at_5
1008
+ value: 67.46
1009
+ - type: ndcg_at_1
1010
+ value: 57.9
1011
+ - type: ndcg_at_10
1012
+ value: 72.555
1013
+ - type: ndcg_at_100
1014
+ value: 74.715
1015
+ - type: ndcg_at_1000
1016
+ value: 75.034
1017
+ - type: ndcg_at_3
1018
+ value: 69.102
1019
+ - type: ndcg_at_5
1020
+ value: 71.004
1021
+ - type: precision_at_1
1022
+ value: 57.9
1023
+ - type: precision_at_10
1024
+ value: 8.63
1025
+ - type: precision_at_100
1026
+ value: 0.963
1027
+ - type: precision_at_1000
1028
+ value: 0.099
1029
+ - type: precision_at_3
1030
+ value: 25.633
1031
+ - type: precision_at_5
1032
+ value: 16.3
1033
+ - type: recall_at_1
1034
+ value: 57.9
1035
+ - type: recall_at_10
1036
+ value: 86.3
1037
+ - type: recall_at_100
1038
+ value: 96.3
1039
+ - type: recall_at_1000
1040
+ value: 98.9
1041
+ - type: recall_at_3
1042
+ value: 76.9
1043
+ - type: recall_at_5
1044
+ value: 81.5
1045
+ - task:
1046
+ type: Classification
1047
+ dataset:
1048
+ type: C-MTEB/waimai-classification
1049
+ name: MTEB Waimai
1050
+ config: default
1051
+ split: test
1052
+ revision: None
1053
+ metrics:
1054
+ - type: accuracy
1055
+ value: 87.27000000000001
1056
+ - type: ap
1057
+ value: 71.10883470119464
1058
+ - type: f1
1059
+ value: 85.76618863591946
1060
+ ---
1061
+
1062
  # 1 开源清单
1063
 
1064
  本次开源2个通用向量编码模型和一个针对dialogue进行编码的向量模型，同时开源全量160万对话重写数据集和20万的难负例的检索数据集。