Hugging Face's logo

BAAI
/
bge-reranker-v2.5-gemma2-lightweight

Text Classification
sentence-transformers
Safetensors
Transformers
multilingual
cost_wise_gemma
text-generation
custom_code
Model card Files Community
7
Shitao commited on
Commit
e1eac59
•
1 Parent(s): 3d1b260

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +39 -39
README.md CHANGED
@@ -362,45 +362,45 @@ with torch.no_grad():
362
 
363
  - **BEIR:**
364
 
365
- | BEIR | bge-large-en-v1.5 | Bge-rearanker v2 m3 | jina-reranker-v2-base-multilingual | bge-reranker-v2.5-gemma2-lightweight | bge-reranker-v2.5-gemma2-lightweight |
366
- | :----------------: | :---------------: | :-----------------: | :--------------------------------: | :----------------------------------: | :----------------------------------: |
367
- | **Save** **Flops** | - | - | - | 60% | 0 |
368
- | **ArguAna** | 63.54 | 37.7 | 52.23 | 86.04 | 86.16 |
369
- | **ClimateFEVER** | 36.49 | 37.99 | 34.65 | 48.41 | 48.48 |
370
- | **CQA** | 42.23 | 38.24 | 40.21 | 49.18 | 48.9 |
371
- | **DBPedia** | 44.16 | 48.15 | 49.31 | 51.98 | 52.11 |
372
- | **FEVER** | 87.17 | 90.15 | 92.44 | 94.71 | 94.69 |
373
- | **FiQA2018** | 44.97 | 49.32 | 45.88 | 60.48 | 60.95 |
374
- | **HotpotQA** | 74.11 | 84.51 | 81.81 | 87.84 | 87.89 |
375
- | **MSMARCO** | 42.48 | 47.79 | 47.83 | 47.23 | 47.26 |
376
- | **NFCorpus** | 38.12 | 34.85 | 37.73 | 41.4 | 41.64 |
377
- | **NQ** | 55.04 | 69.37 | 67.35 | 75.37 | 75.58 |
378
- | **QuoraRetrieval** | 89.06 | 89.13 | 87.81 | 91.25 | 91.18 |
379
- | **SCIDOCS** | 22.62 | 18.25 | 20.21 | 23.71 | 23.87 |
380
- | **SciFact** | 74.64 | 73.08 | 76.93 | 80.5 | 80.38 |
381
- | **Touche2020** | 25.08 | 35.68 | 32.45 | 30.64 | 31.09 |
382
- | **TRECCOVID** | 74.89 | 83.39 | 80.89 | 84.26 | 84.85 |
383
- | **Mean** | 54.31 | 55.36 | 56.52 | 63.1 | **63.67** |
384
-
385
- | BEIR | e5-mistral-7b-instruct | Bge-rearanker v2 m3 | bge-reranker-v2.5-gemma-lightweight | bge-reranker-v2.5-gemma-lightweight |
386
- | :----------------: | :--------------------: | :-----------------: | :---------------------------------: | :---------------------------------: |
387
- | **Save Flops** | - | - | 60% | 0 |
388
- | **ArguAna** | 61.8 | 79.05 | 86.02 | 86.58 |
389
- | **ClimateFEVER** | 38.37 | 37.66 | 47.27 | 47.13 |
390
- | **CQA** | 42.97 | 46.16 | 49.06 | 49.53 |
391
- | **DBPedia** | 48.84 | 50.77 | 52.45 | 52.87 |
392
- | **FEVER** | 87.82 | 91.36 | 94.85 | 95.19 |
393
- | **FiQA2018** | 56.58 | 50.96 | 58.81 | 61.19 |
394
- | **HotpotQA** | 75.72 | 86.99 | 88.49 | 88.82 |
395
- | **MSMARCO** | 43.06 | 48.35 | 47.65 | 47.4 |
396
- | **NFCorpus** | 38.58 | 39.25 | 42.28 | 42.17 |
397
- | **NQ** | 63.56 | 73.44 | 75 | 76.28 |
398
- | **QuoraRetrieval** | 89.59 | 90.44 | 91.09 | 91.18 |
399
- | **SCIDOCS** | 16.3 | 20.77 | 22.2 | 22.69 |
400
- | **SciFact** | 76.26 | 77.78 | 79.94 | 80.98 |
401
- | **Touche2020** | 26.24 | 35.79 | 28.69 | 31.17 |
402
- | **TRECCOVID** | 87.07 | 88.13 | 86.61 | 87.36 |
403
- | **Mean** | 56.85 | 61.13 | 63.36 | **64.04** |
404
 
405
  - **MIRACL**:
406
 
 
362
 
363
  - **BEIR:**
364
 
365
+ | BEIR | bge-large-en-v1.5 | Bge-rearanker v2 m3 | jina-reranker-v2-base-multilingual | bge-reranker-v2-gemma | bge-reranker-v2.5-gemma2-lightweight | bge-reranker-v2.5-gemma2-lightweight |
366
+ | :----------------: | :---------------: | :-----------------: | :--------------------------------: | :-------------------: | :----------------------------------: | :----------------------------------: |
367
+ | **Save** **Flops** | - | - | - | - | 60% | 0 |
368
+ | **ArguAna** | 63.54 | 37.7 | 52.23 | 78.68 | 86.04 | 86.16 |
369
+ | **ClimateFEVER** | 36.49 | 37.99 | 34.65 | 39.07 | 48.41 | 48.48 |
370
+ | **CQA** | 42.23 | 38.24 | 40.21 | 45.85 | 49.18 | 48.9 |
371
+ | **DBPedia** | 44.16 | 48.15 | 49.31 | 49.92 | 51.98 | 52.11 |
372
+ | **FEVER** | 87.17 | 90.15 | 92.44 | 90.15 | 94.71 | 94.69 |
373
+ | **FiQA2018** | 44.97 | 49.32 | 45.88 | 49.32 | 60.48 | 60.95 |
374
+ | **HotpotQA** | 74.11 | 84.51 | 81.81 | 86.15 | 87.84 | 87.89 |
375
+ | **MSMARCO** | 42.48 | 47.79 | 47.83 | 48.07 | 47.23 | 47.26 |
376
+ | **NFCorpus** | 38.12 | 34.85 | 37.73 | 39.73 | 41.4 | 41.64 |
377
+ | **NQ** | 55.04 | 69.37 | 67.35 | 72.6 | 75.37 | 75.58 |
378
+ | **QuoraRetrieval** | 89.06 | 89.13 | 87.81 | 90.37 | 91.25 | 91.18 |
379
+ | **SCIDOCS** | 22.62 | 18.25 | 20.21 | 21.65 | 23.71 | 23.87 |
380
+ | **SciFact** | 74.64 | 73.08 | 76.93 | 77.22 | 80.5 | 80.38 |
381
+ | **Touche2020** | 25.08 | 35.68 | 32.45 | 35.68 | 30.64 | 31.09 |
382
+ | **TRECCOVID** | 74.89 | 83.39 | 80.89 | 85.51 | 84.26 | 84.85 |
383
+ | **Mean** | 54.31 | 55.36 | 56.52 | 60.71 | 63.1 | **63.67** |
384
+
385
+ | BEIR | e5-mistral-7b-instruct | bge-reranker-v2-gemma | bge-reranker-v2.5-gemma-lightweight | bge-reranker-v2.5-gemma-lightweight |
386
+ | :----------------: | :--------------------: | :-------------------: | :---------------------------------: | :---------------------------------: |
387
+ | **Save Flops** | - | - | 60% | 0 |
388
+ | **ArguAna** | 61.8 | 79.05 | 86.02 | 86.58 |
389
+ | **ClimateFEVER** | 38.37 | 37.66 | 47.27 | 47.13 |
390
+ | **CQA** | 42.97 | 46.16 | 49.06 | 49.53 |
391
+ | **DBPedia** | 48.84 | 50.77 | 52.45 | 52.87 |
392
+ | **FEVER** | 87.82 | 91.36 | 94.85 | 95.19 |
393
+ | **FiQA2018** | 56.58 | 50.96 | 58.81 | 61.19 |
394
+ | **HotpotQA** | 75.72 | 86.99 | 88.49 | 88.82 |
395
+ | **MSMARCO** | 43.06 | 48.35 | 47.65 | 47.4 |
396
+ | **NFCorpus** | 38.58 | 39.25 | 42.28 | 42.17 |
397
+ | **NQ** | 63.56 | 73.44 | 75 | 76.28 |
398
+ | **QuoraRetrieval** | 89.59 | 90.44 | 91.09 | 91.18 |
399
+ | **SCIDOCS** | 16.3 | 20.77 | 22.2 | 22.69 |
400
+ | **SciFact** | 76.26 | 77.78 | 79.94 | 80.98 |
401
+ | **Touche2020** | 26.24 | 35.79 | 28.69 | 31.17 |
402
+ | **TRECCOVID** | 87.07 | 88.13 | 86.61 | 87.36 |
403
+ | **Mean** | 56.85 | 61.13 | 63.36 | **64.04** |
404
 
405
  - **MIRACL**:
406