antoinelouis
commited on
Commit
•
45157f8
1
Parent(s):
b4c344a
Update README.md
Browse files
README.md
CHANGED
@@ -83,12 +83,12 @@ RAG.search(query="Comment effectuer une recherche avec ColBERT ?", k=10)
|
|
83 |
## Evaluation
|
84 |
|
85 |
The model is evaluated on the smaller development set of mMARCO-fr, which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compared its
|
86 |
-
performance with other publicly available 🇫🇷 ColBERT models (as well as one single-vector representation model) fine-tuned on the same dataset. We report the
|
87 |
-
|
88 |
|
89 |
| model | #Param.(↓) | Size | Dim. | Index | R@1000 | R@500 | R@100 | R@10 | MRR@10 |
|
90 |
|:-----------------------------------------------------------------------------------------------------------|-----------:|------:|-----:|------:|-------:|------:|------:|-----:|-------:|
|
91 |
-
| **colbertv2-camembert-L4-mmarcoFR** | 54M | 0.2GB | 32 |
|
92 |
| [FraColBERTv2](https://huggingface.co/bclavie/FraColBERTv2) | 111M | 0.4GB | 128 | 28GB | 90.0 | 88.9 | 81.2 | 57.1 | 32.4 |
|
93 |
| [colbertv1-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/colbertv1-camembert-base-mmarcoFR) | 111M | 0.4GB | 128 | 28GB | 89.7 | 88.4 | 80.0 | 54.2 | 29.5 |
|
94 |
| [biencoder-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-base-mmarcoFR) | 111M | 0.4GB | 128 | 28GB | - | 89.1 | 77.8 | 51.5 | 28.5 |
|
|
|
83 |
## Evaluation
|
84 |
|
85 |
The model is evaluated on the smaller development set of mMARCO-fr, which consists of 6,980 queries for a corpus of 8.8M candidate passages. Below, we compared its
|
86 |
+
performance with other publicly available 🇫🇷 ColBERT models (as well as one single-vector representation model) fine-tuned on the same dataset. We report the recall
|
87 |
+
at various cut-offs (R@k) and the mean reciprocal rank at cut-off 10 (MRR@10).
|
88 |
|
89 |
| model | #Param.(↓) | Size | Dim. | Index | R@1000 | R@500 | R@100 | R@10 | MRR@10 |
|
90 |
|:-----------------------------------------------------------------------------------------------------------|-----------:|------:|-----:|------:|-------:|------:|------:|-----:|-------:|
|
91 |
+
| **colbertv2-camembert-L4-mmarcoFR** | 54M | 0.2GB | 32 | 9GB | 91.9 | 90.3 | 81.9 | 56.7 | 32.3 |
|
92 |
| [FraColBERTv2](https://huggingface.co/bclavie/FraColBERTv2) | 111M | 0.4GB | 128 | 28GB | 90.0 | 88.9 | 81.2 | 57.1 | 32.4 |
|
93 |
| [colbertv1-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/colbertv1-camembert-base-mmarcoFR) | 111M | 0.4GB | 128 | 28GB | 89.7 | 88.4 | 80.0 | 54.2 | 29.5 |
|
94 |
| [biencoder-camembert-base-mmarcoFR](https://huggingface.co/antoinelouis/biencoder-camembert-base-mmarcoFR) | 111M | 0.4GB | 128 | 28GB | - | 89.1 | 77.8 | 51.5 | 28.5 |
|