Improve model metadata
Browse filesHello!
## Pull Request overview
* Improve model metadata
## Details
This should make it a bit easier to find the model.
- Tom Aarsen
README.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
| 1 |
---
|
| 2 |
tags:
|
| 3 |
- mteb
|
|
|
|
|
|
|
| 4 |
model-index:
|
| 5 |
- name: NV-Embed-v2
|
| 6 |
results:
|
|
@@ -89,17 +91,17 @@ model-index:
|
|
| 89 |
- type: map_at_5
|
| 90 |
value: 61.027
|
| 91 |
- type: mrr_at_1
|
| 92 |
-
value: 0
|
| 93 |
- type: mrr_at_10
|
| 94 |
-
value: 0
|
| 95 |
- type: mrr_at_100
|
| 96 |
-
value: 0
|
| 97 |
- type: mrr_at_1000
|
| 98 |
-
value: 0
|
| 99 |
- type: mrr_at_3
|
| 100 |
-
value: 0
|
| 101 |
- type: mrr_at_5
|
| 102 |
-
value: 0
|
| 103 |
- type: ndcg_at_1
|
| 104 |
value: 46.515
|
| 105 |
- type: ndcg_at_10
|
|
@@ -277,17 +279,17 @@ model-index:
|
|
| 277 |
- type: map_at_5
|
| 278 |
value: 42.317083333333336
|
| 279 |
- type: mrr_at_1
|
| 280 |
-
value: 0
|
| 281 |
- type: mrr_at_10
|
| 282 |
-
value: 0
|
| 283 |
- type: mrr_at_100
|
| 284 |
-
value: 0
|
| 285 |
- type: mrr_at_1000
|
| 286 |
-
value: 0
|
| 287 |
- type: mrr_at_3
|
| 288 |
-
value: 0
|
| 289 |
- type: mrr_at_5
|
| 290 |
-
value: 0
|
| 291 |
- type: ndcg_at_1
|
| 292 |
value: 38.30616666666667
|
| 293 |
- type: ndcg_at_10
|
|
@@ -348,17 +350,17 @@ model-index:
|
|
| 348 |
- type: map_at_5
|
| 349 |
value: 31.955
|
| 350 |
- type: mrr_at_1
|
| 351 |
-
value: 0
|
| 352 |
- type: mrr_at_10
|
| 353 |
-
value: 0
|
| 354 |
- type: mrr_at_100
|
| 355 |
-
value: 0
|
| 356 |
- type: mrr_at_1000
|
| 357 |
-
value: 0
|
| 358 |
- type: mrr_at_3
|
| 359 |
-
value: 0
|
| 360 |
- type: mrr_at_5
|
| 361 |
-
value: 0
|
| 362 |
- type: ndcg_at_1
|
| 363 |
value: 44.104
|
| 364 |
- type: ndcg_at_10
|
|
@@ -419,19 +421,19 @@ model-index:
|
|
| 419 |
- type: map_at_5
|
| 420 |
value: 21.062
|
| 421 |
- type: mrr_at_1
|
| 422 |
-
value: 0
|
| 423 |
- type: mrr_at_10
|
| 424 |
-
value: 0
|
| 425 |
- type: mrr_at_100
|
| 426 |
-
value: 0
|
| 427 |
- type: mrr_at_1000
|
| 428 |
-
value: 0
|
| 429 |
- type: mrr_at_3
|
| 430 |
-
value: 0
|
| 431 |
- type: mrr_at_5
|
| 432 |
-
value: 0
|
| 433 |
- type: ndcg_at_1
|
| 434 |
-
value: 66
|
| 435 |
- type: ndcg_at_10
|
| 436 |
value: 53.496
|
| 437 |
- type: ndcg_at_100
|
|
@@ -451,7 +453,7 @@ model-index:
|
|
| 451 |
- type: precision_at_1000
|
| 452 |
value: 2.5940000000000003
|
| 453 |
- type: precision_at_3
|
| 454 |
-
value: 61
|
| 455 |
- type: precision_at_5
|
| 456 |
value: 54.65
|
| 457 |
- type: recall_at_1
|
|
@@ -509,17 +511,17 @@ model-index:
|
|
| 509 |
- type: map_at_5
|
| 510 |
value: 91.262
|
| 511 |
- type: mrr_at_1
|
| 512 |
-
value: 0
|
| 513 |
- type: mrr_at_10
|
| 514 |
-
value: 0
|
| 515 |
- type: mrr_at_100
|
| 516 |
-
value: 0
|
| 517 |
- type: mrr_at_1000
|
| 518 |
-
value: 0
|
| 519 |
- type: mrr_at_3
|
| 520 |
-
value: 0
|
| 521 |
- type: mrr_at_5
|
| 522 |
-
value: 0
|
| 523 |
- type: ndcg_at_1
|
| 524 |
value: 91.20899999999999
|
| 525 |
- type: ndcg_at_10
|
|
@@ -580,17 +582,17 @@ model-index:
|
|
| 580 |
- type: map_at_5
|
| 581 |
value: 55.054
|
| 582 |
- type: mrr_at_1
|
| 583 |
-
value: 0
|
| 584 |
- type: mrr_at_10
|
| 585 |
-
value: 0
|
| 586 |
- type: mrr_at_100
|
| 587 |
-
value: 0
|
| 588 |
- type: mrr_at_1000
|
| 589 |
-
value: 0
|
| 590 |
- type: mrr_at_3
|
| 591 |
-
value: 0
|
| 592 |
- type: mrr_at_5
|
| 593 |
-
value: 0
|
| 594 |
- type: ndcg_at_1
|
| 595 |
value: 64.815
|
| 596 |
- type: ndcg_at_10
|
|
@@ -651,17 +653,17 @@ model-index:
|
|
| 651 |
- type: map_at_5
|
| 652 |
value: 78.935
|
| 653 |
- type: mrr_at_1
|
| 654 |
-
value: 0
|
| 655 |
- type: mrr_at_10
|
| 656 |
-
value: 0
|
| 657 |
- type: mrr_at_100
|
| 658 |
-
value: 0
|
| 659 |
- type: mrr_at_1000
|
| 660 |
-
value: 0
|
| 661 |
- type: mrr_at_3
|
| 662 |
-
value: 0
|
| 663 |
- type: mrr_at_5
|
| 664 |
-
value: 0
|
| 665 |
- type: ndcg_at_1
|
| 666 |
value: 89.305
|
| 667 |
- type: ndcg_at_10
|
|
@@ -733,65 +735,65 @@ model-index:
|
|
| 733 |
type: mteb/msmarco
|
| 734 |
metrics:
|
| 735 |
- type: map_at_1
|
| 736 |
-
value: 0
|
| 737 |
- type: map_at_10
|
| 738 |
value: 38.342
|
| 739 |
- type: map_at_100
|
| 740 |
-
value: 0
|
| 741 |
- type: map_at_1000
|
| 742 |
-
value: 0
|
| 743 |
- type: map_at_3
|
| 744 |
-
value: 0
|
| 745 |
- type: map_at_5
|
| 746 |
-
value: 0
|
| 747 |
- type: mrr_at_1
|
| 748 |
-
value: 0
|
| 749 |
- type: mrr_at_10
|
| 750 |
-
value: 0
|
| 751 |
- type: mrr_at_100
|
| 752 |
-
value: 0
|
| 753 |
- type: mrr_at_1000
|
| 754 |
-
value: 0
|
| 755 |
- type: mrr_at_3
|
| 756 |
-
value: 0
|
| 757 |
- type: mrr_at_5
|
| 758 |
-
value: 0
|
| 759 |
- type: ndcg_at_1
|
| 760 |
-
value: 0
|
| 761 |
- type: ndcg_at_10
|
| 762 |
value: 45.629999999999995
|
| 763 |
- type: ndcg_at_100
|
| 764 |
-
value: 0
|
| 765 |
- type: ndcg_at_1000
|
| 766 |
-
value: 0
|
| 767 |
- type: ndcg_at_3
|
| 768 |
-
value: 0
|
| 769 |
- type: ndcg_at_5
|
| 770 |
-
value: 0
|
| 771 |
- type: precision_at_1
|
| 772 |
-
value: 0
|
| 773 |
- type: precision_at_10
|
| 774 |
value: 7.119000000000001
|
| 775 |
- type: precision_at_100
|
| 776 |
-
value: 0
|
| 777 |
- type: precision_at_1000
|
| 778 |
-
value: 0
|
| 779 |
- type: precision_at_3
|
| 780 |
-
value: 0
|
| 781 |
- type: precision_at_5
|
| 782 |
-
value: 0
|
| 783 |
- type: recall_at_1
|
| 784 |
-
value: 0
|
| 785 |
- type: recall_at_10
|
| 786 |
value: 67.972
|
| 787 |
- type: recall_at_100
|
| 788 |
-
value: 0
|
| 789 |
- type: recall_at_1000
|
| 790 |
-
value: 0
|
| 791 |
- type: recall_at_3
|
| 792 |
-
value: 0
|
| 793 |
- type: recall_at_5
|
| 794 |
-
value: 0
|
| 795 |
- type: main_score
|
| 796 |
value: 45.629999999999995
|
| 797 |
task:
|
|
@@ -937,17 +939,17 @@ model-index:
|
|
| 937 |
- type: map_at_5
|
| 938 |
value: 15.171000000000001
|
| 939 |
- type: mrr_at_1
|
| 940 |
-
value: 0
|
| 941 |
- type: mrr_at_10
|
| 942 |
-
value: 0
|
| 943 |
- type: mrr_at_100
|
| 944 |
-
value: 0
|
| 945 |
- type: mrr_at_1000
|
| 946 |
-
value: 0
|
| 947 |
- type: mrr_at_3
|
| 948 |
-
value: 0
|
| 949 |
- type: mrr_at_5
|
| 950 |
-
value: 0
|
| 951 |
- type: ndcg_at_1
|
| 952 |
value: 55.728
|
| 953 |
- type: ndcg_at_10
|
|
@@ -1008,17 +1010,17 @@ model-index:
|
|
| 1008 |
- type: map_at_5
|
| 1009 |
value: 65.364
|
| 1010 |
- type: mrr_at_1
|
| 1011 |
-
value: 0
|
| 1012 |
- type: mrr_at_10
|
| 1013 |
-
value: 0
|
| 1014 |
- type: mrr_at_100
|
| 1015 |
-
value: 0
|
| 1016 |
- type: mrr_at_1000
|
| 1017 |
-
value: 0
|
| 1018 |
- type: mrr_at_3
|
| 1019 |
-
value: 0
|
| 1020 |
- type: mrr_at_5
|
| 1021 |
-
value: 0
|
| 1022 |
- type: ndcg_at_1
|
| 1023 |
value: 55.417
|
| 1024 |
- type: ndcg_at_10
|
|
@@ -1079,17 +1081,17 @@ model-index:
|
|
| 1079 |
- type: map_at_5
|
| 1080 |
value: 84.396
|
| 1081 |
- type: mrr_at_1
|
| 1082 |
-
value: 0
|
| 1083 |
- type: mrr_at_10
|
| 1084 |
-
value: 0
|
| 1085 |
- type: mrr_at_100
|
| 1086 |
-
value: 0
|
| 1087 |
- type: mrr_at_1000
|
| 1088 |
-
value: 0
|
| 1089 |
- type: mrr_at_3
|
| 1090 |
-
value: 0
|
| 1091 |
- type: mrr_at_5
|
| 1092 |
-
value: 0
|
| 1093 |
- type: ndcg_at_1
|
| 1094 |
value: 82.12
|
| 1095 |
- type: ndcg_at_10
|
|
@@ -1180,17 +1182,17 @@ model-index:
|
|
| 1180 |
- type: map_at_5
|
| 1181 |
value: 11.158
|
| 1182 |
- type: mrr_at_1
|
| 1183 |
-
value: 0
|
| 1184 |
- type: mrr_at_10
|
| 1185 |
-
value: 0
|
| 1186 |
- type: mrr_at_100
|
| 1187 |
-
value: 0
|
| 1188 |
- type: mrr_at_1000
|
| 1189 |
-
value: 0
|
| 1190 |
- type: mrr_at_3
|
| 1191 |
-
value: 0
|
| 1192 |
- type: mrr_at_5
|
| 1193 |
-
value: 0
|
| 1194 |
- type: ndcg_at_1
|
| 1195 |
value: 26.3
|
| 1196 |
- type: ndcg_at_10
|
|
@@ -1473,19 +1475,19 @@ model-index:
|
|
| 1473 |
- type: map_at_5
|
| 1474 |
value: 74.74
|
| 1475 |
- type: mrr_at_1
|
| 1476 |
-
value: 0
|
| 1477 |
- type: mrr_at_10
|
| 1478 |
-
value: 0
|
| 1479 |
- type: mrr_at_100
|
| 1480 |
-
value: 0
|
| 1481 |
- type: mrr_at_1000
|
| 1482 |
-
value: 0
|
| 1483 |
- type: mrr_at_3
|
| 1484 |
-
value: 0
|
| 1485 |
- type: mrr_at_5
|
| 1486 |
-
value: 0
|
| 1487 |
- type: ndcg_at_1
|
| 1488 |
-
value: 66
|
| 1489 |
- type: ndcg_at_10
|
| 1490 |
value: 80.12700000000001
|
| 1491 |
- type: ndcg_at_100
|
|
@@ -1497,7 +1499,7 @@ model-index:
|
|
| 1497 |
- type: ndcg_at_5
|
| 1498 |
value: 78.827
|
| 1499 |
- type: precision_at_1
|
| 1500 |
-
value: 66
|
| 1501 |
- type: precision_at_10
|
| 1502 |
value: 10.567
|
| 1503 |
- type: precision_at_100
|
|
@@ -1515,7 +1517,7 @@ model-index:
|
|
| 1515 |
- type: recall_at_100
|
| 1516 |
value: 98.667
|
| 1517 |
- type: recall_at_1000
|
| 1518 |
-
value: 100
|
| 1519 |
- type: recall_at_3
|
| 1520 |
value: 83.322
|
| 1521 |
- type: recall_at_5
|
|
@@ -1679,19 +1681,19 @@ model-index:
|
|
| 1679 |
- type: map_at_5
|
| 1680 |
value: 1.185
|
| 1681 |
- type: mrr_at_1
|
| 1682 |
-
value: 0
|
| 1683 |
- type: mrr_at_10
|
| 1684 |
-
value: 0
|
| 1685 |
- type: mrr_at_100
|
| 1686 |
-
value: 0
|
| 1687 |
- type: mrr_at_1000
|
| 1688 |
-
value: 0
|
| 1689 |
- type: mrr_at_3
|
| 1690 |
-
value: 0
|
| 1691 |
- type: mrr_at_5
|
| 1692 |
-
value: 0
|
| 1693 |
- type: ndcg_at_1
|
| 1694 |
-
value: 91
|
| 1695 |
- type: ndcg_at_10
|
| 1696 |
value: 88.442
|
| 1697 |
- type: ndcg_at_100
|
|
@@ -1703,7 +1705,7 @@ model-index:
|
|
| 1703 |
- type: ndcg_at_5
|
| 1704 |
value: 89.562
|
| 1705 |
- type: precision_at_1
|
| 1706 |
-
value: 92
|
| 1707 |
- type: precision_at_10
|
| 1708 |
value: 92.60000000000001
|
| 1709 |
- type: precision_at_100
|
|
@@ -1711,7 +1713,7 @@ model-index:
|
|
| 1711 |
- type: precision_at_1000
|
| 1712 |
value: 28.222
|
| 1713 |
- type: precision_at_3
|
| 1714 |
-
value: 94
|
| 1715 |
- type: precision_at_5
|
| 1716 |
value: 93.60000000000001
|
| 1717 |
- type: recall_at_1
|
|
@@ -1750,17 +1752,17 @@ model-index:
|
|
| 1750 |
- type: map_at_5
|
| 1751 |
value: 9.49
|
| 1752 |
- type: mrr_at_1
|
| 1753 |
-
value: 0
|
| 1754 |
- type: mrr_at_10
|
| 1755 |
-
value: 0
|
| 1756 |
- type: mrr_at_100
|
| 1757 |
-
value: 0
|
| 1758 |
- type: mrr_at_1000
|
| 1759 |
-
value: 0
|
| 1760 |
- type: mrr_at_3
|
| 1761 |
-
value: 0
|
| 1762 |
- type: mrr_at_5
|
| 1763 |
-
value: 0
|
| 1764 |
- type: ndcg_at_1
|
| 1765 |
value: 47.959
|
| 1766 |
- type: ndcg_at_10
|
|
@@ -2003,6 +2005,7 @@ model-index:
|
|
| 2003 |
language:
|
| 2004 |
- en
|
| 2005 |
license: cc-by-nc-4.0
|
|
|
|
| 2006 |
---
|
| 2007 |
## Introduction
|
| 2008 |
We present NV-Embed-v2, a generalist embedding model that ranks No. 1 on the Massive Text Embedding Benchmark ([MTEB benchmark](https://huggingface.co/spaces/mteb/leaderboard))(as of Aug 30, 2024) with a score of 72.31 across 56 text embedding tasks. It also holds the No. 1 in the retrieval sub-category (a score of 62.65 across 15 tasks) in the leaderboard, which is essential to the development of RAG technology.
|
|
@@ -2183,4 +2186,4 @@ cd sentence-transformers
|
|
| 2183 |
git checkout v2.7-release
|
| 2184 |
# Modify L353 in SentenceTransformer.py to **'extra_features["prompt_length"] = tokenized_prompt["input_ids"].shape[-1]'**.
|
| 2185 |
pip install -e .
|
| 2186 |
-
```
|
|
|
|
| 1 |
---
|
| 2 |
tags:
|
| 3 |
- mteb
|
| 4 |
+
- transformers
|
| 5 |
+
- sentence-transformers
|
| 6 |
model-index:
|
| 7 |
- name: NV-Embed-v2
|
| 8 |
results:
|
|
|
|
| 91 |
- type: map_at_5
|
| 92 |
value: 61.027
|
| 93 |
- type: mrr_at_1
|
| 94 |
+
value: 0
|
| 95 |
- type: mrr_at_10
|
| 96 |
+
value: 0
|
| 97 |
- type: mrr_at_100
|
| 98 |
+
value: 0
|
| 99 |
- type: mrr_at_1000
|
| 100 |
+
value: 0
|
| 101 |
- type: mrr_at_3
|
| 102 |
+
value: 0
|
| 103 |
- type: mrr_at_5
|
| 104 |
+
value: 0
|
| 105 |
- type: ndcg_at_1
|
| 106 |
value: 46.515
|
| 107 |
- type: ndcg_at_10
|
|
|
|
| 279 |
- type: map_at_5
|
| 280 |
value: 42.317083333333336
|
| 281 |
- type: mrr_at_1
|
| 282 |
+
value: 0
|
| 283 |
- type: mrr_at_10
|
| 284 |
+
value: 0
|
| 285 |
- type: mrr_at_100
|
| 286 |
+
value: 0
|
| 287 |
- type: mrr_at_1000
|
| 288 |
+
value: 0
|
| 289 |
- type: mrr_at_3
|
| 290 |
+
value: 0
|
| 291 |
- type: mrr_at_5
|
| 292 |
+
value: 0
|
| 293 |
- type: ndcg_at_1
|
| 294 |
value: 38.30616666666667
|
| 295 |
- type: ndcg_at_10
|
|
|
|
| 350 |
- type: map_at_5
|
| 351 |
value: 31.955
|
| 352 |
- type: mrr_at_1
|
| 353 |
+
value: 0
|
| 354 |
- type: mrr_at_10
|
| 355 |
+
value: 0
|
| 356 |
- type: mrr_at_100
|
| 357 |
+
value: 0
|
| 358 |
- type: mrr_at_1000
|
| 359 |
+
value: 0
|
| 360 |
- type: mrr_at_3
|
| 361 |
+
value: 0
|
| 362 |
- type: mrr_at_5
|
| 363 |
+
value: 0
|
| 364 |
- type: ndcg_at_1
|
| 365 |
value: 44.104
|
| 366 |
- type: ndcg_at_10
|
|
|
|
| 421 |
- type: map_at_5
|
| 422 |
value: 21.062
|
| 423 |
- type: mrr_at_1
|
| 424 |
+
value: 0
|
| 425 |
- type: mrr_at_10
|
| 426 |
+
value: 0
|
| 427 |
- type: mrr_at_100
|
| 428 |
+
value: 0
|
| 429 |
- type: mrr_at_1000
|
| 430 |
+
value: 0
|
| 431 |
- type: mrr_at_3
|
| 432 |
+
value: 0
|
| 433 |
- type: mrr_at_5
|
| 434 |
+
value: 0
|
| 435 |
- type: ndcg_at_1
|
| 436 |
+
value: 66
|
| 437 |
- type: ndcg_at_10
|
| 438 |
value: 53.496
|
| 439 |
- type: ndcg_at_100
|
|
|
|
| 453 |
- type: precision_at_1000
|
| 454 |
value: 2.5940000000000003
|
| 455 |
- type: precision_at_3
|
| 456 |
+
value: 61
|
| 457 |
- type: precision_at_5
|
| 458 |
value: 54.65
|
| 459 |
- type: recall_at_1
|
|
|
|
| 511 |
- type: map_at_5
|
| 512 |
value: 91.262
|
| 513 |
- type: mrr_at_1
|
| 514 |
+
value: 0
|
| 515 |
- type: mrr_at_10
|
| 516 |
+
value: 0
|
| 517 |
- type: mrr_at_100
|
| 518 |
+
value: 0
|
| 519 |
- type: mrr_at_1000
|
| 520 |
+
value: 0
|
| 521 |
- type: mrr_at_3
|
| 522 |
+
value: 0
|
| 523 |
- type: mrr_at_5
|
| 524 |
+
value: 0
|
| 525 |
- type: ndcg_at_1
|
| 526 |
value: 91.20899999999999
|
| 527 |
- type: ndcg_at_10
|
|
|
|
| 582 |
- type: map_at_5
|
| 583 |
value: 55.054
|
| 584 |
- type: mrr_at_1
|
| 585 |
+
value: 0
|
| 586 |
- type: mrr_at_10
|
| 587 |
+
value: 0
|
| 588 |
- type: mrr_at_100
|
| 589 |
+
value: 0
|
| 590 |
- type: mrr_at_1000
|
| 591 |
+
value: 0
|
| 592 |
- type: mrr_at_3
|
| 593 |
+
value: 0
|
| 594 |
- type: mrr_at_5
|
| 595 |
+
value: 0
|
| 596 |
- type: ndcg_at_1
|
| 597 |
value: 64.815
|
| 598 |
- type: ndcg_at_10
|
|
|
|
| 653 |
- type: map_at_5
|
| 654 |
value: 78.935
|
| 655 |
- type: mrr_at_1
|
| 656 |
+
value: 0
|
| 657 |
- type: mrr_at_10
|
| 658 |
+
value: 0
|
| 659 |
- type: mrr_at_100
|
| 660 |
+
value: 0
|
| 661 |
- type: mrr_at_1000
|
| 662 |
+
value: 0
|
| 663 |
- type: mrr_at_3
|
| 664 |
+
value: 0
|
| 665 |
- type: mrr_at_5
|
| 666 |
+
value: 0
|
| 667 |
- type: ndcg_at_1
|
| 668 |
value: 89.305
|
| 669 |
- type: ndcg_at_10
|
|
|
|
| 735 |
type: mteb/msmarco
|
| 736 |
metrics:
|
| 737 |
- type: map_at_1
|
| 738 |
+
value: 0
|
| 739 |
- type: map_at_10
|
| 740 |
value: 38.342
|
| 741 |
- type: map_at_100
|
| 742 |
+
value: 0
|
| 743 |
- type: map_at_1000
|
| 744 |
+
value: 0
|
| 745 |
- type: map_at_3
|
| 746 |
+
value: 0
|
| 747 |
- type: map_at_5
|
| 748 |
+
value: 0
|
| 749 |
- type: mrr_at_1
|
| 750 |
+
value: 0
|
| 751 |
- type: mrr_at_10
|
| 752 |
+
value: 0
|
| 753 |
- type: mrr_at_100
|
| 754 |
+
value: 0
|
| 755 |
- type: mrr_at_1000
|
| 756 |
+
value: 0
|
| 757 |
- type: mrr_at_3
|
| 758 |
+
value: 0
|
| 759 |
- type: mrr_at_5
|
| 760 |
+
value: 0
|
| 761 |
- type: ndcg_at_1
|
| 762 |
+
value: 0
|
| 763 |
- type: ndcg_at_10
|
| 764 |
value: 45.629999999999995
|
| 765 |
- type: ndcg_at_100
|
| 766 |
+
value: 0
|
| 767 |
- type: ndcg_at_1000
|
| 768 |
+
value: 0
|
| 769 |
- type: ndcg_at_3
|
| 770 |
+
value: 0
|
| 771 |
- type: ndcg_at_5
|
| 772 |
+
value: 0
|
| 773 |
- type: precision_at_1
|
| 774 |
+
value: 0
|
| 775 |
- type: precision_at_10
|
| 776 |
value: 7.119000000000001
|
| 777 |
- type: precision_at_100
|
| 778 |
+
value: 0
|
| 779 |
- type: precision_at_1000
|
| 780 |
+
value: 0
|
| 781 |
- type: precision_at_3
|
| 782 |
+
value: 0
|
| 783 |
- type: precision_at_5
|
| 784 |
+
value: 0
|
| 785 |
- type: recall_at_1
|
| 786 |
+
value: 0
|
| 787 |
- type: recall_at_10
|
| 788 |
value: 67.972
|
| 789 |
- type: recall_at_100
|
| 790 |
+
value: 0
|
| 791 |
- type: recall_at_1000
|
| 792 |
+
value: 0
|
| 793 |
- type: recall_at_3
|
| 794 |
+
value: 0
|
| 795 |
- type: recall_at_5
|
| 796 |
+
value: 0
|
| 797 |
- type: main_score
|
| 798 |
value: 45.629999999999995
|
| 799 |
task:
|
|
|
|
| 939 |
- type: map_at_5
|
| 940 |
value: 15.171000000000001
|
| 941 |
- type: mrr_at_1
|
| 942 |
+
value: 0
|
| 943 |
- type: mrr_at_10
|
| 944 |
+
value: 0
|
| 945 |
- type: mrr_at_100
|
| 946 |
+
value: 0
|
| 947 |
- type: mrr_at_1000
|
| 948 |
+
value: 0
|
| 949 |
- type: mrr_at_3
|
| 950 |
+
value: 0
|
| 951 |
- type: mrr_at_5
|
| 952 |
+
value: 0
|
| 953 |
- type: ndcg_at_1
|
| 954 |
value: 55.728
|
| 955 |
- type: ndcg_at_10
|
|
|
|
| 1010 |
- type: map_at_5
|
| 1011 |
value: 65.364
|
| 1012 |
- type: mrr_at_1
|
| 1013 |
+
value: 0
|
| 1014 |
- type: mrr_at_10
|
| 1015 |
+
value: 0
|
| 1016 |
- type: mrr_at_100
|
| 1017 |
+
value: 0
|
| 1018 |
- type: mrr_at_1000
|
| 1019 |
+
value: 0
|
| 1020 |
- type: mrr_at_3
|
| 1021 |
+
value: 0
|
| 1022 |
- type: mrr_at_5
|
| 1023 |
+
value: 0
|
| 1024 |
- type: ndcg_at_1
|
| 1025 |
value: 55.417
|
| 1026 |
- type: ndcg_at_10
|
|
|
|
| 1081 |
- type: map_at_5
|
| 1082 |
value: 84.396
|
| 1083 |
- type: mrr_at_1
|
| 1084 |
+
value: 0
|
| 1085 |
- type: mrr_at_10
|
| 1086 |
+
value: 0
|
| 1087 |
- type: mrr_at_100
|
| 1088 |
+
value: 0
|
| 1089 |
- type: mrr_at_1000
|
| 1090 |
+
value: 0
|
| 1091 |
- type: mrr_at_3
|
| 1092 |
+
value: 0
|
| 1093 |
- type: mrr_at_5
|
| 1094 |
+
value: 0
|
| 1095 |
- type: ndcg_at_1
|
| 1096 |
value: 82.12
|
| 1097 |
- type: ndcg_at_10
|
|
|
|
| 1182 |
- type: map_at_5
|
| 1183 |
value: 11.158
|
| 1184 |
- type: mrr_at_1
|
| 1185 |
+
value: 0
|
| 1186 |
- type: mrr_at_10
|
| 1187 |
+
value: 0
|
| 1188 |
- type: mrr_at_100
|
| 1189 |
+
value: 0
|
| 1190 |
- type: mrr_at_1000
|
| 1191 |
+
value: 0
|
| 1192 |
- type: mrr_at_3
|
| 1193 |
+
value: 0
|
| 1194 |
- type: mrr_at_5
|
| 1195 |
+
value: 0
|
| 1196 |
- type: ndcg_at_1
|
| 1197 |
value: 26.3
|
| 1198 |
- type: ndcg_at_10
|
|
|
|
| 1475 |
- type: map_at_5
|
| 1476 |
value: 74.74
|
| 1477 |
- type: mrr_at_1
|
| 1478 |
+
value: 0
|
| 1479 |
- type: mrr_at_10
|
| 1480 |
+
value: 0
|
| 1481 |
- type: mrr_at_100
|
| 1482 |
+
value: 0
|
| 1483 |
- type: mrr_at_1000
|
| 1484 |
+
value: 0
|
| 1485 |
- type: mrr_at_3
|
| 1486 |
+
value: 0
|
| 1487 |
- type: mrr_at_5
|
| 1488 |
+
value: 0
|
| 1489 |
- type: ndcg_at_1
|
| 1490 |
+
value: 66
|
| 1491 |
- type: ndcg_at_10
|
| 1492 |
value: 80.12700000000001
|
| 1493 |
- type: ndcg_at_100
|
|
|
|
| 1499 |
- type: ndcg_at_5
|
| 1500 |
value: 78.827
|
| 1501 |
- type: precision_at_1
|
| 1502 |
+
value: 66
|
| 1503 |
- type: precision_at_10
|
| 1504 |
value: 10.567
|
| 1505 |
- type: precision_at_100
|
|
|
|
| 1517 |
- type: recall_at_100
|
| 1518 |
value: 98.667
|
| 1519 |
- type: recall_at_1000
|
| 1520 |
+
value: 100
|
| 1521 |
- type: recall_at_3
|
| 1522 |
value: 83.322
|
| 1523 |
- type: recall_at_5
|
|
|
|
| 1681 |
- type: map_at_5
|
| 1682 |
value: 1.185
|
| 1683 |
- type: mrr_at_1
|
| 1684 |
+
value: 0
|
| 1685 |
- type: mrr_at_10
|
| 1686 |
+
value: 0
|
| 1687 |
- type: mrr_at_100
|
| 1688 |
+
value: 0
|
| 1689 |
- type: mrr_at_1000
|
| 1690 |
+
value: 0
|
| 1691 |
- type: mrr_at_3
|
| 1692 |
+
value: 0
|
| 1693 |
- type: mrr_at_5
|
| 1694 |
+
value: 0
|
| 1695 |
- type: ndcg_at_1
|
| 1696 |
+
value: 91
|
| 1697 |
- type: ndcg_at_10
|
| 1698 |
value: 88.442
|
| 1699 |
- type: ndcg_at_100
|
|
|
|
| 1705 |
- type: ndcg_at_5
|
| 1706 |
value: 89.562
|
| 1707 |
- type: precision_at_1
|
| 1708 |
+
value: 92
|
| 1709 |
- type: precision_at_10
|
| 1710 |
value: 92.60000000000001
|
| 1711 |
- type: precision_at_100
|
|
|
|
| 1713 |
- type: precision_at_1000
|
| 1714 |
value: 28.222
|
| 1715 |
- type: precision_at_3
|
| 1716 |
+
value: 94
|
| 1717 |
- type: precision_at_5
|
| 1718 |
value: 93.60000000000001
|
| 1719 |
- type: recall_at_1
|
|
|
|
| 1752 |
- type: map_at_5
|
| 1753 |
value: 9.49
|
| 1754 |
- type: mrr_at_1
|
| 1755 |
+
value: 0
|
| 1756 |
- type: mrr_at_10
|
| 1757 |
+
value: 0
|
| 1758 |
- type: mrr_at_100
|
| 1759 |
+
value: 0
|
| 1760 |
- type: mrr_at_1000
|
| 1761 |
+
value: 0
|
| 1762 |
- type: mrr_at_3
|
| 1763 |
+
value: 0
|
| 1764 |
- type: mrr_at_5
|
| 1765 |
+
value: 0
|
| 1766 |
- type: ndcg_at_1
|
| 1767 |
value: 47.959
|
| 1768 |
- type: ndcg_at_10
|
|
|
|
| 2005 |
language:
|
| 2006 |
- en
|
| 2007 |
license: cc-by-nc-4.0
|
| 2008 |
+
base_model: mistralai/Mistral-7B-v0.1
|
| 2009 |
---
|
| 2010 |
## Introduction
|
| 2011 |
We present NV-Embed-v2, a generalist embedding model that ranks No. 1 on the Massive Text Embedding Benchmark ([MTEB benchmark](https://huggingface.co/spaces/mteb/leaderboard))(as of Aug 30, 2024) with a score of 72.31 across 56 text embedding tasks. It also holds the No. 1 in the retrieval sub-category (a score of 62.65 across 15 tasks) in the leaderboard, which is essential to the development of RAG technology.
|
|
|
|
| 2186 |
git checkout v2.7-release
|
| 2187 |
# Modify L353 in SentenceTransformer.py to **'extra_features["prompt_length"] = tokenized_prompt["input_ids"].shape[-1]'**.
|
| 2188 |
pip install -e .
|
| 2189 |
+
```
|