diff --git "a/README.md" "b/README.md" new file mode 100644--- /dev/null +++ "b/README.md" @@ -0,0 +1,9032 @@ +--- +library_name: model2vec +license: mit +model-index: +- name: potion-base-2M + results: + - dataset: + config: en-ext + name: MTEB AmazonCounterfactualClassification (en-ext) + revision: e8379541af4e31359cca9fbcf4b00f2671dba205 + split: test + type: mteb/amazon_counterfactual + metrics: + - type: accuracy + value: 64.09295352323838 + - type: ap + value: 15.890517627297978 + - type: ap_weighted + value: 15.890517627297978 + - type: f1 + value: 52.38020164592307 + - type: f1_weighted + value: 71.02083973787023 + - type: main_score + value: 64.09295352323838 + task: + type: Classification + - dataset: + config: en + name: MTEB AmazonCounterfactualClassification (en) + revision: e8379541af4e31359cca9fbcf4b00f2671dba205 + split: test + type: mteb/amazon_counterfactual + metrics: + - type: accuracy + value: 65.44776119402985 + - type: ap + value: 27.63770109073248 + - type: ap_weighted + value: 27.63770109073248 + - type: f1 + value: 58.90706680555824 + - type: f1_weighted + value: 68.76825531256598 + - type: main_score + value: 65.44776119402985 + task: + type: Classification + - dataset: + config: default + name: MTEB AmazonPolarityClassification (default) + revision: e2d317d38cd51312af73b3d32a06d1a08b442046 + split: test + type: mteb/amazon_polarity + metrics: + - type: accuracy + value: 70.8279 + - type: ap + value: 65.28490580225534 + - type: ap_weighted + value: 65.28490580225534 + - type: f1 + value: 70.57831663695143 + - type: f1_weighted + value: 70.5783166369514 + - type: main_score + value: 70.8279 + task: + type: Classification + - dataset: + config: en + name: MTEB AmazonReviewsClassification (en) + revision: 1399c76144fd37290681b995c656ef9b2e06e26d + split: test + type: mteb/amazon_reviews_multi + metrics: + - type: accuracy + value: 32.996 + - type: f1 + value: 32.31726739771069 + - type: f1_weighted + value: 32.31726739771067 + - type: main_score + value: 32.996 + task: + type: Classification + - dataset: + config: default + name: MTEB ArguAna (default) + revision: c22ab2a51041ffd869aaddef7af8d8215647e41a + split: test + type: mteb/arguana + metrics: + - type: main_score + value: 32.622 + - type: map_at_1 + value: 15.931999999999999 + - type: map_at_10 + value: 26.404 + - type: map_at_100 + value: 27.697 + - type: map_at_1000 + value: 27.755000000000003 + - type: map_at_20 + value: 27.195000000000004 + - type: map_at_3 + value: 22.819 + - type: map_at_5 + value: 24.714 + - type: mrr_at_1 + value: 16.35846372688478 + - type: mrr_at_10 + value: 26.554703199440034 + - type: mrr_at_100 + value: 27.847935657025847 + - type: mrr_at_1000 + value: 27.90589599760921 + - type: mrr_at_20 + value: 27.34605509902883 + - type: mrr_at_3 + value: 22.97297297297296 + - type: mrr_at_5 + value: 24.839971550497825 + - type: nauc_map_at_1000_diff1 + value: 9.371299425433035 + - type: nauc_map_at_1000_max + value: 2.9574792255471287 + - type: nauc_map_at_1000_std + value: 11.687859698836608 + - type: nauc_map_at_100_diff1 + value: 9.371053628047203 + - type: nauc_map_at_100_max + value: 2.9809791205622322 + - type: nauc_map_at_100_std + value: 11.75333361929675 + - type: nauc_map_at_10_diff1 + value: 9.252241357696322 + - type: nauc_map_at_10_max + value: 2.742017052724334 + - type: nauc_map_at_10_std + value: 11.558508764008751 + - type: nauc_map_at_1_diff1 + value: 12.16316227081504 + - type: nauc_map_at_1_max + value: -0.5028808771807869 + - type: nauc_map_at_1_std + value: 6.962721581652243 + - type: nauc_map_at_20_diff1 + value: 9.300115601627917 + - type: nauc_map_at_20_max + value: 2.9569034588818686 + - type: nauc_map_at_20_std + value: 11.691557039773556 + - type: nauc_map_at_3_diff1 + value: 8.848778277311055 + - type: nauc_map_at_3_max + value: 1.423235443130067 + - type: nauc_map_at_3_std + value: 8.992118754001654 + - type: nauc_map_at_5_diff1 + value: 8.57377738557976 + - type: nauc_map_at_5_max + value: 1.6847244703988815 + - type: nauc_map_at_5_std + value: 10.164700751752981 + - type: nauc_mrr_at_1000_diff1 + value: 7.83234430121121 + - type: nauc_mrr_at_1000_max + value: 2.791250763636318 + - type: nauc_mrr_at_1000_std + value: 11.622332535373612 + - type: nauc_mrr_at_100_diff1 + value: 7.834999518447015 + - type: nauc_mrr_at_100_max + value: 2.8149797974038266 + - type: nauc_mrr_at_100_std + value: 11.687441656533483 + - type: nauc_mrr_at_10_diff1 + value: 7.762034718278307 + - type: nauc_mrr_at_10_max + value: 2.588154258263355 + - type: nauc_mrr_at_10_std + value: 11.496120973626677 + - type: nauc_mrr_at_1_diff1 + value: 10.195687129188947 + - type: nauc_mrr_at_1_max + value: 0.7959780949834511 + - type: nauc_mrr_at_1_std + value: 6.909734959320861 + - type: nauc_mrr_at_20_diff1 + value: 7.782191305042387 + - type: nauc_mrr_at_20_max + value: 2.793166533360494 + - type: nauc_mrr_at_20_std + value: 11.627007183010022 + - type: nauc_mrr_at_3_diff1 + value: 7.24231818837065 + - type: nauc_mrr_at_3_max + value: 1.1207446053044912 + - type: nauc_mrr_at_3_std + value: 8.80503737855467 + - type: nauc_mrr_at_5_diff1 + value: 6.979716982325695 + - type: nauc_mrr_at_5_max + value: 1.4170822735031223 + - type: nauc_mrr_at_5_std + value: 10.1236436941688 + - type: nauc_ndcg_at_1000_diff1 + value: 9.442383068205185 + - type: nauc_ndcg_at_1000_max + value: 5.123695329561358 + - type: nauc_ndcg_at_1000_std + value: 14.437791974669423 + - type: nauc_ndcg_at_100_diff1 + value: 9.708550273174609 + - type: nauc_ndcg_at_100_max + value: 6.051680933998194 + - type: nauc_ndcg_at_100_std + value: 16.356231738002254 + - type: nauc_ndcg_at_10_diff1 + value: 9.067738290559637 + - type: nauc_ndcg_at_10_max + value: 5.051562540309537 + - type: nauc_ndcg_at_10_std + value: 14.981456308721722 + - type: nauc_ndcg_at_1_diff1 + value: 12.16316227081504 + - type: nauc_ndcg_at_1_max + value: -0.5028808771807869 + - type: nauc_ndcg_at_1_std + value: 6.962721581652243 + - type: nauc_ndcg_at_20_diff1 + value: 9.231508996935032 + - type: nauc_ndcg_at_20_max + value: 5.908400445912541 + - type: nauc_ndcg_at_20_std + value: 15.59551518646074 + - type: nauc_ndcg_at_3_diff1 + value: 8.176316410933907 + - type: nauc_ndcg_at_3_max + value: 2.274031400606708 + - type: nauc_ndcg_at_3_std + value: 9.692557288413111 + - type: nauc_ndcg_at_5_diff1 + value: 7.594758622405593 + - type: nauc_ndcg_at_5_max + value: 2.6813579129239575 + - type: nauc_ndcg_at_5_std + value: 11.657985683999456 + - type: nauc_precision_at_1000_diff1 + value: 10.05696671180073 + - type: nauc_precision_at_1000_max + value: 38.62495739567785 + - type: nauc_precision_at_1000_std + value: 48.13074377805322 + - type: nauc_precision_at_100_diff1 + value: 14.277579964693818 + - type: nauc_precision_at_100_max + value: 25.831877593548068 + - type: nauc_precision_at_100_std + value: 48.257692065653586 + - type: nauc_precision_at_10_diff1 + value: 8.967213665268718 + - type: nauc_precision_at_10_max + value: 11.661929553714575 + - type: nauc_precision_at_10_std + value: 24.98165263864555 + - type: nauc_precision_at_1_diff1 + value: 12.16316227081504 + - type: nauc_precision_at_1_max + value: -0.5028808771807869 + - type: nauc_precision_at_1_std + value: 6.962721581652243 + - type: nauc_precision_at_20_diff1 + value: 9.596515632249602 + - type: nauc_precision_at_20_max + value: 16.17218527097958 + - type: nauc_precision_at_20_std + value: 29.12126867872034 + - type: nauc_precision_at_3_diff1 + value: 6.602134310609642 + - type: nauc_precision_at_3_max + value: 4.422669583221979 + - type: nauc_precision_at_3_std + value: 11.430754758385522 + - type: nauc_precision_at_5_diff1 + value: 5.2002210955386445 + - type: nauc_precision_at_5_max + value: 5.146945531205673 + - type: nauc_precision_at_5_std + value: 15.431582644403685 + - type: nauc_recall_at_1000_diff1 + value: 10.056966711801207 + - type: nauc_recall_at_1000_max + value: 38.62495739567756 + - type: nauc_recall_at_1000_std + value: 48.1307437780539 + - type: nauc_recall_at_100_diff1 + value: 14.277579964693778 + - type: nauc_recall_at_100_max + value: 25.83187759354808 + - type: nauc_recall_at_100_std + value: 48.257692065653615 + - type: nauc_recall_at_10_diff1 + value: 8.967213665268691 + - type: nauc_recall_at_10_max + value: 11.661929553714536 + - type: nauc_recall_at_10_std + value: 24.98165263864557 + - type: nauc_recall_at_1_diff1 + value: 12.16316227081504 + - type: nauc_recall_at_1_max + value: -0.5028808771807869 + - type: nauc_recall_at_1_std + value: 6.962721581652243 + - type: nauc_recall_at_20_diff1 + value: 9.596515632249584 + - type: nauc_recall_at_20_max + value: 16.17218527097958 + - type: nauc_recall_at_20_std + value: 29.121268678720334 + - type: nauc_recall_at_3_diff1 + value: 6.602134310609631 + - type: nauc_recall_at_3_max + value: 4.4226695832219765 + - type: nauc_recall_at_3_std + value: 11.430754758385536 + - type: nauc_recall_at_5_diff1 + value: 5.200221095538592 + - type: nauc_recall_at_5_max + value: 5.146945531205657 + - type: nauc_recall_at_5_std + value: 15.431582644403655 + - type: ndcg_at_1 + value: 15.931999999999999 + - type: ndcg_at_10 + value: 32.622 + - type: ndcg_at_100 + value: 39.182 + - type: ndcg_at_1000 + value: 40.814 + - type: ndcg_at_20 + value: 35.538 + - type: ndcg_at_3 + value: 25.119000000000003 + - type: ndcg_at_5 + value: 28.541 + - type: precision_at_1 + value: 15.931999999999999 + - type: precision_at_10 + value: 5.27 + - type: precision_at_100 + value: 0.84 + - type: precision_at_1000 + value: 0.097 + - type: precision_at_20 + value: 3.215 + - type: precision_at_3 + value: 10.597 + - type: precision_at_5 + value: 8.023 + - type: recall_at_1 + value: 15.931999999999999 + - type: recall_at_10 + value: 52.703 + - type: recall_at_100 + value: 83.997 + - type: recall_at_1000 + value: 97.013 + - type: recall_at_20 + value: 64.29599999999999 + - type: recall_at_3 + value: 31.791999999999998 + - type: recall_at_5 + value: 40.114 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ArxivClusteringP2P (default) + revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d + split: test + type: mteb/arxiv-clustering-p2p + metrics: + - type: main_score + value: 29.870127302809124 + - type: v_measure + value: 29.870127302809124 + - type: v_measure_std + value: 14.791231720290682 + task: + type: Clustering + - dataset: + config: default + name: MTEB ArxivClusteringS2S (default) + revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 + split: test + type: mteb/arxiv-clustering-s2s + metrics: + - type: main_score + value: 20.120157976895523 + - type: v_measure + value: 20.120157976895523 + - type: v_measure_std + value: 15.985610307944178 + task: + type: Clustering + - dataset: + config: default + name: MTEB AskUbuntuDupQuestions (default) + revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 + split: test + type: mteb/askubuntudupquestions-reranking + metrics: + - type: main_score + value: 52.90637925416103 + - type: map + value: 52.90637925416103 + - type: mrr + value: 66.15365167304226 + - type: nAUC_map_diff1 + value: 9.30133487550967 + - type: nAUC_map_max + value: 18.70463131454981 + - type: nAUC_map_std + value: -0.14109850923583017 + - type: nAUC_mrr_diff1 + value: 14.055448816273671 + - type: nAUC_mrr_max + value: 24.008690838618477 + - type: nAUC_mrr_std + value: 4.127979271888478 + task: + type: Reranking + - dataset: + config: default + name: MTEB BIOSSES (default) + revision: d3fb88f8f02e40887cd149695127462bbcf29b4a + split: test + type: mteb/biosses-sts + metrics: + - type: cosine_pearson + value: 69.51991057082712 + - type: cosine_spearman + value: 64.10808725228159 + - type: euclidean_pearson + value: 68.61144541101957 + - type: euclidean_spearman + value: 64.10808725228159 + - type: main_score + value: 64.10808725228159 + - type: manhattan_pearson + value: 68.51780712764004 + - type: manhattan_spearman + value: 66.6352532692086 + - type: pearson + value: 69.51991057082712 + - type: spearman + value: 64.10808725228159 + task: + type: STS + - dataset: + config: default + name: MTEB Banking77Classification (default) + revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 + split: test + type: mteb/banking77 + metrics: + - type: accuracy + value: 65.17207792207793 + - type: f1 + value: 63.62144343754335 + - type: f1_weighted + value: 63.62144343754335 + - type: main_score + value: 65.17207792207793 + task: + type: Classification + - dataset: + config: default + name: MTEB BiorxivClusteringP2P (default) + revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 + split: test + type: mteb/biorxiv-clustering-p2p + metrics: + - type: main_score + value: 25.780291675770933 + - type: v_measure + value: 25.780291675770933 + - type: v_measure_std + value: 0.5140442536046052 + task: + type: Clustering + - dataset: + config: default + name: MTEB BiorxivClusteringS2S (default) + revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 + split: test + type: mteb/biorxiv-clustering-s2s + metrics: + - type: main_score + value: 14.938305313404305 + - type: v_measure + value: 14.938305313404305 + - type: v_measure_std + value: 0.6925176157191298 + task: + type: Clustering + - dataset: + config: default + name: MTEB CQADupstackAndroidRetrieval (default) + revision: f46a197baaae43b4f621051089b82a364682dfeb + split: test + type: mteb/cqadupstack-android + metrics: + - type: main_score + value: 25.330000000000002 + - type: map_at_1 + value: 14.652000000000001 + - type: map_at_10 + value: 20.697 + - type: map_at_100 + value: 21.601 + - type: map_at_1000 + value: 21.741 + - type: map_at_20 + value: 21.182000000000002 + - type: map_at_3 + value: 18.176000000000002 + - type: map_at_5 + value: 19.43 + - type: mrr_at_1 + value: 18.88412017167382 + - type: mrr_at_10 + value: 25.220780252969078 + - type: mrr_at_100 + value: 25.948898254396696 + - type: mrr_at_1000 + value: 26.026660358556764 + - type: mrr_at_20 + value: 25.57115097233221 + - type: mrr_at_3 + value: 22.746781115879823 + - type: mrr_at_5 + value: 23.984263233190266 + - type: nauc_map_at_1000_diff1 + value: 42.12290261948617 + - type: nauc_map_at_1000_max + value: 25.055151779659003 + - type: nauc_map_at_1000_std + value: -2.210774158888407 + - type: nauc_map_at_100_diff1 + value: 42.103292224612055 + - type: nauc_map_at_100_max + value: 25.024932512461167 + - type: nauc_map_at_100_std + value: -2.2434491051465666 + - type: nauc_map_at_10_diff1 + value: 42.43897329865005 + - type: nauc_map_at_10_max + value: 24.900985870032148 + - type: nauc_map_at_10_std + value: -3.1698848459287547 + - type: nauc_map_at_1_diff1 + value: 50.52605319846898 + - type: nauc_map_at_1_max + value: 26.416683733723296 + - type: nauc_map_at_1_std + value: -5.956774956817916 + - type: nauc_map_at_20_diff1 + value: 42.16171369615511 + - type: nauc_map_at_20_max + value: 24.885527318331327 + - type: nauc_map_at_20_std + value: -2.5912307656872033 + - type: nauc_map_at_3_diff1 + value: 45.269307730870345 + - type: nauc_map_at_3_max + value: 24.920202708925796 + - type: nauc_map_at_3_std + value: -3.8211698120619553 + - type: nauc_map_at_5_diff1 + value: 43.518058300179305 + - type: nauc_map_at_5_max + value: 24.525800979439225 + - type: nauc_map_at_5_std + value: -3.7992949173792248 + - type: nauc_mrr_at_1000_diff1 + value: 40.614363171344095 + - type: nauc_mrr_at_1000_max + value: 25.461648381508994 + - type: nauc_mrr_at_1000_std + value: -3.0170351047974697 + - type: nauc_mrr_at_100_diff1 + value: 40.60930462215041 + - type: nauc_mrr_at_100_max + value: 25.438960747808164 + - type: nauc_mrr_at_100_std + value: -2.9834713770918477 + - type: nauc_mrr_at_10_diff1 + value: 40.73347197221245 + - type: nauc_mrr_at_10_max + value: 25.462525362516896 + - type: nauc_mrr_at_10_std + value: -3.6376857514606833 + - type: nauc_mrr_at_1_diff1 + value: 47.29968987535835 + - type: nauc_mrr_at_1_max + value: 28.554321882610132 + - type: nauc_mrr_at_1_std + value: -7.160060969495176 + - type: nauc_mrr_at_20_diff1 + value: 40.609457206778366 + - type: nauc_mrr_at_20_max + value: 25.385706784725997 + - type: nauc_mrr_at_20_std + value: -3.2196439087267446 + - type: nauc_mrr_at_3_diff1 + value: 42.40692706796631 + - type: nauc_mrr_at_3_max + value: 25.77574851888572 + - type: nauc_mrr_at_3_std + value: -4.8469351394956135 + - type: nauc_mrr_at_5_diff1 + value: 41.22146669637393 + - type: nauc_mrr_at_5_max + value: 25.328517631105935 + - type: nauc_mrr_at_5_std + value: -4.572547216622222 + - type: nauc_ndcg_at_1000_diff1 + value: 38.07049187253007 + - type: nauc_ndcg_at_1000_max + value: 25.746482146110424 + - type: nauc_ndcg_at_1000_std + value: 3.058232130723082 + - type: nauc_ndcg_at_100_diff1 + value: 37.72243748027699 + - type: nauc_ndcg_at_100_max + value: 24.739589495666532 + - type: nauc_ndcg_at_100_std + value: 3.1186035597586583 + - type: nauc_ndcg_at_10_diff1 + value: 38.986427400509506 + - type: nauc_ndcg_at_10_max + value: 24.145586682086602 + - type: nauc_ndcg_at_10_std + value: -1.3984860683398967 + - type: nauc_ndcg_at_1_diff1 + value: 47.29968987535835 + - type: nauc_ndcg_at_1_max + value: 28.554321882610132 + - type: nauc_ndcg_at_1_std + value: -7.160060969495176 + - type: nauc_ndcg_at_20_diff1 + value: 38.340556985152666 + - type: nauc_ndcg_at_20_max + value: 23.852700466978412 + - type: nauc_ndcg_at_20_std + value: 0.39156203034556514 + - type: nauc_ndcg_at_3_diff1 + value: 42.73433715617584 + - type: nauc_ndcg_at_3_max + value: 24.171983374932484 + - type: nauc_ndcg_at_3_std + value: -3.2561069261029916 + - type: nauc_ndcg_at_5_diff1 + value: 40.6293880411303 + - type: nauc_ndcg_at_5_max + value: 23.470270149785282 + - type: nauc_ndcg_at_5_std + value: -3.25295826799678 + - type: nauc_precision_at_1000_diff1 + value: 14.532481445869708 + - type: nauc_precision_at_1000_max + value: 2.4063597129179874 + - type: nauc_precision_at_1000_std + value: 5.08743191359027 + - type: nauc_precision_at_100_diff1 + value: 16.229283902136586 + - type: nauc_precision_at_100_max + value: 14.68488845425874 + - type: nauc_precision_at_100_std + value: 11.53673782607175 + - type: nauc_precision_at_10_diff1 + value: 22.658121443906644 + - type: nauc_precision_at_10_max + value: 20.458468048702898 + - type: nauc_precision_at_10_std + value: 2.549916225317396 + - type: nauc_precision_at_1_diff1 + value: 47.29968987535835 + - type: nauc_precision_at_1_max + value: 28.554321882610132 + - type: nauc_precision_at_1_std + value: -7.160060969495176 + - type: nauc_precision_at_20_diff1 + value: 20.64109922418946 + - type: nauc_precision_at_20_max + value: 18.105770725108478 + - type: nauc_precision_at_20_std + value: 6.61515398984593 + - type: nauc_precision_at_3_diff1 + value: 34.10042771425271 + - type: nauc_precision_at_3_max + value: 22.672104646433855 + - type: nauc_precision_at_3_std + value: -2.2992673003241033 + - type: nauc_precision_at_5_diff1 + value: 28.555115169767404 + - type: nauc_precision_at_5_max + value: 21.203888592575485 + - type: nauc_precision_at_5_std + value: -1.7718460407125416 + - type: nauc_recall_at_1000_diff1 + value: 15.35267593542552 + - type: nauc_recall_at_1000_max + value: 33.67891449185779 + - type: nauc_recall_at_1000_std + value: 33.03686303481503 + - type: nauc_recall_at_100_diff1 + value: 22.251888567786047 + - type: nauc_recall_at_100_max + value: 22.200691564139312 + - type: nauc_recall_at_100_std + value: 21.990679346902265 + - type: nauc_recall_at_10_diff1 + value: 28.995014251584863 + - type: nauc_recall_at_10_max + value: 20.072781093522806 + - type: nauc_recall_at_10_std + value: 3.72566933318097 + - type: nauc_recall_at_1_diff1 + value: 50.52605319846898 + - type: nauc_recall_at_1_max + value: 26.416683733723296 + - type: nauc_recall_at_1_std + value: -5.956774956817916 + - type: nauc_recall_at_20_diff1 + value: 26.562818478114973 + - type: nauc_recall_at_20_max + value: 19.231280299972486 + - type: nauc_recall_at_20_std + value: 8.678225293725014 + - type: nauc_recall_at_3_diff1 + value: 39.31561233983533 + - type: nauc_recall_at_3_max + value: 20.930101667953906 + - type: nauc_recall_at_3_std + value: -0.6675587418511502 + - type: nauc_recall_at_5_diff1 + value: 34.09061124292363 + - type: nauc_recall_at_5_max + value: 18.66212328855417 + - type: nauc_recall_at_5_std + value: -0.7309156199742269 + - type: ndcg_at_1 + value: 18.884 + - type: ndcg_at_10 + value: 25.330000000000002 + - type: ndcg_at_100 + value: 29.803 + - type: ndcg_at_1000 + value: 33.22 + - type: ndcg_at_20 + value: 26.817 + - type: ndcg_at_3 + value: 20.903 + - type: ndcg_at_5 + value: 22.653000000000002 + - type: precision_at_1 + value: 18.884 + - type: precision_at_10 + value: 5.165 + - type: precision_at_100 + value: 0.9169999999999999 + - type: precision_at_1000 + value: 0.152 + - type: precision_at_20 + value: 3.0620000000000003 + - type: precision_at_3 + value: 10.205 + - type: precision_at_5 + value: 7.668 + - type: recall_at_1 + value: 14.652000000000001 + - type: recall_at_10 + value: 34.839999999999996 + - type: recall_at_100 + value: 55.254999999999995 + - type: recall_at_1000 + value: 79.499 + - type: recall_at_20 + value: 40.519 + - type: recall_at_3 + value: 21.951999999999998 + - type: recall_at_5 + value: 26.866 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackEnglishRetrieval (default) + revision: ad9991cb51e31e31e430383c75ffb2885547b5f0 + split: test + type: mteb/cqadupstack-english + metrics: + - type: main_score + value: 19.03 + - type: map_at_1 + value: 12.038 + - type: map_at_10 + value: 16.209 + - type: map_at_100 + value: 16.852 + - type: map_at_1000 + value: 16.957 + - type: map_at_20 + value: 16.503 + - type: map_at_3 + value: 14.918999999999999 + - type: map_at_5 + value: 15.628 + - type: mrr_at_1 + value: 14.522292993630574 + - type: mrr_at_10 + value: 19.049211404306938 + - type: mrr_at_100 + value: 19.68616846000299 + - type: mrr_at_1000 + value: 19.76021263547599 + - type: mrr_at_20 + value: 19.357303026531586 + - type: mrr_at_3 + value: 17.6963906581741 + - type: mrr_at_5 + value: 18.42887473460721 + - type: nauc_map_at_1000_diff1 + value: 40.668452256157586 + - type: nauc_map_at_1000_max + value: 12.03068593940894 + - type: nauc_map_at_1000_std + value: 8.282626956376456 + - type: nauc_map_at_100_diff1 + value: 40.69689331326516 + - type: nauc_map_at_100_max + value: 11.975907653133852 + - type: nauc_map_at_100_std + value: 8.200723943582632 + - type: nauc_map_at_10_diff1 + value: 40.92675356589961 + - type: nauc_map_at_10_max + value: 12.162793044434746 + - type: nauc_map_at_10_std + value: 7.416703287646554 + - type: nauc_map_at_1_diff1 + value: 50.680707097806824 + - type: nauc_map_at_1_max + value: 14.950571652895167 + - type: nauc_map_at_1_std + value: 7.7584022650339355 + - type: nauc_map_at_20_diff1 + value: 40.83556248327022 + - type: nauc_map_at_20_max + value: 12.10303655820891 + - type: nauc_map_at_20_std + value: 7.902382326647041 + - type: nauc_map_at_3_diff1 + value: 43.05716913197717 + - type: nauc_map_at_3_max + value: 13.059797217109528 + - type: nauc_map_at_3_std + value: 7.086563876751212 + - type: nauc_map_at_5_diff1 + value: 41.805198491838716 + - type: nauc_map_at_5_max + value: 12.76978465765044 + - type: nauc_map_at_5_std + value: 7.062456814961006 + - type: nauc_mrr_at_1000_diff1 + value: 39.04694353040439 + - type: nauc_mrr_at_1000_max + value: 12.815481060933905 + - type: nauc_mrr_at_1000_std + value: 9.1263110333354 + - type: nauc_mrr_at_100_diff1 + value: 39.048743374446275 + - type: nauc_mrr_at_100_max + value: 12.798690926603784 + - type: nauc_mrr_at_100_std + value: 9.122224737593758 + - type: nauc_mrr_at_10_diff1 + value: 39.31192280486829 + - type: nauc_mrr_at_10_max + value: 12.98896180731353 + - type: nauc_mrr_at_10_std + value: 8.589942965121546 + - type: nauc_mrr_at_1_diff1 + value: 48.851762997569374 + - type: nauc_mrr_at_1_max + value: 17.448204383555886 + - type: nauc_mrr_at_1_std + value: 9.957963740437934 + - type: nauc_mrr_at_20_diff1 + value: 39.194781081736885 + - type: nauc_mrr_at_20_max + value: 12.96253606083205 + - type: nauc_mrr_at_20_std + value: 8.972866893546797 + - type: nauc_mrr_at_3_diff1 + value: 41.01782917626344 + - type: nauc_mrr_at_3_max + value: 13.603475495611184 + - type: nauc_mrr_at_3_std + value: 8.897651838362023 + - type: nauc_mrr_at_5_diff1 + value: 40.19192790045331 + - type: nauc_mrr_at_5_max + value: 13.548960496378678 + - type: nauc_mrr_at_5_std + value: 8.41651532405131 + - type: nauc_ndcg_at_1000_diff1 + value: 35.354979509339366 + - type: nauc_ndcg_at_1000_max + value: 10.336450702196894 + - type: nauc_ndcg_at_1000_std + value: 10.554226856804888 + - type: nauc_ndcg_at_100_diff1 + value: 35.76413836339596 + - type: nauc_ndcg_at_100_max + value: 9.670128921989843 + - type: nauc_ndcg_at_100_std + value: 10.17476854541832 + - type: nauc_ndcg_at_10_diff1 + value: 36.94656640954636 + - type: nauc_ndcg_at_10_max + value: 10.643065285351891 + - type: nauc_ndcg_at_10_std + value: 7.806040737820652 + - type: nauc_ndcg_at_1_diff1 + value: 48.851762997569374 + - type: nauc_ndcg_at_1_max + value: 17.448204383555886 + - type: nauc_ndcg_at_1_std + value: 9.957963740437934 + - type: nauc_ndcg_at_20_diff1 + value: 36.73652715597696 + - type: nauc_ndcg_at_20_max + value: 10.555021619182972 + - type: nauc_ndcg_at_20_std + value: 9.079373122526354 + - type: nauc_ndcg_at_3_diff1 + value: 40.4128640487012 + - type: nauc_ndcg_at_3_max + value: 12.91614218382092 + - type: nauc_ndcg_at_3_std + value: 7.903843158502924 + - type: nauc_ndcg_at_5_diff1 + value: 38.74411532365887 + - type: nauc_ndcg_at_5_max + value: 12.179514645439847 + - type: nauc_ndcg_at_5_std + value: 7.336617986486788 + - type: nauc_precision_at_1000_diff1 + value: 1.9011163507603661 + - type: nauc_precision_at_1000_max + value: 7.990884111944549 + - type: nauc_precision_at_1000_std + value: 21.061967908108002 + - type: nauc_precision_at_100_diff1 + value: 12.33281445055559 + - type: nauc_precision_at_100_max + value: 8.880349742023368 + - type: nauc_precision_at_100_std + value: 21.013372083124928 + - type: nauc_precision_at_10_diff1 + value: 21.650071006760747 + - type: nauc_precision_at_10_max + value: 9.318883411833756 + - type: nauc_precision_at_10_std + value: 11.730645859949483 + - type: nauc_precision_at_1_diff1 + value: 48.851762997569374 + - type: nauc_precision_at_1_max + value: 17.448204383555886 + - type: nauc_precision_at_1_std + value: 9.957963740437934 + - type: nauc_precision_at_20_diff1 + value: 20.35610346048067 + - type: nauc_precision_at_20_max + value: 9.924634585296353 + - type: nauc_precision_at_20_std + value: 16.14897238644223 + - type: nauc_precision_at_3_diff1 + value: 31.173178334965346 + - type: nauc_precision_at_3_max + value: 13.51252773710169 + - type: nauc_precision_at_3_std + value: 10.580704859270702 + - type: nauc_precision_at_5_diff1 + value: 26.650379822403686 + - type: nauc_precision_at_5_max + value: 12.638452808323358 + - type: nauc_precision_at_5_std + value: 10.624172954473973 + - type: nauc_recall_at_1000_diff1 + value: 20.894977442792207 + - type: nauc_recall_at_1000_max + value: 4.351275412385671 + - type: nauc_recall_at_1000_std + value: 13.465338160416145 + - type: nauc_recall_at_100_diff1 + value: 23.773806443855143 + - type: nauc_recall_at_100_max + value: 1.7786515196265826 + - type: nauc_recall_at_100_std + value: 13.316142806464812 + - type: nauc_recall_at_10_diff1 + value: 28.238969776585414 + - type: nauc_recall_at_10_max + value: 5.795998046983078 + - type: nauc_recall_at_10_std + value: 6.472324467861674 + - type: nauc_recall_at_1_diff1 + value: 50.680707097806824 + - type: nauc_recall_at_1_max + value: 14.950571652895167 + - type: nauc_recall_at_1_std + value: 7.7584022650339355 + - type: nauc_recall_at_20_diff1 + value: 27.607217008926234 + - type: nauc_recall_at_20_max + value: 5.759176414257366 + - type: nauc_recall_at_20_std + value: 10.465779842554717 + - type: nauc_recall_at_3_diff1 + value: 36.11483064950007 + - type: nauc_recall_at_3_max + value: 9.989993667730074 + - type: nauc_recall_at_3_std + value: 5.790709363191019 + - type: nauc_recall_at_5_diff1 + value: 32.48686329315466 + - type: nauc_recall_at_5_max + value: 9.036821419906465 + - type: nauc_recall_at_5_std + value: 4.936776376797586 + - type: ndcg_at_1 + value: 14.521999999999998 + - type: ndcg_at_10 + value: 19.03 + - type: ndcg_at_100 + value: 22.418 + - type: ndcg_at_1000 + value: 25.174000000000003 + - type: ndcg_at_20 + value: 20.008 + - type: ndcg_at_3 + value: 16.699 + - type: ndcg_at_5 + value: 17.718999999999998 + - type: precision_at_1 + value: 14.521999999999998 + - type: precision_at_10 + value: 3.49 + - type: precision_at_100 + value: 0.641 + - type: precision_at_1000 + value: 0.11 + - type: precision_at_20 + value: 2.057 + - type: precision_at_3 + value: 7.856000000000001 + - type: precision_at_5 + value: 5.631 + - type: recall_at_1 + value: 12.038 + - type: recall_at_10 + value: 24.585 + - type: recall_at_100 + value: 40.032000000000004 + - type: recall_at_1000 + value: 59.343999999999994 + - type: recall_at_20 + value: 28.224 + - type: recall_at_3 + value: 17.95 + - type: recall_at_5 + value: 20.605 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackGamingRetrieval (default) + revision: 4885aa143210c98657558c04aaf3dc47cfb54340 + split: test + type: mteb/cqadupstack-gaming + metrics: + - type: main_score + value: 29.583 + - type: map_at_1 + value: 18.041 + - type: map_at_10 + value: 25.406000000000002 + - type: map_at_100 + value: 26.375 + - type: map_at_1000 + value: 26.473999999999997 + - type: map_at_20 + value: 25.936999999999998 + - type: map_at_3 + value: 23.250999999999998 + - type: map_at_5 + value: 24.503 + - type: mrr_at_1 + value: 21.065830721003135 + - type: mrr_at_10 + value: 28.189505896402416 + - type: mrr_at_100 + value: 29.027452479410755 + - type: mrr_at_1000 + value: 29.097293909258422 + - type: mrr_at_20 + value: 28.662884995323772 + - type: mrr_at_3 + value: 26.175548589341673 + - type: mrr_at_5 + value: 27.363636363636324 + - type: nauc_map_at_1000_diff1 + value: 37.796326336571276 + - type: nauc_map_at_1000_max + value: 22.220500792581 + - type: nauc_map_at_1000_std + value: -2.194799091612798 + - type: nauc_map_at_100_diff1 + value: 37.790675383791196 + - type: nauc_map_at_100_max + value: 22.18614999731695 + - type: nauc_map_at_100_std + value: -2.2470039249188556 + - type: nauc_map_at_10_diff1 + value: 38.05208350844174 + - type: nauc_map_at_10_max + value: 21.89935352381032 + - type: nauc_map_at_10_std + value: -2.98869512787041 + - type: nauc_map_at_1_diff1 + value: 43.17642739404867 + - type: nauc_map_at_1_max + value: 20.42250394733745 + - type: nauc_map_at_1_std + value: -5.1467721201242815 + - type: nauc_map_at_20_diff1 + value: 37.88392749116593 + - type: nauc_map_at_20_max + value: 22.112588076399714 + - type: nauc_map_at_20_std + value: -2.5454188620897344 + - type: nauc_map_at_3_diff1 + value: 38.95611714844621 + - type: nauc_map_at_3_max + value: 21.111778563401586 + - type: nauc_map_at_3_std + value: -4.030551789691853 + - type: nauc_map_at_5_diff1 + value: 38.21256789270451 + - type: nauc_map_at_5_max + value: 21.630014652906855 + - type: nauc_map_at_5_std + value: -3.590513047822652 + - type: nauc_mrr_at_1000_diff1 + value: 36.87803483972353 + - type: nauc_mrr_at_1000_max + value: 23.879462874038296 + - type: nauc_mrr_at_1000_std + value: -0.22072500218761223 + - type: nauc_mrr_at_100_diff1 + value: 36.855693968543044 + - type: nauc_mrr_at_100_max + value: 23.86918435310746 + - type: nauc_mrr_at_100_std + value: -0.2373419919431428 + - type: nauc_mrr_at_10_diff1 + value: 37.101094342552734 + - type: nauc_mrr_at_10_max + value: 23.891367825494495 + - type: nauc_mrr_at_10_std + value: -0.5793444512066471 + - type: nauc_mrr_at_1_diff1 + value: 42.42316114487945 + - type: nauc_mrr_at_1_max + value: 22.68818224435017 + - type: nauc_mrr_at_1_std + value: -3.303022083526107 + - type: nauc_mrr_at_20_diff1 + value: 36.89206331322041 + - type: nauc_mrr_at_20_max + value: 23.89115604434676 + - type: nauc_mrr_at_20_std + value: -0.3403716687972552 + - type: nauc_mrr_at_3_diff1 + value: 37.521708109041334 + - type: nauc_mrr_at_3_max + value: 23.449137234856188 + - type: nauc_mrr_at_3_std + value: -1.4620893509854138 + - type: nauc_mrr_at_5_diff1 + value: 37.025745633521915 + - type: nauc_mrr_at_5_max + value: 23.75830801312528 + - type: nauc_mrr_at_5_std + value: -1.0488330632921128 + - type: nauc_ndcg_at_1000_diff1 + value: 35.587510810642115 + - type: nauc_ndcg_at_1000_max + value: 24.203987312028563 + - type: nauc_ndcg_at_1000_std + value: 2.620871846305902 + - type: nauc_ndcg_at_100_diff1 + value: 35.12129268842065 + - type: nauc_ndcg_at_100_max + value: 23.65718178818282 + - type: nauc_ndcg_at_100_std + value: 1.8047464668718496 + - type: nauc_ndcg_at_10_diff1 + value: 36.010772727724536 + - type: nauc_ndcg_at_10_max + value: 22.90551457795281 + - type: nauc_ndcg_at_10_std + value: -1.0357681514957635 + - type: nauc_ndcg_at_1_diff1 + value: 42.42316114487945 + - type: nauc_ndcg_at_1_max + value: 22.68818224435017 + - type: nauc_ndcg_at_1_std + value: -3.303022083526107 + - type: nauc_ndcg_at_20_diff1 + value: 35.40541309259456 + - type: nauc_ndcg_at_20_max + value: 23.11326844908543 + - type: nauc_ndcg_at_20_std + value: 0.1368197147455866 + - type: nauc_ndcg_at_3_diff1 + value: 37.13532824824766 + - type: nauc_ndcg_at_3_max + value: 22.0500118152094 + - type: nauc_ndcg_at_3_std + value: -2.9320120660361626 + - type: nauc_ndcg_at_5_diff1 + value: 36.07908412831893 + - type: nauc_ndcg_at_5_max + value: 22.602680088691933 + - type: nauc_ndcg_at_5_std + value: -2.3155102885765224 + - type: nauc_precision_at_1000_diff1 + value: 4.577550609194703 + - type: nauc_precision_at_1000_max + value: 26.78992905112731 + - type: nauc_precision_at_1000_std + value: 25.97267966871786 + - type: nauc_precision_at_100_diff1 + value: 13.994381284162882 + - type: nauc_precision_at_100_max + value: 28.409033283319435 + - type: nauc_precision_at_100_std + value: 19.54360923075281 + - type: nauc_precision_at_10_diff1 + value: 26.12260312750217 + - type: nauc_precision_at_10_max + value: 26.95132125829464 + - type: nauc_precision_at_10_std + value: 6.680006112016357 + - type: nauc_precision_at_1_diff1 + value: 42.42316114487945 + - type: nauc_precision_at_1_max + value: 22.68818224435017 + - type: nauc_precision_at_1_std + value: -3.303022083526107 + - type: nauc_precision_at_20_diff1 + value: 22.016448533332603 + - type: nauc_precision_at_20_max + value: 27.456103557217652 + - type: nauc_precision_at_20_std + value: 11.45451026915214 + - type: nauc_precision_at_3_diff1 + value: 32.23575594151647 + - type: nauc_precision_at_3_max + value: 24.64944574707706 + - type: nauc_precision_at_3_std + value: 0.4270460916222822 + - type: nauc_precision_at_5_diff1 + value: 28.184806696208724 + - type: nauc_precision_at_5_max + value: 26.024592101042877 + - type: nauc_precision_at_5_std + value: 2.7638864153340785 + - type: nauc_recall_at_1000_diff1 + value: 28.164928645360614 + - type: nauc_recall_at_1000_max + value: 31.31853591627895 + - type: nauc_recall_at_1000_std + value: 29.79149359900253 + - type: nauc_recall_at_100_diff1 + value: 26.247172017074462 + - type: nauc_recall_at_100_max + value: 25.072512911620848 + - type: nauc_recall_at_100_std + value: 14.124550084941525 + - type: nauc_recall_at_10_diff1 + value: 30.58061655649643 + - type: nauc_recall_at_10_max + value: 22.667693055140635 + - type: nauc_recall_at_10_std + value: 1.9018004971795854 + - type: nauc_recall_at_1_diff1 + value: 43.17642739404867 + - type: nauc_recall_at_1_max + value: 20.42250394733745 + - type: nauc_recall_at_1_std + value: -5.1467721201242815 + - type: nauc_recall_at_20_diff1 + value: 27.988315641021494 + - type: nauc_recall_at_20_max + value: 22.796849228410014 + - type: nauc_recall_at_20_std + value: 5.50275370960407 + - type: nauc_recall_at_3_diff1 + value: 33.43292455380229 + - type: nauc_recall_at_3_max + value: 20.805193280031343 + - type: nauc_recall_at_3_std + value: -2.581776145023966 + - type: nauc_recall_at_5_diff1 + value: 30.835768599716566 + - type: nauc_recall_at_5_max + value: 21.93578745163432 + - type: nauc_recall_at_5_std + value: -1.268567019686916 + - type: ndcg_at_1 + value: 21.066 + - type: ndcg_at_10 + value: 29.583 + - type: ndcg_at_100 + value: 34.281 + - type: ndcg_at_1000 + value: 36.876 + - type: ndcg_at_20 + value: 31.339 + - type: ndcg_at_3 + value: 25.572 + - type: ndcg_at_5 + value: 27.61 + - type: precision_at_1 + value: 21.066 + - type: precision_at_10 + value: 5.009 + - type: precision_at_100 + value: 0.8109999999999999 + - type: precision_at_1000 + value: 0.11100000000000002 + - type: precision_at_20 + value: 2.972 + - type: precision_at_3 + value: 11.661000000000001 + - type: precision_at_5 + value: 8.364 + - type: recall_at_1 + value: 18.041 + - type: recall_at_10 + value: 39.545 + - type: recall_at_100 + value: 60.827 + - type: recall_at_1000 + value: 80.255 + - type: recall_at_20 + value: 46.076 + - type: recall_at_3 + value: 28.872999999999998 + - type: recall_at_5 + value: 33.771 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackGisRetrieval (default) + revision: 5003b3064772da1887988e05400cf3806fe491f2 + split: test + type: mteb/cqadupstack-gis + metrics: + - type: main_score + value: 14.183000000000002 + - type: map_at_1 + value: 9.024000000000001 + - type: map_at_10 + value: 12.02 + - type: map_at_100 + value: 12.692999999999998 + - type: map_at_1000 + value: 12.797 + - type: map_at_20 + value: 12.353 + - type: map_at_3 + value: 10.932 + - type: map_at_5 + value: 11.296000000000001 + - type: mrr_at_1 + value: 9.717514124293785 + - type: mrr_at_10 + value: 12.945879293336926 + - type: mrr_at_100 + value: 13.611537580723049 + - type: mrr_at_1000 + value: 13.711450596451852 + - type: mrr_at_20 + value: 13.25786962702532 + - type: mrr_at_3 + value: 11.770244821092277 + - type: mrr_at_5 + value: 12.171374764595104 + - type: nauc_map_at_1000_diff1 + value: 29.694382411892878 + - type: nauc_map_at_1000_max + value: 8.928542212540712 + - type: nauc_map_at_1000_std + value: -8.388892090698437 + - type: nauc_map_at_100_diff1 + value: 29.71725096884429 + - type: nauc_map_at_100_max + value: 8.920381106437004 + - type: nauc_map_at_100_std + value: -8.43007776575813 + - type: nauc_map_at_10_diff1 + value: 30.15550030704659 + - type: nauc_map_at_10_max + value: 8.77661748937144 + - type: nauc_map_at_10_std + value: -9.270013574850484 + - type: nauc_map_at_1_diff1 + value: 35.9745839417964 + - type: nauc_map_at_1_max + value: 8.983726756853555 + - type: nauc_map_at_1_std + value: -11.543240200322865 + - type: nauc_map_at_20_diff1 + value: 29.937715854896386 + - type: nauc_map_at_20_max + value: 8.928620938274623 + - type: nauc_map_at_20_std + value: -8.728423357602786 + - type: nauc_map_at_3_diff1 + value: 32.944694198956206 + - type: nauc_map_at_3_max + value: 8.952369834670906 + - type: nauc_map_at_3_std + value: -10.766696316204687 + - type: nauc_map_at_5_diff1 + value: 32.631999531576085 + - type: nauc_map_at_5_max + value: 8.918457144741136 + - type: nauc_map_at_5_std + value: -10.136893335843391 + - type: nauc_mrr_at_1000_diff1 + value: 28.618528255212343 + - type: nauc_mrr_at_1000_max + value: 9.826758587627383 + - type: nauc_mrr_at_1000_std + value: -7.368476011211808 + - type: nauc_mrr_at_100_diff1 + value: 28.632394726892514 + - type: nauc_mrr_at_100_max + value: 9.839544124304876 + - type: nauc_mrr_at_100_std + value: -7.39471497056017 + - type: nauc_mrr_at_10_diff1 + value: 28.93313107887261 + - type: nauc_mrr_at_10_max + value: 9.691269483479616 + - type: nauc_mrr_at_10_std + value: -8.100092578456541 + - type: nauc_mrr_at_1_diff1 + value: 34.18603002613899 + - type: nauc_mrr_at_1_max + value: 8.97972973713456 + - type: nauc_mrr_at_1_std + value: -9.556153409751639 + - type: nauc_mrr_at_20_diff1 + value: 28.880085769654883 + - type: nauc_mrr_at_20_max + value: 9.83539863841401 + - type: nauc_mrr_at_20_std + value: -7.6711928068068715 + - type: nauc_mrr_at_3_diff1 + value: 31.40284372995198 + - type: nauc_mrr_at_3_max + value: 9.594465679874684 + - type: nauc_mrr_at_3_std + value: -9.400155113213387 + - type: nauc_mrr_at_5_diff1 + value: 31.18905330799594 + - type: nauc_mrr_at_5_max + value: 9.787223698234424 + - type: nauc_mrr_at_5_std + value: -8.944299902971345 + - type: nauc_ndcg_at_1000_diff1 + value: 23.98343656497298 + - type: nauc_ndcg_at_1000_max + value: 9.548662140754342 + - type: nauc_ndcg_at_1000_std + value: -3.7183620023361126 + - type: nauc_ndcg_at_100_diff1 + value: 24.482138056294662 + - type: nauc_ndcg_at_100_max + value: 9.287712920341493 + - type: nauc_ndcg_at_100_std + value: -4.2098225628994586 + - type: nauc_ndcg_at_10_diff1 + value: 26.272731221058486 + - type: nauc_ndcg_at_10_max + value: 8.935260059702577 + - type: nauc_ndcg_at_10_std + value: -7.4748482166321235 + - type: nauc_ndcg_at_1_diff1 + value: 34.18603002613899 + - type: nauc_ndcg_at_1_max + value: 8.97972973713456 + - type: nauc_ndcg_at_1_std + value: -9.556153409751639 + - type: nauc_ndcg_at_20_diff1 + value: 26.042310377055955 + - type: nauc_ndcg_at_20_max + value: 9.515567707399725 + - type: nauc_ndcg_at_20_std + value: -5.846263691928579 + - type: nauc_ndcg_at_3_diff1 + value: 31.72486596127279 + - type: nauc_ndcg_at_3_max + value: 9.163918049738104 + - type: nauc_ndcg_at_3_std + value: -10.2293347637829 + - type: nauc_ndcg_at_5_diff1 + value: 31.345187267291596 + - type: nauc_ndcg_at_5_max + value: 9.251871069906997 + - type: nauc_ndcg_at_5_std + value: -9.190517533933997 + - type: nauc_precision_at_1000_diff1 + value: 1.6736869988815313 + - type: nauc_precision_at_1000_max + value: 14.364386740236068 + - type: nauc_precision_at_1000_std + value: 10.502570067944601 + - type: nauc_precision_at_100_diff1 + value: 9.796375940479075 + - type: nauc_precision_at_100_max + value: 11.6759128601933 + - type: nauc_precision_at_100_std + value: 6.451807159550276 + - type: nauc_precision_at_10_diff1 + value: 14.956196135825833 + - type: nauc_precision_at_10_max + value: 10.647897707252016 + - type: nauc_precision_at_10_std + value: -1.8947853747077736 + - type: nauc_precision_at_1_diff1 + value: 34.18603002613899 + - type: nauc_precision_at_1_max + value: 8.97972973713456 + - type: nauc_precision_at_1_std + value: -9.556153409751639 + - type: nauc_precision_at_20_diff1 + value: 14.948011109939843 + - type: nauc_precision_at_20_max + value: 11.83845188031389 + - type: nauc_precision_at_20_std + value: 1.8405747345723766 + - type: nauc_precision_at_3_diff1 + value: 26.918031903045808 + - type: nauc_precision_at_3_max + value: 10.123941318105489 + - type: nauc_precision_at_3_std + value: -7.702246246477078 + - type: nauc_precision_at_5_diff1 + value: 26.346862759257117 + - type: nauc_precision_at_5_max + value: 11.828023841927628 + - type: nauc_precision_at_5_std + value: -4.953822315909637 + - type: nauc_recall_at_1000_diff1 + value: 8.185631273725106 + - type: nauc_recall_at_1000_max + value: 9.208792138920922 + - type: nauc_recall_at_1000_std + value: 8.851170225853041 + - type: nauc_recall_at_100_diff1 + value: 13.184052895257976 + - type: nauc_recall_at_100_max + value: 8.34697668610321 + - type: nauc_recall_at_100_std + value: 5.133453080128742 + - type: nauc_recall_at_10_diff1 + value: 18.03232365141138 + - type: nauc_recall_at_10_max + value: 8.097991374932626 + - type: nauc_recall_at_10_std + value: -4.213914606258432 + - type: nauc_recall_at_1_diff1 + value: 35.9745839417964 + - type: nauc_recall_at_1_max + value: 8.983726756853555 + - type: nauc_recall_at_1_std + value: -11.543240200322865 + - type: nauc_recall_at_20_diff1 + value: 18.1972141095434 + - type: nauc_recall_at_20_max + value: 9.80580980272139 + - type: nauc_recall_at_20_std + value: 0.20096255057340812 + - type: nauc_recall_at_3_diff1 + value: 29.839268769004594 + - type: nauc_recall_at_3_max + value: 8.768732586863983 + - type: nauc_recall_at_3_std + value: -10.018089902140288 + - type: nauc_recall_at_5_diff1 + value: 29.23297408189115 + - type: nauc_recall_at_5_max + value: 8.544522709961912 + - type: nauc_recall_at_5_std + value: -7.932279670012872 + - type: ndcg_at_1 + value: 9.718 + - type: ndcg_at_10 + value: 14.183000000000002 + - type: ndcg_at_100 + value: 17.863 + - type: ndcg_at_1000 + value: 21.016000000000002 + - type: ndcg_at_20 + value: 15.354999999999999 + - type: ndcg_at_3 + value: 11.745 + - type: ndcg_at_5 + value: 12.427000000000001 + - type: precision_at_1 + value: 9.718 + - type: precision_at_10 + value: 2.294 + - type: precision_at_100 + value: 0.434 + - type: precision_at_1000 + value: 0.075 + - type: precision_at_20 + value: 1.401 + - type: precision_at_3 + value: 4.859 + - type: precision_at_5 + value: 3.345 + - type: recall_at_1 + value: 9.024000000000001 + - type: recall_at_10 + value: 20.125 + - type: recall_at_100 + value: 37.833 + - type: recall_at_1000 + value: 62.426 + - type: recall_at_20 + value: 24.616 + - type: recall_at_3 + value: 13.221 + - type: recall_at_5 + value: 14.895 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackMathematicaRetrieval (default) + revision: 90fceea13679c63fe563ded68f3b6f06e50061de + split: test + type: mteb/cqadupstack-mathematica + metrics: + - type: main_score + value: 8.94 + - type: map_at_1 + value: 4.475 + - type: map_at_10 + value: 7.015000000000001 + - type: map_at_100 + value: 7.697 + - type: map_at_1000 + value: 7.804 + - type: map_at_20 + value: 7.367999999999999 + - type: map_at_3 + value: 6.099 + - type: map_at_5 + value: 6.68 + - type: mrr_at_1 + value: 5.597014925373134 + - type: mrr_at_10 + value: 8.630113322277507 + - type: mrr_at_100 + value: 9.353274610091166 + - type: mrr_at_1000 + value: 9.448219998278597 + - type: mrr_at_20 + value: 8.994812422579749 + - type: mrr_at_3 + value: 7.462686567164181 + - type: mrr_at_5 + value: 8.196517412935329 + - type: nauc_map_at_1000_diff1 + value: 20.699138033783353 + - type: nauc_map_at_1000_max + value: 3.4880186481745263 + - type: nauc_map_at_1000_std + value: 4.155266389156031 + - type: nauc_map_at_100_diff1 + value: 20.789490863398864 + - type: nauc_map_at_100_max + value: 3.5101164215354155 + - type: nauc_map_at_100_std + value: 4.047378961340708 + - type: nauc_map_at_10_diff1 + value: 21.067672790599165 + - type: nauc_map_at_10_max + value: 3.0190585833019683 + - type: nauc_map_at_10_std + value: 3.4097840153233765 + - type: nauc_map_at_1_diff1 + value: 32.44381415995889 + - type: nauc_map_at_1_max + value: 1.605779939955904 + - type: nauc_map_at_1_std + value: 7.853049501980802 + - type: nauc_map_at_20_diff1 + value: 20.742651462391994 + - type: nauc_map_at_20_max + value: 3.1031371838146424 + - type: nauc_map_at_20_std + value: 3.3972203680764763 + - type: nauc_map_at_3_diff1 + value: 23.377850907655887 + - type: nauc_map_at_3_max + value: 3.43705458976893 + - type: nauc_map_at_3_std + value: 2.3052940647032365 + - type: nauc_map_at_5_diff1 + value: 21.68540176297294 + - type: nauc_map_at_5_max + value: 2.6845256027684927 + - type: nauc_map_at_5_std + value: 3.6379258535324013 + - type: nauc_mrr_at_1000_diff1 + value: 18.179331618128824 + - type: nauc_mrr_at_1000_max + value: 6.7180152872525145 + - type: nauc_mrr_at_1000_std + value: 8.300550321256708 + - type: nauc_mrr_at_100_diff1 + value: 18.200682367921388 + - type: nauc_mrr_at_100_max + value: 6.723979433081779 + - type: nauc_mrr_at_100_std + value: 8.268658628379946 + - type: nauc_mrr_at_10_diff1 + value: 18.53091497875312 + - type: nauc_mrr_at_10_max + value: 6.714211207518613 + - type: nauc_mrr_at_10_std + value: 8.254586191549585 + - type: nauc_mrr_at_1_diff1 + value: 28.509273697241255 + - type: nauc_mrr_at_1_max + value: 7.443060099776353 + - type: nauc_mrr_at_1_std + value: 11.346899929828002 + - type: nauc_mrr_at_20_diff1 + value: 18.13128713477393 + - type: nauc_mrr_at_20_max + value: 6.441290959694762 + - type: nauc_mrr_at_20_std + value: 7.8537386011502 + - type: nauc_mrr_at_3_diff1 + value: 20.289638131876536 + - type: nauc_mrr_at_3_max + value: 7.51846286598189 + - type: nauc_mrr_at_3_std + value: 6.97629908313753 + - type: nauc_mrr_at_5_diff1 + value: 19.057446686788765 + - type: nauc_mrr_at_5_max + value: 6.194041004063384 + - type: nauc_mrr_at_5_std + value: 8.266439336767421 + - type: nauc_ndcg_at_1000_diff1 + value: 16.126133453218237 + - type: nauc_ndcg_at_1000_max + value: 6.473172599722209 + - type: nauc_ndcg_at_1000_std + value: 8.69652115727476 + - type: nauc_ndcg_at_100_diff1 + value: 17.319872374514915 + - type: nauc_ndcg_at_100_max + value: 5.849495042439249 + - type: nauc_ndcg_at_100_std + value: 6.4265443584796245 + - type: nauc_ndcg_at_10_diff1 + value: 17.125738796368445 + - type: nauc_ndcg_at_10_max + value: 4.0772304333748455 + - type: nauc_ndcg_at_10_std + value: 4.004536808476969 + - type: nauc_ndcg_at_1_diff1 + value: 28.509273697241255 + - type: nauc_ndcg_at_1_max + value: 7.443060099776353 + - type: nauc_ndcg_at_1_std + value: 11.346899929828002 + - type: nauc_ndcg_at_20_diff1 + value: 16.475293282383653 + - type: nauc_ndcg_at_20_max + value: 4.193159703014229 + - type: nauc_ndcg_at_20_std + value: 3.96542774314758 + - type: nauc_ndcg_at_3_diff1 + value: 20.350265482388675 + - type: nauc_ndcg_at_3_max + value: 5.258781547082042 + - type: nauc_ndcg_at_3_std + value: 2.6739128400027274 + - type: nauc_ndcg_at_5_diff1 + value: 18.21156852832094 + - type: nauc_ndcg_at_5_max + value: 3.4009566876713664 + - type: nauc_ndcg_at_5_std + value: 4.529302308172549 + - type: nauc_precision_at_1000_diff1 + value: 2.6694472389688273 + - type: nauc_precision_at_1000_max + value: 10.292839463409427 + - type: nauc_precision_at_1000_std + value: 12.12168608682668 + - type: nauc_precision_at_100_diff1 + value: 10.552417708360135 + - type: nauc_precision_at_100_max + value: 9.353011709773758 + - type: nauc_precision_at_100_std + value: 10.78398814531876 + - type: nauc_precision_at_10_diff1 + value: 10.06090578402067 + - type: nauc_precision_at_10_max + value: 5.515400837825047 + - type: nauc_precision_at_10_std + value: 6.443354281004256 + - type: nauc_precision_at_1_diff1 + value: 28.509273697241255 + - type: nauc_precision_at_1_max + value: 7.443060099776353 + - type: nauc_precision_at_1_std + value: 11.346899929828002 + - type: nauc_precision_at_20_diff1 + value: 9.428352118377042 + - type: nauc_precision_at_20_max + value: 4.33555861374056 + - type: nauc_precision_at_20_std + value: 4.388987611299773 + - type: nauc_precision_at_3_diff1 + value: 13.301255810657844 + - type: nauc_precision_at_3_max + value: 6.908538912369242 + - type: nauc_precision_at_3_std + value: 3.004805130717702 + - type: nauc_precision_at_5_diff1 + value: 11.317272340249335 + - type: nauc_precision_at_5_max + value: 3.4111964514747863 + - type: nauc_precision_at_5_std + value: 7.497163702295884 + - type: nauc_recall_at_1000_diff1 + value: 10.014724636274634 + - type: nauc_recall_at_1000_max + value: 11.16254057075483 + - type: nauc_recall_at_1000_std + value: 17.899753204645734 + - type: nauc_recall_at_100_diff1 + value: 14.341707844085244 + - type: nauc_recall_at_100_max + value: 8.212409271393616 + - type: nauc_recall_at_100_std + value: 8.327605677358545 + - type: nauc_recall_at_10_diff1 + value: 11.00540827579114 + - type: nauc_recall_at_10_max + value: 4.297950741770129 + - type: nauc_recall_at_10_std + value: 1.865764269075481 + - type: nauc_recall_at_1_diff1 + value: 32.44381415995889 + - type: nauc_recall_at_1_max + value: 1.605779939955904 + - type: nauc_recall_at_1_std + value: 7.853049501980802 + - type: nauc_recall_at_20_diff1 + value: 10.733904905390727 + - type: nauc_recall_at_20_max + value: 5.038326152107519 + - type: nauc_recall_at_20_std + value: 2.896829465891877 + - type: nauc_recall_at_3_diff1 + value: 16.293656417216383 + - type: nauc_recall_at_3_max + value: 5.288540184568237 + - type: nauc_recall_at_3_std + value: -1.316531044326585 + - type: nauc_recall_at_5_diff1 + value: 12.627860206383975 + - type: nauc_recall_at_5_max + value: 2.7818463087922063 + - type: nauc_recall_at_5_std + value: 3.183638834186147 + - type: ndcg_at_1 + value: 5.5969999999999995 + - type: ndcg_at_10 + value: 8.94 + - type: ndcg_at_100 + value: 12.7 + - type: ndcg_at_1000 + value: 16.137999999999998 + - type: ndcg_at_20 + value: 10.184999999999999 + - type: ndcg_at_3 + value: 7.031999999999999 + - type: ndcg_at_5 + value: 8.083 + - type: precision_at_1 + value: 5.5969999999999995 + - type: precision_at_10 + value: 1.7160000000000002 + - type: precision_at_100 + value: 0.418 + - type: precision_at_1000 + value: 0.086 + - type: precision_at_20 + value: 1.157 + - type: precision_at_3 + value: 3.4410000000000003 + - type: precision_at_5 + value: 2.711 + - type: recall_at_1 + value: 4.475 + - type: recall_at_10 + value: 13.354 + - type: recall_at_100 + value: 30.723 + - type: recall_at_1000 + value: 56.426 + - type: recall_at_20 + value: 17.980999999999998 + - type: recall_at_3 + value: 8.260000000000002 + - type: recall_at_5 + value: 10.82 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackPhysicsRetrieval (default) + revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4 + split: test + type: mteb/cqadupstack-physics + metrics: + - type: main_score + value: 20.294999999999998 + - type: map_at_1 + value: 12.431000000000001 + - type: map_at_10 + value: 17.011000000000003 + - type: map_at_100 + value: 17.990000000000002 + - type: map_at_1000 + value: 18.117 + - type: map_at_20 + value: 17.524 + - type: map_at_3 + value: 15.459 + - type: map_at_5 + value: 16.220000000000002 + - type: mrr_at_1 + value: 15.014436958614052 + - type: mrr_at_10 + value: 20.13447759597904 + - type: mrr_at_100 + value: 21.092552475264426 + - type: mrr_at_1000 + value: 21.183384542127783 + - type: mrr_at_20 + value: 20.697513105665596 + - type: mrr_at_3 + value: 18.511389156239982 + - type: mrr_at_5 + value: 19.39204363169715 + - type: nauc_map_at_1000_diff1 + value: 40.995356165457636 + - type: nauc_map_at_1000_max + value: 26.332034350632583 + - type: nauc_map_at_1000_std + value: 1.1702089486357359 + - type: nauc_map_at_100_diff1 + value: 40.99769475618877 + - type: nauc_map_at_100_max + value: 26.243799694692854 + - type: nauc_map_at_100_std + value: 1.0963613460061759 + - type: nauc_map_at_10_diff1 + value: 41.59053494619633 + - type: nauc_map_at_10_max + value: 26.368323616840865 + - type: nauc_map_at_10_std + value: 0.005351397334266719 + - type: nauc_map_at_1_diff1 + value: 50.48591275806671 + - type: nauc_map_at_1_max + value: 28.027862494196544 + - type: nauc_map_at_1_std + value: -2.5124831594155737 + - type: nauc_map_at_20_diff1 + value: 41.05475708589465 + - type: nauc_map_at_20_max + value: 26.145989770614918 + - type: nauc_map_at_20_std + value: 0.491490393489295 + - type: nauc_map_at_3_diff1 + value: 44.34725985166202 + - type: nauc_map_at_3_max + value: 26.72607401939832 + - type: nauc_map_at_3_std + value: -0.7356644837153518 + - type: nauc_map_at_5_diff1 + value: 42.89323870377218 + - type: nauc_map_at_5_max + value: 26.64092778112793 + - type: nauc_map_at_5_std + value: -0.3252651730879754 + - type: nauc_mrr_at_1000_diff1 + value: 38.54868139280313 + - type: nauc_mrr_at_1000_max + value: 27.87286530236558 + - type: nauc_mrr_at_1000_std + value: 2.9794274560669813 + - type: nauc_mrr_at_100_diff1 + value: 38.52426785096177 + - type: nauc_mrr_at_100_max + value: 27.84756143748855 + - type: nauc_mrr_at_100_std + value: 2.9843775744142267 + - type: nauc_mrr_at_10_diff1 + value: 39.02760076727665 + - type: nauc_mrr_at_10_max + value: 28.11011491417654 + - type: nauc_mrr_at_10_std + value: 2.263143490304297 + - type: nauc_mrr_at_1_diff1 + value: 47.46793071753645 + - type: nauc_mrr_at_1_max + value: 30.632650480338175 + - type: nauc_mrr_at_1_std + value: 0.7118428827724745 + - type: nauc_mrr_at_20_diff1 + value: 38.500695077836966 + - type: nauc_mrr_at_20_max + value: 27.813613143586462 + - type: nauc_mrr_at_20_std + value: 2.598650654867692 + - type: nauc_mrr_at_3_diff1 + value: 41.3716661699587 + - type: nauc_mrr_at_3_max + value: 29.053477693340696 + - type: nauc_mrr_at_3_std + value: 1.808746724928547 + - type: nauc_mrr_at_5_diff1 + value: 39.961419554830485 + - type: nauc_mrr_at_5_max + value: 28.935701693774877 + - type: nauc_mrr_at_5_std + value: 2.210123564134845 + - type: nauc_ndcg_at_1000_diff1 + value: 35.79875968568294 + - type: nauc_ndcg_at_1000_max + value: 26.45802178490949 + - type: nauc_ndcg_at_1000_std + value: 7.215707993785419 + - type: nauc_ndcg_at_100_diff1 + value: 35.644370540541075 + - type: nauc_ndcg_at_100_max + value: 25.34353569408452 + - type: nauc_ndcg_at_100_std + value: 6.393901524581685 + - type: nauc_ndcg_at_10_diff1 + value: 37.29085262377084 + - type: nauc_ndcg_at_10_max + value: 25.72198363571514 + - type: nauc_ndcg_at_10_std + value: 1.7544615128354837 + - type: nauc_ndcg_at_1_diff1 + value: 47.46793071753645 + - type: nauc_ndcg_at_1_max + value: 30.632650480338175 + - type: nauc_ndcg_at_1_std + value: 0.7118428827724745 + - type: nauc_ndcg_at_20_diff1 + value: 35.647868902287165 + - type: nauc_ndcg_at_20_max + value: 24.821600201323278 + - type: nauc_ndcg_at_20_std + value: 3.1415069731821412 + - type: nauc_ndcg_at_3_diff1 + value: 41.596694545162094 + - type: nauc_ndcg_at_3_max + value: 27.40177916659524 + - type: nauc_ndcg_at_3_std + value: 0.6589628153571406 + - type: nauc_ndcg_at_5_diff1 + value: 39.56724224478506 + - type: nauc_ndcg_at_5_max + value: 27.020558127051846 + - type: nauc_ndcg_at_5_std + value: 1.3523747955809573 + - type: nauc_precision_at_1000_diff1 + value: 4.302657386063613 + - type: nauc_precision_at_1000_max + value: 18.2796045022123 + - type: nauc_precision_at_1000_std + value: 18.118747871890772 + - type: nauc_precision_at_100_diff1 + value: 13.618119375734416 + - type: nauc_precision_at_100_max + value: 22.894335594340422 + - type: nauc_precision_at_100_std + value: 20.49716188566563 + - type: nauc_precision_at_10_diff1 + value: 21.78252168291007 + - type: nauc_precision_at_10_max + value: 26.275127596576407 + - type: nauc_precision_at_10_std + value: 8.954746260059663 + - type: nauc_precision_at_1_diff1 + value: 47.46793071753645 + - type: nauc_precision_at_1_max + value: 30.632650480338175 + - type: nauc_precision_at_1_std + value: 0.7118428827724745 + - type: nauc_precision_at_20_diff1 + value: 17.998498363354663 + - type: nauc_precision_at_20_max + value: 22.79803013558884 + - type: nauc_precision_at_20_std + value: 11.734660275051407 + - type: nauc_precision_at_3_diff1 + value: 32.910356757577446 + - type: nauc_precision_at_3_max + value: 29.355200743687078 + - type: nauc_precision_at_3_std + value: 5.099200729327598 + - type: nauc_precision_at_5_diff1 + value: 27.588662253074265 + - type: nauc_precision_at_5_max + value: 29.922214255182038 + - type: nauc_precision_at_5_std + value: 7.133551104463415 + - type: nauc_recall_at_1000_diff1 + value: 19.241952529490536 + - type: nauc_recall_at_1000_max + value: 22.945537589056595 + - type: nauc_recall_at_1000_std + value: 29.961714655551148 + - type: nauc_recall_at_100_diff1 + value: 22.155127117270283 + - type: nauc_recall_at_100_max + value: 18.625383308175493 + - type: nauc_recall_at_100_std + value: 18.98245591409261 + - type: nauc_recall_at_10_diff1 + value: 27.850417430803375 + - type: nauc_recall_at_10_max + value: 21.675692250461214 + - type: nauc_recall_at_10_std + value: 3.679179920639074 + - type: nauc_recall_at_1_diff1 + value: 50.48591275806671 + - type: nauc_recall_at_1_max + value: 28.027862494196544 + - type: nauc_recall_at_1_std + value: -2.5124831594155737 + - type: nauc_recall_at_20_diff1 + value: 22.512641196134968 + - type: nauc_recall_at_20_max + value: 18.380335694614793 + - type: nauc_recall_at_20_std + value: 7.435706290910056 + - type: nauc_recall_at_3_diff1 + value: 38.23610349854204 + - type: nauc_recall_at_3_max + value: 25.121072891517997 + - type: nauc_recall_at_3_std + value: 1.6735898854406979 + - type: nauc_recall_at_5_diff1 + value: 33.45573937210992 + - type: nauc_recall_at_5_max + value: 24.32998084089149 + - type: nauc_recall_at_5_std + value: 3.0358523743841666 + - type: ndcg_at_1 + value: 15.014 + - type: ndcg_at_10 + value: 20.294999999999998 + - type: ndcg_at_100 + value: 25.351000000000003 + - type: ndcg_at_1000 + value: 28.576 + - type: ndcg_at_20 + value: 22.113 + - type: ndcg_at_3 + value: 17.47 + - type: ndcg_at_5 + value: 18.644 + - type: precision_at_1 + value: 15.014 + - type: precision_at_10 + value: 3.811 + - type: precision_at_100 + value: 0.758 + - type: precision_at_1000 + value: 0.122 + - type: precision_at_20 + value: 2.411 + - type: precision_at_3 + value: 8.341 + - type: precision_at_5 + value: 5.9479999999999995 + - type: recall_at_1 + value: 12.431000000000001 + - type: recall_at_10 + value: 27.188000000000002 + - type: recall_at_100 + value: 49.826 + - type: recall_at_1000 + value: 72.483 + - type: recall_at_20 + value: 33.859 + - type: recall_at_3 + value: 19.167 + - type: recall_at_5 + value: 22.281000000000002 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackProgrammersRetrieval (default) + revision: 6184bc1440d2dbc7612be22b50686b8826d22b32 + split: test + type: mteb/cqadupstack-programmers + metrics: + - type: main_score + value: 14.094999999999999 + - type: map_at_1 + value: 7.303 + - type: map_at_10 + value: 11.117 + - type: map_at_100 + value: 12.032 + - type: map_at_1000 + value: 12.163 + - type: map_at_20 + value: 11.573 + - type: map_at_3 + value: 9.649000000000001 + - type: map_at_5 + value: 10.363999999999999 + - type: mrr_at_1 + value: 9.1324200913242 + - type: mrr_at_10 + value: 13.651110748713483 + - type: mrr_at_100 + value: 14.51197163225672 + - type: mrr_at_1000 + value: 14.613337993483293 + - type: mrr_at_20 + value: 14.100106685630351 + - type: mrr_at_3 + value: 12.005327245053268 + - type: mrr_at_5 + value: 12.815829528158297 + - type: nauc_map_at_1000_diff1 + value: 29.862867281596824 + - type: nauc_map_at_1000_max + value: 18.28531520368194 + - type: nauc_map_at_1000_std + value: 5.80584733659641 + - type: nauc_map_at_100_diff1 + value: 29.87327043708194 + - type: nauc_map_at_100_max + value: 18.163395665550443 + - type: nauc_map_at_100_std + value: 5.676097483977982 + - type: nauc_map_at_10_diff1 + value: 30.350916526673387 + - type: nauc_map_at_10_max + value: 17.846298150443882 + - type: nauc_map_at_10_std + value: 4.3226731665874345 + - type: nauc_map_at_1_diff1 + value: 41.53532999423742 + - type: nauc_map_at_1_max + value: 20.989977874238626 + - type: nauc_map_at_1_std + value: 0.12919073079860827 + - type: nauc_map_at_20_diff1 + value: 30.06844081159391 + - type: nauc_map_at_20_max + value: 17.873194514535488 + - type: nauc_map_at_20_std + value: 4.872720284268709 + - type: nauc_map_at_3_diff1 + value: 33.6174080151037 + - type: nauc_map_at_3_max + value: 18.166456594456175 + - type: nauc_map_at_3_std + value: 2.896530825289501 + - type: nauc_map_at_5_diff1 + value: 31.504566510177302 + - type: nauc_map_at_5_max + value: 17.93591120644573 + - type: nauc_map_at_5_std + value: 2.254193364885245 + - type: nauc_mrr_at_1000_diff1 + value: 26.320408655131395 + - type: nauc_mrr_at_1000_max + value: 21.953529111617648 + - type: nauc_mrr_at_1000_std + value: 7.37046085302909 + - type: nauc_mrr_at_100_diff1 + value: 26.302748717403563 + - type: nauc_mrr_at_100_max + value: 21.889357474358334 + - type: nauc_mrr_at_100_std + value: 7.338435428167231 + - type: nauc_mrr_at_10_diff1 + value: 26.555927614522894 + - type: nauc_mrr_at_10_max + value: 22.257898944461783 + - type: nauc_mrr_at_10_std + value: 6.8052301035062035 + - type: nauc_mrr_at_1_diff1 + value: 36.684553884221955 + - type: nauc_mrr_at_1_max + value: 25.48530151556953 + - type: nauc_mrr_at_1_std + value: 3.048877367303359 + - type: nauc_mrr_at_20_diff1 + value: 26.326859097141593 + - type: nauc_mrr_at_20_max + value: 21.919318670390712 + - type: nauc_mrr_at_20_std + value: 7.125641468085978 + - type: nauc_mrr_at_3_diff1 + value: 29.69932296050628 + - type: nauc_mrr_at_3_max + value: 23.683386112606282 + - type: nauc_mrr_at_3_std + value: 5.397563170400955 + - type: nauc_mrr_at_5_diff1 + value: 27.701563882738522 + - type: nauc_mrr_at_5_max + value: 22.955245303254017 + - type: nauc_mrr_at_5_std + value: 5.231835251158974 + - type: nauc_ndcg_at_1000_diff1 + value: 24.297424985001015 + - type: nauc_ndcg_at_1000_max + value: 20.306966661874245 + - type: nauc_ndcg_at_1000_std + value: 13.873545348303818 + - type: nauc_ndcg_at_100_diff1 + value: 24.093566196853757 + - type: nauc_ndcg_at_100_max + value: 17.873825740316427 + - type: nauc_ndcg_at_100_std + value: 11.617123418364676 + - type: nauc_ndcg_at_10_diff1 + value: 25.263815241083986 + - type: nauc_ndcg_at_10_max + value: 17.53213955708432 + - type: nauc_ndcg_at_10_std + value: 7.18466149501028 + - type: nauc_ndcg_at_1_diff1 + value: 36.684553884221955 + - type: nauc_ndcg_at_1_max + value: 25.48530151556953 + - type: nauc_ndcg_at_1_std + value: 3.048877367303359 + - type: nauc_ndcg_at_20_diff1 + value: 24.51658540263249 + - type: nauc_ndcg_at_20_max + value: 17.15525149947788 + - type: nauc_ndcg_at_20_std + value: 8.47477375794873 + - type: nauc_ndcg_at_3_diff1 + value: 30.477243993172383 + - type: nauc_ndcg_at_3_max + value: 19.362585630411125 + - type: nauc_ndcg_at_3_std + value: 4.36632933674639 + - type: nauc_ndcg_at_5_diff1 + value: 27.26848481494293 + - type: nauc_ndcg_at_5_max + value: 18.462159616033276 + - type: nauc_ndcg_at_5_std + value: 3.213137910126468 + - type: nauc_precision_at_1000_diff1 + value: 1.0736355105318376 + - type: nauc_precision_at_1000_max + value: 18.51072739593544 + - type: nauc_precision_at_1000_std + value: 18.09096260033513 + - type: nauc_precision_at_100_diff1 + value: 10.91522849112623 + - type: nauc_precision_at_100_max + value: 20.046044177346676 + - type: nauc_precision_at_100_std + value: 22.757282561231293 + - type: nauc_precision_at_10_diff1 + value: 15.502562368567993 + - type: nauc_precision_at_10_max + value: 19.710006841646 + - type: nauc_precision_at_10_std + value: 15.785469668328542 + - type: nauc_precision_at_1_diff1 + value: 36.684553884221955 + - type: nauc_precision_at_1_max + value: 25.48530151556953 + - type: nauc_precision_at_1_std + value: 3.048877367303359 + - type: nauc_precision_at_20_diff1 + value: 13.971940104916381 + - type: nauc_precision_at_20_max + value: 18.273116307710737 + - type: nauc_precision_at_20_std + value: 18.36985028918322 + - type: nauc_precision_at_3_diff1 + value: 24.258746372971096 + - type: nauc_precision_at_3_max + value: 21.58783837985128 + - type: nauc_precision_at_3_std + value: 7.400932652411214 + - type: nauc_precision_at_5_diff1 + value: 18.7761846559401 + - type: nauc_precision_at_5_max + value: 19.42020311187233 + - type: nauc_precision_at_5_std + value: 5.341062590691995 + - type: nauc_recall_at_1000_diff1 + value: 13.643070685078346 + - type: nauc_recall_at_1000_max + value: 24.738113172451285 + - type: nauc_recall_at_1000_std + value: 37.04460245864719 + - type: nauc_recall_at_100_diff1 + value: 13.667185609958683 + - type: nauc_recall_at_100_max + value: 12.525118917774652 + - type: nauc_recall_at_100_std + value: 21.04251260586679 + - type: nauc_recall_at_10_diff1 + value: 16.739434654375483 + - type: nauc_recall_at_10_max + value: 12.717329215582637 + - type: nauc_recall_at_10_std + value: 10.731719143575397 + - type: nauc_recall_at_1_diff1 + value: 41.53532999423742 + - type: nauc_recall_at_1_max + value: 20.989977874238626 + - type: nauc_recall_at_1_std + value: 0.12919073079860827 + - type: nauc_recall_at_20_diff1 + value: 14.868890539957777 + - type: nauc_recall_at_20_max + value: 11.729101949172534 + - type: nauc_recall_at_20_std + value: 12.813638534198143 + - type: nauc_recall_at_3_diff1 + value: 25.86824535860269 + - type: nauc_recall_at_3_max + value: 15.894774632237532 + - type: nauc_recall_at_3_std + value: 4.443073134833879 + - type: nauc_recall_at_5_diff1 + value: 20.89067633508717 + - type: nauc_recall_at_5_max + value: 15.049382971341338 + - type: nauc_recall_at_5_std + value: 2.887694159444617 + - type: ndcg_at_1 + value: 9.132 + - type: ndcg_at_10 + value: 14.094999999999999 + - type: ndcg_at_100 + value: 18.762999999999998 + - type: ndcg_at_1000 + value: 22.141 + - type: ndcg_at_20 + value: 15.673 + - type: ndcg_at_3 + value: 11.129 + - type: ndcg_at_5 + value: 12.289 + - type: precision_at_1 + value: 9.132 + - type: precision_at_10 + value: 2.8770000000000002 + - type: precision_at_100 + value: 0.639 + - type: precision_at_1000 + value: 0.109 + - type: precision_at_20 + value: 1.9120000000000001 + - type: precision_at_3 + value: 5.441 + - type: precision_at_5 + value: 4.087 + - type: recall_at_1 + value: 7.303 + - type: recall_at_10 + value: 20.74 + - type: recall_at_100 + value: 41.465999999999994 + - type: recall_at_1000 + value: 65.818 + - type: recall_at_20 + value: 26.395000000000003 + - type: recall_at_3 + value: 12.546 + - type: recall_at_5 + value: 15.453 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackRetrieval (default) + revision: CQADupstackRetrieval_is_a_combined_dataset + split: test + type: CQADupstackRetrieval_is_a_combined_dataset + metrics: + - type: main_score + value: 16.292583333333337 + - type: ndcg_at_10 + value: 16.292583333333337 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackStatsRetrieval (default) + revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a + split: test + type: mteb/cqadupstack-stats + metrics: + - type: main_score + value: 11.084 + - type: map_at_1 + value: 5.473999999999999 + - type: map_at_10 + value: 8.647 + - type: map_at_100 + value: 9.254999999999999 + - type: map_at_1000 + value: 9.343 + - type: map_at_20 + value: 8.917 + - type: map_at_3 + value: 7.116 + - type: map_at_5 + value: 7.994999999999999 + - type: mrr_at_1 + value: 6.901840490797547 + - type: mrr_at_10 + value: 10.09019865614958 + - type: mrr_at_100 + value: 10.710876089302603 + - type: mrr_at_1000 + value: 10.80105115646488 + - type: mrr_at_20 + value: 10.370791231604123 + - type: mrr_at_3 + value: 8.384458077709613 + - type: mrr_at_5 + value: 9.435071574642132 + - type: nauc_map_at_1000_diff1 + value: 30.435533125669867 + - type: nauc_map_at_1000_max + value: 12.005700207406719 + - type: nauc_map_at_1000_std + value: 11.98304586760235 + - type: nauc_map_at_100_diff1 + value: 30.39651861345863 + - type: nauc_map_at_100_max + value: 11.958667333522488 + - type: nauc_map_at_100_std + value: 11.988463526455364 + - type: nauc_map_at_10_diff1 + value: 31.61134419975441 + - type: nauc_map_at_10_max + value: 12.466187898649492 + - type: nauc_map_at_10_std + value: 11.313817239575812 + - type: nauc_map_at_1_diff1 + value: 40.459086633032356 + - type: nauc_map_at_1_max + value: 15.968191561744455 + - type: nauc_map_at_1_std + value: 11.53791310770506 + - type: nauc_map_at_20_diff1 + value: 30.924210732306122 + - type: nauc_map_at_20_max + value: 12.25287847619258 + - type: nauc_map_at_20_std + value: 11.55166826589565 + - type: nauc_map_at_3_diff1 + value: 34.474904520349476 + - type: nauc_map_at_3_max + value: 12.362217289453296 + - type: nauc_map_at_3_std + value: 11.91602389729683 + - type: nauc_map_at_5_diff1 + value: 33.02009096009877 + - type: nauc_map_at_5_max + value: 12.293024493317091 + - type: nauc_map_at_5_std + value: 11.115164915945979 + - type: nauc_mrr_at_1000_diff1 + value: 30.201837733428693 + - type: nauc_mrr_at_1000_max + value: 13.810857624668744 + - type: nauc_mrr_at_1000_std + value: 13.424949091741523 + - type: nauc_mrr_at_100_diff1 + value: 30.154981139066106 + - type: nauc_mrr_at_100_max + value: 13.768835955825477 + - type: nauc_mrr_at_100_std + value: 13.437344407365057 + - type: nauc_mrr_at_10_diff1 + value: 31.32911731857685 + - type: nauc_mrr_at_10_max + value: 14.372259338280156 + - type: nauc_mrr_at_10_std + value: 12.954575872352802 + - type: nauc_mrr_at_1_diff1 + value: 40.183370781496436 + - type: nauc_mrr_at_1_max + value: 18.521164314158963 + - type: nauc_mrr_at_1_std + value: 13.876661499833192 + - type: nauc_mrr_at_20_diff1 + value: 30.659255034067307 + - type: nauc_mrr_at_20_max + value: 14.208264685122343 + - type: nauc_mrr_at_20_std + value: 13.247575342063858 + - type: nauc_mrr_at_3_diff1 + value: 33.81220382547985 + - type: nauc_mrr_at_3_max + value: 14.739175293962711 + - type: nauc_mrr_at_3_std + value: 13.51393470776092 + - type: nauc_mrr_at_5_diff1 + value: 32.70503122094358 + - type: nauc_mrr_at_5_max + value: 14.58301653112657 + - type: nauc_mrr_at_5_std + value: 12.688382187193964 + - type: nauc_ndcg_at_1000_diff1 + value: 23.898238338589813 + - type: nauc_ndcg_at_1000_max + value: 11.114129082978721 + - type: nauc_ndcg_at_1000_std + value: 13.731675765819507 + - type: nauc_ndcg_at_100_diff1 + value: 23.37647877218008 + - type: nauc_ndcg_at_100_max + value: 9.752186087015916 + - type: nauc_ndcg_at_100_std + value: 14.038238814077786 + - type: nauc_ndcg_at_10_diff1 + value: 27.862970249084874 + - type: nauc_ndcg_at_10_max + value: 12.080245279627151 + - type: nauc_ndcg_at_10_std + value: 11.163917047664574 + - type: nauc_ndcg_at_1_diff1 + value: 40.183370781496436 + - type: nauc_ndcg_at_1_max + value: 18.521164314158963 + - type: nauc_ndcg_at_1_std + value: 13.876661499833192 + - type: nauc_ndcg_at_20_diff1 + value: 26.004324461514095 + - type: nauc_ndcg_at_20_max + value: 11.489468188078314 + - type: nauc_ndcg_at_20_std + value: 11.996507006587706 + - type: nauc_ndcg_at_3_diff1 + value: 32.598511559625955 + - type: nauc_ndcg_at_3_max + value: 11.4749311612369 + - type: nauc_ndcg_at_3_std + value: 12.255358545075135 + - type: nauc_ndcg_at_5_diff1 + value: 30.40071630731615 + - type: nauc_ndcg_at_5_max + value: 11.564972341416748 + - type: nauc_ndcg_at_5_std + value: 10.673835410901216 + - type: nauc_precision_at_1000_diff1 + value: 15.885458614190167 + - type: nauc_precision_at_1000_max + value: 13.531762379826272 + - type: nauc_precision_at_1000_std + value: 18.76316740635119 + - type: nauc_precision_at_100_diff1 + value: 13.514892278384458 + - type: nauc_precision_at_100_max + value: 12.69922516770082 + - type: nauc_precision_at_100_std + value: 21.78469261226693 + - type: nauc_precision_at_10_diff1 + value: 23.955298612073094 + - type: nauc_precision_at_10_max + value: 14.65966194981769 + - type: nauc_precision_at_10_std + value: 13.163937767486209 + - type: nauc_precision_at_1_diff1 + value: 40.183370781496436 + - type: nauc_precision_at_1_max + value: 18.521164314158963 + - type: nauc_precision_at_1_std + value: 13.876661499833192 + - type: nauc_precision_at_20_diff1 + value: 20.256887655625906 + - type: nauc_precision_at_20_max + value: 12.444330057218782 + - type: nauc_precision_at_20_std + value: 14.516083967147559 + - type: nauc_precision_at_3_diff1 + value: 30.673101233143047 + - type: nauc_precision_at_3_max + value: 12.749036514728967 + - type: nauc_precision_at_3_std + value: 13.791131287286138 + - type: nauc_precision_at_5_diff1 + value: 28.23647732368974 + - type: nauc_precision_at_5_max + value: 12.463909834151341 + - type: nauc_precision_at_5_std + value: 10.939907985087165 + - type: nauc_recall_at_1000_diff1 + value: 10.109855189399731 + - type: nauc_recall_at_1000_max + value: 8.877789681216209 + - type: nauc_recall_at_1000_std + value: 14.575603895088566 + - type: nauc_recall_at_100_diff1 + value: 10.19885264652248 + - type: nauc_recall_at_100_max + value: 2.6983137370833488 + - type: nauc_recall_at_100_std + value: 16.498875224858587 + - type: nauc_recall_at_10_diff1 + value: 20.468620592125365 + - type: nauc_recall_at_10_max + value: 9.468638893767974 + - type: nauc_recall_at_10_std + value: 9.106303262865776 + - type: nauc_recall_at_1_diff1 + value: 40.459086633032356 + - type: nauc_recall_at_1_max + value: 15.968191561744455 + - type: nauc_recall_at_1_std + value: 11.53791310770506 + - type: nauc_recall_at_20_diff1 + value: 16.298246884560854 + - type: nauc_recall_at_20_max + value: 7.988135441382633 + - type: nauc_recall_at_20_std + value: 11.127243159943715 + - type: nauc_recall_at_3_diff1 + value: 27.53910781445452 + - type: nauc_recall_at_3_max + value: 8.222622822120936 + - type: nauc_recall_at_3_std + value: 11.272104373840055 + - type: nauc_recall_at_5_diff1 + value: 23.86996501085239 + - type: nauc_recall_at_5_max + value: 8.392455457234703 + - type: nauc_recall_at_5_std + value: 8.326116528002244 + - type: ndcg_at_1 + value: 6.902 + - type: ndcg_at_10 + value: 11.084 + - type: ndcg_at_100 + value: 14.585999999999999 + - type: ndcg_at_1000 + value: 17.299 + - type: ndcg_at_20 + value: 12.073 + - type: ndcg_at_3 + value: 8.03 + - type: ndcg_at_5 + value: 9.583 + - type: precision_at_1 + value: 6.902 + - type: precision_at_10 + value: 2.147 + - type: precision_at_100 + value: 0.426 + - type: precision_at_1000 + value: 0.07200000000000001 + - type: precision_at_20 + value: 1.2959999999999998 + - type: precision_at_3 + value: 3.7830000000000004 + - type: precision_at_5 + value: 3.19 + - type: recall_at_1 + value: 5.473999999999999 + - type: recall_at_10 + value: 17.432 + - type: recall_at_100 + value: 34.385 + - type: recall_at_1000 + value: 55.132000000000005 + - type: recall_at_20 + value: 21.305 + - type: recall_at_3 + value: 9.052 + - type: recall_at_5 + value: 12.891 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackTexRetrieval (default) + revision: 46989137a86843e03a6195de44b09deda022eec7 + split: test + type: mteb/cqadupstack-tex + metrics: + - type: main_score + value: 9.0 + - type: map_at_1 + value: 4.699 + - type: map_at_10 + value: 7.098 + - type: map_at_100 + value: 7.605 + - type: map_at_1000 + value: 7.712 + - type: map_at_20 + value: 7.337000000000001 + - type: map_at_3 + value: 6.138 + - type: map_at_5 + value: 6.5809999999999995 + - type: mrr_at_1 + value: 5.815554026152787 + - type: mrr_at_10 + value: 8.60257649744917 + - type: mrr_at_100 + value: 9.178358604435788 + - type: mrr_at_1000 + value: 9.273358449979913 + - type: mrr_at_20 + value: 8.890048577862194 + - type: mrr_at_3 + value: 7.501720578114253 + - type: mrr_at_5 + value: 8.014452856159672 + - type: nauc_map_at_1000_diff1 + value: 30.593503138765428 + - type: nauc_map_at_1000_max + value: 18.453660255547693 + - type: nauc_map_at_1000_std + value: -1.7685567473582366 + - type: nauc_map_at_100_diff1 + value: 30.650406650792867 + - type: nauc_map_at_100_max + value: 18.373623725444453 + - type: nauc_map_at_100_std + value: -1.9681957834935826 + - type: nauc_map_at_10_diff1 + value: 31.811653014506152 + - type: nauc_map_at_10_max + value: 18.21651480605788 + - type: nauc_map_at_10_std + value: -2.9289522062108784 + - type: nauc_map_at_1_diff1 + value: 41.22007664054066 + - type: nauc_map_at_1_max + value: 21.46970375146556 + - type: nauc_map_at_1_std + value: -1.2270094063971864 + - type: nauc_map_at_20_diff1 + value: 30.950280855804262 + - type: nauc_map_at_20_max + value: 18.20426095422395 + - type: nauc_map_at_20_std + value: -2.2813322264387317 + - type: nauc_map_at_3_diff1 + value: 34.836258994809555 + - type: nauc_map_at_3_max + value: 20.008809414227088 + - type: nauc_map_at_3_std + value: -2.4831995200462877 + - type: nauc_map_at_5_diff1 + value: 33.53955304108325 + - type: nauc_map_at_5_max + value: 19.249532084724976 + - type: nauc_map_at_5_std + value: -2.7248191532377017 + - type: nauc_mrr_at_1000_diff1 + value: 28.970055122260934 + - type: nauc_mrr_at_1000_max + value: 19.51589259479635 + - type: nauc_mrr_at_1000_std + value: -0.6191130680144826 + - type: nauc_mrr_at_100_diff1 + value: 28.99415333990028 + - type: nauc_mrr_at_100_max + value: 19.489286318028913 + - type: nauc_mrr_at_100_std + value: -0.6904166831553156 + - type: nauc_mrr_at_10_diff1 + value: 29.849897127551994 + - type: nauc_mrr_at_10_max + value: 19.34919336545091 + - type: nauc_mrr_at_10_std + value: -1.6338498086246527 + - type: nauc_mrr_at_1_diff1 + value: 38.11981597192683 + - type: nauc_mrr_at_1_max + value: 23.254943869402304 + - type: nauc_mrr_at_1_std + value: -0.912901935471625 + - type: nauc_mrr_at_20_diff1 + value: 29.188353833212986 + - type: nauc_mrr_at_20_max + value: 19.359637185038142 + - type: nauc_mrr_at_20_std + value: -0.9198376575664564 + - type: nauc_mrr_at_3_diff1 + value: 32.588159647991255 + - type: nauc_mrr_at_3_max + value: 20.740531692877195 + - type: nauc_mrr_at_3_std + value: -1.5083070506201077 + - type: nauc_mrr_at_5_diff1 + value: 31.539888866743084 + - type: nauc_mrr_at_5_max + value: 20.44348881279962 + - type: nauc_mrr_at_5_std + value: -1.6873820593904274 + - type: nauc_ndcg_at_1000_diff1 + value: 23.557578195316097 + - type: nauc_ndcg_at_1000_max + value: 18.510335075222166 + - type: nauc_ndcg_at_1000_std + value: 3.469591902714254 + - type: nauc_ndcg_at_100_diff1 + value: 24.458067231136006 + - type: nauc_ndcg_at_100_max + value: 17.298674134219844 + - type: nauc_ndcg_at_100_std + value: 0.5501100894117191 + - type: nauc_ndcg_at_10_diff1 + value: 27.689420705932218 + - type: nauc_ndcg_at_10_max + value: 16.38882750521239 + - type: nauc_ndcg_at_10_std + value: -3.0544713516417197 + - type: nauc_ndcg_at_1_diff1 + value: 38.11981597192683 + - type: nauc_ndcg_at_1_max + value: 23.254943869402304 + - type: nauc_ndcg_at_1_std + value: -0.912901935471625 + - type: nauc_ndcg_at_20_diff1 + value: 25.61748620708531 + - type: nauc_ndcg_at_20_max + value: 16.382425400354414 + - type: nauc_ndcg_at_20_std + value: -1.1038549983732726 + - type: nauc_ndcg_at_3_diff1 + value: 32.54369670243574 + - type: nauc_ndcg_at_3_max + value: 19.579663222392483 + - type: nauc_ndcg_at_3_std + value: -2.39761396687425 + - type: nauc_ndcg_at_5_diff1 + value: 30.92127124752099 + - type: nauc_ndcg_at_5_max + value: 18.48113539417429 + - type: nauc_ndcg_at_5_std + value: -2.773343405545779 + - type: nauc_precision_at_1000_diff1 + value: 7.4306683973787955 + - type: nauc_precision_at_1000_max + value: 24.159650238977886 + - type: nauc_precision_at_1000_std + value: 20.763060943130444 + - type: nauc_precision_at_100_diff1 + value: 13.977127543915302 + - type: nauc_precision_at_100_max + value: 20.869226948933484 + - type: nauc_precision_at_100_std + value: 9.661028336511375 + - type: nauc_precision_at_10_diff1 + value: 19.380945900490023 + - type: nauc_precision_at_10_max + value: 15.448645443238792 + - type: nauc_precision_at_10_std + value: -2.40779497372621 + - type: nauc_precision_at_1_diff1 + value: 38.11981597192683 + - type: nauc_precision_at_1_max + value: 23.254943869402304 + - type: nauc_precision_at_1_std + value: -0.912901935471625 + - type: nauc_precision_at_20_diff1 + value: 16.160123874138144 + - type: nauc_precision_at_20_max + value: 17.48755145780663 + - type: nauc_precision_at_20_std + value: 4.165695881782506 + - type: nauc_precision_at_3_diff1 + value: 29.147993451561664 + - type: nauc_precision_at_3_max + value: 19.906375084344262 + - type: nauc_precision_at_3_std + value: -2.963281652492904 + - type: nauc_precision_at_5_diff1 + value: 26.37656688810861 + - type: nauc_precision_at_5_max + value: 19.490814194233902 + - type: nauc_precision_at_5_std + value: -2.807622324723641 + - type: nauc_recall_at_1000_diff1 + value: 11.055980578698891 + - type: nauc_recall_at_1000_max + value: 17.067913162609578 + - type: nauc_recall_at_1000_std + value: 13.86839682763791 + - type: nauc_recall_at_100_diff1 + value: 14.616493061478572 + - type: nauc_recall_at_100_max + value: 14.082658622676192 + - type: nauc_recall_at_100_std + value: 4.29554835873666 + - type: nauc_recall_at_10_diff1 + value: 20.59282910525744 + - type: nauc_recall_at_10_max + value: 11.294371411877117 + - type: nauc_recall_at_10_std + value: -4.076776396400985 + - type: nauc_recall_at_1_diff1 + value: 41.22007664054066 + - type: nauc_recall_at_1_max + value: 21.46970375146556 + - type: nauc_recall_at_1_std + value: -1.2270094063971864 + - type: nauc_recall_at_20_diff1 + value: 16.590417625375398 + - type: nauc_recall_at_20_max + value: 11.496290687792293 + - type: nauc_recall_at_20_std + value: 0.2718324385330241 + - type: nauc_recall_at_3_diff1 + value: 29.739297600177927 + - type: nauc_recall_at_3_max + value: 17.456812549646664 + - type: nauc_recall_at_3_std + value: -3.330718505394941 + - type: nauc_recall_at_5_diff1 + value: 26.526643928548303 + - type: nauc_recall_at_5_max + value: 14.970019231755732 + - type: nauc_recall_at_5_std + value: -4.104709617473779 + - type: ndcg_at_1 + value: 5.816000000000001 + - type: ndcg_at_10 + value: 9.0 + - type: ndcg_at_100 + value: 11.966000000000001 + - type: ndcg_at_1000 + value: 15.136 + - type: ndcg_at_20 + value: 9.895 + - type: ndcg_at_3 + value: 7.029000000000001 + - type: ndcg_at_5 + value: 7.7490000000000006 + - type: precision_at_1 + value: 5.816000000000001 + - type: precision_at_10 + value: 1.786 + - type: precision_at_100 + value: 0.392 + - type: precision_at_1000 + value: 0.08099999999999999 + - type: precision_at_20 + value: 1.139 + - type: precision_at_3 + value: 3.338 + - type: precision_at_5 + value: 2.512 + - type: recall_at_1 + value: 4.699 + - type: recall_at_10 + value: 13.444 + - type: recall_at_100 + value: 27.338 + - type: recall_at_1000 + value: 50.958000000000006 + - type: recall_at_20 + value: 16.789 + - type: recall_at_3 + value: 7.965999999999999 + - type: recall_at_5 + value: 9.795 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackUnixRetrieval (default) + revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53 + split: test + type: mteb/cqadupstack-unix + metrics: + - type: main_score + value: 14.642 + - type: map_at_1 + value: 8.594 + - type: map_at_10 + value: 12.043 + - type: map_at_100 + value: 12.628 + - type: map_at_1000 + value: 12.748000000000001 + - type: map_at_20 + value: 12.328999999999999 + - type: map_at_3 + value: 10.825 + - type: map_at_5 + value: 11.456 + - type: mrr_at_1 + value: 10.167910447761194 + - type: mrr_at_10 + value: 14.230632551528075 + - type: mrr_at_100 + value: 14.84826533644871 + - type: mrr_at_1000 + value: 14.952253325630801 + - type: mrr_at_20 + value: 14.559963665127185 + - type: mrr_at_3 + value: 12.795398009950246 + - type: mrr_at_5 + value: 13.583644278606963 + - type: nauc_map_at_1000_diff1 + value: 34.931894084707906 + - type: nauc_map_at_1000_max + value: 26.011453817858175 + - type: nauc_map_at_1000_std + value: -5.259310316806391 + - type: nauc_map_at_100_diff1 + value: 34.97132262158893 + - type: nauc_map_at_100_max + value: 25.977925092139316 + - type: nauc_map_at_100_std + value: -5.363299933468949 + - type: nauc_map_at_10_diff1 + value: 35.413614063892055 + - type: nauc_map_at_10_max + value: 25.996042532353293 + - type: nauc_map_at_10_std + value: -5.828052448730221 + - type: nauc_map_at_1_diff1 + value: 45.43577340681606 + - type: nauc_map_at_1_max + value: 31.35451790693192 + - type: nauc_map_at_1_std + value: -6.804647167393954 + - type: nauc_map_at_20_diff1 + value: 35.22634384251006 + - type: nauc_map_at_20_max + value: 26.0791417390376 + - type: nauc_map_at_20_std + value: -5.568206354119795 + - type: nauc_map_at_3_diff1 + value: 37.88264382017105 + - type: nauc_map_at_3_max + value: 26.58464964762371 + - type: nauc_map_at_3_std + value: -7.143200449464618 + - type: nauc_map_at_5_diff1 + value: 36.87555416284773 + - type: nauc_map_at_5_max + value: 26.187259454362504 + - type: nauc_map_at_5_std + value: -6.369340817625665 + - type: nauc_mrr_at_1000_diff1 + value: 32.95386882016721 + - type: nauc_mrr_at_1000_max + value: 25.415545409989647 + - type: nauc_mrr_at_1000_std + value: -3.2966559932534576 + - type: nauc_mrr_at_100_diff1 + value: 32.929024101483584 + - type: nauc_mrr_at_100_max + value: 25.410922727779607 + - type: nauc_mrr_at_100_std + value: -3.3602828495013846 + - type: nauc_mrr_at_10_diff1 + value: 33.33184163667248 + - type: nauc_mrr_at_10_max + value: 25.23251644013973 + - type: nauc_mrr_at_10_std + value: -3.9425879290670562 + - type: nauc_mrr_at_1_diff1 + value: 42.82444714568402 + - type: nauc_mrr_at_1_max + value: 31.162164601016823 + - type: nauc_mrr_at_1_std + value: -3.5959462505495385 + - type: nauc_mrr_at_20_diff1 + value: 32.98077540397205 + - type: nauc_mrr_at_20_max + value: 25.377444590764863 + - type: nauc_mrr_at_20_std + value: -3.5942289573803903 + - type: nauc_mrr_at_3_diff1 + value: 35.661719035464436 + - type: nauc_mrr_at_3_max + value: 25.822835021311068 + - type: nauc_mrr_at_3_std + value: -5.167396128886139 + - type: nauc_mrr_at_5_diff1 + value: 34.415035085200515 + - type: nauc_mrr_at_5_max + value: 25.28881508345901 + - type: nauc_mrr_at_5_std + value: -4.235402582912731 + - type: nauc_ndcg_at_1000_diff1 + value: 28.03863791690561 + - type: nauc_ndcg_at_1000_max + value: 24.6958953334643 + - type: nauc_ndcg_at_1000_std + value: 0.2069763422184006 + - type: nauc_ndcg_at_100_diff1 + value: 28.85548056591013 + - type: nauc_ndcg_at_100_max + value: 24.19122237527082 + - type: nauc_ndcg_at_100_std + value: -2.2423685304285166 + - type: nauc_ndcg_at_10_diff1 + value: 30.880622024820205 + - type: nauc_ndcg_at_10_max + value: 24.18898174097302 + - type: nauc_ndcg_at_10_std + value: -4.469335009355912 + - type: nauc_ndcg_at_1_diff1 + value: 42.82444714568402 + - type: nauc_ndcg_at_1_max + value: 31.162164601016823 + - type: nauc_ndcg_at_1_std + value: -3.5959462505495385 + - type: nauc_ndcg_at_20_diff1 + value: 30.20323649965523 + - type: nauc_ndcg_at_20_max + value: 24.593772143811695 + - type: nauc_ndcg_at_20_std + value: -3.5991057238283384 + - type: nauc_ndcg_at_3_diff1 + value: 34.91373670076786 + - type: nauc_ndcg_at_3_max + value: 25.19536568941237 + - type: nauc_ndcg_at_3_std + value: -6.633997595341361 + - type: nauc_ndcg_at_5_diff1 + value: 33.63758824952721 + - type: nauc_ndcg_at_5_max + value: 24.49292307627649 + - type: nauc_ndcg_at_5_std + value: -5.4472189787652745 + - type: nauc_precision_at_1000_diff1 + value: 1.894326402664092 + - type: nauc_precision_at_1000_max + value: 18.837828888922346 + - type: nauc_precision_at_1000_std + value: 17.376768802392526 + - type: nauc_precision_at_100_diff1 + value: 10.353766835996908 + - type: nauc_precision_at_100_max + value: 20.365565090207998 + - type: nauc_precision_at_100_std + value: 7.2204931423322805 + - type: nauc_precision_at_10_diff1 + value: 16.79988263352762 + - type: nauc_precision_at_10_max + value: 21.063528845406292 + - type: nauc_precision_at_10_std + value: 0.13932249989411608 + - type: nauc_precision_at_1_diff1 + value: 42.82444714568402 + - type: nauc_precision_at_1_max + value: 31.162164601016823 + - type: nauc_precision_at_1_std + value: -3.5959462505495385 + - type: nauc_precision_at_20_diff1 + value: 14.92905968759458 + - type: nauc_precision_at_20_max + value: 22.36507590889619 + - type: nauc_precision_at_20_std + value: 2.331429468555827 + - type: nauc_precision_at_3_diff1 + value: 26.951382943694664 + - type: nauc_precision_at_3_max + value: 22.522880130849387 + - type: nauc_precision_at_3_std + value: -4.497059354718661 + - type: nauc_precision_at_5_diff1 + value: 23.34961270184579 + - type: nauc_precision_at_5_max + value: 21.853489705418728 + - type: nauc_precision_at_5_std + value: -1.902919557653842 + - type: nauc_recall_at_1000_diff1 + value: 8.111241040499726 + - type: nauc_recall_at_1000_max + value: 19.23752105197241 + - type: nauc_recall_at_1000_std + value: 17.91949865825608 + - type: nauc_recall_at_100_diff1 + value: 15.786966865491731 + - type: nauc_recall_at_100_max + value: 18.78573902011604 + - type: nauc_recall_at_100_std + value: 3.605693949233441 + - type: nauc_recall_at_10_diff1 + value: 21.754181878336514 + - type: nauc_recall_at_10_max + value: 19.791211404418977 + - type: nauc_recall_at_10_std + value: -2.515188856014479 + - type: nauc_recall_at_1_diff1 + value: 45.43577340681606 + - type: nauc_recall_at_1_max + value: 31.35451790693192 + - type: nauc_recall_at_1_std + value: -6.804647167393954 + - type: nauc_recall_at_20_diff1 + value: 20.17297025573394 + - type: nauc_recall_at_20_max + value: 20.664495991798958 + - type: nauc_recall_at_20_std + value: -0.4228284630643226 + - type: nauc_recall_at_3_diff1 + value: 30.62553553336479 + - type: nauc_recall_at_3_max + value: 21.35160745104226 + - type: nauc_recall_at_3_std + value: -7.871793761319987 + - type: nauc_recall_at_5_diff1 + value: 28.572307121700735 + - type: nauc_recall_at_5_max + value: 20.62477116223967 + - type: nauc_recall_at_5_std + value: -4.906118790698553 + - type: ndcg_at_1 + value: 10.168000000000001 + - type: ndcg_at_10 + value: 14.642 + - type: ndcg_at_100 + value: 17.97 + - type: ndcg_at_1000 + value: 21.528 + - type: ndcg_at_20 + value: 15.675 + - type: ndcg_at_3 + value: 12.116 + - type: ndcg_at_5 + value: 13.217 + - type: precision_at_1 + value: 10.168000000000001 + - type: precision_at_10 + value: 2.6310000000000002 + - type: precision_at_100 + value: 0.48700000000000004 + - type: precision_at_1000 + value: 0.09 + - type: precision_at_20 + value: 1.5859999999999999 + - type: precision_at_3 + value: 5.5969999999999995 + - type: precision_at_5 + value: 4.123 + - type: recall_at_1 + value: 8.594 + - type: recall_at_10 + value: 20.544999999999998 + - type: recall_at_100 + value: 36.232 + - type: recall_at_1000 + value: 62.958999999999996 + - type: recall_at_20 + value: 24.248 + - type: recall_at_3 + value: 13.555 + - type: recall_at_5 + value: 16.259 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackWebmastersRetrieval (default) + revision: 160c094312a0e1facb97e55eeddb698c0abe3571 + split: test + type: mteb/cqadupstack-webmasters + metrics: + - type: main_score + value: 17.971999999999998 + - type: map_at_1 + value: 9.227 + - type: map_at_10 + value: 14.176 + - type: map_at_100 + value: 14.982999999999999 + - type: map_at_1000 + value: 15.165999999999999 + - type: map_at_20 + value: 14.523 + - type: map_at_3 + value: 12.273 + - type: map_at_5 + value: 13.254 + - type: mrr_at_1 + value: 11.6600790513834 + - type: mrr_at_10 + value: 17.24355354790138 + - type: mrr_at_100 + value: 17.97458464711609 + - type: mrr_at_1000 + value: 18.083881916544176 + - type: mrr_at_20 + value: 17.592255982425357 + - type: mrr_at_3 + value: 15.316205533596843 + - type: mrr_at_5 + value: 16.442687747035578 + - type: nauc_map_at_1000_diff1 + value: 28.293578671148907 + - type: nauc_map_at_1000_max + value: 13.73183648632115 + - type: nauc_map_at_1000_std + value: 0.6513907283453153 + - type: nauc_map_at_100_diff1 + value: 28.359982175889286 + - type: nauc_map_at_100_max + value: 13.735767207497823 + - type: nauc_map_at_100_std + value: 0.636220564456525 + - type: nauc_map_at_10_diff1 + value: 28.87472863753925 + - type: nauc_map_at_10_max + value: 13.300236729031244 + - type: nauc_map_at_10_std + value: -0.21966965850894107 + - type: nauc_map_at_1_diff1 + value: 34.280945985083164 + - type: nauc_map_at_1_max + value: 15.24775858038184 + - type: nauc_map_at_1_std + value: -3.2611316928574468 + - type: nauc_map_at_20_diff1 + value: 28.467795424891552 + - type: nauc_map_at_20_max + value: 13.103030925180414 + - type: nauc_map_at_20_std + value: 0.16211034905789024 + - type: nauc_map_at_3_diff1 + value: 30.771049312852856 + - type: nauc_map_at_3_max + value: 14.117084438478287 + - type: nauc_map_at_3_std + value: -2.2303908982405245 + - type: nauc_map_at_5_diff1 + value: 29.33000695676755 + - type: nauc_map_at_5_max + value: 15.098774022935096 + - type: nauc_map_at_5_std + value: -0.7416053986126886 + - type: nauc_mrr_at_1000_diff1 + value: 26.580718195411514 + - type: nauc_mrr_at_1000_max + value: 11.543767496741827 + - type: nauc_mrr_at_1000_std + value: -0.08729361203690514 + - type: nauc_mrr_at_100_diff1 + value: 26.591028172685533 + - type: nauc_mrr_at_100_max + value: 11.467614402235903 + - type: nauc_mrr_at_100_std + value: -0.123955657400379 + - type: nauc_mrr_at_10_diff1 + value: 26.798913138334534 + - type: nauc_mrr_at_10_max + value: 11.078083355015591 + - type: nauc_mrr_at_10_std + value: -0.8852609502403819 + - type: nauc_mrr_at_1_diff1 + value: 32.436384851266354 + - type: nauc_mrr_at_1_max + value: 12.9913980702892 + - type: nauc_mrr_at_1_std + value: -3.9949543285156777 + - type: nauc_mrr_at_20_diff1 + value: 26.612951871381856 + - type: nauc_mrr_at_20_max + value: 10.930917683032405 + - type: nauc_mrr_at_20_std + value: -0.46554240191712054 + - type: nauc_mrr_at_3_diff1 + value: 28.55986693321283 + - type: nauc_mrr_at_3_max + value: 11.798572580625544 + - type: nauc_mrr_at_3_std + value: -2.1716364631291873 + - type: nauc_mrr_at_5_diff1 + value: 27.40320377813637 + - type: nauc_mrr_at_5_max + value: 12.131216799357102 + - type: nauc_mrr_at_5_std + value: -1.553808132785284 + - type: nauc_ndcg_at_1000_diff1 + value: 25.29494004035333 + - type: nauc_ndcg_at_1000_max + value: 15.467396480816747 + - type: nauc_ndcg_at_1000_std + value: 5.582839175419764 + - type: nauc_ndcg_at_100_diff1 + value: 25.618005514824066 + - type: nauc_ndcg_at_100_max + value: 14.133983313569171 + - type: nauc_ndcg_at_100_std + value: 5.178074286634601 + - type: nauc_ndcg_at_10_diff1 + value: 26.182869933214597 + - type: nauc_ndcg_at_10_max + value: 10.826305051726441 + - type: nauc_ndcg_at_10_std + value: 1.1448249930047905 + - type: nauc_ndcg_at_1_diff1 + value: 32.436384851266354 + - type: nauc_ndcg_at_1_max + value: 12.9913980702892 + - type: nauc_ndcg_at_1_std + value: -3.9949543285156777 + - type: nauc_ndcg_at_20_diff1 + value: 25.47014527965502 + - type: nauc_ndcg_at_20_max + value: 10.494689973913854 + - type: nauc_ndcg_at_20_std + value: 2.5743266762961925 + - type: nauc_ndcg_at_3_diff1 + value: 28.30011273776709 + - type: nauc_ndcg_at_3_max + value: 12.409101049392858 + - type: nauc_ndcg_at_3_std + value: -1.989338069256987 + - type: nauc_ndcg_at_5_diff1 + value: 26.93375179240513 + - type: nauc_ndcg_at_5_max + value: 14.247006851983018 + - type: nauc_ndcg_at_5_std + value: -0.08501751009489864 + - type: nauc_precision_at_1000_diff1 + value: 2.8681342841906576 + - type: nauc_precision_at_1000_max + value: 8.341013508588487 + - type: nauc_precision_at_1000_std + value: 5.822810476383136 + - type: nauc_precision_at_100_diff1 + value: 10.768484095563934 + - type: nauc_precision_at_100_max + value: 9.704449885569526 + - type: nauc_precision_at_100_std + value: 8.928506635459463 + - type: nauc_precision_at_10_diff1 + value: 17.05139532059708 + - type: nauc_precision_at_10_max + value: 3.9583770881813787 + - type: nauc_precision_at_10_std + value: 2.4925097171199555 + - type: nauc_precision_at_1_diff1 + value: 32.436384851266354 + - type: nauc_precision_at_1_max + value: 12.9913980702892 + - type: nauc_precision_at_1_std + value: -3.9949543285156777 + - type: nauc_precision_at_20_diff1 + value: 13.937332184712863 + - type: nauc_precision_at_20_max + value: 1.8030522940312594 + - type: nauc_precision_at_20_std + value: 4.717403639427268 + - type: nauc_precision_at_3_diff1 + value: 24.877912406786166 + - type: nauc_precision_at_3_max + value: 10.485969186043016 + - type: nauc_precision_at_3_std + value: -0.904450839848222 + - type: nauc_precision_at_5_diff1 + value: 22.445190926200702 + - type: nauc_precision_at_5_max + value: 12.540817149569952 + - type: nauc_precision_at_5_std + value: 1.4289427473397038 + - type: nauc_recall_at_1000_diff1 + value: 17.226074022952993 + - type: nauc_recall_at_1000_max + value: 30.215724888141928 + - type: nauc_recall_at_1000_std + value: 25.363518496790032 + - type: nauc_recall_at_100_diff1 + value: 19.98699952435803 + - type: nauc_recall_at_100_max + value: 18.145945965591014 + - type: nauc_recall_at_100_std + value: 16.845190921447113 + - type: nauc_recall_at_10_diff1 + value: 21.378756963830913 + - type: nauc_recall_at_10_max + value: 6.684867375302075 + - type: nauc_recall_at_10_std + value: 4.930618483880454 + - type: nauc_recall_at_1_diff1 + value: 34.280945985083164 + - type: nauc_recall_at_1_max + value: 15.24775858038184 + - type: nauc_recall_at_1_std + value: -3.2611316928574468 + - type: nauc_recall_at_20_diff1 + value: 18.380399265092144 + - type: nauc_recall_at_20_max + value: 5.801348964041679 + - type: nauc_recall_at_20_std + value: 8.335943218907563 + - type: nauc_recall_at_3_diff1 + value: 28.312640631315944 + - type: nauc_recall_at_3_max + value: 12.616717752139403 + - type: nauc_recall_at_3_std + value: -1.4581980410781947 + - type: nauc_recall_at_5_diff1 + value: 23.215808038829948 + - type: nauc_recall_at_5_max + value: 14.815584298180424 + - type: nauc_recall_at_5_std + value: 2.3437599877590856 + - type: ndcg_at_1 + value: 11.66 + - type: ndcg_at_10 + value: 17.971999999999998 + - type: ndcg_at_100 + value: 22.047 + - type: ndcg_at_1000 + value: 25.948 + - type: ndcg_at_20 + value: 19.09 + - type: ndcg_at_3 + value: 14.512 + - type: ndcg_at_5 + value: 16.062 + - type: precision_at_1 + value: 11.66 + - type: precision_at_10 + value: 3.7150000000000003 + - type: precision_at_100 + value: 0.808 + - type: precision_at_1000 + value: 0.17099999999999999 + - type: precision_at_20 + value: 2.3120000000000003 + - type: precision_at_3 + value: 7.181 + - type: precision_at_5 + value: 5.494000000000001 + - type: recall_at_1 + value: 9.227 + - type: recall_at_10 + value: 26.179999999999996 + - type: recall_at_100 + value: 45.37 + - type: recall_at_1000 + value: 72.20299999999999 + - type: recall_at_20 + value: 30.45 + - type: recall_at_3 + value: 15.906 + - type: recall_at_5 + value: 20.16 + task: + type: Retrieval + - dataset: + config: default + name: MTEB CQADupstackWordpressRetrieval (default) + revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4 + split: test + type: mteb/cqadupstack-wordpress + metrics: + - type: main_score + value: 11.357000000000001 + - type: map_at_1 + value: 5.829 + - type: map_at_10 + value: 9.134 + - type: map_at_100 + value: 9.817 + - type: map_at_1000 + value: 9.947000000000001 + - type: map_at_20 + value: 9.446 + - type: map_at_3 + value: 7.898 + - type: map_at_5 + value: 8.674 + - type: mrr_at_1 + value: 6.284658040665435 + - type: mrr_at_10 + value: 10.019364492562275 + - type: mrr_at_100 + value: 10.722417463000754 + - type: mrr_at_1000 + value: 10.835465799161899 + - type: mrr_at_20 + value: 10.357338079063702 + - type: mrr_at_3 + value: 8.687615526802217 + - type: mrr_at_5 + value: 9.500924214417745 + - type: nauc_map_at_1000_diff1 + value: 22.78724774966205 + - type: nauc_map_at_1000_max + value: 16.34437163744556 + - type: nauc_map_at_1000_std + value: -1.4735193183859276 + - type: nauc_map_at_100_diff1 + value: 22.82517267745717 + - type: nauc_map_at_100_max + value: 16.284531033348436 + - type: nauc_map_at_100_std + value: -1.589549517811522 + - type: nauc_map_at_10_diff1 + value: 23.49210604391011 + - type: nauc_map_at_10_max + value: 16.698351930811523 + - type: nauc_map_at_10_std + value: -2.126619934067786 + - type: nauc_map_at_1_diff1 + value: 33.25537251862352 + - type: nauc_map_at_1_max + value: 21.059212756563113 + - type: nauc_map_at_1_std + value: -3.8230994890517693 + - type: nauc_map_at_20_diff1 + value: 23.175992838887694 + - type: nauc_map_at_20_max + value: 16.456651106073693 + - type: nauc_map_at_20_std + value: -2.0476493700504066 + - type: nauc_map_at_3_diff1 + value: 24.4113337992372 + - type: nauc_map_at_3_max + value: 18.630728856868554 + - type: nauc_map_at_3_std + value: -3.209291836815527 + - type: nauc_map_at_5_diff1 + value: 23.277532213263274 + - type: nauc_map_at_5_max + value: 17.62493140717765 + - type: nauc_map_at_5_std + value: -1.7098531700261095 + - type: nauc_mrr_at_1000_diff1 + value: 24.26971376692725 + - type: nauc_mrr_at_1000_max + value: 17.440070924898812 + - type: nauc_mrr_at_1000_std + value: -1.4904182677524227 + - type: nauc_mrr_at_100_diff1 + value: 24.28090305523701 + - type: nauc_mrr_at_100_max + value: 17.38475182429859 + - type: nauc_mrr_at_100_std + value: -1.5763033409306217 + - type: nauc_mrr_at_10_diff1 + value: 25.065955666592476 + - type: nauc_mrr_at_10_max + value: 17.76938293185054 + - type: nauc_mrr_at_10_std + value: -1.9095223519038518 + - type: nauc_mrr_at_1_diff1 + value: 35.50030652953492 + - type: nauc_mrr_at_1_max + value: 22.074248230325683 + - type: nauc_mrr_at_1_std + value: -2.715605927207539 + - type: nauc_mrr_at_20_diff1 + value: 24.628291881254953 + - type: nauc_mrr_at_20_max + value: 17.486410808513845 + - type: nauc_mrr_at_20_std + value: -1.9572884876186265 + - type: nauc_mrr_at_3_diff1 + value: 26.601750153333132 + - type: nauc_mrr_at_3_max + value: 19.784337786810045 + - type: nauc_mrr_at_3_std + value: -3.0884478868833742 + - type: nauc_mrr_at_5_diff1 + value: 25.34229521812873 + - type: nauc_mrr_at_5_max + value: 18.677143772462735 + - type: nauc_mrr_at_5_std + value: -1.819327747550439 + - type: nauc_ndcg_at_1000_diff1 + value: 17.97058129000062 + - type: nauc_ndcg_at_1000_max + value: 14.443351643128269 + - type: nauc_ndcg_at_1000_std + value: 3.320385091285511 + - type: nauc_ndcg_at_100_diff1 + value: 18.231929337260787 + - type: nauc_ndcg_at_100_max + value: 13.581295908170954 + - type: nauc_ndcg_at_100_std + value: 1.42405813280858 + - type: nauc_ndcg_at_10_diff1 + value: 20.708409240714495 + - type: nauc_ndcg_at_10_max + value: 14.65955381947667 + - type: nauc_ndcg_at_10_std + value: -1.6167091304922117 + - type: nauc_ndcg_at_1_diff1 + value: 35.50030652953492 + - type: nauc_ndcg_at_1_max + value: 22.074248230325683 + - type: nauc_ndcg_at_1_std + value: -2.715605927207539 + - type: nauc_ndcg_at_20_diff1 + value: 19.794733794450234 + - type: nauc_ndcg_at_20_max + value: 13.936007085907942 + - type: nauc_ndcg_at_20_std + value: -1.6063508013870402 + - type: nauc_ndcg_at_3_diff1 + value: 22.567689849957002 + - type: nauc_ndcg_at_3_max + value: 18.56916278259968 + - type: nauc_ndcg_at_3_std + value: -3.028792144928301 + - type: nauc_ndcg_at_5_diff1 + value: 20.601733525569706 + - type: nauc_ndcg_at_5_max + value: 16.676847473988563 + - type: nauc_ndcg_at_5_std + value: -1.0024946114419166 + - type: nauc_precision_at_1000_diff1 + value: 6.751633858887363 + - type: nauc_precision_at_1000_max + value: 8.574094346486792 + - type: nauc_precision_at_1000_std + value: 10.961823016288378 + - type: nauc_precision_at_100_diff1 + value: 9.185536248366825 + - type: nauc_precision_at_100_max + value: 11.154969903889436 + - type: nauc_precision_at_100_std + value: 8.849680313441572 + - type: nauc_precision_at_10_diff1 + value: 16.631977514574494 + - type: nauc_precision_at_10_max + value: 10.891738735128701 + - type: nauc_precision_at_10_std + value: -0.5881313108064528 + - type: nauc_precision_at_1_diff1 + value: 35.50030652953492 + - type: nauc_precision_at_1_max + value: 22.074248230325683 + - type: nauc_precision_at_1_std + value: -2.715605927207539 + - type: nauc_precision_at_20_diff1 + value: 13.755455474334418 + - type: nauc_precision_at_20_max + value: 11.143176815424624 + - type: nauc_precision_at_20_std + value: 0.22127860009316463 + - type: nauc_precision_at_3_diff1 + value: 18.850494736038712 + - type: nauc_precision_at_3_max + value: 18.3734160357334 + - type: nauc_precision_at_3_std + value: -3.570335275359161 + - type: nauc_precision_at_5_diff1 + value: 16.116395366767875 + - type: nauc_precision_at_5_max + value: 15.199475728220216 + - type: nauc_precision_at_5_std + value: -0.1646903087985237 + - type: nauc_recall_at_1000_diff1 + value: 7.064519186534486 + - type: nauc_recall_at_1000_max + value: 10.537702917035265 + - type: nauc_recall_at_1000_std + value: 19.24374744927976 + - type: nauc_recall_at_100_diff1 + value: 9.673538463478982 + - type: nauc_recall_at_100_max + value: 7.750546515055119 + - type: nauc_recall_at_100_std + value: 8.300080212088146 + - type: nauc_recall_at_10_diff1 + value: 14.59536923133317 + - type: nauc_recall_at_10_max + value: 9.898224821325174 + - type: nauc_recall_at_10_std + value: -0.5852780000420884 + - type: nauc_recall_at_1_diff1 + value: 33.25537251862352 + - type: nauc_recall_at_1_max + value: 21.059212756563113 + - type: nauc_recall_at_1_std + value: -3.8230994890517693 + - type: nauc_recall_at_20_diff1 + value: 12.823131452573252 + - type: nauc_recall_at_20_max + value: 7.83963802432653 + - type: nauc_recall_at_20_std + value: -1.092413321137494 + - type: nauc_recall_at_3_diff1 + value: 15.884549077272911 + - type: nauc_recall_at_3_max + value: 16.64076323497644 + - type: nauc_recall_at_3_std + value: -2.971396404621954 + - type: nauc_recall_at_5_diff1 + value: 13.683727065301637 + - type: nauc_recall_at_5_max + value: 13.899539774976596 + - type: nauc_recall_at_5_std + value: 0.9343056640766644 + - type: ndcg_at_1 + value: 6.285 + - type: ndcg_at_10 + value: 11.357000000000001 + - type: ndcg_at_100 + value: 15.296000000000001 + - type: ndcg_at_1000 + value: 18.945999999999998 + - type: ndcg_at_20 + value: 12.494 + - type: ndcg_at_3 + value: 8.852 + - type: ndcg_at_5 + value: 10.228 + - type: precision_at_1 + value: 6.285 + - type: precision_at_10 + value: 1.959 + - type: precision_at_100 + value: 0.434 + - type: precision_at_1000 + value: 0.08099999999999999 + - type: precision_at_20 + value: 1.257 + - type: precision_at_3 + value: 4.005 + - type: precision_at_5 + value: 3.105 + - type: recall_at_1 + value: 5.829 + - type: recall_at_10 + value: 17.449 + - type: recall_at_100 + value: 36.439 + - type: recall_at_1000 + value: 64.549 + - type: recall_at_20 + value: 21.667 + - type: recall_at_3 + value: 10.789 + - type: recall_at_5 + value: 14.116000000000001 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ClimateFEVER (default) + revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380 + split: test + type: mteb/climate-fever + metrics: + - type: main_score + value: 8.665000000000001 + - type: map_at_1 + value: 3.431 + - type: map_at_10 + value: 5.548 + - type: map_at_100 + value: 6.451999999999999 + - type: map_at_1000 + value: 6.572 + - type: map_at_20 + value: 6.0 + - type: map_at_3 + value: 4.517 + - type: map_at_5 + value: 4.989 + - type: mrr_at_1 + value: 7.752442996742672 + - type: mrr_at_10 + value: 12.347164055633106 + - type: mrr_at_100 + value: 13.47855237102118 + - type: mrr_at_1000 + value: 13.549653634356204 + - type: mrr_at_20 + value: 12.991485758879193 + - type: mrr_at_3 + value: 10.542888165038002 + - type: mrr_at_5 + value: 11.500542888165036 + - type: nauc_map_at_1000_diff1 + value: 20.20268497242339 + - type: nauc_map_at_1000_max + value: 24.18193699020533 + - type: nauc_map_at_1000_std + value: 24.50439660861598 + - type: nauc_map_at_100_diff1 + value: 20.24249901794732 + - type: nauc_map_at_100_max + value: 23.967201952298748 + - type: nauc_map_at_100_std + value: 24.15000750583459 + - type: nauc_map_at_10_diff1 + value: 20.03966291824178 + - type: nauc_map_at_10_max + value: 21.755785801673685 + - type: nauc_map_at_10_std + value: 20.282919204579944 + - type: nauc_map_at_1_diff1 + value: 29.695690051340613 + - type: nauc_map_at_1_max + value: 20.865617103351067 + - type: nauc_map_at_1_std + value: 14.29329137354716 + - type: nauc_map_at_20_diff1 + value: 20.728143069074356 + - type: nauc_map_at_20_max + value: 22.90305825311414 + - type: nauc_map_at_20_std + value: 22.11399310632219 + - type: nauc_map_at_3_diff1 + value: 22.380408121971325 + - type: nauc_map_at_3_max + value: 19.75817554487646 + - type: nauc_map_at_3_std + value: 15.90373516686131 + - type: nauc_map_at_5_diff1 + value: 21.162868524231556 + - type: nauc_map_at_5_max + value: 20.83194239461333 + - type: nauc_map_at_5_std + value: 17.646811142371032 + - type: nauc_mrr_at_1000_diff1 + value: 18.72010631814857 + - type: nauc_mrr_at_1000_max + value: 23.109557240302276 + - type: nauc_mrr_at_1000_std + value: 24.817749690786577 + - type: nauc_mrr_at_100_diff1 + value: 18.72349343780436 + - type: nauc_mrr_at_100_max + value: 23.114096614704422 + - type: nauc_mrr_at_100_std + value: 24.797054308635044 + - type: nauc_mrr_at_10_diff1 + value: 18.064637218620152 + - type: nauc_mrr_at_10_max + value: 22.076095274822805 + - type: nauc_mrr_at_10_std + value: 23.01088518773615 + - type: nauc_mrr_at_1_diff1 + value: 26.01054304903776 + - type: nauc_mrr_at_1_max + value: 20.13170457353963 + - type: nauc_mrr_at_1_std + value: 16.267938736930734 + - type: nauc_mrr_at_20_diff1 + value: 18.845964611635978 + - type: nauc_mrr_at_20_max + value: 22.864659670290468 + - type: nauc_mrr_at_20_std + value: 24.27391107280338 + - type: nauc_mrr_at_3_diff1 + value: 19.460522034655536 + - type: nauc_mrr_at_3_max + value: 20.850086228271273 + - type: nauc_mrr_at_3_std + value: 20.09277671782743 + - type: nauc_mrr_at_5_diff1 + value: 18.48847312194798 + - type: nauc_mrr_at_5_max + value: 21.678625281995373 + - type: nauc_mrr_at_5_std + value: 21.459857528465935 + - type: nauc_ndcg_at_1000_diff1 + value: 17.22440190516704 + - type: nauc_ndcg_at_1000_max + value: 30.81923311154538 + - type: nauc_ndcg_at_1000_std + value: 39.35963524379091 + - type: nauc_ndcg_at_100_diff1 + value: 17.457054645637687 + - type: nauc_ndcg_at_100_max + value: 28.843983901862085 + - type: nauc_ndcg_at_100_std + value: 35.96968009544363 + - type: nauc_ndcg_at_10_diff1 + value: 17.06876854851489 + - type: nauc_ndcg_at_10_max + value: 23.111301663700726 + - type: nauc_ndcg_at_10_std + value: 25.079720024466596 + - type: nauc_ndcg_at_1_diff1 + value: 26.01054304903776 + - type: nauc_ndcg_at_1_max + value: 20.13170457353963 + - type: nauc_ndcg_at_1_std + value: 16.267938736930734 + - type: nauc_ndcg_at_20_diff1 + value: 19.159383706288963 + - type: nauc_ndcg_at_20_max + value: 25.748625998594076 + - type: nauc_ndcg_at_20_std + value: 29.2777966585303 + - type: nauc_ndcg_at_3_diff1 + value: 19.76246562466077 + - type: nauc_ndcg_at_3_max + value: 21.020808382794733 + - type: nauc_ndcg_at_3_std + value: 19.113271468015192 + - type: nauc_ndcg_at_5_diff1 + value: 18.36759011393604 + - type: nauc_ndcg_at_5_max + value: 21.650284537673105 + - type: nauc_ndcg_at_5_std + value: 20.85439010070303 + - type: nauc_precision_at_1000_diff1 + value: 7.866460573005927 + - type: nauc_precision_at_1000_max + value: 36.3237806073285 + - type: nauc_precision_at_1000_std + value: 53.341836980739984 + - type: nauc_precision_at_100_diff1 + value: 10.119925830770402 + - type: nauc_precision_at_100_max + value: 34.98290066245935 + - type: nauc_precision_at_100_std + value: 51.722408165202694 + - type: nauc_precision_at_10_diff1 + value: 10.295201659904015 + - type: nauc_precision_at_10_max + value: 28.09830932510505 + - type: nauc_precision_at_10_std + value: 35.82169548156545 + - type: nauc_precision_at_1_diff1 + value: 26.01054304903776 + - type: nauc_precision_at_1_max + value: 20.13170457353963 + - type: nauc_precision_at_1_std + value: 16.267938736930734 + - type: nauc_precision_at_20_diff1 + value: 15.485906980500427 + - type: nauc_precision_at_20_max + value: 32.34263330026811 + - type: nauc_precision_at_20_std + value: 42.658087739955434 + - type: nauc_precision_at_3_diff1 + value: 13.939250177065949 + - type: nauc_precision_at_3_max + value: 21.825298086308017 + - type: nauc_precision_at_3_std + value: 23.371972971620732 + - type: nauc_precision_at_5_diff1 + value: 11.885509516199035 + - type: nauc_precision_at_5_max + value: 24.896726997531367 + - type: nauc_precision_at_5_std + value: 28.093898617550252 + - type: nauc_recall_at_1000_diff1 + value: 11.60220525255486 + - type: nauc_recall_at_1000_max + value: 33.43532086300828 + - type: nauc_recall_at_1000_std + value: 51.14547402627655 + - type: nauc_recall_at_100_diff1 + value: 12.590256364068992 + - type: nauc_recall_at_100_max + value: 30.79008080487411 + - type: nauc_recall_at_100_std + value: 45.42391176816372 + - type: nauc_recall_at_10_diff1 + value: 13.539869516256994 + - type: nauc_recall_at_10_max + value: 23.294485598950875 + - type: nauc_recall_at_10_std + value: 29.061101403756684 + - type: nauc_recall_at_1_diff1 + value: 29.695690051340613 + - type: nauc_recall_at_1_max + value: 20.865617103351067 + - type: nauc_recall_at_1_std + value: 14.29329137354716 + - type: nauc_recall_at_20_diff1 + value: 17.845867931336173 + - type: nauc_recall_at_20_max + value: 27.404059514695874 + - type: nauc_recall_at_20_std + value: 35.2508074045364 + - type: nauc_recall_at_3_diff1 + value: 17.838823438292053 + - type: nauc_recall_at_3_max + value: 19.309869178987764 + - type: nauc_recall_at_3_std + value: 18.81309403950389 + - type: nauc_recall_at_5_diff1 + value: 15.536078861120748 + - type: nauc_recall_at_5_max + value: 21.78126721208168 + - type: nauc_recall_at_5_std + value: 22.671176234569295 + - type: ndcg_at_1 + value: 7.752000000000001 + - type: ndcg_at_10 + value: 8.665000000000001 + - type: ndcg_at_100 + value: 13.478000000000002 + - type: ndcg_at_1000 + value: 16.463 + - type: ndcg_at_20 + value: 10.357 + - type: ndcg_at_3 + value: 6.526 + - type: ndcg_at_5 + value: 7.204000000000001 + - type: precision_at_1 + value: 7.752000000000001 + - type: precision_at_10 + value: 2.932 + - type: precision_at_100 + value: 0.7939999999999999 + - type: precision_at_1000 + value: 0.134 + - type: precision_at_20 + value: 2.137 + - type: precision_at_3 + value: 4.951 + - type: precision_at_5 + value: 3.987 + - type: recall_at_1 + value: 3.431 + - type: recall_at_10 + value: 10.982 + - type: recall_at_100 + value: 28.276 + - type: recall_at_1000 + value: 45.938 + - type: recall_at_20 + value: 16.005 + - type: recall_at_3 + value: 5.862 + - type: recall_at_5 + value: 7.595000000000001 + task: + type: Retrieval + - dataset: + config: default + name: MTEB DBPedia (default) + revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659 + split: test + type: mteb/dbpedia + metrics: + - type: main_score + value: 18.317 + - type: map_at_1 + value: 3.4450000000000003 + - type: map_at_10 + value: 7.106 + - type: map_at_100 + value: 9.631 + - type: map_at_1000 + value: 10.252 + - type: map_at_20 + value: 8.045 + - type: map_at_3 + value: 5.438 + - type: map_at_5 + value: 6.075 + - type: mrr_at_1 + value: 33.25 + - type: mrr_at_10 + value: 43.1984126984127 + - type: mrr_at_100 + value: 44.10105314669371 + - type: mrr_at_1000 + value: 44.14652177743905 + - type: mrr_at_20 + value: 43.764106802854876 + - type: mrr_at_3 + value: 41.16666666666667 + - type: mrr_at_5 + value: 42.07916666666668 + - type: nauc_map_at_1000_diff1 + value: 32.89274359578512 + - type: nauc_map_at_1000_max + value: 26.604828696525264 + - type: nauc_map_at_1000_std + value: 31.04310644401098 + - type: nauc_map_at_100_diff1 + value: 33.39909855555811 + - type: nauc_map_at_100_max + value: 24.426258776321095 + - type: nauc_map_at_100_std + value: 27.950162866259888 + - type: nauc_map_at_10_diff1 + value: 38.01633731897987 + - type: nauc_map_at_10_max + value: 10.864204272144072 + - type: nauc_map_at_10_std + value: 13.002257173637139 + - type: nauc_map_at_1_diff1 + value: 51.974783929669066 + - type: nauc_map_at_1_max + value: 2.1087384419233546 + - type: nauc_map_at_1_std + value: 3.876223529954049 + - type: nauc_map_at_20_diff1 + value: 36.578511302087925 + - type: nauc_map_at_20_max + value: 16.12600615737132 + - type: nauc_map_at_20_std + value: 19.17629858668846 + - type: nauc_map_at_3_diff1 + value: 42.100527162189024 + - type: nauc_map_at_3_max + value: 4.583854234482867 + - type: nauc_map_at_3_std + value: 4.043752599363613 + - type: nauc_map_at_5_diff1 + value: 41.25097930082802 + - type: nauc_map_at_5_max + value: 6.748191433066141 + - type: nauc_map_at_5_std + value: 6.834355643360842 + - type: nauc_mrr_at_1000_diff1 + value: 31.808593974374478 + - type: nauc_mrr_at_1000_max + value: 38.189881128516284 + - type: nauc_mrr_at_1000_std + value: 29.26318169221298 + - type: nauc_mrr_at_100_diff1 + value: 31.78571608370332 + - type: nauc_mrr_at_100_max + value: 38.19998391171405 + - type: nauc_mrr_at_100_std + value: 29.23827479906131 + - type: nauc_mrr_at_10_diff1 + value: 31.786867448793465 + - type: nauc_mrr_at_10_max + value: 37.99600295959577 + - type: nauc_mrr_at_10_std + value: 28.829996904095655 + - type: nauc_mrr_at_1_diff1 + value: 38.05179996742141 + - type: nauc_mrr_at_1_max + value: 36.79685915741111 + - type: nauc_mrr_at_1_std + value: 28.350691962727403 + - type: nauc_mrr_at_20_diff1 + value: 31.808590310476177 + - type: nauc_mrr_at_20_max + value: 38.10937432755284 + - type: nauc_mrr_at_20_std + value: 29.159630421397104 + - type: nauc_mrr_at_3_diff1 + value: 31.831109798177128 + - type: nauc_mrr_at_3_max + value: 37.160264447610786 + - type: nauc_mrr_at_3_std + value: 28.541784432773664 + - type: nauc_mrr_at_5_diff1 + value: 32.265744410577796 + - type: nauc_mrr_at_5_max + value: 37.1185356423142 + - type: nauc_mrr_at_5_std + value: 28.542067443171383 + - type: nauc_ndcg_at_1000_diff1 + value: 26.2354937722142 + - type: nauc_ndcg_at_1000_max + value: 35.15647109201309 + - type: nauc_ndcg_at_1000_std + value: 43.72983297081998 + - type: nauc_ndcg_at_100_diff1 + value: 27.4257708149693 + - type: nauc_ndcg_at_100_max + value: 29.44362350882367 + - type: nauc_ndcg_at_100_std + value: 35.47194671187406 + - type: nauc_ndcg_at_10_diff1 + value: 26.83748894960956 + - type: nauc_ndcg_at_10_max + value: 26.324600583375148 + - type: nauc_ndcg_at_10_std + value: 26.839688509046074 + - type: nauc_ndcg_at_1_diff1 + value: 33.91534683952059 + - type: nauc_ndcg_at_1_max + value: 27.781900866901438 + - type: nauc_ndcg_at_1_std + value: 20.609436435124508 + - type: nauc_ndcg_at_20_diff1 + value: 27.33782331818608 + - type: nauc_ndcg_at_20_max + value: 25.655998678526053 + - type: nauc_ndcg_at_20_std + value: 28.815358762679 + - type: nauc_ndcg_at_3_diff1 + value: 24.911857448124124 + - type: nauc_ndcg_at_3_max + value: 28.44218389736495 + - type: nauc_ndcg_at_3_std + value: 22.814032559889135 + - type: nauc_ndcg_at_5_diff1 + value: 26.99854276880701 + - type: nauc_ndcg_at_5_max + value: 26.70673289401602 + - type: nauc_ndcg_at_5_std + value: 23.875749510070484 + - type: nauc_precision_at_1000_diff1 + value: -4.944236798264726 + - type: nauc_precision_at_1000_max + value: 40.121224630291145 + - type: nauc_precision_at_1000_std + value: 41.39631390461634 + - type: nauc_precision_at_100_diff1 + value: 3.378479753980361 + - type: nauc_precision_at_100_max + value: 48.32256441737045 + - type: nauc_precision_at_100_std + value: 49.393335268436736 + - type: nauc_precision_at_10_diff1 + value: 11.158417038139243 + - type: nauc_precision_at_10_max + value: 41.57441754310071 + - type: nauc_precision_at_10_std + value: 41.773169149009064 + - type: nauc_precision_at_1_diff1 + value: 38.05179996742141 + - type: nauc_precision_at_1_max + value: 36.79685915741111 + - type: nauc_precision_at_1_std + value: 28.350691962727403 + - type: nauc_precision_at_20_diff1 + value: 8.363894422086595 + - type: nauc_precision_at_20_max + value: 43.897592804770255 + - type: nauc_precision_at_20_std + value: 45.898546020673685 + - type: nauc_precision_at_3_diff1 + value: 17.496170798036925 + - type: nauc_precision_at_3_max + value: 35.77099809321373 + - type: nauc_precision_at_3_std + value: 28.270294770404174 + - type: nauc_precision_at_5_diff1 + value: 17.512866815049172 + - type: nauc_precision_at_5_max + value: 35.87345026620194 + - type: nauc_precision_at_5_std + value: 31.14539151046845 + - type: nauc_recall_at_1000_diff1 + value: 12.042358990194806 + - type: nauc_recall_at_1000_max + value: 27.794792931251482 + - type: nauc_recall_at_1000_std + value: 48.936601177086885 + - type: nauc_recall_at_100_diff1 + value: 17.74014614328557 + - type: nauc_recall_at_100_max + value: 23.649469832153112 + - type: nauc_recall_at_100_std + value: 35.91554178778231 + - type: nauc_recall_at_10_diff1 + value: 24.602172763255464 + - type: nauc_recall_at_10_max + value: 3.2087715731960933 + - type: nauc_recall_at_10_std + value: 8.001266920970215 + - type: nauc_recall_at_1_diff1 + value: 51.974783929669066 + - type: nauc_recall_at_1_max + value: 2.1087384419233546 + - type: nauc_recall_at_1_std + value: 3.876223529954049 + - type: nauc_recall_at_20_diff1 + value: 23.722094513884095 + - type: nauc_recall_at_20_max + value: 10.321708038535022 + - type: nauc_recall_at_20_std + value: 18.772263734208646 + - type: nauc_recall_at_3_diff1 + value: 33.0581116651206 + - type: nauc_recall_at_3_max + value: -1.4200131955084976 + - type: nauc_recall_at_3_std + value: -1.0154219924636907 + - type: nauc_recall_at_5_diff1 + value: 31.665054222106033 + - type: nauc_recall_at_5_max + value: 0.41784903940174156 + - type: nauc_recall_at_5_std + value: 1.821099304694692 + - type: ndcg_at_1 + value: 24.5 + - type: ndcg_at_10 + value: 18.317 + - type: ndcg_at_100 + value: 20.737 + - type: ndcg_at_1000 + value: 26.124000000000002 + - type: ndcg_at_20 + value: 17.91 + - type: ndcg_at_3 + value: 21.468999999999998 + - type: ndcg_at_5 + value: 19.557 + - type: precision_at_1 + value: 33.25 + - type: precision_at_10 + value: 15.925 + - type: precision_at_100 + value: 5.015 + - type: precision_at_1000 + value: 1.009 + - type: precision_at_20 + value: 12.063 + - type: precision_at_3 + value: 25.833000000000002 + - type: precision_at_5 + value: 20.7 + - type: recall_at_1 + value: 3.4450000000000003 + - type: recall_at_10 + value: 10.914 + - type: recall_at_100 + value: 25.511 + - type: recall_at_1000 + value: 44.506 + - type: recall_at_20 + value: 14.252999999999998 + - type: recall_at_3 + value: 6.6530000000000005 + - type: recall_at_5 + value: 7.93 + task: + type: Retrieval + - dataset: + config: default + name: MTEB EmotionClassification (default) + revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 + split: test + type: mteb/emotion + metrics: + - type: accuracy + value: 42.035 + - type: f1 + value: 37.86640884786291 + - type: f1_weighted + value: 44.414257303205105 + - type: main_score + value: 42.035 + task: + type: Classification + - dataset: + config: default + name: MTEB FEVER (default) + revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 + split: test + type: mteb/fever + metrics: + - type: main_score + value: 18.761 + - type: map_at_1 + value: 10.476 + - type: map_at_10 + value: 15.404000000000002 + - type: map_at_100 + value: 16.253999999999998 + - type: map_at_1000 + value: 16.331 + - type: map_at_20 + value: 15.853 + - type: map_at_3 + value: 13.575000000000001 + - type: map_at_5 + value: 14.521 + - type: mrr_at_1 + value: 11.086108610861087 + - type: mrr_at_10 + value: 16.355343867720112 + - type: mrr_at_100 + value: 17.231932946772645 + - type: mrr_at_1000 + value: 17.306050360630675 + - type: mrr_at_20 + value: 16.824638510255593 + - type: mrr_at_3 + value: 14.421442144214469 + - type: mrr_at_5 + value: 15.419791979198013 + - type: nauc_map_at_1000_diff1 + value: 21.483384414650857 + - type: nauc_map_at_1000_max + value: 14.658845058298656 + - type: nauc_map_at_1000_std + value: 1.4230966250967994 + - type: nauc_map_at_100_diff1 + value: 21.478059331590387 + - type: nauc_map_at_100_max + value: 14.631728769027703 + - type: nauc_map_at_100_std + value: 1.3881789289506472 + - type: nauc_map_at_10_diff1 + value: 21.720905300450458 + - type: nauc_map_at_10_max + value: 14.076908554314246 + - type: nauc_map_at_10_std + value: 0.5551386401263194 + - type: nauc_map_at_1_diff1 + value: 27.661685422401224 + - type: nauc_map_at_1_max + value: 11.99044345858924 + - type: nauc_map_at_1_std + value: -3.3979769351402322 + - type: nauc_map_at_20_diff1 + value: 21.59196261739584 + - type: nauc_map_at_20_max + value: 14.385540847050335 + - type: nauc_map_at_20_std + value: 0.9895830524590538 + - type: nauc_map_at_3_diff1 + value: 23.20704312937525 + - type: nauc_map_at_3_max + value: 12.810161113474871 + - type: nauc_map_at_3_std + value: -1.411771996719974 + - type: nauc_map_at_5_diff1 + value: 22.356183179583493 + - type: nauc_map_at_5_max + value: 13.36852816007986 + - type: nauc_map_at_5_std + value: -0.48512124846138643 + - type: nauc_mrr_at_1000_diff1 + value: 21.435605535073744 + - type: nauc_mrr_at_1000_max + value: 15.102897427738405 + - type: nauc_mrr_at_1000_std + value: 1.8946939279941968 + - type: nauc_mrr_at_100_diff1 + value: 21.421114034979606 + - type: nauc_mrr_at_100_max + value: 15.083160957472696 + - type: nauc_mrr_at_100_std + value: 1.8744293097807236 + - type: nauc_mrr_at_10_diff1 + value: 21.649184972259103 + - type: nauc_mrr_at_10_max + value: 14.533371443998226 + - type: nauc_mrr_at_10_std + value: 1.1004240892945276 + - type: nauc_mrr_at_1_diff1 + value: 27.75425157234291 + - type: nauc_mrr_at_1_max + value: 12.264180225035945 + - type: nauc_mrr_at_1_std + value: -2.9856311667661473 + - type: nauc_mrr_at_20_diff1 + value: 21.50477816444254 + - type: nauc_mrr_at_20_max + value: 14.839198606656051 + - type: nauc_mrr_at_20_std + value: 1.504911445582658 + - type: nauc_mrr_at_3_diff1 + value: 23.118987947282125 + - type: nauc_mrr_at_3_max + value: 13.268686916605809 + - type: nauc_mrr_at_3_std + value: -0.8973701423333541 + - type: nauc_mrr_at_5_diff1 + value: 22.31906380361573 + - type: nauc_mrr_at_5_max + value: 13.879335206291863 + - type: nauc_mrr_at_5_std + value: 0.0743953082913858 + - type: nauc_ndcg_at_1000_diff1 + value: 18.130594943202155 + - type: nauc_ndcg_at_1000_max + value: 18.618199152782932 + - type: nauc_ndcg_at_1000_std + value: 7.73800191626906 + - type: nauc_ndcg_at_100_diff1 + value: 18.112919952254646 + - type: nauc_ndcg_at_100_max + value: 18.025448330998607 + - type: nauc_ndcg_at_100_std + value: 6.91218512382701 + - type: nauc_ndcg_at_10_diff1 + value: 19.25411748879802 + - type: nauc_ndcg_at_10_max + value: 15.50666163176253 + - type: nauc_ndcg_at_10_std + value: 3.1415591079404876 + - type: nauc_ndcg_at_1_diff1 + value: 27.75425157234291 + - type: nauc_ndcg_at_1_max + value: 12.264180225035945 + - type: nauc_ndcg_at_1_std + value: -2.9856311667661473 + - type: nauc_ndcg_at_20_diff1 + value: 18.851607836867498 + - type: nauc_ndcg_at_20_max + value: 16.48340412282649 + - type: nauc_ndcg_at_20_std + value: 4.4126572393461165 + - type: nauc_ndcg_at_3_diff1 + value: 21.91877312160879 + - type: nauc_ndcg_at_3_max + value: 13.094685444889004 + - type: nauc_ndcg_at_3_std + value: -0.6318460963618564 + - type: nauc_ndcg_at_5_diff1 + value: 20.572894306217556 + - type: nauc_ndcg_at_5_max + value: 14.021287944779818 + - type: nauc_ndcg_at_5_std + value: 0.883274716789248 + - type: nauc_precision_at_1000_diff1 + value: 4.720401672882462 + - type: nauc_precision_at_1000_max + value: 32.576463495142036 + - type: nauc_precision_at_1000_std + value: 31.992921949716685 + - type: nauc_precision_at_100_diff1 + value: 8.668918137307568 + - type: nauc_precision_at_100_max + value: 28.43637657585084 + - type: nauc_precision_at_100_std + value: 24.081259792345715 + - type: nauc_precision_at_10_diff1 + value: 13.4834870121441 + - type: nauc_precision_at_10_max + value: 19.174654923689058 + - type: nauc_precision_at_10_std + value: 9.740431350499781 + - type: nauc_precision_at_1_diff1 + value: 27.75425157234291 + - type: nauc_precision_at_1_max + value: 12.264180225035945 + - type: nauc_precision_at_1_std + value: -2.9856311667661473 + - type: nauc_precision_at_20_diff1 + value: 12.350150417728162 + - type: nauc_precision_at_20_max + value: 22.212511486344233 + - type: nauc_precision_at_20_std + value: 13.526492272122145 + - type: nauc_precision_at_3_diff1 + value: 18.64161060878928 + - type: nauc_precision_at_3_max + value: 13.97868776386616 + - type: nauc_precision_at_3_std + value: 1.5199256063275335 + - type: nauc_precision_at_5_diff1 + value: 16.302163063725423 + - type: nauc_precision_at_5_max + value: 15.854519904730397 + - type: nauc_precision_at_5_std + value: 4.58110916862716 + - type: nauc_recall_at_1000_diff1 + value: 6.38722839005945 + - type: nauc_recall_at_1000_max + value: 31.894646548853515 + - type: nauc_recall_at_1000_std + value: 30.783921718347955 + - type: nauc_recall_at_100_diff1 + value: 9.226926697109567 + - type: nauc_recall_at_100_max + value: 26.16213840995931 + - type: nauc_recall_at_100_std + value: 21.293733397810367 + - type: nauc_recall_at_10_diff1 + value: 13.76020081492309 + - type: nauc_recall_at_10_max + value: 18.19732247560085 + - type: nauc_recall_at_10_std + value: 8.53316271133414 + - type: nauc_recall_at_1_diff1 + value: 27.661685422401224 + - type: nauc_recall_at_1_max + value: 11.99044345858924 + - type: nauc_recall_at_1_std + value: -3.3979769351402322 + - type: nauc_recall_at_20_diff1 + value: 12.650003144424666 + - type: nauc_recall_at_20_max + value: 20.557776180489952 + - type: nauc_recall_at_20_std + value: 11.639706607747087 + - type: nauc_recall_at_3_diff1 + value: 18.999448979579647 + - type: nauc_recall_at_3_max + value: 13.339340670906067 + - type: nauc_recall_at_3_std + value: 0.7757658802091059 + - type: nauc_recall_at_5_diff1 + value: 16.30330865863684 + - type: nauc_recall_at_5_max + value: 14.848069915205242 + - type: nauc_recall_at_5_std + value: 3.5287752456847232 + - type: ndcg_at_1 + value: 11.086 + - type: ndcg_at_10 + value: 18.761 + - type: ndcg_at_100 + value: 23.362 + - type: ndcg_at_1000 + value: 25.651000000000003 + - type: ndcg_at_20 + value: 20.406 + - type: ndcg_at_3 + value: 14.865 + - type: ndcg_at_5 + value: 16.594 + - type: precision_at_1 + value: 11.086 + - type: precision_at_10 + value: 3.072 + - type: precision_at_100 + value: 0.553 + - type: precision_at_1000 + value: 0.076 + - type: precision_at_20 + value: 1.891 + - type: precision_at_3 + value: 6.331 + - type: precision_at_5 + value: 4.707 + - type: recall_at_1 + value: 10.476 + - type: recall_at_10 + value: 28.544999999999998 + - type: recall_at_100 + value: 50.602999999999994 + - type: recall_at_1000 + value: 68.609 + - type: recall_at_20 + value: 34.919 + - type: recall_at_3 + value: 17.79 + - type: recall_at_5 + value: 21.956 + task: + type: Retrieval + - dataset: + config: default + name: MTEB FiQA2018 (default) + revision: 27a168819829fe9bcd655c2df245fb19452e8e06 + split: test + type: mteb/fiqa + metrics: + - type: main_score + value: 10.219000000000001 + - type: map_at_1 + value: 4.202 + - type: map_at_10 + value: 7.047000000000001 + - type: map_at_100 + value: 7.879 + - type: map_at_1000 + value: 8.062999999999999 + - type: map_at_20 + value: 7.393 + - type: map_at_3 + value: 5.8229999999999995 + - type: map_at_5 + value: 6.398 + - type: mrr_at_1 + value: 8.641975308641975 + - type: mrr_at_10 + value: 12.682184499314136 + - type: mrr_at_100 + value: 13.641850480305543 + - type: mrr_at_1000 + value: 13.764256983049062 + - type: mrr_at_20 + value: 13.167533189337608 + - type: mrr_at_3 + value: 10.956790123456795 + - type: mrr_at_5 + value: 11.697530864197537 + - type: nauc_map_at_1000_diff1 + value: 14.386120645400236 + - type: nauc_map_at_1000_max + value: 7.810435452171063 + - type: nauc_map_at_1000_std + value: 6.279009499496127 + - type: nauc_map_at_100_diff1 + value: 14.394099339804242 + - type: nauc_map_at_100_max + value: 7.5482313825844365 + - type: nauc_map_at_100_std + value: 6.073231086542136 + - type: nauc_map_at_10_diff1 + value: 15.019233622476264 + - type: nauc_map_at_10_max + value: 6.89561121695038 + - type: nauc_map_at_10_std + value: 4.69688965053592 + - type: nauc_map_at_1_diff1 + value: 15.673097607542335 + - type: nauc_map_at_1_max + value: 7.8728446291021905 + - type: nauc_map_at_1_std + value: 6.4648760315306255 + - type: nauc_map_at_20_diff1 + value: 14.300009454569937 + - type: nauc_map_at_20_max + value: 7.249872455283453 + - type: nauc_map_at_20_std + value: 5.268402065325031 + - type: nauc_map_at_3_diff1 + value: 15.80071370296546 + - type: nauc_map_at_3_max + value: 6.339581216563252 + - type: nauc_map_at_3_std + value: 4.61618681520698 + - type: nauc_map_at_5_diff1 + value: 14.770168963543034 + - type: nauc_map_at_5_max + value: 5.82190695283646 + - type: nauc_map_at_5_std + value: 4.041144061189678 + - type: nauc_mrr_at_1000_diff1 + value: 13.947508055836414 + - type: nauc_mrr_at_1000_max + value: 8.016461541914124 + - type: nauc_mrr_at_1000_std + value: 2.3739399442305404 + - type: nauc_mrr_at_100_diff1 + value: 13.916158963907396 + - type: nauc_mrr_at_100_max + value: 7.954544708752525 + - type: nauc_mrr_at_100_std + value: 2.3407189360908487 + - type: nauc_mrr_at_10_diff1 + value: 14.169248704433596 + - type: nauc_mrr_at_10_max + value: 7.250162018740328 + - type: nauc_mrr_at_10_std + value: 1.8219524654513586 + - type: nauc_mrr_at_1_diff1 + value: 14.956230962502085 + - type: nauc_mrr_at_1_max + value: 8.73931836103222 + - type: nauc_mrr_at_1_std + value: 2.64832616615408 + - type: nauc_mrr_at_20_diff1 + value: 14.024588680674979 + - type: nauc_mrr_at_20_max + value: 7.920759304685261 + - type: nauc_mrr_at_20_std + value: 2.060286657771815 + - type: nauc_mrr_at_3_diff1 + value: 15.015616913010701 + - type: nauc_mrr_at_3_max + value: 6.312297156230054 + - type: nauc_mrr_at_3_std + value: 2.696675323128789 + - type: nauc_mrr_at_5_diff1 + value: 14.0352810633434 + - type: nauc_mrr_at_5_max + value: 7.2737096971574156 + - type: nauc_mrr_at_5_std + value: 1.9681344739855215 + - type: nauc_ndcg_at_1000_diff1 + value: 12.794321125652422 + - type: nauc_ndcg_at_1000_max + value: 13.35215380526579 + - type: nauc_ndcg_at_1000_std + value: 11.213644620947537 + - type: nauc_ndcg_at_100_diff1 + value: 13.02582260228888 + - type: nauc_ndcg_at_100_max + value: 9.545082480237982 + - type: nauc_ndcg_at_100_std + value: 8.533517097634686 + - type: nauc_ndcg_at_10_diff1 + value: 14.529247699477052 + - type: nauc_ndcg_at_10_max + value: 7.485652713814595 + - type: nauc_ndcg_at_10_std + value: 3.75099655360646 + - type: nauc_ndcg_at_1_diff1 + value: 14.956230962502085 + - type: nauc_ndcg_at_1_max + value: 8.73931836103222 + - type: nauc_ndcg_at_1_std + value: 2.64832616615408 + - type: nauc_ndcg_at_20_diff1 + value: 13.005656656807135 + - type: nauc_ndcg_at_20_max + value: 8.412698057403183 + - type: nauc_ndcg_at_20_std + value: 4.9112354043743 + - type: nauc_ndcg_at_3_diff1 + value: 14.363695670994714 + - type: nauc_ndcg_at_3_max + value: 6.793799720301805 + - type: nauc_ndcg_at_3_std + value: 3.568348121928968 + - type: nauc_ndcg_at_5_diff1 + value: 13.518402147118861 + - type: nauc_ndcg_at_5_max + value: 6.463128522589964 + - type: nauc_ndcg_at_5_std + value: 2.6290251894206036 + - type: nauc_precision_at_1000_diff1 + value: -0.14859656704447735 + - type: nauc_precision_at_1000_max + value: 22.305443744434182 + - type: nauc_precision_at_1000_std + value: 8.879840465533688 + - type: nauc_precision_at_100_diff1 + value: 6.760575881719946 + - type: nauc_precision_at_100_max + value: 15.29737164075807 + - type: nauc_precision_at_100_std + value: 11.230012315493347 + - type: nauc_precision_at_10_diff1 + value: 11.759182955072145 + - type: nauc_precision_at_10_max + value: 8.660562486174117 + - type: nauc_precision_at_10_std + value: 3.3549292515779094 + - type: nauc_precision_at_1_diff1 + value: 14.956230962502085 + - type: nauc_precision_at_1_max + value: 8.73931836103222 + - type: nauc_precision_at_1_std + value: 2.64832616615408 + - type: nauc_precision_at_20_diff1 + value: 7.401208860694865 + - type: nauc_precision_at_20_max + value: 12.51957393165894 + - type: nauc_precision_at_20_std + value: 3.7015352831870416 + - type: nauc_precision_at_3_diff1 + value: 14.491922438510684 + - type: nauc_precision_at_3_max + value: 7.32351776738068 + - type: nauc_precision_at_3_std + value: 1.9199578473009384 + - type: nauc_precision_at_5_diff1 + value: 12.119924723755787 + - type: nauc_precision_at_5_max + value: 8.55933421537529 + - type: nauc_precision_at_5_std + value: 0.9457844262487164 + - type: nauc_recall_at_1000_diff1 + value: 8.83089587152667 + - type: nauc_recall_at_1000_max + value: 20.396552995861764 + - type: nauc_recall_at_1000_std + value: 26.716191067517524 + - type: nauc_recall_at_100_diff1 + value: 9.43183012992213 + - type: nauc_recall_at_100_max + value: 9.501501571448282 + - type: nauc_recall_at_100_std + value: 14.285058832848089 + - type: nauc_recall_at_10_diff1 + value: 13.839808835810432 + - type: nauc_recall_at_10_max + value: 7.933505226644632 + - type: nauc_recall_at_10_std + value: 3.1966135029140164 + - type: nauc_recall_at_1_diff1 + value: 15.673097607542335 + - type: nauc_recall_at_1_max + value: 7.8728446291021905 + - type: nauc_recall_at_1_std + value: 6.4648760315306255 + - type: nauc_recall_at_20_diff1 + value: 10.437563158014042 + - type: nauc_recall_at_20_max + value: 8.311799162247189 + - type: nauc_recall_at_20_std + value: 5.85366764191572 + - type: nauc_recall_at_3_diff1 + value: 14.407773253315103 + - type: nauc_recall_at_3_max + value: 4.396111262809052 + - type: nauc_recall_at_3_std + value: 3.8372791660540058 + - type: nauc_recall_at_5_diff1 + value: 11.54279108068964 + - type: nauc_recall_at_5_max + value: 4.945221910869192 + - type: nauc_recall_at_5_std + value: 1.7566860385328107 + - type: ndcg_at_1 + value: 8.642 + - type: ndcg_at_10 + value: 10.219000000000001 + - type: ndcg_at_100 + value: 14.838000000000001 + - type: ndcg_at_1000 + value: 19.335 + - type: ndcg_at_20 + value: 11.495 + - type: ndcg_at_3 + value: 8.021 + - type: ndcg_at_5 + value: 8.663 + - type: precision_at_1 + value: 8.642 + - type: precision_at_10 + value: 3.086 + - type: precision_at_100 + value: 0.747 + - type: precision_at_1000 + value: 0.152 + - type: precision_at_20 + value: 2.022 + - type: precision_at_3 + value: 5.453 + - type: precision_at_5 + value: 4.198 + - type: recall_at_1 + value: 4.202 + - type: recall_at_10 + value: 14.086000000000002 + - type: recall_at_100 + value: 32.757999999999996 + - type: recall_at_1000 + value: 60.649 + - type: recall_at_20 + value: 18.129 + - type: recall_at_3 + value: 7.8549999999999995 + - type: recall_at_5 + value: 9.861 + task: + type: Retrieval + - dataset: + config: default + name: MTEB HotpotQA (default) + revision: ab518f4d6fcca38d87c25209f94beba119d02014 + split: test + type: mteb/hotpotqa + metrics: + - type: main_score + value: 21.614 + - type: map_at_1 + value: 11.789 + - type: map_at_10 + value: 16.298000000000002 + - type: map_at_100 + value: 16.965 + - type: map_at_1000 + value: 17.052 + - type: map_at_20 + value: 16.669 + - type: map_at_3 + value: 14.963000000000001 + - type: map_at_5 + value: 15.709000000000001 + - type: mrr_at_1 + value: 23.578663065496286 + - type: mrr_at_10 + value: 29.639073770403908 + - type: mrr_at_100 + value: 30.359813527624098 + - type: mrr_at_1000 + value: 30.427786306549432 + - type: mrr_at_20 + value: 30.057356684477064 + - type: mrr_at_3 + value: 27.91357191087095 + - type: mrr_at_5 + value: 28.895340985820354 + - type: nauc_map_at_1000_diff1 + value: 44.917394348472826 + - type: nauc_map_at_1000_max + value: 31.80330450852885 + - type: nauc_map_at_1000_std + value: 14.578989457609914 + - type: nauc_map_at_100_diff1 + value: 44.946269171547904 + - type: nauc_map_at_100_max + value: 31.75547980049172 + - type: nauc_map_at_100_std + value: 14.497740666511097 + - type: nauc_map_at_10_diff1 + value: 45.63183405960151 + - type: nauc_map_at_10_max + value: 31.57684331750769 + - type: nauc_map_at_10_std + value: 13.720611937996688 + - type: nauc_map_at_1_diff1 + value: 54.88588090350412 + - type: nauc_map_at_1_max + value: 32.01539186380243 + - type: nauc_map_at_1_std + value: 9.711864312852528 + - type: nauc_map_at_20_diff1 + value: 45.183673410307875 + - type: nauc_map_at_20_max + value: 31.675889031272824 + - type: nauc_map_at_20_std + value: 14.117826246559353 + - type: nauc_map_at_3_diff1 + value: 47.52208598209566 + - type: nauc_map_at_3_max + value: 31.86158421243083 + - type: nauc_map_at_3_std + value: 12.381248746187193 + - type: nauc_map_at_5_diff1 + value: 46.442184401322145 + - type: nauc_map_at_5_max + value: 31.63017703410021 + - type: nauc_map_at_5_std + value: 13.132735114121214 + - type: nauc_mrr_at_1000_diff1 + value: 47.54619690126628 + - type: nauc_mrr_at_1000_max + value: 30.657047435440386 + - type: nauc_mrr_at_1000_std + value: 12.157743555031063 + - type: nauc_mrr_at_100_diff1 + value: 47.53279072131234 + - type: nauc_mrr_at_100_max + value: 30.644795488492903 + - type: nauc_mrr_at_100_std + value: 12.148316668565558 + - type: nauc_mrr_at_10_diff1 + value: 47.80371467312708 + - type: nauc_mrr_at_10_max + value: 30.63631672644867 + - type: nauc_mrr_at_10_std + value: 11.850411637055567 + - type: nauc_mrr_at_1_diff1 + value: 54.88588090350412 + - type: nauc_mrr_at_1_max + value: 32.01539186380243 + - type: nauc_mrr_at_1_std + value: 9.711864312852528 + - type: nauc_mrr_at_20_diff1 + value: 47.59914593566423 + - type: nauc_mrr_at_20_max + value: 30.64820791166759 + - type: nauc_mrr_at_20_std + value: 12.025245568167318 + - type: nauc_mrr_at_3_diff1 + value: 49.07945193378141 + - type: nauc_mrr_at_3_max + value: 31.101665157446384 + - type: nauc_mrr_at_3_std + value: 11.261770295831171 + - type: nauc_mrr_at_5_diff1 + value: 48.31599574171794 + - type: nauc_mrr_at_5_max + value: 30.758029728579356 + - type: nauc_mrr_at_5_std + value: 11.57586015346866 + - type: nauc_ndcg_at_1000_diff1 + value: 40.59730013454305 + - type: nauc_ndcg_at_1000_max + value: 31.858727171630406 + - type: nauc_ndcg_at_1000_std + value: 18.38911731839912 + - type: nauc_ndcg_at_100_diff1 + value: 41.05727875473311 + - type: nauc_ndcg_at_100_max + value: 31.200209607115507 + - type: nauc_ndcg_at_100_std + value: 17.143920588351886 + - type: nauc_ndcg_at_10_diff1 + value: 43.438534881275444 + - type: nauc_ndcg_at_10_max + value: 30.737946990772997 + - type: nauc_ndcg_at_10_std + value: 14.449960369509911 + - type: nauc_ndcg_at_1_diff1 + value: 54.88588090350412 + - type: nauc_ndcg_at_1_max + value: 32.01539186380243 + - type: nauc_ndcg_at_1_std + value: 9.711864312852528 + - type: nauc_ndcg_at_20_diff1 + value: 42.25718271284509 + - type: nauc_ndcg_at_20_max + value: 30.897435243868905 + - type: nauc_ndcg_at_20_std + value: 15.412549120054193 + - type: nauc_ndcg_at_3_diff1 + value: 46.53638838597477 + - type: nauc_ndcg_at_3_max + value: 31.390495377790177 + - type: nauc_ndcg_at_3_std + value: 12.468335527090519 + - type: nauc_ndcg_at_5_diff1 + value: 44.92693354952539 + - type: nauc_ndcg_at_5_max + value: 30.887428625960915 + - type: nauc_ndcg_at_5_std + value: 13.403669618286864 + - type: nauc_precision_at_1000_diff1 + value: 14.046084009393786 + - type: nauc_precision_at_1000_max + value: 26.025160058476338 + - type: nauc_precision_at_1000_std + value: 29.157031067483256 + - type: nauc_precision_at_100_diff1 + value: 21.990659256216876 + - type: nauc_precision_at_100_max + value: 26.93221065873268 + - type: nauc_precision_at_100_std + value: 25.151535987506747 + - type: nauc_precision_at_10_diff1 + value: 33.711273609447844 + - type: nauc_precision_at_10_max + value: 28.4857023531615 + - type: nauc_precision_at_10_std + value: 17.99141308322003 + - type: nauc_precision_at_1_diff1 + value: 54.88588090350412 + - type: nauc_precision_at_1_max + value: 32.01539186380243 + - type: nauc_precision_at_1_std + value: 9.711864312852528 + - type: nauc_precision_at_20_diff1 + value: 29.06191496521685 + - type: nauc_precision_at_20_max + value: 27.913633624163765 + - type: nauc_precision_at_20_std + value: 20.103925765892363 + - type: nauc_precision_at_3_diff1 + value: 41.89101661752835 + - type: nauc_precision_at_3_max + value: 30.936389232946116 + - type: nauc_precision_at_3_std + value: 13.983206968178264 + - type: nauc_precision_at_5_diff1 + value: 38.15960353436429 + - type: nauc_precision_at_5_max + value: 29.48701633210831 + - type: nauc_precision_at_5_std + value: 15.725313714018691 + - type: nauc_recall_at_1000_diff1 + value: 14.046084009393903 + - type: nauc_recall_at_1000_max + value: 26.02516005847642 + - type: nauc_recall_at_1000_std + value: 29.157031067483334 + - type: nauc_recall_at_100_diff1 + value: 21.990659256216883 + - type: nauc_recall_at_100_max + value: 26.932210658732647 + - type: nauc_recall_at_100_std + value: 25.15153598750676 + - type: nauc_recall_at_10_diff1 + value: 33.71127360944788 + - type: nauc_recall_at_10_max + value: 28.485702353161496 + - type: nauc_recall_at_10_std + value: 17.991413083220063 + - type: nauc_recall_at_1_diff1 + value: 54.88588090350412 + - type: nauc_recall_at_1_max + value: 32.01539186380243 + - type: nauc_recall_at_1_std + value: 9.711864312852528 + - type: nauc_recall_at_20_diff1 + value: 29.061914965216857 + - type: nauc_recall_at_20_max + value: 27.913633624163786 + - type: nauc_recall_at_20_std + value: 20.103925765892363 + - type: nauc_recall_at_3_diff1 + value: 41.89101661752835 + - type: nauc_recall_at_3_max + value: 30.936389232946127 + - type: nauc_recall_at_3_std + value: 13.983206968178255 + - type: nauc_recall_at_5_diff1 + value: 38.15960353436428 + - type: nauc_recall_at_5_max + value: 29.487016332108308 + - type: nauc_recall_at_5_std + value: 15.72531371401871 + - type: ndcg_at_1 + value: 23.579 + - type: ndcg_at_10 + value: 21.614 + - type: ndcg_at_100 + value: 25.008000000000003 + - type: ndcg_at_1000 + value: 27.394000000000002 + - type: ndcg_at_20 + value: 22.909 + - type: ndcg_at_3 + value: 18.806 + - type: ndcg_at_5 + value: 20.180999999999997 + - type: precision_at_1 + value: 23.579 + - type: precision_at_10 + value: 4.8469999999999995 + - type: precision_at_100 + value: 0.757 + - type: precision_at_1000 + value: 0.108 + - type: precision_at_20 + value: 2.841 + - type: precision_at_3 + value: 11.915000000000001 + - type: precision_at_5 + value: 8.243 + - type: recall_at_1 + value: 11.789 + - type: recall_at_10 + value: 24.234 + - type: recall_at_100 + value: 37.833 + - type: recall_at_1000 + value: 53.869 + - type: recall_at_20 + value: 28.406 + - type: recall_at_3 + value: 17.873 + - type: recall_at_5 + value: 20.608 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ImdbClassification (default) + revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 + split: test + type: mteb/imdb + metrics: + - type: accuracy + value: 70.35639999999998 + - type: ap + value: 64.70837553635555 + - type: ap_weighted + value: 64.70837553635555 + - type: f1 + value: 70.18257490051944 + - type: f1_weighted + value: 70.18257490051944 + - type: main_score + value: 70.35639999999998 + task: + type: Classification + - dataset: + config: default + name: MTEB MSMARCO (default) + revision: c5a29a104738b98a9e76336939199e264163d4a0 + split: test + type: mteb/msmarco + metrics: + - type: main_score + value: 29.474 + - type: map_at_1 + value: 0.844 + - type: map_at_10 + value: 5.276 + - type: map_at_100 + value: 12.017999999999999 + - type: map_at_1000 + value: 15.082 + - type: map_at_20 + value: 7.545 + - type: map_at_3 + value: 2.102 + - type: map_at_5 + value: 3.3000000000000003 + - type: mrr_at_1 + value: 44.18604651162791 + - type: mrr_at_10 + value: 54.360465116279066 + - type: mrr_at_100 + value: 54.99847310312427 + - type: mrr_at_1000 + value: 55.02059372381039 + - type: mrr_at_20 + value: 54.799741602067186 + - type: mrr_at_3 + value: 50.775193798449614 + - type: mrr_at_5 + value: 53.68217054263565 + - type: nauc_map_at_1000_diff1 + value: -2.021143214503584 + - type: nauc_map_at_1000_max + value: 52.31379907413224 + - type: nauc_map_at_1000_std + value: 49.2308667371812 + - type: nauc_map_at_100_diff1 + value: -3.8937842066417074 + - type: nauc_map_at_100_max + value: 42.865843640140625 + - type: nauc_map_at_100_std + value: 37.95585233504797 + - type: nauc_map_at_10_diff1 + value: 1.606824138727677 + - type: nauc_map_at_10_max + value: 23.890927075059928 + - type: nauc_map_at_10_std + value: 16.257680972641474 + - type: nauc_map_at_1_diff1 + value: 22.570349150082496 + - type: nauc_map_at_1_max + value: 19.238395165822904 + - type: nauc_map_at_1_std + value: 17.550539838923076 + - type: nauc_map_at_20_diff1 + value: -4.951486071114569 + - type: nauc_map_at_20_max + value: 27.583006880684685 + - type: nauc_map_at_20_std + value: 20.615620122179987 + - type: nauc_map_at_3_diff1 + value: 14.786952703553915 + - type: nauc_map_at_3_max + value: 20.55790784157572 + - type: nauc_map_at_3_std + value: 19.79232931312015 + - type: nauc_map_at_5_diff1 + value: 12.076877755500902 + - type: nauc_map_at_5_max + value: 24.663714787605002 + - type: nauc_map_at_5_std + value: 20.483862838332758 + - type: nauc_mrr_at_1000_diff1 + value: -2.0990215138353774 + - type: nauc_mrr_at_1000_max + value: 63.648044801780216 + - type: nauc_mrr_at_1000_std + value: 52.77427870744277 + - type: nauc_mrr_at_100_diff1 + value: -2.168718494363165 + - type: nauc_mrr_at_100_max + value: 63.67099864984754 + - type: nauc_mrr_at_100_std + value: 52.80928825893113 + - type: nauc_mrr_at_10_diff1 + value: -1.792364177532816 + - type: nauc_mrr_at_10_max + value: 64.24673279475928 + - type: nauc_mrr_at_10_std + value: 53.00600172156902 + - type: nauc_mrr_at_1_diff1 + value: -3.3449341568902606 + - type: nauc_mrr_at_1_max + value: 53.62962555265127 + - type: nauc_mrr_at_1_std + value: 41.51614972127939 + - type: nauc_mrr_at_20_diff1 + value: -2.15540611512638 + - type: nauc_mrr_at_20_max + value: 63.819872925158805 + - type: nauc_mrr_at_20_std + value: 52.7961849237132 + - type: nauc_mrr_at_3_diff1 + value: -0.43541432598439994 + - type: nauc_mrr_at_3_max + value: 59.7829402577313 + - type: nauc_mrr_at_3_std + value: 50.90856959435809 + - type: nauc_mrr_at_5_diff1 + value: -2.3053362706782323 + - type: nauc_mrr_at_5_max + value: 64.96293039839213 + - type: nauc_mrr_at_5_std + value: 54.140790167234734 + - type: nauc_ndcg_at_1000_diff1 + value: 4.829519342110107 + - type: nauc_ndcg_at_1000_max + value: 61.07451345243899 + - type: nauc_ndcg_at_1000_std + value: 57.92106611562103 + - type: nauc_ndcg_at_100_diff1 + value: 2.04263990794124 + - type: nauc_ndcg_at_100_max + value: 55.358917987828036 + - type: nauc_ndcg_at_100_std + value: 49.87187447439087 + - type: nauc_ndcg_at_10_diff1 + value: -3.004323448668936 + - type: nauc_ndcg_at_10_max + value: 52.83608965686278 + - type: nauc_ndcg_at_10_std + value: 41.53186594077247 + - type: nauc_ndcg_at_1_diff1 + value: 8.54881714077641 + - type: nauc_ndcg_at_1_max + value: 49.57842930523978 + - type: nauc_ndcg_at_1_std + value: 36.75885193295566 + - type: nauc_ndcg_at_20_diff1 + value: -5.839572654994822 + - type: nauc_ndcg_at_20_max + value: 55.613971289427624 + - type: nauc_ndcg_at_20_std + value: 44.3490091376206 + - type: nauc_ndcg_at_3_diff1 + value: 0.9619430931504145 + - type: nauc_ndcg_at_3_max + value: 47.53601312984855 + - type: nauc_ndcg_at_3_std + value: 40.4176409277924 + - type: nauc_ndcg_at_5_diff1 + value: 2.4738911967465573 + - type: nauc_ndcg_at_5_max + value: 54.82024302253763 + - type: nauc_ndcg_at_5_std + value: 45.60241734712396 + - type: nauc_precision_at_1000_diff1 + value: 0.3013953483568884 + - type: nauc_precision_at_1000_max + value: 57.70808930624168 + - type: nauc_precision_at_1000_std + value: 58.201127413689 + - type: nauc_precision_at_100_diff1 + value: -2.741499937534905 + - type: nauc_precision_at_100_max + value: 63.181504513834575 + - type: nauc_precision_at_100_std + value: 61.10964102659246 + - type: nauc_precision_at_10_diff1 + value: -7.539781228254228 + - type: nauc_precision_at_10_max + value: 55.52784878560168 + - type: nauc_precision_at_10_std + value: 44.104151562310946 + - type: nauc_precision_at_1_diff1 + value: -3.3449341568902606 + - type: nauc_precision_at_1_max + value: 53.62962555265127 + - type: nauc_precision_at_1_std + value: 41.51614972127939 + - type: nauc_precision_at_20_diff1 + value: -9.570969327978647 + - type: nauc_precision_at_20_max + value: 57.27619865280717 + - type: nauc_precision_at_20_std + value: 47.014271773458766 + - type: nauc_precision_at_3_diff1 + value: -8.650135123593332 + - type: nauc_precision_at_3_max + value: 52.75623365854728 + - type: nauc_precision_at_3_std + value: 45.17073452831896 + - type: nauc_precision_at_5_diff1 + value: -5.93704448023575 + - type: nauc_precision_at_5_max + value: 59.65088855595867 + - type: nauc_precision_at_5_std + value: 49.423110379770065 + - type: nauc_recall_at_1000_diff1 + value: 9.797434639403033 + - type: nauc_recall_at_1000_max + value: 56.59565673453876 + - type: nauc_recall_at_1000_std + value: 55.7036657099338 + - type: nauc_recall_at_100_diff1 + value: 3.5802050256517317 + - type: nauc_recall_at_100_max + value: 44.92647077291689 + - type: nauc_recall_at_100_std + value: 40.98080812630615 + - type: nauc_recall_at_10_diff1 + value: 3.3855332934188818 + - type: nauc_recall_at_10_max + value: 24.87343302186072 + - type: nauc_recall_at_10_std + value: 15.077881728039621 + - type: nauc_recall_at_1_diff1 + value: 22.570349150082496 + - type: nauc_recall_at_1_max + value: 19.238395165822904 + - type: nauc_recall_at_1_std + value: 17.550539838923076 + - type: nauc_recall_at_20_diff1 + value: -2.129517961134834 + - type: nauc_recall_at_20_max + value: 29.231226886425212 + - type: nauc_recall_at_20_std + value: 21.112189894866912 + - type: nauc_recall_at_3_diff1 + value: 11.57541674939924 + - type: nauc_recall_at_3_max + value: 19.503507707855817 + - type: nauc_recall_at_3_std + value: 19.46878131021193 + - type: nauc_recall_at_5_diff1 + value: 11.802964146201003 + - type: nauc_recall_at_5_max + value: 25.973454165368093 + - type: nauc_recall_at_5_std + value: 20.259629965777922 + - type: ndcg_at_1 + value: 25.968999999999998 + - type: ndcg_at_10 + value: 29.474 + - type: ndcg_at_100 + value: 26.762999999999998 + - type: ndcg_at_1000 + value: 35.032000000000004 + - type: ndcg_at_20 + value: 28.141 + - type: ndcg_at_3 + value: 28.58 + - type: ndcg_at_5 + value: 30.213 + - type: precision_at_1 + value: 44.186 + - type: precision_at_10 + value: 40.465 + - type: precision_at_100 + value: 17.023 + - type: precision_at_1000 + value: 3.881 + - type: precision_at_20 + value: 33.721000000000004 + - type: precision_at_3 + value: 43.411 + - type: precision_at_5 + value: 45.116 + - type: recall_at_1 + value: 0.844 + - type: recall_at_10 + value: 6.4670000000000005 + - type: recall_at_100 + value: 21.831 + - type: recall_at_1000 + value: 44.84 + - type: recall_at_20 + value: 10.042 + - type: recall_at_3 + value: 2.2800000000000002 + - type: recall_at_5 + value: 3.94 + task: + type: Retrieval + - dataset: + config: en + name: MTEB MTOPDomainClassification (en) + revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf + split: test + type: mteb/mtop_domain + metrics: + - type: accuracy + value: 79.27268581851345 + - type: f1 + value: 77.91734002694149 + - type: f1_weighted + value: 79.14088602852584 + - type: main_score + value: 79.27268581851345 + task: + type: Classification + - dataset: + config: en + name: MTEB MTOPIntentClassification (en) + revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba + split: test + type: mteb/mtop_intent + metrics: + - type: accuracy + value: 42.49886000911992 + - type: f1 + value: 27.161113280134728 + - type: f1_weighted + value: 46.29236281595424 + - type: main_score + value: 42.49886000911992 + task: + type: Classification + - dataset: + config: en + name: MTEB MassiveIntentClassification (en) + revision: 4672e20407010da34463acc759c162ca9734bca6 + split: test + type: mteb/amazon_massive_intent + metrics: + - type: accuracy + value: 54.065232010759914 + - type: f1 + value: 52.64846159370178 + - type: f1_weighted + value: 52.69815077422998 + - type: main_score + value: 54.065232010759914 + task: + type: Classification + - dataset: + config: en + name: MTEB MassiveScenarioClassification (en) + revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 + split: test + type: mteb/amazon_massive_scenario + metrics: + - type: accuracy + value: 59.596503026227296 + - type: f1 + value: 58.454086272367725 + - type: f1_weighted + value: 59.23698301210568 + - type: main_score + value: 59.596503026227296 + task: + type: Classification + - dataset: + config: default + name: MTEB MedrxivClusteringP2P (default) + revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 + split: test + type: mteb/medrxiv-clustering-p2p + metrics: + - type: main_score + value: 25.59161751046095 + - type: v_measure + value: 25.59161751046095 + - type: v_measure_std + value: 1.4816189134361553 + task: + type: Clustering + - dataset: + config: default + name: MTEB MedrxivClusteringS2S (default) + revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 + split: test + type: mteb/medrxiv-clustering-s2s + metrics: + - type: main_score + value: 21.396391045777328 + - type: v_measure + value: 21.396391045777328 + - type: v_measure_std + value: 1.6103207158789596 + task: + type: Clustering + - dataset: + config: default + name: MTEB MindSmallReranking (default) + revision: 59042f120c80e8afa9cdbb224f67076cec0fc9a7 + split: test + type: mteb/mind_small + metrics: + - type: main_score + value: 28.017817065141404 + - type: map + value: 28.017817065141404 + - type: mrr + value: 28.540519062700398 + - type: nAUC_map_diff1 + value: 15.461550063785692 + - type: nAUC_map_max + value: -25.32105536328766 + - type: nAUC_map_std + value: -8.329979908589804 + - type: nAUC_mrr_diff1 + value: 14.686110906248775 + - type: nAUC_mrr_max + value: -19.527290469919414 + - type: nAUC_mrr_std + value: -6.772185014428633 + task: + type: Reranking + - dataset: + config: default + name: MTEB NFCorpus (default) + revision: ec0fa4fe99da2ff19ca1214b7966684033a58814 + split: test + type: mteb/nfcorpus + metrics: + - type: main_score + value: 18.958 + - type: map_at_1 + value: 2.6790000000000003 + - type: map_at_10 + value: 5.232 + - type: map_at_100 + value: 6.662 + - type: map_at_1000 + value: 7.611999999999999 + - type: map_at_20 + value: 5.753 + - type: map_at_3 + value: 4.037 + - type: map_at_5 + value: 4.611 + - type: mrr_at_1 + value: 28.482972136222912 + - type: mrr_at_10 + value: 36.405597326649946 + - type: mrr_at_100 + value: 37.22595173312844 + - type: mrr_at_1000 + value: 37.31683387820029 + - type: mrr_at_20 + value: 36.93095174519322 + - type: mrr_at_3 + value: 34.26212590299277 + - type: mrr_at_5 + value: 35.33023735810114 + - type: nauc_map_at_1000_diff1 + value: 34.55475823314036 + - type: nauc_map_at_1000_max + value: 27.34025191906842 + - type: nauc_map_at_1000_std + value: 27.342181927439913 + - type: nauc_map_at_100_diff1 + value: 35.89756716732043 + - type: nauc_map_at_100_max + value: 25.674881665083976 + - type: nauc_map_at_100_std + value: 26.198892926722237 + - type: nauc_map_at_10_diff1 + value: 39.22190087054285 + - type: nauc_map_at_10_max + value: 21.95676384880576 + - type: nauc_map_at_10_std + value: 23.963311484529996 + - type: nauc_map_at_1_diff1 + value: 37.24773940905471 + - type: nauc_map_at_1_max + value: 14.271803984288198 + - type: nauc_map_at_1_std + value: 29.688383585943097 + - type: nauc_map_at_20_diff1 + value: 37.68453649833976 + - type: nauc_map_at_20_max + value: 24.07159414296253 + - type: nauc_map_at_20_std + value: 25.447472109181398 + - type: nauc_map_at_3_diff1 + value: 41.58967433401796 + - type: nauc_map_at_3_max + value: 18.968008374733863 + - type: nauc_map_at_3_std + value: 21.28294335040504 + - type: nauc_map_at_5_diff1 + value: 40.49610097499772 + - type: nauc_map_at_5_max + value: 19.882044394766137 + - type: nauc_map_at_5_std + value: 22.896905648859477 + - type: nauc_mrr_at_1000_diff1 + value: 25.809484398850618 + - type: nauc_mrr_at_1000_max + value: 26.414231307244414 + - type: nauc_mrr_at_1000_std + value: 17.985986694919003 + - type: nauc_mrr_at_100_diff1 + value: 25.78606536353402 + - type: nauc_mrr_at_100_max + value: 26.4528757137163 + - type: nauc_mrr_at_100_std + value: 17.999133117747803 + - type: nauc_mrr_at_10_diff1 + value: 25.779300906162796 + - type: nauc_mrr_at_10_max + value: 26.213232859542853 + - type: nauc_mrr_at_10_std + value: 18.08183257670132 + - type: nauc_mrr_at_1_diff1 + value: 27.598875670159266 + - type: nauc_mrr_at_1_max + value: 20.32658251250794 + - type: nauc_mrr_at_1_std + value: 18.18823377326309 + - type: nauc_mrr_at_20_diff1 + value: 25.636959155407197 + - type: nauc_mrr_at_20_max + value: 26.464954542586 + - type: nauc_mrr_at_20_std + value: 18.000213123428896 + - type: nauc_mrr_at_3_diff1 + value: 26.204839589208444 + - type: nauc_mrr_at_3_max + value: 24.408100393220273 + - type: nauc_mrr_at_3_std + value: 17.5978010208717 + - type: nauc_mrr_at_5_diff1 + value: 26.40893924955858 + - type: nauc_mrr_at_5_max + value: 25.12051846945133 + - type: nauc_mrr_at_5_std + value: 17.88757695915964 + - type: nauc_ndcg_at_1000_diff1 + value: 28.382776725135717 + - type: nauc_ndcg_at_1000_max + value: 35.17640492284041 + - type: nauc_ndcg_at_1000_std + value: 21.960947306809935 + - type: nauc_ndcg_at_100_diff1 + value: 29.15650285424262 + - type: nauc_ndcg_at_100_max + value: 28.93369196915297 + - type: nauc_ndcg_at_100_std + value: 22.98450747640745 + - type: nauc_ndcg_at_10_diff1 + value: 23.285486018526193 + - type: nauc_ndcg_at_10_max + value: 25.356509158119785 + - type: nauc_ndcg_at_10_std + value: 21.47481325489274 + - type: nauc_ndcg_at_1_diff1 + value: 28.30393191665423 + - type: nauc_ndcg_at_1_max + value: 19.280395074712168 + - type: nauc_ndcg_at_1_std + value: 20.264338440493027 + - type: nauc_ndcg_at_20_diff1 + value: 25.056396490391524 + - type: nauc_ndcg_at_20_max + value: 27.345120936004818 + - type: nauc_ndcg_at_20_std + value: 23.96666766743033 + - type: nauc_ndcg_at_3_diff1 + value: 23.30190701865977 + - type: nauc_ndcg_at_3_max + value: 23.229020303623322 + - type: nauc_ndcg_at_3_std + value: 20.04669638404903 + - type: nauc_ndcg_at_5_diff1 + value: 23.09298233563678 + - type: nauc_ndcg_at_5_max + value: 24.961046635562553 + - type: nauc_ndcg_at_5_std + value: 20.761704284437112 + - type: nauc_precision_at_1000_diff1 + value: 1.0960840257544657 + - type: nauc_precision_at_1000_max + value: 18.092406576627578 + - type: nauc_precision_at_1000_std + value: 18.90288850498279 + - type: nauc_precision_at_100_diff1 + value: 6.267759440812813 + - type: nauc_precision_at_100_max + value: 24.652404577181972 + - type: nauc_precision_at_100_std + value: 22.887811997352166 + - type: nauc_precision_at_10_diff1 + value: 9.718986333358377 + - type: nauc_precision_at_10_max + value: 29.836283256489544 + - type: nauc_precision_at_10_std + value: 20.462452851651605 + - type: nauc_precision_at_1_diff1 + value: 27.598875670159266 + - type: nauc_precision_at_1_max + value: 20.32658251250794 + - type: nauc_precision_at_1_std + value: 18.18823377326309 + - type: nauc_precision_at_20_diff1 + value: 8.676606543268425 + - type: nauc_precision_at_20_max + value: 30.399108802564502 + - type: nauc_precision_at_20_std + value: 23.861676907877747 + - type: nauc_precision_at_3_diff1 + value: 17.45339550106393 + - type: nauc_precision_at_3_max + value: 26.216766421583156 + - type: nauc_precision_at_3_std + value: 19.277741416054468 + - type: nauc_precision_at_5_diff1 + value: 14.548311075529996 + - type: nauc_precision_at_5_max + value: 28.578357432043095 + - type: nauc_precision_at_5_std + value: 20.06284061666006 + - type: nauc_recall_at_1000_diff1 + value: 15.468783472460082 + - type: nauc_recall_at_1000_max + value: 18.76028457005554 + - type: nauc_recall_at_1000_std + value: 11.263359191726643 + - type: nauc_recall_at_100_diff1 + value: 24.242089843241242 + - type: nauc_recall_at_100_max + value: 16.5727197700428 + - type: nauc_recall_at_100_std + value: 11.874947318154511 + - type: nauc_recall_at_10_diff1 + value: 32.05672140211618 + - type: nauc_recall_at_10_max + value: 15.422864149105198 + - type: nauc_recall_at_10_std + value: 16.485136884962905 + - type: nauc_recall_at_1_diff1 + value: 37.24773940905471 + - type: nauc_recall_at_1_max + value: 14.271803984288198 + - type: nauc_recall_at_1_std + value: 29.688383585943097 + - type: nauc_recall_at_20_diff1 + value: 26.72558411584015 + - type: nauc_recall_at_20_max + value: 18.71742158521011 + - type: nauc_recall_at_20_std + value: 16.30979019801867 + - type: nauc_recall_at_3_diff1 + value: 42.18802462874201 + - type: nauc_recall_at_3_max + value: 15.849348165931904 + - type: nauc_recall_at_3_std + value: 15.265602449011315 + - type: nauc_recall_at_5_diff1 + value: 36.73549549313766 + - type: nauc_recall_at_5_max + value: 14.244134216875928 + - type: nauc_recall_at_5_std + value: 16.831447694292613 + - type: ndcg_at_1 + value: 27.245 + - type: ndcg_at_10 + value: 18.958 + - type: ndcg_at_100 + value: 17.480999999999998 + - type: ndcg_at_1000 + value: 26.590000000000003 + - type: ndcg_at_20 + value: 17.732999999999997 + - type: ndcg_at_3 + value: 22.745 + - type: ndcg_at_5 + value: 20.845 + - type: precision_at_1 + value: 28.483000000000004 + - type: precision_at_10 + value: 13.963000000000001 + - type: precision_at_100 + value: 4.926 + - type: precision_at_1000 + value: 1.7129999999999999 + - type: precision_at_20 + value: 10.867 + - type: precision_at_3 + value: 21.259 + - type: precision_at_5 + value: 17.647 + - type: recall_at_1 + value: 2.6790000000000003 + - type: recall_at_10 + value: 8.35 + - type: recall_at_100 + value: 18.846 + - type: recall_at_1000 + value: 51.906 + - type: recall_at_20 + value: 10.713000000000001 + - type: recall_at_3 + value: 5.005 + - type: recall_at_5 + value: 6.3950000000000005 + task: + type: Retrieval + - dataset: + config: default + name: MTEB NQ (default) + revision: b774495ed302d8c44a3a7ea25c90dbce03968f31 + split: test + type: mteb/nq + metrics: + - type: main_score + value: 13.048000000000002 + - type: map_at_1 + value: 5.733 + - type: map_at_10 + value: 10.041 + - type: map_at_100 + value: 10.93 + - type: map_at_1000 + value: 11.026 + - type: map_at_20 + value: 10.506 + - type: map_at_3 + value: 8.271 + - type: map_at_5 + value: 9.245000000000001 + - type: mrr_at_1 + value: 6.60486674391657 + - type: mrr_at_10 + value: 11.158530136658742 + - type: mrr_at_100 + value: 12.026649413998268 + - type: mrr_at_1000 + value: 12.113535659557408 + - type: mrr_at_20 + value: 11.630352356549999 + - type: mrr_at_3 + value: 9.385863267670912 + - type: mrr_at_5 + value: 10.349073001158729 + - type: nauc_map_at_1000_diff1 + value: 18.601922397825142 + - type: nauc_map_at_1000_max + value: 15.538804960648637 + - type: nauc_map_at_1000_std + value: 8.932607273344003 + - type: nauc_map_at_100_diff1 + value: 18.620680660067194 + - type: nauc_map_at_100_max + value: 15.484338018985905 + - type: nauc_map_at_100_std + value: 8.819973524636717 + - type: nauc_map_at_10_diff1 + value: 19.13487178673801 + - type: nauc_map_at_10_max + value: 14.806496007734665 + - type: nauc_map_at_10_std + value: 7.298071664899234 + - type: nauc_map_at_1_diff1 + value: 21.9930605817395 + - type: nauc_map_at_1_max + value: 11.133712801824862 + - type: nauc_map_at_1_std + value: 2.131487062602308 + - type: nauc_map_at_20_diff1 + value: 18.845986559595225 + - type: nauc_map_at_20_max + value: 14.84348513576822 + - type: nauc_map_at_20_std + value: 7.856790248019083 + - type: nauc_map_at_3_diff1 + value: 19.755782126040426 + - type: nauc_map_at_3_max + value: 12.255007439868141 + - type: nauc_map_at_3_std + value: 4.958398044298864 + - type: nauc_map_at_5_diff1 + value: 19.022537070708342 + - type: nauc_map_at_5_max + value: 13.17042573872807 + - type: nauc_map_at_5_std + value: 5.482903489537978 + - type: nauc_mrr_at_1000_diff1 + value: 17.632222622653444 + - type: nauc_mrr_at_1000_max + value: 15.42476571014277 + - type: nauc_mrr_at_1000_std + value: 10.153638750166555 + - type: nauc_mrr_at_100_diff1 + value: 17.646689914747668 + - type: nauc_mrr_at_100_max + value: 15.38756697506547 + - type: nauc_mrr_at_100_std + value: 10.083635499306897 + - type: nauc_mrr_at_10_diff1 + value: 18.028117672644154 + - type: nauc_mrr_at_10_max + value: 14.860588727254278 + - type: nauc_mrr_at_10_std + value: 8.896662029848459 + - type: nauc_mrr_at_1_diff1 + value: 21.40810772236307 + - type: nauc_mrr_at_1_max + value: 11.946611379314245 + - type: nauc_mrr_at_1_std + value: 4.892375669408125 + - type: nauc_mrr_at_20_diff1 + value: 17.804802947324127 + - type: nauc_mrr_at_20_max + value: 14.878056026236205 + - type: nauc_mrr_at_20_std + value: 9.377905847506275 + - type: nauc_mrr_at_3_diff1 + value: 18.414257987511295 + - type: nauc_mrr_at_3_max + value: 12.761434587966134 + - type: nauc_mrr_at_3_std + value: 7.11744205502733 + - type: nauc_mrr_at_5_diff1 + value: 17.685888476834307 + - type: nauc_mrr_at_5_max + value: 13.357657290287806 + - type: nauc_mrr_at_5_std + value: 7.41999209613162 + - type: nauc_ndcg_at_1000_diff1 + value: 15.737671084966395 + - type: nauc_ndcg_at_1000_max + value: 21.300001109022485 + - type: nauc_ndcg_at_1000_std + value: 18.420598093162717 + - type: nauc_ndcg_at_100_diff1 + value: 16.10827621536761 + - type: nauc_ndcg_at_100_max + value: 19.937939696831897 + - type: nauc_ndcg_at_100_std + value: 16.25333603534063 + - type: nauc_ndcg_at_10_diff1 + value: 17.91787045098329 + - type: nauc_ndcg_at_10_max + value: 17.054099236569016 + - type: nauc_ndcg_at_10_std + value: 10.235961770803096 + - type: nauc_ndcg_at_1_diff1 + value: 21.40810772236307 + - type: nauc_ndcg_at_1_max + value: 11.946611379314245 + - type: nauc_ndcg_at_1_std + value: 4.892375669408125 + - type: nauc_ndcg_at_20_diff1 + value: 17.19832761450297 + - type: nauc_ndcg_at_20_max + value: 16.87970880397001 + - type: nauc_ndcg_at_20_std + value: 11.592453610359893 + - type: nauc_ndcg_at_3_diff1 + value: 18.758841337291486 + - type: nauc_ndcg_at_3_max + value: 12.647106844067164 + - type: nauc_ndcg_at_3_std + value: 6.233821611512539 + - type: nauc_ndcg_at_5_diff1 + value: 17.58950136961931 + - type: nauc_ndcg_at_5_max + value: 13.932670021270935 + - type: nauc_ndcg_at_5_std + value: 6.858307184055891 + - type: nauc_precision_at_1000_diff1 + value: 5.363101838479181 + - type: nauc_precision_at_1000_max + value: 29.23404219454856 + - type: nauc_precision_at_1000_std + value: 38.77321422404564 + - type: nauc_precision_at_100_diff1 + value: 10.391704346995839 + - type: nauc_precision_at_100_max + value: 27.03066371996945 + - type: nauc_precision_at_100_std + value: 32.09682685301781 + - type: nauc_precision_at_10_diff1 + value: 16.048191019042747 + - type: nauc_precision_at_10_max + value: 21.207538845729886 + - type: nauc_precision_at_10_std + value: 17.019629271943458 + - type: nauc_precision_at_1_diff1 + value: 21.40810772236307 + - type: nauc_precision_at_1_max + value: 11.946611379314245 + - type: nauc_precision_at_1_std + value: 4.892375669408125 + - type: nauc_precision_at_20_diff1 + value: 13.977058772500564 + - type: nauc_precision_at_20_max + value: 19.839732387000677 + - type: nauc_precision_at_20_std + value: 19.57373741788698 + - type: nauc_precision_at_3_diff1 + value: 17.421929652197804 + - type: nauc_precision_at_3_max + value: 13.424075887007348 + - type: nauc_precision_at_3_std + value: 9.341912396168489 + - type: nauc_precision_at_5_diff1 + value: 15.501119413583092 + - type: nauc_precision_at_5_max + value: 15.459130320595197 + - type: nauc_precision_at_5_std + value: 10.159860224374993 + - type: nauc_recall_at_1000_diff1 + value: 8.82421580754989 + - type: nauc_recall_at_1000_max + value: 37.506204045082434 + - type: nauc_recall_at_1000_std + value: 43.482870689484166 + - type: nauc_recall_at_100_diff1 + value: 11.324486503492425 + - type: nauc_recall_at_100_max + value: 28.076656459901706 + - type: nauc_recall_at_100_std + value: 29.50463229683909 + - type: nauc_recall_at_10_diff1 + value: 16.00096630993718 + - type: nauc_recall_at_10_max + value: 20.75175370679478 + - type: nauc_recall_at_10_std + value: 14.211472195848762 + - type: nauc_recall_at_1_diff1 + value: 21.9930605817395 + - type: nauc_recall_at_1_max + value: 11.133712801824862 + - type: nauc_recall_at_1_std + value: 2.131487062602308 + - type: nauc_recall_at_20_diff1 + value: 14.608826437651182 + - type: nauc_recall_at_20_max + value: 19.689200636397832 + - type: nauc_recall_at_20_std + value: 16.665580013229857 + - type: nauc_recall_at_3_diff1 + value: 16.98767313511421 + - type: nauc_recall_at_3_max + value: 12.842939106207622 + - type: nauc_recall_at_3_std + value: 7.3018200578662 + - type: nauc_recall_at_5_diff1 + value: 15.399790005324546 + - type: nauc_recall_at_5_max + value: 15.12565840699527 + - type: nauc_recall_at_5_std + value: 8.093512618178716 + - type: ndcg_at_1 + value: 6.6049999999999995 + - type: ndcg_at_10 + value: 13.048000000000002 + - type: ndcg_at_100 + value: 17.723 + - type: ndcg_at_1000 + value: 20.497 + - type: ndcg_at_20 + value: 14.724 + - type: ndcg_at_3 + value: 9.402000000000001 + - type: ndcg_at_5 + value: 11.125 + - type: precision_at_1 + value: 6.6049999999999995 + - type: precision_at_10 + value: 2.5090000000000003 + - type: precision_at_100 + value: 0.523 + - type: precision_at_1000 + value: 0.079 + - type: precision_at_20 + value: 1.641 + - type: precision_at_3 + value: 4.519 + - type: precision_at_5 + value: 3.662 + - type: recall_at_1 + value: 5.733 + - type: recall_at_10 + value: 21.407999999999998 + - type: recall_at_100 + value: 43.197 + - type: recall_at_1000 + value: 64.786 + - type: recall_at_20 + value: 27.752 + - type: recall_at_3 + value: 11.584999999999999 + - type: recall_at_5 + value: 15.662 + task: + type: Retrieval + - dataset: + config: default + name: MTEB QuoraRetrieval (default) + revision: e4e08e0b7dbe3c8700f0daef558ff32256715259 + split: test + type: mteb/quora + metrics: + - type: main_score + value: 75.889 + - type: map_at_1 + value: 59.521 + - type: map_at_10 + value: 71.28 + - type: map_at_100 + value: 72.1 + - type: map_at_1000 + value: 72.139 + - type: map_at_20 + value: 71.788 + - type: map_at_3 + value: 68.462 + - type: map_at_5 + value: 70.143 + - type: mrr_at_1 + value: 68.46 + - type: mrr_at_10 + value: 75.60400396825362 + - type: mrr_at_100 + value: 75.94020350652137 + - type: mrr_at_1000 + value: 75.95072803605511 + - type: mrr_at_20 + value: 75.82968447032792 + - type: mrr_at_3 + value: 74.10999999999966 + - type: mrr_at_5 + value: 75.06249999999946 + - type: nauc_map_at_1000_diff1 + value: 70.19386706664726 + - type: nauc_map_at_1000_max + value: 43.44897820040013 + - type: nauc_map_at_1000_std + value: -14.237163216930396 + - type: nauc_map_at_100_diff1 + value: 70.19866967992321 + - type: nauc_map_at_100_max + value: 43.44171960985875 + - type: nauc_map_at_100_std + value: -14.252373890404598 + - type: nauc_map_at_10_diff1 + value: 70.26642450807104 + - type: nauc_map_at_10_max + value: 42.96787858551095 + - type: nauc_map_at_10_std + value: -15.223674656269695 + - type: nauc_map_at_1_diff1 + value: 73.50598798215337 + - type: nauc_map_at_1_max + value: 35.6307293512228 + - type: nauc_map_at_1_std + value: -16.269089125483806 + - type: nauc_map_at_20_diff1 + value: 70.2053600879007 + - type: nauc_map_at_20_max + value: 43.27498690180252 + - type: nauc_map_at_20_std + value: -14.64603003196119 + - type: nauc_map_at_3_diff1 + value: 70.65138430278584 + - type: nauc_map_at_3_max + value: 40.89023340246951 + - type: nauc_map_at_3_std + value: -16.549534070111047 + - type: nauc_map_at_5_diff1 + value: 70.349373734752 + - type: nauc_map_at_5_max + value: 42.24082396168828 + - type: nauc_map_at_5_std + value: -15.811500680254115 + - type: nauc_mrr_at_1000_diff1 + value: 71.29203504603552 + - type: nauc_mrr_at_1000_max + value: 46.42690559992034 + - type: nauc_mrr_at_1000_std + value: -11.828704627498078 + - type: nauc_mrr_at_100_diff1 + value: 71.2900226340849 + - type: nauc_mrr_at_100_max + value: 46.435253826921844 + - type: nauc_mrr_at_100_std + value: -11.814735763142298 + - type: nauc_mrr_at_10_diff1 + value: 71.21844201081325 + - type: nauc_mrr_at_10_max + value: 46.421627982430884 + - type: nauc_mrr_at_10_std + value: -12.030751233110733 + - type: nauc_mrr_at_1_diff1 + value: 73.05505007326735 + - type: nauc_mrr_at_1_max + value: 44.42599826502435 + - type: nauc_mrr_at_1_std + value: -13.238066635762205 + - type: nauc_mrr_at_20_diff1 + value: 71.25293148556567 + - type: nauc_mrr_at_20_max + value: 46.41621835298584 + - type: nauc_mrr_at_20_std + value: -11.876661055767109 + - type: nauc_mrr_at_3_diff1 + value: 71.37331205958733 + - type: nauc_mrr_at_3_max + value: 45.98413603164503 + - type: nauc_mrr_at_3_std + value: -12.574574509695696 + - type: nauc_mrr_at_5_diff1 + value: 71.24542402884832 + - type: nauc_mrr_at_5_max + value: 46.38452436382147 + - type: nauc_mrr_at_5_std + value: -12.226266259767742 + - type: nauc_ndcg_at_1000_diff1 + value: 69.77356705929742 + - type: nauc_ndcg_at_1000_max + value: 45.769266206351325 + - type: nauc_ndcg_at_1000_std + value: -11.368815656652979 + - type: nauc_ndcg_at_100_diff1 + value: 69.75368757543434 + - type: nauc_ndcg_at_100_max + value: 45.92974089305349 + - type: nauc_ndcg_at_100_std + value: -10.962000777691044 + - type: nauc_ndcg_at_10_diff1 + value: 69.43855250047541 + - type: nauc_ndcg_at_10_max + value: 44.87412123405316 + - type: nauc_ndcg_at_10_std + value: -13.581485619007081 + - type: nauc_ndcg_at_1_diff1 + value: 72.98830746442073 + - type: nauc_ndcg_at_1_max + value: 44.52285306872999 + - type: nauc_ndcg_at_1_std + value: -13.140713028099874 + - type: nauc_ndcg_at_20_diff1 + value: 69.47101741777516 + - type: nauc_ndcg_at_20_max + value: 45.363995866856655 + - type: nauc_ndcg_at_20_std + value: -12.31761653051956 + - type: nauc_ndcg_at_3_diff1 + value: 69.63995301777895 + - type: nauc_ndcg_at_3_max + value: 43.385900881945936 + - type: nauc_ndcg_at_3_std + value: -14.646876750798473 + - type: nauc_ndcg_at_5_diff1 + value: 69.43625651308415 + - type: nauc_ndcg_at_5_max + value: 44.28219675438962 + - type: nauc_ndcg_at_5_std + value: -14.245771527733861 + - type: nauc_precision_at_1000_diff1 + value: -33.91911759045305 + - type: nauc_precision_at_1000_max + value: -1.6255596813223787 + - type: nauc_precision_at_1000_std + value: 22.114523481372608 + - type: nauc_precision_at_100_diff1 + value: -29.22957093123771 + - type: nauc_precision_at_100_max + value: 3.3573621623107597 + - type: nauc_precision_at_100_std + value: 22.64243055590217 + - type: nauc_precision_at_10_diff1 + value: -10.768120433843423 + - type: nauc_precision_at_10_max + value: 16.372356342933074 + - type: nauc_precision_at_10_std + value: 11.106047707692019 + - type: nauc_precision_at_1_diff1 + value: 72.98830746442073 + - type: nauc_precision_at_1_max + value: 44.52285306872999 + - type: nauc_precision_at_1_std + value: -13.140713028099874 + - type: nauc_precision_at_20_diff1 + value: -19.60328656850837 + - type: nauc_precision_at_20_max + value: 11.263698603553324 + - type: nauc_precision_at_20_std + value: 17.12862112761346 + - type: nauc_precision_at_3_diff1 + value: 16.396535763529837 + - type: nauc_precision_at_3_max + value: 27.52510331770848 + - type: nauc_precision_at_3_std + value: -0.1431320144068112 + - type: nauc_precision_at_5_diff1 + value: 1.9559899327174441 + - type: nauc_precision_at_5_max + value: 23.115797894537156 + - type: nauc_precision_at_5_std + value: 5.587976439020372 + - type: nauc_recall_at_1000_diff1 + value: 52.165335899073426 + - type: nauc_recall_at_1000_max + value: 67.68332681999831 + - type: nauc_recall_at_1000_std + value: 48.44483006588368 + - type: nauc_recall_at_100_diff1 + value: 59.452660123413004 + - type: nauc_recall_at_100_max + value: 57.21697880811637 + - type: nauc_recall_at_100_std + value: 23.194172294428036 + - type: nauc_recall_at_10_diff1 + value: 62.06320665783282 + - type: nauc_recall_at_10_max + value: 44.97632618214612 + - type: nauc_recall_at_10_std + value: -11.783932871643586 + - type: nauc_recall_at_1_diff1 + value: 73.50598798215337 + - type: nauc_recall_at_1_max + value: 35.6307293512228 + - type: nauc_recall_at_1_std + value: -16.269089125483806 + - type: nauc_recall_at_20_diff1 + value: 60.096792659466615 + - type: nauc_recall_at_20_max + value: 47.36791373671932 + - type: nauc_recall_at_20_std + value: -3.617358805963005 + - type: nauc_recall_at_3_diff1 + value: 66.4817738935685 + - type: nauc_recall_at_3_max + value: 40.07846085049393 + - type: nauc_recall_at_3_std + value: -17.019266805305357 + - type: nauc_recall_at_5_diff1 + value: 64.29294958329528 + - type: nauc_recall_at_5_max + value: 42.80590704093666 + - type: nauc_recall_at_5_std + value: -14.96426642947079 + - type: ndcg_at_1 + value: 68.49 + - type: ndcg_at_10 + value: 75.889 + - type: ndcg_at_100 + value: 78.452 + - type: ndcg_at_1000 + value: 79.022 + - type: ndcg_at_20 + value: 77.095 + - type: ndcg_at_3 + value: 72.336 + - type: ndcg_at_5 + value: 74.122 + - type: precision_at_1 + value: 68.49 + - type: precision_at_10 + value: 11.478 + - type: precision_at_100 + value: 1.4040000000000001 + - type: precision_at_1000 + value: 0.152 + - type: precision_at_20 + value: 6.232 + - type: precision_at_3 + value: 31.287 + - type: precision_at_5 + value: 20.712 + - type: recall_at_1 + value: 59.521 + - type: recall_at_10 + value: 84.768 + - type: recall_at_100 + value: 94.906 + - type: recall_at_1000 + value: 98.437 + - type: recall_at_20 + value: 88.919 + - type: recall_at_3 + value: 74.507 + - type: recall_at_5 + value: 79.487 + task: + type: Retrieval + - dataset: + config: default + name: MTEB RedditClustering (default) + revision: 24640382cdbf8abc73003fb0fa6d111a705499eb + split: test + type: mteb/reddit-clustering + metrics: + - type: main_score + value: 29.134297978095674 + - type: v_measure + value: 29.134297978095674 + - type: v_measure_std + value: 3.9934034124121185 + task: + type: Clustering + - dataset: + config: default + name: MTEB RedditClusteringP2P (default) + revision: 385e3cb46b4cfa89021f56c4380204149d0efe33 + split: test + type: mteb/reddit-clustering-p2p + metrics: + - type: main_score + value: 39.215421675518 + - type: v_measure + value: 39.215421675518 + - type: v_measure_std + value: 10.607286582764162 + task: + type: Clustering + - dataset: + config: default + name: MTEB SCIDOCS (default) + revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88 + split: test + type: mteb/scidocs + metrics: + - type: main_score + value: 8.163 + - type: map_at_1 + value: 2.06 + - type: map_at_10 + value: 4.349 + - type: map_at_100 + value: 5.29 + - type: map_at_1000 + value: 5.502 + - type: map_at_20 + value: 4.757 + - type: map_at_3 + value: 3.3320000000000003 + - type: map_at_5 + value: 3.789 + - type: mrr_at_1 + value: 10.2 + - type: mrr_at_10 + value: 15.777301587301585 + - type: mrr_at_100 + value: 16.87040130235826 + - type: mrr_at_1000 + value: 16.99318092347203 + - type: mrr_at_20 + value: 16.33887346227904 + - type: mrr_at_3 + value: 13.816666666666663 + - type: mrr_at_5 + value: 14.876666666666672 + - type: nauc_map_at_1000_diff1 + value: 16.820399860912577 + - type: nauc_map_at_1000_max + value: 11.036291872368661 + - type: nauc_map_at_1000_std + value: 28.197293404910596 + - type: nauc_map_at_100_diff1 + value: 16.974714726054092 + - type: nauc_map_at_100_max + value: 10.865585523763155 + - type: nauc_map_at_100_std + value: 27.743692421138462 + - type: nauc_map_at_10_diff1 + value: 17.114604628918144 + - type: nauc_map_at_10_max + value: 9.375565363093843 + - type: nauc_map_at_10_std + value: 24.780455052145346 + - type: nauc_map_at_1_diff1 + value: 20.658116587695112 + - type: nauc_map_at_1_max + value: 13.155907394830615 + - type: nauc_map_at_1_std + value: 22.426847527670628 + - type: nauc_map_at_20_diff1 + value: 16.898082467709294 + - type: nauc_map_at_20_max + value: 9.988559872619556 + - type: nauc_map_at_20_std + value: 26.307229888534273 + - type: nauc_map_at_3_diff1 + value: 16.76206057854526 + - type: nauc_map_at_3_max + value: 10.440545699729146 + - type: nauc_map_at_3_std + value: 22.958859601482796 + - type: nauc_map_at_5_diff1 + value: 16.530876411848187 + - type: nauc_map_at_5_max + value: 9.311867261847425 + - type: nauc_map_at_5_std + value: 24.387141896077 + - type: nauc_mrr_at_1000_diff1 + value: 16.73600946942673 + - type: nauc_mrr_at_1000_max + value: 12.676243402896537 + - type: nauc_mrr_at_1000_std + value: 22.6143458461655 + - type: nauc_mrr_at_100_diff1 + value: 16.71570698771131 + - type: nauc_mrr_at_100_max + value: 12.669159209039286 + - type: nauc_mrr_at_100_std + value: 22.628552541531114 + - type: nauc_mrr_at_10_diff1 + value: 16.724205099190893 + - type: nauc_mrr_at_10_max + value: 12.284407084892688 + - type: nauc_mrr_at_10_std + value: 22.029318681643513 + - type: nauc_mrr_at_1_diff1 + value: 20.65518464968726 + - type: nauc_mrr_at_1_max + value: 13.380415958308845 + - type: nauc_mrr_at_1_std + value: 23.027057944994354 + - type: nauc_mrr_at_20_diff1 + value: 16.743321127999945 + - type: nauc_mrr_at_20_max + value: 12.497444652373034 + - type: nauc_mrr_at_20_std + value: 22.523374071799278 + - type: nauc_mrr_at_3_diff1 + value: 17.429706373434005 + - type: nauc_mrr_at_3_max + value: 11.567635068970057 + - type: nauc_mrr_at_3_std + value: 21.792853845733678 + - type: nauc_mrr_at_5_diff1 + value: 17.067362660541164 + - type: nauc_mrr_at_5_max + value: 11.838591994727086 + - type: nauc_mrr_at_5_std + value: 21.89903870786229 + - type: nauc_ndcg_at_1000_diff1 + value: 13.976383558618032 + - type: nauc_ndcg_at_1000_max + value: 15.004200828026185 + - type: nauc_ndcg_at_1000_std + value: 32.29145506934514 + - type: nauc_ndcg_at_100_diff1 + value: 15.151338074531035 + - type: nauc_ndcg_at_100_max + value: 13.467821215240413 + - type: nauc_ndcg_at_100_std + value: 29.802198591954856 + - type: nauc_ndcg_at_10_diff1 + value: 15.922085162704697 + - type: nauc_ndcg_at_10_max + value: 10.447859010783938 + - type: nauc_ndcg_at_10_std + value: 23.90947332043245 + - type: nauc_ndcg_at_1_diff1 + value: 20.65518464968726 + - type: nauc_ndcg_at_1_max + value: 13.380415958308845 + - type: nauc_ndcg_at_1_std + value: 23.027057944994354 + - type: nauc_ndcg_at_20_diff1 + value: 15.55874475345454 + - type: nauc_ndcg_at_20_max + value: 11.437273797119275 + - type: nauc_ndcg_at_20_std + value: 26.552304709089604 + - type: nauc_ndcg_at_3_diff1 + value: 16.552004746479756 + - type: nauc_ndcg_at_3_max + value: 10.804357733167999 + - type: nauc_ndcg_at_3_std + value: 22.49302563529031 + - type: nauc_ndcg_at_5_diff1 + value: 15.941541214122312 + - type: nauc_ndcg_at_5_max + value: 9.999332076127734 + - type: nauc_ndcg_at_5_std + value: 23.38227609142699 + - type: nauc_precision_at_1000_diff1 + value: 7.058340373874923 + - type: nauc_precision_at_1000_max + value: 17.258245524006963 + - type: nauc_precision_at_1000_std + value: 36.45652142925109 + - type: nauc_precision_at_100_diff1 + value: 11.432632810176315 + - type: nauc_precision_at_100_max + value: 14.706743744286266 + - type: nauc_precision_at_100_std + value: 33.509148405438516 + - type: nauc_precision_at_10_diff1 + value: 13.899621711899094 + - type: nauc_precision_at_10_max + value: 9.288924597831762 + - type: nauc_precision_at_10_std + value: 23.887371820866647 + - type: nauc_precision_at_1_diff1 + value: 20.65518464968726 + - type: nauc_precision_at_1_max + value: 13.380415958308845 + - type: nauc_precision_at_1_std + value: 23.027057944994354 + - type: nauc_precision_at_20_diff1 + value: 13.143970754077383 + - type: nauc_precision_at_20_max + value: 11.347036397273031 + - type: nauc_precision_at_20_std + value: 28.728122625727842 + - type: nauc_precision_at_3_diff1 + value: 15.17229479556843 + - type: nauc_precision_at_3_max + value: 9.942412630398362 + - type: nauc_precision_at_3_std + value: 22.56878491369083 + - type: nauc_precision_at_5_diff1 + value: 14.110634156514475 + - type: nauc_precision_at_5_max + value: 8.484616766016119 + - type: nauc_precision_at_5_std + value: 24.207087120488005 + - type: nauc_recall_at_1000_diff1 + value: 7.1431971152880624 + - type: nauc_recall_at_1000_max + value: 17.940414755613098 + - type: nauc_recall_at_1000_std + value: 36.52036034800652 + - type: nauc_recall_at_100_diff1 + value: 11.564422539945994 + - type: nauc_recall_at_100_max + value: 14.768866414576125 + - type: nauc_recall_at_100_std + value: 33.57594828721061 + - type: nauc_recall_at_10_diff1 + value: 13.826760252646993 + - type: nauc_recall_at_10_max + value: 9.167497311121016 + - type: nauc_recall_at_10_std + value: 23.658037032839953 + - type: nauc_recall_at_1_diff1 + value: 20.658116587695112 + - type: nauc_recall_at_1_max + value: 13.155907394830615 + - type: nauc_recall_at_1_std + value: 22.426847527670628 + - type: nauc_recall_at_20_diff1 + value: 13.053984329111262 + - type: nauc_recall_at_20_max + value: 11.261855881829511 + - type: nauc_recall_at_20_std + value: 28.59453401494739 + - type: nauc_recall_at_3_diff1 + value: 15.176171372019025 + - type: nauc_recall_at_3_max + value: 9.927743665598976 + - type: nauc_recall_at_3_std + value: 22.263215956695852 + - type: nauc_recall_at_5_diff1 + value: 14.117753921764933 + - type: nauc_recall_at_5_max + value: 8.407325978866647 + - type: nauc_recall_at_5_std + value: 23.973302948861026 + - type: ndcg_at_1 + value: 10.2 + - type: ndcg_at_10 + value: 8.163 + - type: ndcg_at_100 + value: 13.093 + - type: ndcg_at_1000 + value: 18.129 + - type: ndcg_at_20 + value: 9.589 + - type: ndcg_at_3 + value: 7.85 + - type: ndcg_at_5 + value: 6.701 + - type: precision_at_1 + value: 10.2 + - type: precision_at_10 + value: 4.19 + - type: precision_at_100 + value: 1.1520000000000001 + - type: precision_at_1000 + value: 0.23800000000000002 + - type: precision_at_20 + value: 2.9250000000000003 + - type: precision_at_3 + value: 7.167 + - type: precision_at_5 + value: 5.7 + - type: recall_at_1 + value: 2.06 + - type: recall_at_10 + value: 8.488 + - type: recall_at_100 + value: 23.383000000000003 + - type: recall_at_1000 + value: 48.312 + - type: recall_at_20 + value: 11.843 + - type: recall_at_3 + value: 4.35 + - type: recall_at_5 + value: 5.765 + task: + type: Retrieval + - dataset: + config: default + name: MTEB SICK-R (default) + revision: 20a6d6f312dd54037fe07a32d58e5e168867909d + split: test + type: mteb/sickr-sts + metrics: + - type: cosine_pearson + value: 74.06014749723313 + - type: cosine_spearman + value: 62.85583348143348 + - type: euclidean_pearson + value: 67.90829663189977 + - type: euclidean_spearman + value: 62.855829168697966 + - type: main_score + value: 62.85583348143348 + - type: manhattan_pearson + value: 68.16895231463954 + - type: manhattan_spearman + value: 62.942131884343155 + - type: pearson + value: 74.06014749723313 + - type: spearman + value: 62.85583348143348 + task: + type: STS + - dataset: + config: default + name: MTEB STS12 (default) + revision: a0d554a64d88156834ff5ae9920b964011b16384 + split: test + type: mteb/sts12-sts + metrics: + - type: cosine_pearson + value: 71.71587397454503 + - type: cosine_spearman + value: 62.07913034464432 + - type: euclidean_pearson + value: 68.38396254056258 + - type: euclidean_spearman + value: 62.07912362817025 + - type: main_score + value: 62.07913034464432 + - type: manhattan_pearson + value: 68.80285040680448 + - type: manhattan_spearman + value: 62.48427594825191 + - type: pearson + value: 71.71587397454503 + - type: spearman + value: 62.07913034464432 + task: + type: STS + - dataset: + config: default + name: MTEB STS13 (default) + revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca + split: test + type: mteb/sts13-sts + metrics: + - type: cosine_pearson + value: 74.00146491973214 + - type: cosine_spearman + value: 75.73113726697468 + - type: euclidean_pearson + value: 75.17723244913799 + - type: euclidean_spearman + value: 75.73113726697468 + - type: main_score + value: 75.73113726697468 + - type: manhattan_pearson + value: 75.04817185215164 + - type: manhattan_spearman + value: 75.56882534866682 + - type: pearson + value: 74.00146491973214 + - type: spearman + value: 75.73113726697468 + task: + type: STS + - dataset: + config: default + name: MTEB STS14 (default) + revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 + split: test + type: mteb/sts14-sts + metrics: + - type: cosine_pearson + value: 73.18799052289306 + - type: cosine_spearman + value: 69.27997439795548 + - type: euclidean_pearson + value: 72.01699560068344 + - type: euclidean_spearman + value: 69.27997439795548 + - type: main_score + value: 69.27997439795548 + - type: manhattan_pearson + value: 72.08946320578453 + - type: manhattan_spearman + value: 69.3115876094692 + - type: pearson + value: 73.18799052289306 + - type: spearman + value: 69.27997439795548 + task: + type: STS + - dataset: + config: default + name: MTEB STS15 (default) + revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 + split: test + type: mteb/sts15-sts + metrics: + - type: cosine_pearson + value: 75.05240168700195 + - type: cosine_spearman + value: 76.32976845993336 + - type: euclidean_pearson + value: 76.23332436291838 + - type: euclidean_spearman + value: 76.32976845993336 + - type: main_score + value: 76.32976845993336 + - type: manhattan_pearson + value: 76.0196103445834 + - type: manhattan_spearman + value: 76.11367025445854 + - type: pearson + value: 75.05240168700195 + - type: spearman + value: 76.32976845993336 + task: + type: STS + - dataset: + config: default + name: MTEB STS16 (default) + revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 + split: test + type: mteb/sts16-sts + metrics: + - type: cosine_pearson + value: 71.35240308275529 + - type: cosine_spearman + value: 73.46659216141927 + - type: euclidean_pearson + value: 73.01431094479076 + - type: euclidean_spearman + value: 73.46659216141927 + - type: main_score + value: 73.46659216141927 + - type: manhattan_pearson + value: 73.05673665594625 + - type: manhattan_spearman + value: 73.46097106391906 + - type: pearson + value: 71.35240308275529 + - type: spearman + value: 73.46659216141927 + task: + type: STS + - dataset: + config: en-en + name: MTEB STS17 (en-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 80.32585308865436 + - type: cosine_spearman + value: 82.08042618874391 + - type: euclidean_pearson + value: 81.50315333642457 + - type: euclidean_spearman + value: 82.08042618874391 + - type: main_score + value: 82.08042618874391 + - type: manhattan_pearson + value: 81.50403771658148 + - type: manhattan_spearman + value: 81.86754737875918 + - type: pearson + value: 80.32585308865436 + - type: spearman + value: 82.08042618874391 + task: + type: STS + - dataset: + config: fr-en + name: MTEB STS17 (fr-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 26.492433454505004 + - type: cosine_spearman + value: 25.26192630209604 + - type: euclidean_pearson + value: 26.508248428681146 + - type: euclidean_spearman + value: 25.26192630209604 + - type: main_score + value: 25.26192630209604 + - type: manhattan_pearson + value: 24.577958833043617 + - type: manhattan_spearman + value: 23.107268314361416 + - type: pearson + value: 26.492433454505004 + - type: spearman + value: 25.26192630209604 + task: + type: STS + - dataset: + config: en-ar + name: MTEB STS17 (en-ar) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 4.849502004066215 + - type: cosine_spearman + value: 2.4221360201347566 + - type: euclidean_pearson + value: 4.946775087695405 + - type: euclidean_spearman + value: 2.4221360201347566 + - type: main_score + value: 2.4221360201347566 + - type: manhattan_pearson + value: 2.0088474400844993 + - type: manhattan_spearman + value: 2.501283757283151 + - type: pearson + value: 4.849502004066215 + - type: spearman + value: 2.4221360201347566 + task: + type: STS + - dataset: + config: it-en + name: MTEB STS17 (it-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 17.67430795163699 + - type: cosine_spearman + value: 14.138028269188412 + - type: euclidean_pearson + value: 17.719124540919474 + - type: euclidean_spearman + value: 14.138028269188412 + - type: main_score + value: 14.138028269188412 + - type: manhattan_pearson + value: 15.979360498335673 + - type: manhattan_spearman + value: 13.033003503816229 + - type: pearson + value: 17.67430795163699 + - type: spearman + value: 14.138028269188412 + task: + type: STS + - dataset: + config: en-tr + name: MTEB STS17 (en-tr) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 11.032677618214326 + - type: cosine_spearman + value: 8.819837594034183 + - type: euclidean_pearson + value: 11.212471090722028 + - type: euclidean_spearman + value: 8.819837594034183 + - type: main_score + value: 8.819837594034183 + - type: manhattan_pearson + value: 7.552514418132754 + - type: manhattan_spearman + value: 5.788554277295719 + - type: pearson + value: 11.032677618214326 + - type: spearman + value: 8.819837594034183 + task: + type: STS + - dataset: + config: nl-en + name: MTEB STS17 (nl-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 21.77242194085935 + - type: cosine_spearman + value: 19.564246863458028 + - type: euclidean_pearson + value: 21.997979208209685 + - type: euclidean_spearman + value: 19.564246863458028 + - type: main_score + value: 19.564246863458028 + - type: manhattan_pearson + value: 21.80940866422115 + - type: manhattan_spearman + value: 19.5079327687257 + - type: pearson + value: 21.77242194085935 + - type: spearman + value: 19.564246863458028 + task: + type: STS + - dataset: + config: en-de + name: MTEB STS17 (en-de) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 24.34392722914247 + - type: cosine_spearman + value: 22.516912091222096 + - type: euclidean_pearson + value: 24.860123500651827 + - type: euclidean_spearman + value: 22.516912091222096 + - type: main_score + value: 22.516912091222096 + - type: manhattan_pearson + value: 28.816841514354884 + - type: manhattan_spearman + value: 27.740284625490002 + - type: pearson + value: 24.34392722914247 + - type: spearman + value: 22.516912091222096 + task: + type: STS + - dataset: + config: es-en + name: MTEB STS17 (es-en) + revision: faeb762787bd10488a50c8b5be4a3b82e411949c + split: test + type: mteb/sts17-crosslingual-sts + metrics: + - type: cosine_pearson + value: 11.486165309912764 + - type: cosine_spearman + value: 10.139614392782256 + - type: euclidean_pearson + value: 11.75125169236987 + - type: euclidean_spearman + value: 10.139614392782256 + - type: main_score + value: 10.139614392782256 + - type: manhattan_pearson + value: 12.405185554485518 + - type: manhattan_spearman + value: 10.053617041455068 + - type: pearson + value: 11.486165309912764 + - type: spearman + value: 10.139614392782256 + task: + type: STS + - dataset: + config: en + name: MTEB STS22 (en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 56.1393113780294 + - type: cosine_spearman + value: 62.64707232707212 + - type: euclidean_pearson + value: 60.68784382476653 + - type: euclidean_spearman + value: 62.64707232707212 + - type: main_score + value: 62.64707232707212 + - type: manhattan_pearson + value: 60.849049192788776 + - type: manhattan_spearman + value: 62.50847116906587 + - type: pearson + value: 56.1393113780294 + - type: spearman + value: 62.64707232707212 + task: + type: STS + - dataset: + config: zh-en + name: MTEB STS22 (zh-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 9.554093605202135 + - type: cosine_spearman + value: 11.788855140937605 + - type: euclidean_pearson + value: 8.3590729795075 + - type: euclidean_spearman + value: 11.674682450877647 + - type: main_score + value: 11.788855140937605 + - type: manhattan_pearson + value: 15.34897773929992 + - type: manhattan_spearman + value: 18.76001274530547 + - type: pearson + value: 9.554093605202135 + - type: spearman + value: 11.788855140937605 + task: + type: STS + - dataset: + config: de-en + name: MTEB STS22 (de-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 23.699968394697848 + - type: cosine_spearman + value: 25.635685273215014 + - type: euclidean_pearson + value: 23.30847681564559 + - type: euclidean_spearman + value: 25.62920498344224 + - type: main_score + value: 25.635685273215014 + - type: manhattan_pearson + value: 22.877299914701172 + - type: manhattan_spearman + value: 29.58678616125247 + - type: pearson + value: 23.699968394697848 + - type: spearman + value: 25.635685273215014 + task: + type: STS + - dataset: + config: es-en + name: MTEB STS22 (es-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 9.344287684864119 + - type: cosine_spearman + value: 10.042423712385212 + - type: euclidean_pearson + value: 9.033832457728343 + - type: euclidean_spearman + value: 10.033434689662734 + - type: main_score + value: 10.042423712385212 + - type: manhattan_pearson + value: 6.744871382125734 + - type: manhattan_spearman + value: 6.302888265155408 + - type: pearson + value: 9.344287684864119 + - type: spearman + value: 10.042423712385212 + task: + type: STS + - dataset: + config: pl-en + name: MTEB STS22 (pl-en) + revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 + split: test + type: mteb/sts22-crosslingual-sts + metrics: + - type: cosine_pearson + value: 8.001041267374578 + - type: cosine_spearman + value: 15.127881072012025 + - type: euclidean_pearson + value: 6.362399463770095 + - type: euclidean_spearman + value: 15.127881072012025 + - type: main_score + value: 15.127881072012025 + - type: manhattan_pearson + value: 14.398693320171096 + - type: manhattan_spearman + value: 23.42112716934093 + - type: pearson + value: 8.001041267374578 + - type: spearman + value: 15.127881072012025 + task: + type: STS + - dataset: + config: default + name: MTEB STSBenchmark (default) + revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 + split: test + type: mteb/stsbenchmark-sts + metrics: + - type: cosine_pearson + value: 73.57677681061787 + - type: cosine_spearman + value: 72.80800903257308 + - type: euclidean_pearson + value: 74.4255195885822 + - type: euclidean_spearman + value: 72.80800903257308 + - type: main_score + value: 72.80800903257308 + - type: manhattan_pearson + value: 74.6222587602608 + - type: manhattan_spearman + value: 72.9559654281266 + - type: pearson + value: 73.57677681061787 + - type: spearman + value: 72.80800903257308 + task: + type: STS + - dataset: + config: default + name: MTEB SciDocsRR (default) + revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab + split: test + type: mteb/scidocs-reranking + metrics: + - type: main_score + value: 66.13944998572143 + - type: map + value: 66.13944998572143 + - type: mrr + value: 86.84982682531702 + - type: nAUC_map_diff1 + value: 9.655489527090475 + - type: nAUC_map_max + value: 55.329447225464676 + - type: nAUC_map_std + value: 63.92644562946389 + - type: nAUC_mrr_diff1 + value: 38.69379106400398 + - type: nAUC_mrr_max + value: 69.96013858740152 + - type: nAUC_mrr_std + value: 62.072046098925156 + task: + type: Reranking + - dataset: + config: default + name: MTEB SciFact (default) + revision: 0228b52cf27578f30900b9e5271d331663a030d7 + split: test + type: mteb/scifact + metrics: + - type: main_score + value: 36.55 + - type: map_at_1 + value: 24.166999999999998 + - type: map_at_10 + value: 32.255 + - type: map_at_100 + value: 33.359 + - type: map_at_1000 + value: 33.44 + - type: map_at_20 + value: 32.885999999999996 + - type: map_at_3 + value: 29.866 + - type: map_at_5 + value: 31.557000000000002 + - type: mrr_at_1 + value: 26.0 + - type: mrr_at_10 + value: 33.87486772486773 + - type: mrr_at_100 + value: 34.829580502303436 + - type: mrr_at_1000 + value: 34.90127768602221 + - type: mrr_at_20 + value: 34.44486881510876 + - type: mrr_at_3 + value: 31.5 + - type: mrr_at_5 + value: 33.21666666666667 + - type: nauc_map_at_1000_diff1 + value: 40.27120542337293 + - type: nauc_map_at_1000_max + value: 34.881725395249745 + - type: nauc_map_at_1000_std + value: 14.804128183822776 + - type: nauc_map_at_100_diff1 + value: 40.21945625314103 + - type: nauc_map_at_100_max + value: 34.86632729849179 + - type: nauc_map_at_100_std + value: 14.822072821802628 + - type: nauc_map_at_10_diff1 + value: 40.22503326192557 + - type: nauc_map_at_10_max + value: 34.681031950162314 + - type: nauc_map_at_10_std + value: 13.969909037414721 + - type: nauc_map_at_1_diff1 + value: 46.18145166307877 + - type: nauc_map_at_1_max + value: 32.850938864900506 + - type: nauc_map_at_1_std + value: 9.284621032067 + - type: nauc_map_at_20_diff1 + value: 40.24803710661317 + - type: nauc_map_at_20_max + value: 34.74111106117432 + - type: nauc_map_at_20_std + value: 14.52709156133609 + - type: nauc_map_at_3_diff1 + value: 42.21002367018134 + - type: nauc_map_at_3_max + value: 33.57697582968568 + - type: nauc_map_at_3_std + value: 12.715120187246024 + - type: nauc_map_at_5_diff1 + value: 40.361071576043734 + - type: nauc_map_at_5_max + value: 34.960048160721065 + - type: nauc_map_at_5_std + value: 13.587382041347976 + - type: nauc_mrr_at_1000_diff1 + value: 40.172026071768876 + - type: nauc_mrr_at_1000_max + value: 36.393047482968285 + - type: nauc_mrr_at_1000_std + value: 18.514061967011543 + - type: nauc_mrr_at_100_diff1 + value: 40.10935371489407 + - type: nauc_mrr_at_100_max + value: 36.3941603476526 + - type: nauc_mrr_at_100_std + value: 18.549364999359284 + - type: nauc_mrr_at_10_diff1 + value: 40.0143297539273 + - type: nauc_mrr_at_10_max + value: 36.285300624108174 + - type: nauc_mrr_at_10_std + value: 18.06040618434985 + - type: nauc_mrr_at_1_diff1 + value: 45.94065893825947 + - type: nauc_mrr_at_1_max + value: 34.89250575775973 + - type: nauc_mrr_at_1_std + value: 15.032812003122197 + - type: nauc_mrr_at_20_diff1 + value: 40.04355197303724 + - type: nauc_mrr_at_20_max + value: 36.35951332162351 + - type: nauc_mrr_at_20_std + value: 18.42335404196948 + - type: nauc_mrr_at_3_diff1 + value: 42.30608858184136 + - type: nauc_mrr_at_3_max + value: 35.61225415425098 + - type: nauc_mrr_at_3_std + value: 17.223247492661102 + - type: nauc_mrr_at_5_diff1 + value: 40.485972003959404 + - type: nauc_mrr_at_5_max + value: 36.67688438369464 + - type: nauc_mrr_at_5_std + value: 18.021306525172033 + - type: nauc_ndcg_at_1000_diff1 + value: 38.1448539512143 + - type: nauc_ndcg_at_1000_max + value: 36.41251174091125 + - type: nauc_ndcg_at_1000_std + value: 18.863348298831134 + - type: nauc_ndcg_at_100_diff1 + value: 36.828616223036995 + - type: nauc_ndcg_at_100_max + value: 36.326875743799825 + - type: nauc_ndcg_at_100_std + value: 19.753271271756166 + - type: nauc_ndcg_at_10_diff1 + value: 36.999696509492765 + - type: nauc_ndcg_at_10_max + value: 35.510700157502384 + - type: nauc_ndcg_at_10_std + value: 16.51692902623065 + - type: nauc_ndcg_at_1_diff1 + value: 45.94065893825947 + - type: nauc_ndcg_at_1_max + value: 34.89250575775973 + - type: nauc_ndcg_at_1_std + value: 15.032812003122197 + - type: nauc_ndcg_at_20_diff1 + value: 36.98821200336492 + - type: nauc_ndcg_at_20_max + value: 35.51321933251509 + - type: nauc_ndcg_at_20_std + value: 17.880281629389334 + - type: nauc_ndcg_at_3_diff1 + value: 40.854488421982474 + - type: nauc_ndcg_at_3_max + value: 34.13463997398456 + - type: nauc_ndcg_at_3_std + value: 14.833355685530647 + - type: nauc_ndcg_at_5_diff1 + value: 37.65087893614269 + - type: nauc_ndcg_at_5_max + value: 36.16610548649603 + - type: nauc_ndcg_at_5_std + value: 15.85595394609524 + - type: nauc_precision_at_1000_diff1 + value: -1.2213861207148984 + - type: nauc_precision_at_1000_max + value: 33.792059295415115 + - type: nauc_precision_at_1000_std + value: 51.25358732741295 + - type: nauc_precision_at_100_diff1 + value: 8.942954849181 + - type: nauc_precision_at_100_max + value: 37.44655423710336 + - type: nauc_precision_at_100_std + value: 47.04438370323114 + - type: nauc_precision_at_10_diff1 + value: 22.922592142376622 + - type: nauc_precision_at_10_max + value: 37.716906832835676 + - type: nauc_precision_at_10_std + value: 28.841183482232413 + - type: nauc_precision_at_1_diff1 + value: 45.94065893825947 + - type: nauc_precision_at_1_max + value: 34.89250575775973 + - type: nauc_precision_at_1_std + value: 15.032812003122197 + - type: nauc_precision_at_20_diff1 + value: 20.36859440465984 + - type: nauc_precision_at_20_max + value: 38.34513768246952 + - type: nauc_precision_at_20_std + value: 35.229890026962615 + - type: nauc_precision_at_3_diff1 + value: 34.05345356587694 + - type: nauc_precision_at_3_max + value: 36.40222017855832 + - type: nauc_precision_at_3_std + value: 22.09598528562007 + - type: nauc_precision_at_5_diff1 + value: 25.37065548133688 + - type: nauc_precision_at_5_max + value: 41.03941131544809 + - type: nauc_precision_at_5_std + value: 25.017881172035043 + - type: nauc_recall_at_1000_diff1 + value: 33.09021721896312 + - type: nauc_recall_at_1000_max + value: 47.697443757377194 + - type: nauc_recall_at_1000_std + value: 44.12337261948821 + - type: nauc_recall_at_100_diff1 + value: 22.06710557626714 + - type: nauc_recall_at_100_max + value: 40.18599082452499 + - type: nauc_recall_at_100_std + value: 38.184660672167595 + - type: nauc_recall_at_10_diff1 + value: 27.10204324985591 + - type: nauc_recall_at_10_max + value: 35.30960104403818 + - type: nauc_recall_at_10_std + value: 19.306977699362033 + - type: nauc_recall_at_1_diff1 + value: 46.18145166307877 + - type: nauc_recall_at_1_max + value: 32.850938864900506 + - type: nauc_recall_at_1_std + value: 9.284621032067 + - type: nauc_recall_at_20_diff1 + value: 26.382409406526 + - type: nauc_recall_at_20_max + value: 34.469251680833004 + - type: nauc_recall_at_20_std + value: 23.400056135882995 + - type: nauc_recall_at_3_diff1 + value: 37.2267183565415 + - type: nauc_recall_at_3_max + value: 32.77661901779293 + - type: nauc_recall_at_3_std + value: 15.031686858933885 + - type: nauc_recall_at_5_diff1 + value: 29.257298344591987 + - type: nauc_recall_at_5_max + value: 36.84185394701337 + - type: nauc_recall_at_5_std + value: 17.84731120093588 + - type: ndcg_at_1 + value: 26.0 + - type: ndcg_at_10 + value: 36.55 + - type: ndcg_at_100 + value: 42.049 + - type: ndcg_at_1000 + value: 44.4 + - type: ndcg_at_20 + value: 38.692 + - type: ndcg_at_3 + value: 32.019999999999996 + - type: ndcg_at_5 + value: 34.922 + - type: precision_at_1 + value: 26.0 + - type: precision_at_10 + value: 5.3 + - type: precision_at_100 + value: 0.843 + - type: precision_at_1000 + value: 0.105 + - type: precision_at_20 + value: 3.1329999999999996 + - type: precision_at_3 + value: 13.111 + - type: precision_at_5 + value: 9.4 + - type: recall_at_1 + value: 24.166999999999998 + - type: recall_at_10 + value: 48.167 + - type: recall_at_100 + value: 74.1 + - type: recall_at_1000 + value: 92.93299999999999 + - type: recall_at_20 + value: 56.289 + - type: recall_at_3 + value: 36.306 + - type: recall_at_5 + value: 43.389 + task: + type: Retrieval + - dataset: + config: default + name: MTEB SprintDuplicateQuestions (default) + revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 + split: test + type: mteb/sprintduplicatequestions-pairclassification + metrics: + - type: cosine_accuracy + value: 99.68415841584158 + - type: cosine_accuracy_threshold + value: 81.75174090984304 + - type: cosine_ap + value: 88.17695295243927 + - type: cosine_f1 + value: 83.46293416277865 + - type: cosine_f1_threshold + value: 81.67502920031147 + - type: cosine_precision + value: 86.65231431646933 + - type: cosine_recall + value: 80.5 + - type: dot_accuracy + value: 99.68415841584158 + - type: dot_accuracy_threshold + value: 81.7517430435309 + - type: dot_ap + value: 88.17695295243927 + - type: dot_f1 + value: 83.46293416277865 + - type: dot_f1_threshold + value: 81.67503038678444 + - type: dot_precision + value: 86.65231431646933 + - type: dot_recall + value: 80.5 + - type: euclidean_accuracy + value: 99.68415841584158 + - type: euclidean_accuracy_threshold + value: 60.412346616246396 + - type: euclidean_ap + value: 88.17695295243927 + - type: euclidean_f1 + value: 83.46293416277865 + - type: euclidean_f1_threshold + value: 60.539194123743535 + - type: euclidean_precision + value: 86.65231431646933 + - type: euclidean_recall + value: 80.5 + - type: main_score + value: 88.17695295243927 + - type: manhattan_accuracy + value: 99.67920792079208 + - type: manhattan_accuracy_threshold + value: 366.6920070296328 + - type: manhattan_ap + value: 88.09159390940687 + - type: manhattan_f1 + value: 82.97982410760476 + - type: manhattan_f1_threshold + value: 384.15647450019605 + - type: manhattan_precision + value: 85.95927116827438 + - type: manhattan_recall + value: 80.2 + - type: max_accuracy + value: 99.68415841584158 + - type: max_ap + value: 88.17695295243927 + - type: max_f1 + value: 83.46293416277865 + - type: max_precision + value: 86.65231431646933 + - type: max_recall + value: 80.5 + - type: similarity_accuracy + value: 99.68415841584158 + - type: similarity_accuracy_threshold + value: 81.75174090984304 + - type: similarity_ap + value: 88.17695295243927 + - type: similarity_f1 + value: 83.46293416277865 + - type: similarity_f1_threshold + value: 81.67502920031147 + - type: similarity_precision + value: 86.65231431646933 + - type: similarity_recall + value: 80.5 + task: + type: PairClassification + - dataset: + config: default + name: MTEB StackExchangeClustering (default) + revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 + split: test + type: mteb/stackexchange-clustering + metrics: + - type: main_score + value: 37.82716668016299 + - type: v_measure + value: 37.82716668016299 + - type: v_measure_std + value: 3.9071651545475055 + task: + type: Clustering + - dataset: + config: default + name: MTEB StackExchangeClusteringP2P (default) + revision: 815ca46b2622cec33ccafc3735d572c266efdb44 + split: test + type: mteb/stackexchange-clustering-p2p + metrics: + - type: main_score + value: 31.549916824347523 + - type: v_measure + value: 31.549916824347523 + - type: v_measure_std + value: 1.649284454526032 + task: + type: Clustering + - dataset: + config: default + name: MTEB StackOverflowDupQuestions (default) + revision: e185fbe320c72810689fc5848eb6114e1ef5ec69 + split: test + type: mteb/stackoverflowdupquestions-reranking + metrics: + - type: main_score + value: 40.201162273119 + - type: map + value: 40.201162273119 + - type: mrr + value: 40.4580866437484 + - type: nAUC_map_diff1 + value: 33.47313252548623 + - type: nAUC_map_max + value: 15.023702266852649 + - type: nAUC_map_std + value: 4.389545656603201 + - type: nAUC_mrr_diff1 + value: 32.789106382547125 + - type: nAUC_mrr_max + value: 16.03297395659567 + - type: nAUC_mrr_std + value: 4.5441260195062885 + task: + type: Reranking + - dataset: + config: default + name: MTEB SummEval (default) + revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c + split: test + type: mteb/summeval + metrics: + - type: cosine_pearson + value: 30.982384340344282 + - type: cosine_spearman + value: 31.512077655680574 + - type: dot_pearson + value: 30.98238732795588 + - type: dot_spearman + value: 31.518281673786575 + - type: main_score + value: 31.512077655680574 + - type: pearson + value: 30.982384340344282 + - type: spearman + value: 31.512077655680574 + task: + type: Summarization + - dataset: + config: default + name: MTEB TRECCOVID (default) + revision: bb9466bac8153a0349341eb1b22e06409e78ef4e + split: test + type: mteb/trec-covid + metrics: + - type: main_score + value: 40.475 + - type: map_at_1 + value: 0.13799999999999998 + - type: map_at_10 + value: 0.8420000000000001 + - type: map_at_100 + value: 3.5770000000000004 + - type: map_at_1000 + value: 8.158 + - type: map_at_20 + value: 1.2930000000000001 + - type: map_at_3 + value: 0.314 + - type: map_at_5 + value: 0.469 + - type: mrr_at_1 + value: 52.0 + - type: mrr_at_10 + value: 61.566666666666656 + - type: mrr_at_100 + value: 62.05786242033046 + - type: mrr_at_1000 + value: 62.114468654223785 + - type: mrr_at_20 + value: 61.80277777777777 + - type: mrr_at_3 + value: 59.333333333333336 + - type: mrr_at_5 + value: 61.23333333333333 + - type: nauc_map_at_1000_diff1 + value: -9.841499993688778 + - type: nauc_map_at_1000_max + value: 35.583722623282604 + - type: nauc_map_at_1000_std + value: 49.8512713272345 + - type: nauc_map_at_100_diff1 + value: -9.535301607183984 + - type: nauc_map_at_100_max + value: 24.79493554952309 + - type: nauc_map_at_100_std + value: 30.200092896546565 + - type: nauc_map_at_10_diff1 + value: 8.916577726839806 + - type: nauc_map_at_10_max + value: 29.525597945854425 + - type: nauc_map_at_10_std + value: 19.568384032684456 + - type: nauc_map_at_1_diff1 + value: 26.460797161906434 + - type: nauc_map_at_1_max + value: 2.508967307552916 + - type: nauc_map_at_1_std + value: 1.1681201948586744 + - type: nauc_map_at_20_diff1 + value: 0.9454484237357934 + - type: nauc_map_at_20_max + value: 23.745460933070735 + - type: nauc_map_at_20_std + value: 21.99699798901709 + - type: nauc_map_at_3_diff1 + value: 11.118755215925088 + - type: nauc_map_at_3_max + value: 19.978322544243934 + - type: nauc_map_at_3_std + value: 14.509469952755843 + - type: nauc_map_at_5_diff1 + value: 11.598932237369507 + - type: nauc_map_at_5_max + value: 26.153738721440213 + - type: nauc_map_at_5_std + value: 16.524906956574807 + - type: nauc_mrr_at_1000_diff1 + value: 26.07845182857252 + - type: nauc_mrr_at_1000_max + value: 24.176348172120992 + - type: nauc_mrr_at_1000_std + value: 23.24482013997546 + - type: nauc_mrr_at_100_diff1 + value: 25.984954958619326 + - type: nauc_mrr_at_100_max + value: 24.155747509357905 + - type: nauc_mrr_at_100_std + value: 23.29395979045902 + - type: nauc_mrr_at_10_diff1 + value: 26.129837220948154 + - type: nauc_mrr_at_10_max + value: 24.392744717202998 + - type: nauc_mrr_at_10_std + value: 23.867386603817575 + - type: nauc_mrr_at_1_diff1 + value: 33.149246422547435 + - type: nauc_mrr_at_1_max + value: 27.63480058762716 + - type: nauc_mrr_at_1_std + value: 15.626530279122907 + - type: nauc_mrr_at_20_diff1 + value: 25.83893496030277 + - type: nauc_mrr_at_20_max + value: 24.044445525240917 + - type: nauc_mrr_at_20_std + value: 23.60228538747721 + - type: nauc_mrr_at_3_diff1 + value: 27.270721998857006 + - type: nauc_mrr_at_3_max + value: 24.992229563752698 + - type: nauc_mrr_at_3_std + value: 25.97080321244876 + - type: nauc_mrr_at_5_diff1 + value: 25.725324288076845 + - type: nauc_mrr_at_5_max + value: 24.00232670237143 + - type: nauc_mrr_at_5_std + value: 24.79721341200203 + - type: nauc_ndcg_at_1000_diff1 + value: -4.183105310677864 + - type: nauc_ndcg_at_1000_max + value: 27.180660506931897 + - type: nauc_ndcg_at_1000_std + value: 44.28297994229539 + - type: nauc_ndcg_at_100_diff1 + value: -2.4155068242179603 + - type: nauc_ndcg_at_100_max + value: 31.816956304456312 + - type: nauc_ndcg_at_100_std + value: 46.14561095598332 + - type: nauc_ndcg_at_10_diff1 + value: 2.615898205427684 + - type: nauc_ndcg_at_10_max + value: 35.980044445372805 + - type: nauc_ndcg_at_10_std + value: 33.743474797390846 + - type: nauc_ndcg_at_1_diff1 + value: 32.67671069224133 + - type: nauc_ndcg_at_1_max + value: 26.737904028671945 + - type: nauc_ndcg_at_1_std + value: 17.61598194730206 + - type: nauc_ndcg_at_20_diff1 + value: 0.26491277268362085 + - type: nauc_ndcg_at_20_max + value: 31.281622776028495 + - type: nauc_ndcg_at_20_std + value: 38.724219848828994 + - type: nauc_ndcg_at_3_diff1 + value: 8.936493331684819 + - type: nauc_ndcg_at_3_max + value: 36.02086733359256 + - type: nauc_ndcg_at_3_std + value: 30.338780238893374 + - type: nauc_ndcg_at_5_diff1 + value: 8.045634929889802 + - type: nauc_ndcg_at_5_max + value: 37.327357284632185 + - type: nauc_ndcg_at_5_std + value: 30.957117218415558 + - type: nauc_precision_at_1000_diff1 + value: -4.599639084511409 + - type: nauc_precision_at_1000_max + value: 32.194166261212416 + - type: nauc_precision_at_1000_std + value: 51.93173817114919 + - type: nauc_precision_at_100_diff1 + value: -4.957123210096294 + - type: nauc_precision_at_100_max + value: 30.2670457390949 + - type: nauc_precision_at_100_std + value: 48.68923380370673 + - type: nauc_precision_at_10_diff1 + value: -1.5291339587721136 + - type: nauc_precision_at_10_max + value: 38.17154891824624 + - type: nauc_precision_at_10_std + value: 35.5699151190248 + - type: nauc_precision_at_1_diff1 + value: 33.149246422547435 + - type: nauc_precision_at_1_max + value: 27.63480058762716 + - type: nauc_precision_at_1_std + value: 15.626530279122907 + - type: nauc_precision_at_20_diff1 + value: -4.749129059384092 + - type: nauc_precision_at_20_max + value: 29.86806703620865 + - type: nauc_precision_at_20_std + value: 42.15085625809208 + - type: nauc_precision_at_3_diff1 + value: 0.9399784588269945 + - type: nauc_precision_at_3_max + value: 34.8888671301283 + - type: nauc_precision_at_3_std + value: 32.87183002056205 + - type: nauc_precision_at_5_diff1 + value: 3.555638235202897 + - type: nauc_precision_at_5_max + value: 37.42287539143648 + - type: nauc_precision_at_5_std + value: 31.48792360400585 + - type: nauc_recall_at_1000_diff1 + value: -6.381069093195219 + - type: nauc_recall_at_1000_max + value: 21.417984620002997 + - type: nauc_recall_at_1000_std + value: 39.07231544912849 + - type: nauc_recall_at_100_diff1 + value: -7.26598409041968 + - type: nauc_recall_at_100_max + value: 16.570118087602005 + - type: nauc_recall_at_100_std + value: 26.77146523700651 + - type: nauc_recall_at_10_diff1 + value: 8.714684916963465 + - type: nauc_recall_at_10_max + value: 25.75907022743869 + - type: nauc_recall_at_10_std + value: 19.009377316046564 + - type: nauc_recall_at_1_diff1 + value: 26.460797161906434 + - type: nauc_recall_at_1_max + value: 2.508967307552916 + - type: nauc_recall_at_1_std + value: 1.1681201948586744 + - type: nauc_recall_at_20_diff1 + value: 0.387922884336754 + - type: nauc_recall_at_20_max + value: 16.63869160692463 + - type: nauc_recall_at_20_std + value: 21.613192035516175 + - type: nauc_recall_at_3_diff1 + value: 9.080317655844187 + - type: nauc_recall_at_3_max + value: 17.8202789569972 + - type: nauc_recall_at_3_std + value: 16.410682320537916 + - type: nauc_recall_at_5_diff1 + value: 12.29082747211401 + - type: nauc_recall_at_5_max + value: 21.653811740840258 + - type: nauc_recall_at_5_std + value: 15.799511026732738 + - type: ndcg_at_1 + value: 47.0 + - type: ndcg_at_10 + value: 40.475 + - type: ndcg_at_100 + value: 27.061 + - type: ndcg_at_1000 + value: 23.977999999999998 + - type: ndcg_at_20 + value: 35.676 + - type: ndcg_at_3 + value: 44.285000000000004 + - type: ndcg_at_5 + value: 41.916 + - type: precision_at_1 + value: 52.0 + - type: precision_at_10 + value: 43.2 + - type: precision_at_100 + value: 27.68 + - type: precision_at_1000 + value: 11.318 + - type: precision_at_20 + value: 36.9 + - type: precision_at_3 + value: 46.666999999999994 + - type: precision_at_5 + value: 44.4 + - type: recall_at_1 + value: 0.13799999999999998 + - type: recall_at_10 + value: 1.036 + - type: recall_at_100 + value: 6.024 + - type: recall_at_1000 + value: 22.545 + - type: recall_at_20 + value: 1.729 + - type: recall_at_3 + value: 0.337 + - type: recall_at_5 + value: 0.5329999999999999 + task: + type: Retrieval + - dataset: + config: default + name: MTEB Touche2020 (default) + revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f + split: test + type: mteb/touche2020 + metrics: + - type: main_score + value: 12.869 + - type: map_at_1 + value: 0.662 + - type: map_at_10 + value: 3.971 + - type: map_at_100 + value: 7.363 + - type: map_at_1000 + value: 8.472 + - type: map_at_20 + value: 5.13 + - type: map_at_3 + value: 1.644 + - type: map_at_5 + value: 2.838 + - type: mrr_at_1 + value: 8.16326530612245 + - type: mrr_at_10 + value: 24.049238743116288 + - type: mrr_at_100 + value: 25.12029835148993 + - type: mrr_at_1000 + value: 25.14408894379364 + - type: mrr_at_20 + value: 24.38624196545786 + - type: mrr_at_3 + value: 17.68707482993197 + - type: mrr_at_5 + value: 21.666666666666668 + - type: nauc_map_at_1000_diff1 + value: 12.337448807158625 + - type: nauc_map_at_1000_max + value: -17.318046612869768 + - type: nauc_map_at_1000_std + value: -5.226718750608608 + - type: nauc_map_at_100_diff1 + value: 13.956163090828705 + - type: nauc_map_at_100_max + value: -19.20287240190536 + - type: nauc_map_at_100_std + value: -10.092675511550773 + - type: nauc_map_at_10_diff1 + value: 9.18601548640666 + - type: nauc_map_at_10_max + value: -24.021035071127027 + - type: nauc_map_at_10_std + value: -14.275185624920955 + - type: nauc_map_at_1_diff1 + value: 4.7247342416680995 + - type: nauc_map_at_1_max + value: -34.642931094004034 + - type: nauc_map_at_1_std + value: -11.238414747973119 + - type: nauc_map_at_20_diff1 + value: 12.874639720051235 + - type: nauc_map_at_20_max + value: -24.9080157788591 + - type: nauc_map_at_20_std + value: -13.156578501449037 + - type: nauc_map_at_3_diff1 + value: 7.690290046454848 + - type: nauc_map_at_3_max + value: -23.28618026459301 + - type: nauc_map_at_3_std + value: -12.293774024221053 + - type: nauc_map_at_5_diff1 + value: 12.498873258031583 + - type: nauc_map_at_5_max + value: -21.344480733880943 + - type: nauc_map_at_5_std + value: -8.37592777063943 + - type: nauc_mrr_at_1000_diff1 + value: 7.632497711020806 + - type: nauc_mrr_at_1000_max + value: -18.25120924121576 + - type: nauc_mrr_at_1000_std + value: -6.414648627407907 + - type: nauc_mrr_at_100_diff1 + value: 7.5484282387727655 + - type: nauc_mrr_at_100_max + value: -18.121388473878216 + - type: nauc_mrr_at_100_std + value: -6.297822495739477 + - type: nauc_mrr_at_10_diff1 + value: 8.58584663550636 + - type: nauc_mrr_at_10_max + value: -19.845467444420596 + - type: nauc_mrr_at_10_std + value: -6.690298972830557 + - type: nauc_mrr_at_1_diff1 + value: -16.07306433650747 + - type: nauc_mrr_at_1_max + value: -27.82478676090755 + - type: nauc_mrr_at_1_std + value: -15.97885564257309 + - type: nauc_mrr_at_20_diff1 + value: 8.68636948230222 + - type: nauc_mrr_at_20_max + value: -18.602790322324793 + - type: nauc_mrr_at_20_std + value: -5.764922000928246 + - type: nauc_mrr_at_3_diff1 + value: 8.625611091011928 + - type: nauc_mrr_at_3_max + value: -17.79470893653387 + - type: nauc_mrr_at_3_std + value: -10.458985196270106 + - type: nauc_mrr_at_5_diff1 + value: 9.463099209006096 + - type: nauc_mrr_at_5_max + value: -12.070782626702988 + - type: nauc_mrr_at_5_std + value: -1.7108913341652359 + - type: nauc_ndcg_at_1000_diff1 + value: 17.13876584026755 + - type: nauc_ndcg_at_1000_max + value: -9.658877990859322 + - type: nauc_ndcg_at_1000_std + value: 15.08101976706594 + - type: nauc_ndcg_at_100_diff1 + value: 19.217666462897675 + - type: nauc_ndcg_at_100_max + value: -22.83702381325575 + - type: nauc_ndcg_at_100_std + value: -6.659892970133183 + - type: nauc_ndcg_at_10_diff1 + value: 15.51333602888754 + - type: nauc_ndcg_at_10_max + value: -19.445241461181332 + - type: nauc_ndcg_at_10_std + value: -11.499134493661858 + - type: nauc_ndcg_at_1_diff1 + value: -15.409833668093054 + - type: nauc_ndcg_at_1_max + value: -28.84037358169315 + - type: nauc_ndcg_at_1_std + value: -10.973056606811218 + - type: nauc_ndcg_at_20_diff1 + value: 20.623641958571344 + - type: nauc_ndcg_at_20_max + value: -24.809454049416633 + - type: nauc_ndcg_at_20_std + value: -11.96823517644928 + - type: nauc_ndcg_at_3_diff1 + value: 9.13352117357039 + - type: nauc_ndcg_at_3_max + value: -13.267328789703988 + - type: nauc_ndcg_at_3_std + value: -10.536298335822734 + - type: nauc_ndcg_at_5_diff1 + value: 15.095676598753386 + - type: nauc_ndcg_at_5_max + value: -10.052271783670294 + - type: nauc_ndcg_at_5_std + value: -1.930908878523125 + - type: nauc_precision_at_1000_diff1 + value: -14.11681455108461 + - type: nauc_precision_at_1000_max + value: 45.108803189710365 + - type: nauc_precision_at_1000_std + value: 53.31308692343628 + - type: nauc_precision_at_100_diff1 + value: 6.9028413215024775 + - type: nauc_precision_at_100_max + value: -4.9050592903619785 + - type: nauc_precision_at_100_std + value: 9.735851001561873 + - type: nauc_precision_at_10_diff1 + value: 11.991816062486441 + - type: nauc_precision_at_10_max + value: -19.368511495003602 + - type: nauc_precision_at_10_std + value: -10.990338266181084 + - type: nauc_precision_at_1_diff1 + value: -16.07306433650747 + - type: nauc_precision_at_1_max + value: -27.82478676090755 + - type: nauc_precision_at_1_std + value: -15.97885564257309 + - type: nauc_precision_at_20_diff1 + value: 24.32628917857637 + - type: nauc_precision_at_20_max + value: -19.57542096164371 + - type: nauc_precision_at_20_std + value: -7.99725890197478 + - type: nauc_precision_at_3_diff1 + value: 9.121279719801027 + - type: nauc_precision_at_3_max + value: -11.446415855423215 + - type: nauc_precision_at_3_std + value: -8.639598301111935 + - type: nauc_precision_at_5_diff1 + value: 14.610162733995358 + - type: nauc_precision_at_5_max + value: -6.951865156929523 + - type: nauc_precision_at_5_std + value: 2.6465275964628088 + - type: nauc_recall_at_1000_diff1 + value: 0.83599650194909 + - type: nauc_recall_at_1000_max + value: -3.7204370018748496 + - type: nauc_recall_at_1000_std + value: 36.937019972801764 + - type: nauc_recall_at_100_diff1 + value: 8.73150091660338 + - type: nauc_recall_at_100_max + value: -25.580421701806877 + - type: nauc_recall_at_100_std + value: -4.617850817669842 + - type: nauc_recall_at_10_diff1 + value: 9.577784673082448 + - type: nauc_recall_at_10_max + value: -29.225334501694523 + - type: nauc_recall_at_10_std + value: -16.557978171725594 + - type: nauc_recall_at_1_diff1 + value: 4.7247342416680995 + - type: nauc_recall_at_1_max + value: -34.642931094004034 + - type: nauc_recall_at_1_std + value: -11.238414747973119 + - type: nauc_recall_at_20_diff1 + value: 16.714648634309317 + - type: nauc_recall_at_20_max + value: -30.901938269186104 + - type: nauc_recall_at_20_std + value: -13.257212212629026 + - type: nauc_recall_at_3_diff1 + value: 18.114207560002715 + - type: nauc_recall_at_3_max + value: -20.425807180867242 + - type: nauc_recall_at_3_std + value: -11.309432358198672 + - type: nauc_recall_at_5_diff1 + value: 20.772553996672443 + - type: nauc_recall_at_5_max + value: -17.828048513485772 + - type: nauc_recall_at_5_std + value: -1.0797675761407135 + - type: ndcg_at_1 + value: 7.142999999999999 + - type: ndcg_at_10 + value: 12.869 + - type: ndcg_at_100 + value: 22.469 + - type: ndcg_at_1000 + value: 33.626 + - type: ndcg_at_20 + value: 13.858999999999998 + - type: ndcg_at_3 + value: 11.110000000000001 + - type: ndcg_at_5 + value: 13.208 + - type: precision_at_1 + value: 8.163 + - type: precision_at_10 + value: 14.082 + - type: precision_at_100 + value: 5.449 + - type: precision_at_1000 + value: 1.253 + - type: precision_at_20 + value: 10.816 + - type: precision_at_3 + value: 14.285999999999998 + - type: precision_at_5 + value: 17.551 + - type: recall_at_1 + value: 0.662 + - type: recall_at_10 + value: 9.3 + - type: recall_at_100 + value: 32.952 + - type: recall_at_1000 + value: 67.022 + - type: recall_at_20 + value: 14.485000000000001 + - type: recall_at_3 + value: 2.495 + - type: recall_at_5 + value: 5.4719999999999995 + task: + type: Retrieval + - dataset: + config: default + name: MTEB ToxicConversationsClassification (default) + revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de + split: test + type: mteb/toxic_conversations_50k + metrics: + - type: accuracy + value: 66.591796875 + - type: ap + value: 11.916810235472406 + - type: ap_weighted + value: 11.916810235472406 + - type: f1 + value: 50.82611668132084 + - type: f1_weighted + value: 74.03661967545759 + - type: main_score + value: 66.591796875 + task: + type: Classification + - dataset: + config: default + name: MTEB TweetSentimentExtractionClassification (default) + revision: d604517c81ca91fe16a244d1248fc021f9ecee7a + split: test + type: mteb/tweet_sentiment_extraction + metrics: + - type: accuracy + value: 52.524052065648 + - type: f1 + value: 52.69528334636704 + - type: f1_weighted + value: 51.956916785617736 + - type: main_score + value: 52.524052065648 + task: + type: Classification + - dataset: + config: default + name: MTEB TwentyNewsgroupsClustering (default) + revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 + split: test + type: mteb/twentynewsgroups-clustering + metrics: + - type: main_score + value: 27.072966157648477 + - type: v_measure + value: 27.072966157648477 + - type: v_measure_std + value: 1.563199572918265 + task: + type: Clustering + - dataset: + config: default + name: MTEB TwitterSemEval2015 (default) + revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 + split: test + type: mteb/twittersemeval2015-pairclassification + metrics: + - type: cosine_accuracy + value: 80.6818859152411 + - type: cosine_accuracy_threshold + value: 80.54484987856905 + - type: cosine_ap + value: 53.66110400517361 + - type: cosine_f1 + value: 52.50138045278852 + - type: cosine_f1_threshold + value: 68.20751217239422 + - type: cosine_precision + value: 45.14719848053181 + - type: cosine_recall + value: 62.71767810026385 + - type: dot_accuracy + value: 80.6818859152411 + - type: dot_accuracy_threshold + value: 80.54484833308396 + - type: dot_ap + value: 53.66111257744601 + - type: dot_f1 + value: 52.50138045278852 + - type: dot_f1_threshold + value: 68.20751122368169 + - type: dot_precision + value: 45.14719848053181 + - type: dot_recall + value: 62.71767810026385 + - type: euclidean_accuracy + value: 80.6818859152411 + - type: euclidean_accuracy_threshold + value: 62.37812014295836 + - type: euclidean_ap + value: 53.66110322376674 + - type: euclidean_f1 + value: 52.50138045278852 + - type: euclidean_f1_threshold + value: 79.74018733602558 + - type: euclidean_precision + value: 45.14719848053181 + - type: euclidean_recall + value: 62.71767810026385 + - type: main_score + value: 53.66111257744601 + - type: manhattan_accuracy + value: 80.61631996185254 + - type: manhattan_accuracy_threshold + value: 408.045218404186 + - type: manhattan_ap + value: 53.55565109862251 + - type: manhattan_f1 + value: 52.268926301705875 + - type: manhattan_f1_threshold + value: 502.7589220990194 + - type: manhattan_precision + value: 45.25970264529832 + - type: manhattan_recall + value: 61.84696569920845 + - type: max_accuracy + value: 80.6818859152411 + - type: max_ap + value: 53.66111257744601 + - type: max_f1 + value: 52.50138045278852 + - type: max_precision + value: 45.25970264529832 + - type: max_recall + value: 62.71767810026385 + - type: similarity_accuracy + value: 80.6818859152411 + - type: similarity_accuracy_threshold + value: 80.54484987856905 + - type: similarity_ap + value: 53.66110400517361 + - type: similarity_f1 + value: 52.50138045278852 + - type: similarity_f1_threshold + value: 68.20751217239422 + - type: similarity_precision + value: 45.14719848053181 + - type: similarity_recall + value: 62.71767810026385 + task: + type: PairClassification + - dataset: + config: default + name: MTEB TwitterURLCorpus (default) + revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf + split: test + type: mteb/twitterurlcorpus-pairclassification + metrics: + - type: cosine_accuracy + value: 85.89474909768309 + - type: cosine_accuracy_threshold + value: 66.7208384412081 + - type: cosine_ap + value: 79.33674860685535 + - type: cosine_f1 + value: 71.67923542148885 + - type: cosine_f1_threshold + value: 61.64839965467019 + - type: cosine_precision + value: 69.57530076822728 + - type: cosine_recall + value: 73.91438250692947 + - type: dot_accuracy + value: 85.89474909768309 + - type: dot_accuracy_threshold + value: 66.72083970945883 + - type: dot_ap + value: 79.33671093343192 + - type: dot_f1 + value: 71.67923542148885 + - type: dot_f1_threshold + value: 61.648404274339356 + - type: dot_precision + value: 69.57530076822728 + - type: dot_recall + value: 73.91438250692947 + - type: euclidean_accuracy + value: 85.89474909768309 + - type: euclidean_accuracy_threshold + value: 81.58328528607446 + - type: euclidean_ap + value: 79.33675212612457 + - type: euclidean_f1 + value: 71.67923542148885 + - type: euclidean_f1_threshold + value: 87.58036675764342 + - type: euclidean_precision + value: 69.57530076822728 + - type: euclidean_recall + value: 73.91438250692947 + - type: main_score + value: 79.33675212612457 + - type: manhattan_accuracy + value: 85.89863003065938 + - type: manhattan_accuracy_threshold + value: 513.242605511914 + - type: manhattan_ap + value: 79.06488846026886 + - type: manhattan_f1 + value: 71.42752882790833 + - type: manhattan_f1_threshold + value: 563.6103289492894 + - type: manhattan_precision + value: 67.88984461709212 + - type: manhattan_recall + value: 75.3541730828457 + - type: max_accuracy + value: 85.89863003065938 + - type: max_ap + value: 79.33675212612457 + - type: max_f1 + value: 71.67923542148885 + - type: max_precision + value: 69.57530076822728 + - type: max_recall + value: 75.3541730828457 + - type: similarity_accuracy + value: 85.89474909768309 + - type: similarity_accuracy_threshold + value: 66.7208384412081 + - type: similarity_ap + value: 79.33674860685535 + - type: similarity_f1 + value: 71.67923542148885 + - type: similarity_f1_threshold + value: 61.64839965467019 + - type: similarity_precision + value: 69.57530076822728 + - type: similarity_recall + value: 73.91438250692947 + task: + type: PairClassification +model_name: potion-base-2M +tags: +- embeddings +- static-embeddings +- mteb +--- + +# potion-base-2M Model Card + +
+ Model2Vec logo +
+ +This [Model2Vec](https://github.com/MinishLab/model2vec) model is pre-trained using [Tokenlearn](https://github.com/MinishLab/tokenlearn). It is a distilled version of the [baai/bge-base-en-v1.5](https://huggingface.co/baai/bge-base-en-v1.5) Sentence Transformer. It uses static embeddings, allowing text embeddings to be computed orders of magnitude faster on both GPU and CPU. It is designed for applications where computational resources are limited or where real-time performance is critical. + + + +## Installation + +Install model2vec using pip: +``` +pip install model2vec +``` + +## Usage +Load this model using the `from_pretrained` method: +```python +from model2vec import StaticModel + +# Load a pretrained Model2Vec model +model = StaticModel.from_pretrained("minishlab/potion-base-2M") + +# Compute text embeddings +embeddings = model.encode(["Example sentence"]) +``` + + +## How it works + +Model2vec creates a small, static model that outperforms other static embedding models by a large margin on all tasks on [MTEB](https://huggingface.co/spaces/mteb/leaderboard). This model is pre-trained using [Tokenlearn](https://github.com/MinishLab/tokenlearn). It's created using the following steps: +- Distillation: first, a model is distilled from a sentence transformer model using Model2Vec. +- Training data creation: the sentence transformer model is used to create training data by creating mean output embeddings on a large corpus. +- Training: the distilled model is trained on the training data using Tokenlearn. +- Post-training re-regularization: after training, the model is re-regularized by weighting the tokens based on their frequency, applying PCA, and finally applying [SIF weighting](https://openreview.net/pdf?id=SyK00v5xx). + +The results for this model can be found on the [Model2Vec results page](https://github.com/MinishLab/model2vec/blob/main/results/README.md). + +## Additional Resources + +- [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec) +- [Model2Vec Repo](https://github.com/MinishLab/model2vec) +- [Tokenlearn repo](https://github.com/MinishLab/tokenlearn) +- [Model2Vec Results](https://github.com/MinishLab/model2vec/blob/main/results/README.md) +- [Model2Vec Tutorials](https://github.com/MinishLab/model2vec/tree/main/tutorials) + +## Library Authors + +Model2Vec was developed by the [Minish Lab](https://github.com/MinishLab) team consisting of [Stephan Tulkens](https://github.com/stephantul) and [Thomas van Dongen](https://github.com/Pringled). + +## Citation + +Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work. +``` +@software{minishlab2024model2vec, + authors = {Stephan Tulkens, Thomas van Dongen}, + title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model}, + year = {2024}, + url = {https://github.com/MinishLab/model2vec}, +} +``` \ No newline at end of file