PEFT
Safetensors
Japanese
English
h-iida commited on
Commit
6272daa
1 Parent(s): 58175a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -3
README.md CHANGED
@@ -25,18 +25,31 @@ For the MNTP Adapter, please refer to [this link](https://huggingface.co/uzabase
25
  - **Model type:** PEFT
26
  - **Language(s) (NLP):** Japanese
27
  - **License:** Apache2.0
28
- - **Finetuned from model:** [llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)
29
 
30
  ### Model Sources [optional]
31
 
32
  - **Repository:** https://github.com/McGill-NLP/llm2vec
33
  - **Paper:** https://arxiv.org/abs/2404.05961
34
 
35
- ## Usage
36
 
37
  - Please see [original LLM2Vec repo](https://huggingface.co/McGill-NLP/LLM2Vec-Llama-2-7b-chat-hf-mntp-unsup-simcse#usage)
38
 
39
- ## Training Details
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
41
  ### Training Data
42
 
 
25
  - **Model type:** PEFT
26
  - **Language(s) (NLP):** Japanese
27
  - **License:** Apache2.0
28
+ - **Finetuned from model:** [Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)
29
 
30
  ### Model Sources [optional]
31
 
32
  - **Repository:** https://github.com/McGill-NLP/llm2vec
33
  - **Paper:** https://arxiv.org/abs/2404.05961
34
 
35
+ # Usage
36
 
37
  - Please see [original LLM2Vec repo](https://huggingface.co/McGill-NLP/LLM2Vec-Llama-2-7b-chat-hf-mntp-unsup-simcse#usage)
38
 
39
+ # BenchMark
40
+ - Followings are summaries. Details are [here]()
41
+ ## MTEB(Japansese)
42
+
43
+ | | Classification | Clustering | PairClassification | Reranking | BitextMining | Retrieval | Sts | AVG |
44
+ | --- | ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---|
45
+ | |Llama2-Llm2vec-eng | 0.527 | 0.258 | 0.501 | 0.217 | 0.275 | 0.296 | 0.765 | 0.408 |
46
+ | Llama2-Llm2vec-jpn(This repo) | 0.570 | 0.365 | 0.510 | 0.349 | 0.470 | 0.417 | 0.795 | 0.498 |
47
+ | Swallow-Llm2vec-jpn | 0.621 | 0.391 | 0.510 | 0.475 | 0.475 | 0.491 | 0.832 | 0.523 |
48
+
49
+ ## MTEB(English)
50
+
51
+
52
+ # Training Details
53
 
54
  ### Training Data
55