Update README.md
Browse files
README.md
CHANGED
@@ -25,18 +25,31 @@ For the MNTP Adapter, please refer to [this link](https://huggingface.co/uzabase
|
|
25 |
- **Model type:** PEFT
|
26 |
- **Language(s) (NLP):** Japanese
|
27 |
- **License:** Apache2.0
|
28 |
-
- **Finetuned from model:** [
|
29 |
|
30 |
### Model Sources [optional]
|
31 |
|
32 |
- **Repository:** https://github.com/McGill-NLP/llm2vec
|
33 |
- **Paper:** https://arxiv.org/abs/2404.05961
|
34 |
|
35 |
-
|
36 |
|
37 |
- Please see [original LLM2Vec repo](https://huggingface.co/McGill-NLP/LLM2Vec-Llama-2-7b-chat-hf-mntp-unsup-simcse#usage)
|
38 |
|
39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
|
41 |
### Training Data
|
42 |
|
|
|
25 |
- **Model type:** PEFT
|
26 |
- **Language(s) (NLP):** Japanese
|
27 |
- **License:** Apache2.0
|
28 |
+
- **Finetuned from model:** [Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)
|
29 |
|
30 |
### Model Sources [optional]
|
31 |
|
32 |
- **Repository:** https://github.com/McGill-NLP/llm2vec
|
33 |
- **Paper:** https://arxiv.org/abs/2404.05961
|
34 |
|
35 |
+
# Usage
|
36 |
|
37 |
- Please see [original LLM2Vec repo](https://huggingface.co/McGill-NLP/LLM2Vec-Llama-2-7b-chat-hf-mntp-unsup-simcse#usage)
|
38 |
|
39 |
+
# BenchMark
|
40 |
+
- Followings are summaries. Details are [here]()
|
41 |
+
## MTEB(Japansese)
|
42 |
+
|
43 |
+
| | Classification | Clustering | PairClassification | Reranking | BitextMining | Retrieval | Sts | AVG |
|
44 |
+
| --- | ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---| ---|
|
45 |
+
| |Llama2-Llm2vec-eng | 0.527 | 0.258 | 0.501 | 0.217 | 0.275 | 0.296 | 0.765 | 0.408 |
|
46 |
+
| Llama2-Llm2vec-jpn(This repo) | 0.570 | 0.365 | 0.510 | 0.349 | 0.470 | 0.417 | 0.795 | 0.498 |
|
47 |
+
| Swallow-Llm2vec-jpn | 0.621 | 0.391 | 0.510 | 0.475 | 0.475 | 0.491 | 0.832 | 0.523 |
|
48 |
+
|
49 |
+
## MTEB(English)
|
50 |
+
|
51 |
+
|
52 |
+
# Training Details
|
53 |
|
54 |
### Training Data
|
55 |
|