Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- fr
|
| 5 |
+
- it
|
| 6 |
+
- de
|
| 7 |
+
- es
|
| 8 |
+
- en
|
| 9 |
+
inference: false
|
| 10 |
+
---
|
| 11 |
+
# Model Card for Mixtral-8x7B
|
| 12 |
+
The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested.
|
| 13 |
+
|
| 14 |
+
For full details of this model please read our [release blog post](https://mistral.ai/news/mixtral-of-experts/).
|
| 15 |
+
|
| 16 |
+
## Warning
|
| 17 |
+
This repo contains weights that are compatible with [vLLM](https://github.com/vllm-project/vllm) serving of the model as well as Hugging Face [transformers](https://github.com/huggingface/transformers) library. It is based on the original Mixtral [torrent release](magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%http://2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=http%3A%2F%http://2Ftracker.openbittorrent.com%3A80%2Fannounce), but the file format and parameter names are different. Please note that model cannot (yet) be instantiated with HF.
|
| 18 |
+
|
| 19 |
+
## Instruction format
|
| 20 |
+
|
| 21 |
+
This format must be strictly respected, otherwise the model will generate sub-optimal outputs.
|
| 22 |
+
|
| 23 |
+
The template used to build a prompt for the Instruct model is defined as follows:
|
| 24 |
+
```
|
| 25 |
+
<s> [INST] Instruction [/INST] Model answer</s> [INST] Follow-up instruction [/INST]
|
| 26 |
+
```
|
| 27 |
+
Note that `<s>` and `</s>` are special tokens for beginning of string (BOS) and end of string (EOS) while [INST] and [/INST] are regular strings.
|
| 28 |
+
|
| 29 |
+
As reference, here is the pseudo-code used to tokenize instructions during fine-tuning:
|
| 30 |
+
```python
|
| 31 |
+
def tokenize(text):
|
| 32 |
+
return tok.encode(text, add_special_tokens=False)
|
| 33 |
+
|
| 34 |
+
[BOS_ID] +
|
| 35 |
+
tokenize("[INST]") + tokenize(USER_MESSAGE_1) + tokenize("[/INST]") +
|
| 36 |
+
tokenize(BOT_MESSAGE_1) + [EOS_ID] +
|
| 37 |
+
…
|
| 38 |
+
tokenize("[INST]") + tokenize(USER_MESSAGE_N) + tokenize("[/INST]") +
|
| 39 |
+
tokenize(BOT_MESSAGE_N) + [EOS_ID]
|
| 40 |
+
```
|
| 41 |
+
|
| 42 |
+
In the pseudo-code above, note that the `tokenize` method should not add a BOS or EOS token automatically, but should add a prefix space.
|
| 43 |
+
|
| 44 |
+
## Run the model
|