metadata

license: apache-2.0
library_name: transformers
tags:
  - 4-bit
  - AWQ
  - text-generation
  - autotrain_compatible
  - endpoints_compatible
  - finetuned
pipeline_tag: text-generation
inference: false
quantized_by: Suparious

mistralai/Mistral-7B-Instruct-v0.2 AWQ

Model creator: mistralai
Original model: Mistral-7B-Instruct-v0.2

Model Summary

The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.

Mistral-7B-v0.2 has the following changes compared to Mistral-7B-v0.1

32k context window (vs 8k context in v0.1)
Rope-theta = 1e6
No Sliding-Window Attention

For full details of this model please read our paper and release blog post.

Instruction format

In order to leverage instruction fine-tuning, your prompt should be surrounded by [INST] and [/INST] tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.

E.g.

text = "<s>[INST] What is your favourite condiment? [/INST]"
"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
"[INST] Do you have mayonnaise recipes? [/INST]"

This format is available as a chat template via the apply_chat_template() method.