Description

Quantified mistralai/Mistral-7B-Instruct-v0.2 into Q8_0 and converted to LLaMa CPP (GGUF) format.

GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.

Prompt template: ChatML

<s>[INST] {prompt} [/INST]

GGUF

Model size

7.24B params

Architecture

llama

8-bit

Inference Examples

Unable to determine this model's library. Check the docs .

Base model

Quantized

(79)

this model