Edit model card

Euryale-1.4-L2-70B IQ2-GGUF

Description

IQ2-GGUF quants of Sao10K/Euryale-1.4-L2-70B

Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.

NOTE: Currently you will need experimental branches of Koboldcpp or Ooba for this to work.

More info about IQ2

Models

Models: IQ2-XS, IQ2-XXS

Regular GGUF Quants: Here

Prompt Format

Alpaca:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Input:
{input}

### Response:

Contact

Kooten on discord

Downloads last month
11
GGUF
Model size
69B params
Architecture
llama

2-bit

Inference API
Unable to determine this model's library. Check the docs .

Collection including Kooten/Euryale-1.4-L2-70B-IQ2-GGUF