
# LLAMA-GGML-v2

This is repo for LLaMA models quantised down to 4bit for the latest [llama.cpp](https://github.com/ggerganov/llama.cpp) GGML v2 format.

## THE FILES REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!

llama.cpp recently made a breaking change to its quantisation methods.

I have quantised the GGML files in this repo with the latest version. 

## How to run in `text-generation-webui`

GGML models can be loaded into text-generation-webui by installing the llama.cpp module, then placing the ggml model file in a model folder as usual.

Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).

Note: at this time text-generation-webui may not support the new May 12th llama.cpp quantisation methods.
{"tags": ["LLaMA", "GGML"]}
LLaMA GGML region:us
TAGS
#LLaMA #GGML #region-us 


# LLAMA-GGML-v2

This is repo for LLaMA models quantised down to 4bit for the latest URL GGML v2 format.

## THE FILES REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!

URL recently made a breaking change to its quantisation methods.

I have quantised the GGML files in this repo with the latest version. 

## How to run in 'text-generation-webui'

GGML models can be loaded into text-generation-webui by installing the URL module, then placing the ggml model file in a model folder as usual.

Further instructions here: text-generation-webui/docs/URL.

Note: at this time text-generation-webui may not support the new May 12th URL quantisation methods.
# LLAMA-GGML-v2

This is repo for LLaMA models quantised down to 4bit for the latest URL GGML v2 format. ## THE FILES REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!

URL recently made a breaking change to its quantisation methods.

I have quantised the GGML files in this repo with the latest version. ## How to run in 'text-generation-webui'

GGML models can be loaded into text-generation-webui by installing the URL module, then placing the ggml model file in a model folder as usual.

Further instructions here: text-generation-webui/docs/URL.

Note: at this time text-generation-webui may not support the new May 12th URL quantisation methods.
TAGS
#LLaMA #GGML #region-us 
 # LLAMA-GGML-v2

This is repo for LLaMA models quantised down to 4bit for the latest URL GGML v2 format. ## THE FILES REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!

URL recently made a breaking change to its quantisation methods.

I have quantised the GGML files in this repo with the latest version. ## How to run in 'text-generation-webui'

GGML models can be loaded into text-generation-webui by installing the URL module, then placing the ggml model file in a model folder as usual.

Further instructions here: text-generation-webui/docs/URL.

Note: at this time text-generation-webui may not support the new May 12th URL quantisation methods.