
# ggml versions of OpenLLaMa 3B

- Version: 1T tokens final version
- Project: [OpenLLaMA: An Open Reproduction of LLaMA](https://github.com/openlm-research/open_llama)
- Model: [openlm-research/open_llama_3b](https://huggingface.co/openlm-research/open_llama_3b)
- [llama.cpp](https://github.com/ggerganov/llama.cpp): build 607(ffb06a3) or later

## Use with llama.cpp

Support is now merged to master branch.

## Newer quantizations

There are now more quantization types in llama.cpp, some lower than 4 bits.
Currently these are not supported, maybe because some weights have shapes that don't divide by 256.

## Perplexity on wiki.test.raw

| Q | chunk | 600BT | 1000BT |
| --: | --: | --: | --: |
| F16 | [616] | 8.4656 | 7.7861 |
| Q8_0 | [616] | 8.4667 | 7.7874 |
| Q5_1 | [616] | 8.5072 | 7.8424 |
| Q5_0 | [616] | 8.5156 | 7.8474 |
| Q4_1 | [616] | 8.6102 | 8.0483 |
| Q4_0 | [616] | 8.6674 | 8.0962 |

{"license": "apache-2.0"}
license:apache-2.0 region:us
TAGS
#license-apache-2.0 #region-us 

ggml versions of OpenLLaMa 3B
=============================


* Version: 1T tokens final version
* Project: OpenLLaMA: An Open Reproduction of LLaMA
* Model: openlm-research/open\_llama\_3b
* URL: build 607(ffb06a3) or later


Use with URL
------------


Support is now merged to master branch.


Newer quantizations
-------------------


There are now more quantization types in URL, some lower than 4 bits.
Currently these are not supported, maybe because some weights have shapes that don't divide by 256.


Perplexity on URL
-----------------




TAGS
#license-apache-2.0 #region-us 
