---
license: gpl-3.0
tags:
- text2text-generation
pipeline_tag: text2text-generation
language:
- zh
- en
---


# Model Card for ChatBELLE-int4

## Welcome
4-bit quantized model using [llama.cpp](https://github.com/ggerganov/llama.cpp).
If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE !


## Model description
ChatBELLE-int4 is based on 7B model and quantized to 4-bit.

The code of Chinese data generation and other detailed information can be found in our Github project repository: https://github.com/LianjiaTech/BELLE.


## Download
Should you accept our license and acknowledged the limitations, download the model by clicking [Download](https://huggingface.co/BelleGroup/BELLE-LLaMA-7B-2M-q4/resolve/main/belle-model.bin).


## Model Usage

You can use this model with ChatBELLE, a minimal, cross-platform LLM chat app powered by [BELLE](https://github.com/LianjiaTech/BELLE) 
using quantized on-device offline models and Flutter UI, running on macOS (done), Windows, Android, 
iOS(see [Known Issues](#known-issues)) and more.

### macOS
* Download [chatbelle.dmg](https://github.com/LianjiaTech/BELLE/releases/download/v0.95/chatbelle.dmg) from [Releases](https://github.com/LianjiaTech/BELLE/releases/tag/v0.95) page, double click to open it, then drag `Chat Belle.dmg` into `Applications` folder.
* Open the `Chat Belle` app in `Applications` folder by right click then Ctrl-click `Open`, then click `Open`.
* The app will prompt the intended model file path and fail to load the model. Close the app.
* Download quantized model from [BELLE-LLaMA-7B-2M-q4](https://huggingface.co/BelleGroup/BELLE-LLaMA-7B-2M-q4/blob/main/belle-model.bin).
* Move and rename the model to the path prompted by the app. Defaults to `~/Library/Containers/com.barius.chatbelle/Data/belle-model.bin` .
* Reopen the app again (double clicking is now OK).

### Windows
* Stay tuned

### Android
* Stay tuned

### iOS
* Stay tuned


## Limitations
There still exists a few issues in the model trained on current base model and data:

1. The model might generate factual errors when asked to follow instructions related to facts.

2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions.

3. Needs improvements on reasoning and coding.

Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed.


## Citation

Please cite us when using our code, data or model.

```
@misc{BELLE,
  author = {Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li},
  title = {BELLE: Be Everyone's Large Language model Engine},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/LianjiaTech/BELLE}},
}
```