|
--- |
|
license: gpl-3.0 |
|
tags: |
|
- text2text-generation |
|
pipeline_tag: text2text-generation |
|
language: |
|
- zh |
|
- en |
|
--- |
|
|
|
|
|
# Model Card for ChatBELLE-int4 |
|
|
|
## Welcome |
|
4-bit quantized model using [llama.cpp](https://github.com/ggerganov/llama.cpp). |
|
If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE ! |
|
|
|
|
|
## Model description |
|
ChatBELLE-int4 is based on 7B model and quantized to 4-bit. |
|
|
|
The code of Chinese data generation and other detailed information can be found in our Github project repository: https://github.com/LianjiaTech/BELLE. |
|
|
|
|
|
## Download |
|
Should you accept our license and acknowledged the limitations, download the model by clicking [Download](https://huggingface.co/BelleGroup/BELLE-LLaMA-7B-2M-q4/resolve/main/belle-model.bin). |
|
|
|
|
|
## Model Usage |
|
|
|
You can use this model with ChatBELLE, a minimal, cross-platform LLM chat app powered by [BELLE](https://github.com/LianjiaTech/BELLE) |
|
using quantized on-device offline models and Flutter UI, running on macOS (done), Windows, Android, |
|
iOS(see [Known Issues](#known-issues)) and more. |
|
|
|
### macOS |
|
* Download [chatbelle.dmg](https://github.com/LianjiaTech/BELLE/releases/download/v0.95/chatbelle.dmg) from [Releases](https://github.com/LianjiaTech/BELLE/releases/tag/v0.95) page, double click to open it, then drag `Chat Belle.dmg` into `Applications` folder. |
|
* Open the `Chat Belle` app in `Applications` folder by right click then Ctrl-click `Open`, then click `Open`. |
|
* The app will prompt the intended model file path and fail to load the model. Close the app. |
|
* Download quantized model from [BELLE-LLaMA-7B-2M-q4](https://huggingface.co/BelleGroup/BELLE-LLaMA-7B-2M-q4/blob/main/belle-model.bin). |
|
* Move and rename the model to the path prompted by the app. Defaults to `~/Library/Containers/com.barius.chatbelle/Data/belle-model.bin` . |
|
* Reopen the app again (double clicking is now OK). |
|
|
|
### Windows |
|
* Stay tuned |
|
|
|
### Android |
|
* Stay tuned |
|
|
|
### iOS |
|
* Stay tuned |
|
|
|
|
|
## Limitations |
|
There still exists a few issues in the model trained on current base model and data: |
|
|
|
1. The model might generate factual errors when asked to follow instructions related to facts. |
|
|
|
2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions. |
|
|
|
3. Needs improvements on reasoning and coding. |
|
|
|
Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed. |
|
|
|
|
|
## Citation |
|
|
|
Please cite us when using our code, data or model. |
|
|
|
``` |
|
@misc{BELLE, |
|
author = {Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li}, |
|
title = {BELLE: Be Everyone's Large Language model Engine}, |
|
year = {2023}, |
|
publisher = {GitHub}, |
|
journal = {GitHub repository}, |
|
howpublished = {\url{https://github.com/LianjiaTech/BELLE}}, |
|
} |
|
``` |
