| datasets: | |
| - OpenAssistant/oasst1 | |
| pipeline_tag: text-generation | |
| license: apache-2.0 | |
| --- | |
| # 🚀 Falcon-7b-chat-oasst1 | |
| Falcon-7b-chat-oasst1 is a chatbot-like model for dialogue generation. It was built by fine-tuning [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) on the [OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1) dataset. This repo only includes the LoRA adapters from fine-tuning with 🤗's [peft](https://github.com/huggingface/peft) package. | |
| ## Model Summary | |
| - **Model Type:** Causal decoder-only | |
| - **Language(s):** English | |
| - **Base Model:** [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) (License: [Apache 2.0](https://huggingface.co/tiiuae/falcon-7b#license)) | |
| - **Dataset:** [OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1) (License: [Apache 2.0](https://huggingface.co/datasets/OpenAssistant/oasst1/blob/main/LICENSE)) | |
| - **License(s):** Apache 2.0 inherited from "Base Model" and "Dataset" | |
| ## Model Details | |
| The model was fine-tuned in 8-bit precision using 🤗 `peft` adapters, `transformers`, and `bitsandbytes`. Training relied on a method called "Low Rank Adapters" ([LoRA](https://arxiv.org/pdf/2106.09685.pdf)), specifically the [QLoRA](https://arxiv.org/abs/2305.14314) variant. | |
| ### Model Date | |
| July 3, 2023 | |
| ## Quick Start | |
| To prompt the chat model, use the following format: | |
| ``` | |
| <human>: [Instruction] | |
| <bot>: | |
| ``` |