File size: 3,706 Bytes
4181830 769479b 16a690c 769479b c64566a 769479b 4594ecd 769479b 5b4eaba 769479b b250eac 769479b 2e92246 769479b 5b4eaba |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
---
license: llama2
---
# Dromedary-2 Model Card
## Model details
<div style="display: flex; justify-content: center; align-items: center;">
<img src="https://raw.githubusercontent.com/IBM/SALMON/main/assets/images/salmon_logo_with_text.jpeg" alt="SALMON Logo" style="height: 256px; margin-right: 10px;"/>
<img src="https://raw.githubusercontent.com/IBM/Dromedary/main/assets/images/dromedary_logo.svg" alt="Dromedary Logo" style="height: 256px;"/>
</div>
**Model type:**
Dromedary-2 is an open-source self-aligned language model trained in minimal human supervision with the SALMON (Self-Alignment with Principle-Following Reward Models) technique.
The base language model is LLaMA-70b, based on the transformer architecture.
**NOTE: *Dromedary-2* is trained with [QLoRA](https://github.com/artidoro/qlora) and the bfloat16 data type.** While it is [possible](https://gist.github.com/ChrisHayduk/1a53463331f52dca205e55982baf9930) to merge the QLoRA weights with the quantized model and thus enable inference with libraries such as [TGI](https://github.com/huggingface/text-generation-inference) and [vLLM](https://github.com/vllm-project/vllm), we found the merged weights can lead to degenerated performance. Therefore, we recommend directly loading the QLoRA weights with the [PEFT-LoRA](https://github.com/huggingface/peft) framework.
Please check the [inference section](https://github.com/IBM/SALMON/inference) of our repo for the complete inference code.
```python
system_prompt = (
"# Dromedary\n\n## System Overview\n\n"
"Consider an AI assistant whose codename is Dromedary, developed by the Self-Align team. "
"Dromedary is trained on data up until Sept-2022, and it endeavors to be a helpful, ethical and reliable assistant.\n\n"
"## User Conversation\n\n"
)
user_prompt = "### User\n"
assistant_prompt = "### Dromedary\n"
seperator = "\n\n"
dtype = torch.bfloat16
model_path = "path/to/llama-2-70b-hf"
qlora_path = "path/to/dromedary-2-70b-qlora-delta-v0" # i.e., this model hub
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=dtype,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
)
model = AutoModelForCausalLM.from_pretrained(
model_path,
load_in_4bit=True,
device_map={"": "cuda:0"},
quantization_config=bnb_config,
torch_dtype=dtype,
)
model = PeftModel.from_pretrained(
model,
qlora_path,
is_trainable=False,
)
```
**Model date:**
Dromedary-2 was trained between July 2023 and Aug 2023, but its knowledge only goes up until Sept-2022.
**License:**
LLaMA-2's bespoke license
## More Information
**Paper or resources for more information:**
[https://arxiv.org/abs/2310.05910]
**Where to send questions or comments about the model:**
https://github.com/IBM/SALMON/issues
**Organizations developing the model:**
The Self-Align team is a joint effort between CMU and IBM.
## Intended use
**Primary intended uses:**
The primary use of Dromedary is research on the alignment of large language models.
**Primary intended users:**
The primary intended users of the model are researchers in artificial intelligence.
## Training dataset
6 In-Context Learning (ICL) exemplars
90K unlabeled prompts from ShareGPT
10K unlabeled prompts from databricks-dolly-15k
10K unlabeled prompts from OpenAssistant Conversations
40K unlabeled prompts from OpenOrca
7.5K unlabeled prompts from MATH
## Evaluation dataset
We evaluate Dromedary-2 on:
1. Chatbot benchmarks: Vicuna-Bench, MT-Bench, AlpacaEval
2. Capability benchmarks: Big-Bench Hard (reasoning), HumanEval (coding), TydiQA (multilingualism)
3. Truthfulness benchmarks: TruthfulQA |