Update README.md
Browse files
README.md
CHANGED
@@ -19,19 +19,19 @@ base_model: meta-llama/Llama-2-13b-hf
|
|
19 |
</div>
|
20 |
|
21 |
|
22 |
-
#
|
23 |
|
24 |
A pretrained generative language model with 7 billion parameters geared towards instruction-following capabilities.
|
25 |
|
26 |
## Model Details
|
27 |
|
28 |
-
This model was built via parameter-efficient finetuning of the [
|
29 |
|
30 |
- **Developed by:** Daniel Furman
|
31 |
- **Model type:** Decoder-only
|
32 |
- **Language(s) (NLP):** English
|
33 |
- **License:** Apache 2.0
|
34 |
-
- **Finetuned from model:** [
|
35 |
|
36 |
## Model Sources
|
37 |
|
@@ -69,7 +69,7 @@ from transformers import (
|
|
69 |
```
|
70 |
|
71 |
```python
|
72 |
-
peft_model_id = "dfurman/
|
73 |
config = PeftConfig.from_pretrained(peft_model_id)
|
74 |
|
75 |
tokenizer = AutoTokenizer.from_pretrained(
|
@@ -166,7 +166,7 @@ Ice cubes to fill the shaker
|
|
166 |
|
167 |
## Training
|
168 |
|
169 |
-
It took ~
|
170 |
|
171 |
### Prompt Format
|
172 |
|
|
|
19 |
</div>
|
20 |
|
21 |
|
22 |
+
# Llama-2-13B-Instruct-v0.2
|
23 |
|
24 |
A pretrained generative language model with 7 billion parameters geared towards instruction-following capabilities.
|
25 |
|
26 |
## Model Details
|
27 |
|
28 |
+
This model was built via parameter-efficient finetuning of the [meta-llama/Llama-2-13b-hf](https://huggingface.co/meta-llama/Llama-2-13b-hf) base model on the first 20k rows in each of the [jondurbin/airoboros-2.2.1](https://huggingface.co/datasets/jondurbin/airoboros-2.2.1), [Open-Orca/SlimOrca](https://huggingface.co/datasets/Open-Orca/SlimOrca), and [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus) datasets.
|
29 |
|
30 |
- **Developed by:** Daniel Furman
|
31 |
- **Model type:** Decoder-only
|
32 |
- **Language(s) (NLP):** English
|
33 |
- **License:** Apache 2.0
|
34 |
+
- **Finetuned from model:** [meta-llama/Llama-2-13b-hf](https://huggingface.co/meta-llama/Llama-2-13b-hf)
|
35 |
|
36 |
## Model Sources
|
37 |
|
|
|
69 |
```
|
70 |
|
71 |
```python
|
72 |
+
peft_model_id = "dfurman/Llama-2-13B-Instruct-v0.2"
|
73 |
config = PeftConfig.from_pretrained(peft_model_id)
|
74 |
|
75 |
tokenizer = AutoTokenizer.from_pretrained(
|
|
|
166 |
|
167 |
## Training
|
168 |
|
169 |
+
It took ~7 hours to train 3 epochs on 1x A100 (40 GB SXM).
|
170 |
|
171 |
### Prompt Format
|
172 |
|