|
--- |
|
license: llama2 |
|
language: |
|
- en |
|
library_name: transformers |
|
datasets: |
|
- togethercomputer/llama-instruct |
|
--- |
|
|
|
# LLaMA-2-7B-32K-Chat |
|
|
|
## Model Description |
|
|
|
LLaMA-2-7B-32K-Chat is an open-source, long-context chat model finetuned from [LLaMA-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K), over high-quality instructions and chat data. |
|
We build Llama-2-7B-32K-Chat with less than 200 lines of Python script using [Together API](https://together.ai/blog/api-announcement), and we also make the recipe fully available. |
|
We hope that this can enable everyone to finetune their own version of [LLaMA-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K) — play with [Together API](https://together.ai/blog/api-announcement) and give us feedback! |
|
|
|
|
|
## Data Collection |
|
|
|
LLaMA-2-7B-32K-Chat is fine-tuned over two datasets: (1) 19K single- and multi-round conversations generated by human instructions and |
|
Llama-2-70B-Chat outputs, along with (2) 4K instructions of summarization from the BookSum datasets. |
|
We collected dataset (1) following the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca — producing instructions by querying a powerful LLM. |
|
We release such dataset [here](https://huggingface.co/datasets/togethercomputer/llama-instruct). For more details, please refer to our [Github repo](https://github.com/togethercomputer/LLaMA-2-32K-Chat). |
|
|
|
|
|
## Model Usage |
|
|
|
You can use the [Together API](https://together.ai/blog/api-announcement) to try out LLaMA-2-7B-32K-Chat for inference. |
|
The updated inference stack allows for efficient inference. |
|
The model will also be hosted on [Together Playground](https://api.together.xyz/playground). You can simply play with the model by using prompt formatted by |
|
|
|
``` |
|
[INST] <your instruction here> [\INST]. |
|
``` |
|
|
|
For example, if we query the model with |
|
|
|
``` |
|
[INST] Write a poem about cats [\INST], |
|
``` |
|
|
|
the model will return |
|
``` |
|
Sure, here's a short poem about cats: |
|
|
|
Cats, cats, oh how they enthrall, |
|
With their soft fur and gentle grace, |
|
They purr with joy, their eyes aglow, |
|
And steal our hearts with their charm and grace. |
|
|
|
In their own time, they move about, |
|
With graceful strides, they tiptoe out, |
|
Their whiskers twitching, as they go, |
|
Their curiosity piqued, they want to know. |
|
|
|
They climb, they jump, they play, they chase, |
|
Their agility, a sight to behold, |
|
Their naps, a treat, a break from the chase, |
|
A time to rest, a time to be told. |
|
|
|
Cats, cats, they're unique and special, |
|
A part of our lives, a treasure to hold, |
|
Their personalities, so diverse, |
|
Their charm, a gift, that's forever told. |
|
``` |
|
|
|
## Limitations and Bias |
|
|
|
As with all language models, LLaMA-2-7B-32K-Chat may generate incorrect or biased content. It's important to keep this in mind when using the model. |
|
|
|
## Community |
|
|
|
Join us on [Together Discord](https://discord.gg/6ZVDU8tTD4) |
