|
--- |
|
license: cc-by-sa-3.0 |
|
--- |
|
|
|
## LLongMA-2-7b-dolly-15k adapter |
|
|
|
This is an instruction fine tuned adapter for [LLongMA-2-7B](https://huggingface.co/conceptofmind/LLongMA-2-7b) , trained at **8k context length** using linear positional interpolation scaling. |
|
In order to run this inference with this adapter, you'll need the base [LLongMA-2-7b model](https://huggingface.co/conceptofmind/LLongMA-2-7b) as well. |
|
|
|
The adapter was instruction fined tuned with peft training, using the [dolly-15k dataset](https://huggingface.co/datasets/databricks/databricks-dolly-15k). |
|
|
|
--- |
|
library_name: peft |
|
--- |
|
## Training procedure |
|
|
|
|
|
The following `bitsandbytes` quantization config was used during training: |
|
- load_in_8bit: False |
|
- load_in_4bit: True |
|
- llm_int8_threshold: 6.0 |
|
- llm_int8_skip_modules: None |
|
- llm_int8_enable_fp32_cpu_offload: False |
|
- llm_int8_has_fp16_weight: False |
|
- bnb_4bit_quant_type: nf4 |
|
- bnb_4bit_use_double_quant: False |
|
- bnb_4bit_compute_dtype: bfloat16 |
|
### Framework versions |
|
|
|
|
|
- PEFT 0.4.0 |
|
|
|
### Inference format and example: |
|
|
|
### Human: Write an essay on the effects and impacts of social media. |
|
|
|
### Assistant |
|
Social Media has become a major part in today's society, with over half of the world population having some form of it. It is used for entertainment purposes as well as businesses to promote their products or services. Social Media can also be addictive due to its ability to keep users engaged for long periods of time which leads them spending more time online than doing other activities such as reading books or going outside into nature. This lack of physical activity combined with constant exposure to negative news stories can lead people feeling stressed out and anxious about what’s happening around them at all times - something that wasn't true before when we relied solely on newspapers/television broadcasting etc.. |
|
|
|
There are many benefits associated with using these platforms too; they allow us connect easily with friends & family members living far away from us while sharing photos/videos/stories quickly without any delay! Additionally there are lots opportunities available through networking via LinkedIn etc., allowing individuals access new career paths faster than ever before (provided they possess relevant skills). In conclusion , although there exist certain risks involved within this type technology still remains one most powerful tools available today helping millions achieve goals set forth by themselves every single day . Overall , despite being criticized heavily sometimes even considered dangerous depending upon how user uses it properly always remain safe harmless majority cases . |
|
|
|
|
|
|
|
### Human: Write a poem about love that features sunshine and rainbows. It should rhyme. |
|
|
|
### Assistant |
|
When the Sun shines bright |
|
It warms my heart so tight, |
|
But when it rains I feel sadly, |
|
As all of my joy dissipates. |
|
The storms pass quickly though, |
|
And soon enough there'll be sun once more, |
|
I know this because of what I've learned - |
|
That good times always follow bad ones before. |
|
So even if now things are tough for me, |
|
Soon they will get better as sure as you see! |
|
My spirits lift with every ray of light, |
|
Even on cloudy days I find something to write. |
|
These ups and downs won't last forever long, |
|
They come around from time to time but never stay strong. |
|
Love is patient and kindness too, |
|
This I know despite how hard life can go. |
|
No matter what happens in future years, |
|
Remember these words and don't forget your fears. |
|
You have strength within yourself; believe it or not, |
|
Your spirit knows best where its path leads us to. |
|
Don't let anyone tell you otherwise, |
|
Or try to convince you that you aren't worth much. |
|
Beautiful souls like yours deserve nothing less than peace, |
|
So keep fighting until you reach happiness. |
|
Life isn't easy sometimes but we must persevere, |
|
We learn most through our struggles which makes them dear. |
|
Our hearts may break along way, |
|
Yet still hope remains true regardless of what we say. |
|
Therefore no one else has power over who we become, |
|
For only ourselves decide what matters most at home. |
|
With each passing day comes new opportunities, |
|
To grow stronger while also learning some humility. |
|
Everyone goes through trials big & small, |
|
Which means everyone needs patience and compassion all. |
|
|
|
### Usage |
|
|
|
import torch |
|
import requests |
|
from peft import LoraConfig, PeftModel, get_peft_model |
|
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, AutoTokenizer, GenerationConfig |
|
|
|
torch.manual_seed(100) |
|
|
|
model_name = "conceptofmind/LLongMA-2-7b" |
|
adapter_name = "dreaming-ai/LLongMA-2-7b-dolly-15k" |
|
|
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.float16, |
|
) |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_name, |
|
quantization_config=bnb_config, |
|
trust_remote_code=True |
|
) |
|
model.config.use_cache = False |
|
|
|
generation_config = GenerationConfig.from_pretrained(model_name) |
|
|
|
# Temperature of 1.0 is the default. |
|
generation_config.temperature = 0.95 |
|
# The parameter for repetition penalty. Between 1.0 and infinity. 1.0 means no penalty. Default to 1.0. |
|
generation_config.repetition_penalty = 1.2 |
|
# Exponential penalty to the length. Default to 2. |
|
generation_config.length_penalty = 2.0 |
|
# top-p : Nucleus filtering (top-p) before sampling (<=0.0: no filtering) |
|
generation_config.top_p = 0.6 |
|
|
|
print('generation_config:', generation_config) |
|
|
|
# apply adapter |
|
model = PeftModel.from_pretrained(model, adapter_name) |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) |
|
tokenizer.pad_token = tokenizer.eos_token |
|
|
|
text = '''### Human: Write an essay on the effects and impacts of social media. |
|
|
|
### Assistant''' |
|
|
|
text2 = '''### Human: Write a poem about love that features sunshine and rainbows. It should rhyme. |
|
|
|
### Assistant''' |
|
|
|
device = "cuda:0" |
|
|
|
inputs = tokenizer(text, return_tensors="pt").to(device) |
|
outputs = model.generate(**inputs, max_new_tokens=1024, generation_config=generation_config) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
