# LLM Fine-Tuning with QLoRA

This repository can help to instruct-tune Open LLaMA, RedPajama or StableLM models on consumer hardware using QLoRA (Original implementation [here](https://github.com/artidoro/qlora)). It's mostly based on the original alpaca-lora repo which can be found [here](https://github.com/tloen/alpaca-lora). Please note that this has only been tested on Open LLama 3b and RedPajama 3b Models, but should work with other models. Contributions are welcome!

### Local Setup

1. Install dependencies

   ```bash
   pip install -r requirements.txt
   ```

1. If bitsandbytes doesn't work, [install it from source.](https://github.com/TimDettmers/bitsandbytes/blob/main/compile_from_source.md) Windows users can follow [these instructions](https://github.com/tloen/alpaca-lora/issues/17).

## Training (finetune.py)

This file contains a straightforward application of QLoRA PEFT to the Open LLaMA / RedPajama / StableLM model, as well as some code related to prompt construction and tokenization. PRs adapting this code to support larger models are always welcome.

**Example usage:**

For Open LLaMa

    python finetune.py \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --data_path '../datasets/dolly.json' \
        --num_epochs=3 \
        --cutoff_len=512 \
        --group_by_length \
        --output_dir='./dolly-lora-3b' \
        --lora_r=16 \
        --lora_target_modules='[q_proj,v_proj]'

For RedPajama

    python finetune.py   \
    --base_model='togethercomputer/RedPajama-INCITE-Base-3B-v1' \
    --data_path='../datasets/dolly.json'   \
    --num_epochs=3   \
    --cutoff_len=512   \
    --group_by_length   \
    --output_dir='./dolly-lora-rp-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]' 
    
For StableLM

    python finetune.py  \
    --base_model='stabilityai/stablelm-base-alpha-3b' \
    --data_path='../datasets/dolly.json' \
    --num_epochs=3 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-st-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]'
    
For Pythia

    python finetune.py  \
    --base_model='EleutherAI/pythia-6.9b-deduped' \
    --data_path='../datasets/dolly.json' \
    --num_epochs=1 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-pyt-6b-t1' \
    --lora_r=8 \
    --lora_target_modules='["query_key_value"]'

We can also tweak our hyperparameters (similar to alpaca-lora):

    python finetune.py \
        --base_model 'openlm-research/open_llama_3b_600bt_preview \
        --data_path 'yahma/alpaca-cleaned' \
        --output_dir './lora-alpaca' \
        --batch_size 128 \
        --micro_batch_size 4 \
        --num_epochs 3 \
        --learning_rate 1e-4 \
        --cutoff_len 512 \
        --val_set_size 2000 \
        --lora_r 8 \
        --lora_alpha 16 \
        --lora_dropout 0.05 \
        --lora_target_modules '[q_proj,v_proj]' \
        --train_on_inputs \
        --group_by_length

## Inference (generate.py)
This file reads the foundation model from the Hugging Face model hub and the LoRA weights from trained peft model, and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.

Example usage:    

For Open LLaMa

    python generate.py \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --lora_weights './lora-alpaca'
        
For RedPajama

    python generate.py  \
    --base_model 'togethercomputer/RedPajama-INCITE-Base-3B-v1'  \
    --lora_weights './dolly-lora-rp-3b-t1/'
       
For StableLM

    python generate.py  \
    --base_model 'stabilityai/stablelm-base-alpha-3b' \
    --lora_weights './dolly-lora-st-3b-t1'
    
For Pythia
   
    python generate.py  \
    --base_model 'EleutherAI/pythia-6.9b-deduped'  \
    --lora_weights './dolly-lora-pyt-6b-t1'
    
# Acknowledgements

We would like to express our heartfelt gratitude to **Meta** for releasing LLaMA . Without this pioneering technology, the foundations of projects like **Open Llama** and **Alpaca** wouldn't exist. We sincerely appreciate the immense contributions you've made to the field.

Our acknowledgements also extend to the teams behind **Open LLaMA**, **Together Computer**, **Alpaca** and **Alpaca LoRA**.. You can find more about their excellent work on their respective GitHub repositories:

- [Open Llama](https://github.com/openlm-research/open_llama)
- [Together Computer](https://github.com/togethercomputer)
- [Alpaca](https://github.com/tatsu-lab/stanford_alpaca)
- [Alpaca LoRa](https://github.com/tloen/alpaca-lora)

Lastly, we would like to express our thanks to the developers of **QLoRA** and **bitsandbytes** Your efforts have been instrumental in advancing the field, and we're grateful for your contributions. More information about these projects can be found at:

- [QLoRA](https://github.com/artidoro/qlora)
- [bitsandbytes](https://github.com/TimDettmers/bitsandbytes)


Thank you all for your commitment to innovation and for making these projects possible.


    

{}
tensorboard region:us
TAGS
#tensorboard #region-us 

# LLM Fine-Tuning with QLoRA

This repository can help to instruct-tune Open LLaMA, RedPajama or StableLM models on consumer hardware using QLoRA (Original implementation here). It's mostly based on the original alpaca-lora repo which can be found here. Please note that this has only been tested on Open LLama 3b and RedPajama 3b Models, but should work with other models. Contributions are welcome!

### Local Setup

1. Install dependencies

   

1. If bitsandbytes doesn't work, install it from source. Windows users can follow these instructions.

## Training (URL)

This file contains a straightforward application of QLoRA PEFT to the Open LLaMA / RedPajama / StableLM model, as well as some code related to prompt construction and tokenization. PRs adapting this code to support larger models are always welcome.

Example usage:

For Open LLaMa

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --data_path '../datasets/URL' \
        --num_epochs=3 \
        --cutoff_len=512 \
        --group_by_length \
        --output_dir='./dolly-lora-3b' \
        --lora_r=16 \
        --lora_target_modules='[q_proj,v_proj]'

For RedPajama

    python URL   \
    --base_model='togethercomputer/RedPajama-INCITE-Base-3B-v1' \
    --data_path='../datasets/URL'   \
    --num_epochs=3   \
    --cutoff_len=512   \
    --group_by_length   \
    --output_dir='./dolly-lora-rp-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]' 
    
For StableLM

    python URL  \
    --base_model='stabilityai/stablelm-base-alpha-3b' \
    --data_path='../datasets/URL' \
    --num_epochs=3 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-st-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]'
    
For Pythia

    python URL  \
    --base_model='EleutherAI/pythia-6.9b-deduped' \
    --data_path='../datasets/URL' \
    --num_epochs=1 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-pyt-6b-t1' \
    --lora_r=8 \
    --lora_target_modules='["query_key_value"]'

We can also tweak our hyperparameters (similar to alpaca-lora):

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview \
        --data_path 'yahma/alpaca-cleaned' \
        --output_dir './lora-alpaca' \
        --batch_size 128 \
        --micro_batch_size 4 \
        --num_epochs 3 \
        --learning_rate 1e-4 \
        --cutoff_len 512 \
        --val_set_size 2000 \
        --lora_r 8 \
        --lora_alpha 16 \
        --lora_dropout 0.05 \
        --lora_target_modules '[q_proj,v_proj]' \
        --train_on_inputs \
        --group_by_length

## Inference (URL)
This file reads the foundation model from the Hugging Face model hub and the LoRA weights from trained peft model, and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.

Example usage:    

For Open LLaMa

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --lora_weights './lora-alpaca'
        
For RedPajama

    python URL  \
    --base_model 'togethercomputer/RedPajama-INCITE-Base-3B-v1'  \
    --lora_weights './dolly-lora-rp-3b-t1/'
       
For StableLM

    python URL  \
    --base_model 'stabilityai/stablelm-base-alpha-3b' \
    --lora_weights './dolly-lora-st-3b-t1'
    
For Pythia
   
    python URL  \
    --base_model 'EleutherAI/pythia-6.9b-deduped'  \
    --lora_weights './dolly-lora-pyt-6b-t1'
    
# Acknowledgements

We would like to express our heartfelt gratitude to Meta for releasing LLaMA . Without this pioneering technology, the foundations of projects like Open Llama and Alpaca wouldn't exist. We sincerely appreciate the immense contributions you've made to the field.

Our acknowledgements also extend to the teams behind Open LLaMA, Together Computer, Alpaca and Alpaca LoRA.. You can find more about their excellent work on their respective GitHub repositories:

- Open Llama
- Together Computer
- Alpaca
- Alpaca LoRa

Lastly, we would like to express our thanks to the developers of QLoRA and bitsandbytes Your efforts have been instrumental in advancing the field, and we're grateful for your contributions. More information about these projects can be found at:

- QLoRA
- bitsandbytes


Thank you all for your commitment to innovation and for making these projects possible.


    

# LLM Fine-Tuning with QLoRA

This repository can help to instruct-tune Open LLaMA, RedPajama or StableLM models on consumer hardware using QLoRA (Original implementation here). It's mostly based on the original alpaca-lora repo which can be found here. Please note that this has only been tested on Open LLama 3b and RedPajama 3b Models, but should work with other models. Contributions are welcome! ### Local Setup

1. Install dependencies

   

1. If bitsandbytes doesn't work, install it from source. Windows users can follow these instructions. ## Training (URL)

This file contains a straightforward application of QLoRA PEFT to the Open LLaMA / RedPajama / StableLM model, as well as some code related to prompt construction and tokenization. PRs adapting this code to support larger models are always welcome.

Example usage:

For Open LLaMa

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --data_path '../datasets/URL' \
        --num_epochs=3 \
        --cutoff_len=512 \
        --group_by_length \
        --output_dir='./dolly-lora-3b' \
        --lora_r=16 \
        --lora_target_modules='[q_proj,v_proj]'

For RedPajama

    python URL   \
    --base_model='togethercomputer/RedPajama-INCITE-Base-3B-v1' \
    --data_path='../datasets/URL'   \
    --num_epochs=3   \
    --cutoff_len=512   \
    --group_by_length   \
    --output_dir='./dolly-lora-rp-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]' 
    
For StableLM

    python URL  \
    --base_model='stabilityai/stablelm-base-alpha-3b' \
    --data_path='../datasets/URL' \
    --num_epochs=3 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-st-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]'
    
For Pythia

    python URL  \
    --base_model='EleutherAI/pythia-6.9b-deduped' \
    --data_path='../datasets/URL' \
    --num_epochs=1 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-pyt-6b-t1' \
    --lora_r=8 \
    --lora_target_modules='["query_key_value"]'

We can also tweak our hyperparameters (similar to alpaca-lora):

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview \
        --data_path 'yahma/alpaca-cleaned' \
        --output_dir './lora-alpaca' \
        --batch_size 128 \
        --micro_batch_size 4 \
        --num_epochs 3 \
        --learning_rate 1e-4 \
        --cutoff_len 512 \
        --val_set_size 2000 \
        --lora_r 8 \
        --lora_alpha 16 \
        --lora_dropout 0.05 \
        --lora_target_modules '[q_proj,v_proj]' \
        --train_on_inputs \
        --group_by_length ## Inference (URL)
This file reads the foundation model from the Hugging Face model hub and the LoRA weights from trained peft model, and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.

Example usage:    

For Open LLaMa

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --lora_weights './lora-alpaca'
        
For RedPajama

    python URL  \
    --base_model 'togethercomputer/RedPajama-INCITE-Base-3B-v1'  \
    --lora_weights './dolly-lora-rp-3b-t1/'
       
For StableLM

    python URL  \
    --base_model 'stabilityai/stablelm-base-alpha-3b' \
    --lora_weights './dolly-lora-st-3b-t1'
    
For Pythia
   
    python URL  \
    --base_model 'EleutherAI/pythia-6.9b-deduped'  \
    --lora_weights './dolly-lora-pyt-6b-t1' # Acknowledgements

We would like to express our heartfelt gratitude to Meta for releasing LLaMA . Without this pioneering technology, the foundations of projects like Open Llama and Alpaca wouldn't exist. We sincerely appreciate the immense contributions you've made to the field.

Our acknowledgements also extend to the teams behind Open LLaMA, Together Computer, Alpaca and Alpaca LoRA.. You can find more about their excellent work on their respective GitHub repositories:

- Open Llama
- Together Computer
- Alpaca
- Alpaca LoRa

Lastly, we would like to express our thanks to the developers of QLoRA and bitsandbytes Your efforts have been instrumental in advancing the field, and we're grateful for your contributions. More information about these projects can be found at:

- QLoRA
- bitsandbytes


Thank you all for your commitment to innovation and for making these projects possible.
TAGS
#tensorboard #region-us 
 # LLM Fine-Tuning with QLoRA

This repository can help to instruct-tune Open LLaMA, RedPajama or StableLM models on consumer hardware using QLoRA (Original implementation here). It's mostly based on the original alpaca-lora repo which can be found here. Please note that this has only been tested on Open LLama 3b and RedPajama 3b Models, but should work with other models. Contributions are welcome! ### Local Setup

1. Install dependencies

   

1. If bitsandbytes doesn't work, install it from source. Windows users can follow these instructions. ## Training (URL)

This file contains a straightforward application of QLoRA PEFT to the Open LLaMA / RedPajama / StableLM model, as well as some code related to prompt construction and tokenization. PRs adapting this code to support larger models are always welcome.

Example usage:

For Open LLaMa

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --data_path '../datasets/URL' \
        --num_epochs=3 \
        --cutoff_len=512 \
        --group_by_length \
        --output_dir='./dolly-lora-3b' \
        --lora_r=16 \
        --lora_target_modules='[q_proj,v_proj]'

For RedPajama

    python URL   \
    --base_model='togethercomputer/RedPajama-INCITE-Base-3B-v1' \
    --data_path='../datasets/URL'   \
    --num_epochs=3   \
    --cutoff_len=512   \
    --group_by_length   \
    --output_dir='./dolly-lora-rp-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]' 
    
For StableLM

    python URL  \
    --base_model='stabilityai/stablelm-base-alpha-3b' \
    --data_path='../datasets/URL' \
    --num_epochs=3 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-st-3b-t1' \
    --lora_r=16 \
    --lora_target_modules='["query_key_value"]'
    
For Pythia

    python URL  \
    --base_model='EleutherAI/pythia-6.9b-deduped' \
    --data_path='../datasets/URL' \
    --num_epochs=1 \
    --cutoff_len=512 \
    --group_by_length  \
    --output_dir='./dolly-lora-pyt-6b-t1' \
    --lora_r=8 \
    --lora_target_modules='["query_key_value"]'

We can also tweak our hyperparameters (similar to alpaca-lora):

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview \
        --data_path 'yahma/alpaca-cleaned' \
        --output_dir './lora-alpaca' \
        --batch_size 128 \
        --micro_batch_size 4 \
        --num_epochs 3 \
        --learning_rate 1e-4 \
        --cutoff_len 512 \
        --val_set_size 2000 \
        --lora_r 8 \
        --lora_alpha 16 \
        --lora_dropout 0.05 \
        --lora_target_modules '[q_proj,v_proj]' \
        --train_on_inputs \
        --group_by_length ## Inference (URL)
This file reads the foundation model from the Hugging Face model hub and the LoRA weights from trained peft model, and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.

Example usage:    

For Open LLaMa

    python URL \
        --base_model 'openlm-research/open_llama_3b_600bt_preview' \
        --lora_weights './lora-alpaca'
        
For RedPajama

    python URL  \
    --base_model 'togethercomputer/RedPajama-INCITE-Base-3B-v1'  \
    --lora_weights './dolly-lora-rp-3b-t1/'
       
For StableLM

    python URL  \
    --base_model 'stabilityai/stablelm-base-alpha-3b' \
    --lora_weights './dolly-lora-st-3b-t1'
    
For Pythia
   
    python URL  \
    --base_model 'EleutherAI/pythia-6.9b-deduped'  \
    --lora_weights './dolly-lora-pyt-6b-t1' # Acknowledgements

We would like to express our heartfelt gratitude to Meta for releasing LLaMA . Without this pioneering technology, the foundations of projects like Open Llama and Alpaca wouldn't exist. We sincerely appreciate the immense contributions you've made to the field.

Our acknowledgements also extend to the teams behind Open LLaMA, Together Computer, Alpaca and Alpaca LoRA.. You can find more about their excellent work on their respective GitHub repositories:

- Open Llama
- Together Computer
- Alpaca
- Alpaca LoRa

Lastly, we would like to express our thanks to the developers of QLoRA and bitsandbytes Your efforts have been instrumental in advancing the field, and we're grateful for your contributions. More information about these projects can be found at:

- QLoRA
- bitsandbytes


Thank you all for your commitment to innovation and for making these projects possible.