
# Usage

Find below some example scripts on how to use the model in `transformers`:

## Using the Pytorch model
```python
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

# Load peft config for pre-trained checkpoint etc.
peft_model_id = "rsonavane/flan-t5-xl-alpaca-dolly-lora-peft"
config = PeftConfig.from_pretrained(peft_model_id)

# load base LLM model and tokenizer
model = AutoModelForSeq2SeqLM.from_pretrained(config.base_model_name_or_path,  load_in_8bit=True,  device_map={"":0})
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id, device_map={"":0})
```

## Prompt generation
```python
def generate_prompt(instruction: str, input_ctxt: str = "") -> str:
    if input_ctxt:
        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input_ctxt}

### Response:"""
    else:
        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:"""
```
## Inference
```python

input_ctxt = ""
instruction = ""

input_text = generate_prompt(instruction, input_ctxt)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```

## Training Details
Intended for conversation analysis, closed qna and summarization.
Trained on instructions from doll-15k, alpaca-52k and samsum dataset.
{"language": ["en", "ja", "de", "fr", "multilingual"], "tags": ["t5", "adapter", "flan-t5", "peft", "lora"], "datasets": ["yahma/alpaca-cleaned", "databricks/databricks-dolly-15k", "samsum"], "pipeline_tag": "text2text-generation"}
peft pytorch t5 adapter flan-t5 lora text2text-generation en ja de fr multilingual dataset:yahma/alpaca-cleaned dataset:databricks/databricks-dolly-15k dataset:samsum 8-bit region:us
TAGS
#peft #pytorch #t5 #adapter #flan-t5 #lora #text2text-generation #en #ja #de #fr #multilingual #dataset-yahma/alpaca-cleaned #dataset-databricks/databricks-dolly-15k #dataset-samsum #8-bit #region-us 


# Usage

Find below some example scripts on how to use the model in 'transformers':

## Using the Pytorch model


## Prompt generation

## Inference


## Training Details
Intended for conversation analysis, closed qna and summarization.
Trained on instructions from doll-15k, alpaca-52k and samsum dataset.
# Usage

Find below some example scripts on how to use the model in 'transformers': ## Using the Pytorch model ## Prompt generation ## Inference ## Training Details
Intended for conversation analysis, closed qna and summarization.
Trained on instructions from doll-15k, alpaca-52k and samsum dataset.
TAGS
#peft #pytorch #t5 #adapter #flan-t5 #lora #text2text-generation #en #ja #de #fr #multilingual #dataset-yahma/alpaca-cleaned #dataset-databricks/databricks-dolly-15k #dataset-samsum #8-bit #region-us 
 # Usage

Find below some example scripts on how to use the model in 'transformers': ## Using the Pytorch model ## Prompt generation ## Inference ## Training Details
Intended for conversation analysis, closed qna and summarization.
Trained on instructions from doll-15k, alpaca-52k and samsum dataset.