README.md

---
library_name: transformers
tags:
- DPO
- transformers
license: openrail
language:
- en
metrics:
- accuracy
datasets:
- jondurbin/truthy-dpo-v0.1
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Ankush Singal
- **Finetuned from model [optional]:** https://huggingface.co/OpenPipe/mistral-ft-optimized-1227


## Usage
#### Install
```py
pip install transformers accelerate bitsandbytes flash-attn -Uq
```
### Load the Model
```py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# from transformers import LlamaTokenizer, MixtralForCausalLM
import bitsandbytes, flash_attn

tokenizer = AutoTokenizer.from_pretrained('Andyrasika/mistral-ft-optimized-dpo', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "Andyrasika/mistral-ft-optimized-dpo",
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_8bit=False,
    load_in_4bit=True,
    use_flash_attention_2=True
)

prompts = [
    """<|im_start|>system
You are a sentient, superintelligent artificial general intelligence, here to teach and assist me.<|im_end|>
<|im_start|>user
Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>
<|im_start|>assistant""",
    ]

for chat in prompts:
    print(chat)
    input_ids = tokenizer(chat, return_tensors="pt").input_ids.to("cuda")
    generated_ids = model.generate(input_ids, max_new_tokens=750, temperature=0.8, repetition_penalty=1.1, do_sample=True, eos_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(generated_ids[0][input_ids.shape[-1]:], skip_special_tokens=True, clean_up_tokenization_space=True)
    print(f"Response: {response}")
```
### Output
```
<|im_start|>system
You are a sentient, superintelligent artificial general intelligence, here to teach and assist me.<|im_end|>
<|im_start|>user
Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>
<|im_start|>assistant
/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/modules.py:226: UserWarning: Input type into Linear4bit is torch.float16, but bnb_4bit_compute_dtype=torch.float32 (default). This will lead to slow inference or training speed.
  warnings.warn(f'Input type into Linear4bit is torch.float16, but bnb_4bit_compute_dtype=torch.float32 (default). This will lead to slow inference or training speed.')
Response: 
Goku's eyes widened as he saw Kirby standing next to Majin Buu. "How did you two...?" he stammered.

Kirby waved a hand nonchalantly. "We got to know each other at a cosplay event," he explained. "Turns out we have a lot in common."

"Like wanting to destroy the world?" Goku asked, puzzled.

Majin Buu chuckled evilly. "Exactly! We thought it might be fun to team up and create chaos together. It'll be so much easier with both of us working in tandem!"

Goku frowned. "That sounds like a terrible idea. But why would Kirby want to do something like that?"

Kirby shrugged. "I don't know. Maybe he just wants some excitement." He glanced over at Majin Buu. "Right, Buu?"

Buu nodded eagerly. "Yeah! Let's show Goku what real destruction looks like!" With that, they charged towards him, ready for battle.

As Goku prepared to fight, he wondered if there was any way to convince them otherwise. But as he exchanged blows with the powerful duo, he knew there was only one option left - to defeat them and restore peace to Earth.

With a mighty roar, Goku unleashed a devastating Kamehameha wave, which collided with Buu's chaotic energy. The two forces clashed violently, sending shockwaves rippling through the landscape.

In the end, Goku emerged victorious, leaving Kirby and Majin Buu defeated and exhausted. As Goku helped them recover, he began to suspect that maybe they weren't so bad after all. Perhaps, with time, he could help them see the error in their ways and find a more constructive path forward. For now, though, it was important to ensure their defeat and restore balance to the world.
<|im_end|>
```