Update README.md

README.md

@@ -22,6 +22,102 @@ This llama model was trained 2x faster with [Unsloth](https://github.com/unsloth
 
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
+## Usage with Unsloth
+
+
+```
+from unsloth.chat_templates import get_chat_template
+from unsloth import FastLanguageModel
+
+max_seq_length = 2048
+dtype = None
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "saucam/Saga-8B", # Choose ANY! eg teknium/OpenHermes-2.5-Mistral-7B
+    max_seq_length = max_seq_length,
+    dtype = dtype,
+    load_in_4bit = False,
+    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
+)
+
+tokenizer = get_chat_template(
+    tokenizer,
+    chat_template = "chatml", # Supports zephyr, chatml, mistral, llama, alpaca, vicuna, vicuna_old, unsloth
+    mapping = {"role" : "from", "content" : "value", "user" : "human", "assistant" : "gpt"}, # ShareGPT style
+    map_eos_token = True, # Maps <|im_end|> to </s> instead
+)
+
+FastLanguageModel.for_inference(model) # Enable native 2x faster inference
+
+messages = [
+    {"from": "human", "value": "What is a famous tall tower in Paris?"},
+]
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize = True,
+    add_generation_prompt = True, # Must add for generation
+    return_tensors = "pt",
+).to("cuda")
+
+outputs = model.generate(input_ids = inputs, max_new_tokens = 64, use_cache = True)
+print(tokenizer.batch_decode(outputs))
+```
+
+Output:
+```
+==((====))==  Unsloth: Fast Llama patching release 2024.4
+   \\   /|    GPU: NVIDIA A100 80GB PCIe. Max memory: 79.151 GB. Platform = Linux.
+O^O/ \_/ \    Pytorch: 2.2.0+cu121. CUDA = 8.0. CUDA Toolkit = 12.1.
+\        /    Bfloat16 = TRUE. Xformers = 0.0.24. FA = True.
+ "-____-"     Free Apache license: http://github.com/unslothai/unsloth
+Loading checkpoint shards: 100%|███████████████████████████████████████████████████| 4/4 [00:03<00:00,  1.19it/s]
+Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+Unsloth: Will map <|im_end|> to EOS = <|im_end|>.
+The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
+Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.
+['<|im_start|>user\nWhat is a famous tall tower in Paris?<|im_end|>\n<|im_start|>assistant\nThe Eiffel Tower is the most famous tall tower in Paris. It is a wrought iron tower that was built in 1889 as the entrance to the 1889 Exposition Universelle (Universal Exhibition) of Paris. The tower was named after its designer, engineer Gustave Eiffel. It stands ']
+```
+
+## Usage with Transformers
+
+```
+from transformers import AutoTokenizer
+import transformers
+import torch
+
+model = "saucam/Saga-8B"
+messages = [{"from": "human", "value": "Write a horror story about the monster of eldoria kingdom"}]
+
+tokenizer = AutoTokenizer.from_pretrained(model)
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
+
+outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+print(outputs[0]["generated_text"])
+```
+
+Output:
+```
+Loading checkpoint shards: 100%|███████████████████████████████████████████████████| 4/4 [00:12<00:00,  3.20s/it]
+Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
+<|im_start|>user
+Write a horror story about the monster of eldoria kingdom<|im_end|>
+<|im_start|>assistant
+Title: The Eldorian Beast - A Tale of Eldoria Kingdom
+
+In the heart of Eldoria Kingdom, nestled in the dense forests, lives a creature like no other. It's a tale of survival, love, and betrayal, woven into the intricate narrative of the Eldorian Beast.
+
+The Eldorian Beast, a creature of Eldoria Kingdom, is a symbol of the kingdom's core beliefs and beliefs that reflect its core values. The Eldorian Beast is known for its loyalty, its bravery, and its resilience. Its heart is as big as its kingdom, and like the kingdom, it has its own secrets, challenges, and triumphs, all of which makes it a unique character. 
+
+The Eldorian Beast is a wolf, not just any wolf but one that is a true guardian and protector of the kingdom. It is a wolf that knows the kingdom like no one else does, and knows the kingdom like it's its heart. It's a wolf that knows the kingdom's secrets and mysteries, and it's a wolf that knows the kingdom's strengths and weaknesses. 
+
+The Eldorian Beast is not just a wolf. It's a wolf that has been through many challenges and has survived every obstacle, just like Eldoria Kingdom. It's a wolf that's been
+```
 
 ## Training