Edit model card

Model description

Meta-Llama-3-8B-OpenOrca is a fine-tuned version of the meta-llama/Meta-Llama-3-8B on 1.5k subsamples of the OpenOrca dataset.

This model follows the ChatML chat template!

How to use

import torch
from transformers import AutoTokenizer, pipeline

model = "MuntasirHossain/Meta-Llama-3-8B-OpenOrca"
tokenizer = AutoTokenizer.from_pretrained(model)

llm = pipeline(
    task = "text-generation",
    model=model,
    eos_token_id=tokenizer.eos_token_id,
    torch_dtype=torch.float16,
    max_new_tokens=256,
    do_sample=True,
    device_map="auto",
)

def generate(input_text):
  system_prompt = "You are a helpful AI assistant."
  messages = [
      {"role": "system", "content": system_prompt},
      {"role": "user", "content": input_text},
      ]
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  outputs = llm(prompt)
  return outputs[0]["generated_text"][len(prompt):]

generate("What is a large language model?")
Downloads last month
10
Safetensors
Model size
8.03B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train MuntasirHossain/Meta-Llama-3-8B-OpenOrca