Steps to try the model:
prompt Template
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{}
### Input:
{}
### Response:
{}"""
load the model
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM ,AutoTokenizer
config = PeftConfig.from_pretrained("damerajee/Tinyllama-sft-small")
model = AutoModelForCausalLM.from_pretrained("unsloth/tinyllama")
tokenizer=AutoTokenizer.from_pretrained("damerajee/Tinyllama-sft-small")
model = PeftModel.from_pretrained(model, "damerajee/Tinyllama-sft-small")l")
Inference
inputs = tokenizer(
[
alpaca_prompt.format(
"i want to learn machine learning help me",
"", # input
"", # output
)
]*1, return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens = 312, use_cache = True)
tokenizer.batch_decode(outputs)
Model Information
The base model unsloth/tinyllama-bnb-4bitwas Instruct finetuned using Unsloth
Training Details
The model was trained for 1 epoch on a free goggle colab which took about 1 hour and 30 mins approximately
- Downloads last month
- 1
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for damerajee/Tinyllama-sft-small
Base model
unsloth/tinyllama-bnb-4bit