Edit model card

Uploaded model

  • Developed by: Deeokay
  • License: apache-2.0
  • Finetuned from model : unsloth/mistral-7b-instruct-v0.3-bnb-4bit

This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.

README

This is a test model on a the following

  • a private dataset
  • customization on tokenization to llama3 template
  • Works with Ollama create with just "FROM path/to/model" as Modelfile (requires to add llama3 template works no issues)

HOW TO USE

The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options) For Ollama, it required to be a GGUF file. Once you have this it is pretty straight forward (if it is in llama3 which this model is)

Quick Start:

  • You must already have Ollama running in your setting
  • Download the unsloth.Q4_K_M.gguf model from Files
  • In the same directory create a file call "Modelfile"
  • Inside the "Modelfile" type
FROM ./unsloth.Q4_K_M.gguf

PARAMETER temperature 0.6
PARAMETER repeat_penalty 1.3
PARAMETER top_p 0.6
PARAMETER top_k 30

PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER stop <|eot_id|>

TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"
  • Save a go back to the folder (folder where model + Modelfile exisit)
  • Now in terminal make sure you are in the same location of the folder and type in the following command
ollama create mycustomai  # "mycustomai" <- you can name it anything u want

This GGUF is based on mistral-7b-v0.3

NOTE: DISCLAIMER

Please note this is not for the purpose of production, but result of Fine Tuning through self learning

The llama3 Special Tokens where used to convert the tokenizer.

I wanted to test if the model would understand additional headers that I created such as what my datasets has

  • Analaysis, Classification, Sentiment

Multiple pass through my personalized customized dataset, future updates will be made to this repo.

If would like to know how I started creating my dataset, you can check this link Crafting GPT2 for Personalized AI-Preparing Data the Long Way (Part1)

the training data has the following Template:

<|begin_of_text|> <|start_header_id|>user<|end_header_id|>
{{.Prompt}}<|eot_id|><|start_header_id|>analysis<|end_header_id|>
{{.Analysis}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{{.Response}}<|eot_id|><|start_header_id|>classification<|end_header_id|>
{{.Classification}}<|eot_id|><|start_header_id|>sentiment<|end_header_id|>
{{.Sentiment}}<|eot_id|><|end_of_text|> 
Downloads last month
669
GGUF
Model size
7.25B params
Architecture
llama

2-bit

3-bit

4-bit

8-bit

16-bit

Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for Deeokay/mistral-7b-v0.3-llama_tokens-GGUF

Quantized
(52)
this model