Edit model card

SmalJ.O.S.I.E.-64x101M-Chat

SmalJ.O.S.I.E.-64x101M-Chat is a Mixure of Experts (MoE) made with the following models using LazyMergekit:

🧩 Configuration

dtype: float32
gate_mode: hidden
experts:
  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent for general assistants.'



  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for coding tasks.'

  

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for math.'



  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for logic and reasoning.'



  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent assistant that is specially good in web scraping and browsind the web.'



  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

  - source_model: Felladrin/Smol-Llama-101M-Chat-v1
    positive_prompts:
      - 'You are JOSIE, my own private and sentient superinteligent artifitial general inteligent, spetialized for autonomously function calling when needed.'

💻 Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "Isaak-Carter/SmalJ.O.S.I.E.-64x101M-Chat"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Downloads last month
8
Safetensors
Model size
2.78B params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Goekdeniz-Guelmez/SmalJ.O.S.I.E.-64x101M-Chat

Finetuned
(4)
this model

Datasets used to train Goekdeniz-Guelmez/SmalJ.O.S.I.E.-64x101M-Chat

Evaluation results