README.md

metadata

license: apache-2.0
datasets:
  - ReDiX/xlam-function-calling-60k-ita
language:
  - it
  - code
pipeline_tag: text-generation
library_name: transformers
tags:
  - unsloth
  - trl

ReDiX 1.5B JSON MODE and FUNCTION CALLING

This model is a finetuned version of Qwen2-1.5B on ReDiX/xlam-function-calling-60k-ita.

The function of this model is to perform function calling and creation of structured JSON for integration into simple and complex pipelines

How to use

The model will always generate a response even if the tool does not exist and is not specified in the system prompt among the available tools. it is up to the software and the pipeline to handle erroneous output by parsing the response JSON

System prompt:

Sei un Assistente AI che ha accesso ai seguenti tools:

{TOOL DEFINITIONS} 

Genera in formato JSON la chiamata necessaria per soddisfare la richiesta dell'utente.
{INFORMAZIONI REALTIME (es data di oggi)}

See conversation example

<|im_start|>system
Sei un Assistente AI che ha accesso ai seguenti tools:

Use the function 'stock_search' to: Get stock analisys and values
{
  "name": "stock_search",
  "description": "Get stock values",
  "parameters": {
    "ticker": {
      "param_type": "string",
      "description": "Identifier of the ticker, es: AAPL or list for multiple tickers [‘ticker1’, ‘ticker2’]“,
      "required": true
    },
    "start": {
      "param_type": "string",
      "description": "Range start date, es: 2022-01-01",
      "required": true
    },
    "end": {
      "param_type": "string",
      "description": "Range end date, es: 2022-01-01",
      "required": true
    }
  }
}
Genera in formato JSON la chiamata necessaria per soddisfare la richiesta dell'utente.
Oggi è il 2024-08-01
<|im_end|>
<|im_start|>user
Vorrei sapere com’è andata microsoft (MSFT) e tesla (TSLA) nel corso di luglio<|im_end|>
<|im_start|>assistant

Code example

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer


base_model_name = "ReDiX/ReDiX-1.5B-JSON"


tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(
    base_model_name, 
    device_map="cuda",
    trust_remote_code=True, 
    torch_dtype=torch.bfloat16
)

model.eval()
print("MODELLO PRONTO PER INFERENZA")

input_text = """<|im_start|>system
Sei un Assistente AI che ha accesso ai seguenti tools:

Use the function 'stock_search' to: Get stock analisys and values
{
  "name": "stock_search",
  "description": "Get stock values",
  "parameters": {
    "ticker": {
      "param_type": "string",
      "description": "Identifier of the ticker, es: AAPL or list for multiple tickers [‘ticker1’, ‘ticker2’]“,
      "required": true
    },
    "start": {
      "param_type": "string",
      "description": "Range start date, es: 2022-01-01",
      "required": true
    },
    "end": {
      "param_type": "string",
      "description": "Range end date, es: 2022-01-01",
      "required": true
    }
  }
}
Genera in formato JSON la chiamata necessaria per soddisfare la richiesta dell'utente.<|im_end|>
<|im_start|>user
Vorrei sapere com’è andata microsoft (MSFT) e tesla (TSLA) nel corso di luglio 2024?<|im_end|>
<|im_start|>assistant"""


inputs = tokenizer(input_text, return_tensors="pt").to('cuda')
# tokenizer.eos_token = "<|im_end|>"
# tokenizer.eos_token_id = 151645
print(tokenizer.pad_token)
print(tokenizer.eos_token)

print("=============== inizio generazione ===============")
with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs["input_ids"],
        max_length=100,
        num_return_sequences=1,
        do_sample=False,
        temperature=0.1,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id
    )


generated_text = tokenizer.decode(outputs[0], skip_special_tokens=False)
print(generated_text)

Training

we trained this model on a single RTX A6000 48GB for about 5 hours