README.md

---
license: apache-2.0
datasets:
- ReDiX/xlam-function-calling-60k-ita
language:
- it
- code
pipeline_tag: text-generation
library_name: transformers
tags:
- unsloth
- trl
---

<img src="banner.webp" style="width: 300px; height:auto">

# ReDiX 1.5B JSON MODE and FUNCTION CALLING


This model is a finetuned version of [Qwen2-1.5B](https://huggingface.co/Qwen/Qwen2-1.5B) on [ReDiX/xlam-function-calling-60k-ita](https://huggingface.co/datasets/ReDiX/xlam-function-calling-60k-ita).
> [!NOTE]
> The function of this model is to perform function calling and creation of structured JSON for integration into simple and complex pipelines


## How to use

> [!WARNING]
> The model will always generate a response even if the tool does not exist and is not specified in the system prompt among the available tools. it is up to the software and the pipeline to handle erroneous output by parsing the response JSON

**System prompt**:
```
Sei un Assistente AI che ha accesso ai seguenti tools:

{TOOL DEFINITIONS} 

Genera in formato JSON la chiamata necessaria per soddisfare la richiesta dell'utente.
{INFORMAZIONI REALTIME (es data di oggi)}
```

<details><summary>See conversation example</summary>
  
```
<|im_start|>system
Sei un Assistente AI che ha accesso ai seguenti tools:

Use the function 'stock_search' to: Get stock analisys and values
{
  "name": "stock_search",
  "description": "Get stock values",
  "parameters": {
    "ticker": {
      "param_type": "string",
      "description": "Identifier of the ticker, es: AAPL or list for multiple tickers [‘ticker1’, ‘ticker2’]“,
      "required": true
    },
    "start": {
      "param_type": "string",
      "description": "Range start date, es: 2022-01-01",
      "required": true
    },
    "end": {
      "param_type": "string",
      "description": "Range end date, es: 2022-01-01",
      "required": true
    }
  }
}
Genera in formato JSON la chiamata necessaria per soddisfare la richiesta dell'utente.
Oggi è il 2024-08-01
<|im_end|>
<|im_start|>user
Vorrei sapere com’è andata microsoft (MSFT) e tesla (TSLA) nel corso di luglio<|im_end|>
<|im_start|>assistant
```
</details>


<details><summary>Code example</summary>
    
  ```python
  import torch
  from transformers import AutoModelForCausalLM, AutoTokenizer
  
  
  base_model_name = "ReDiX/ReDiX-1.5B-JSON"
  
  
  tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
  
  model = AutoModelForCausalLM.from_pretrained(
      base_model_name, 
      device_map="cuda",
      trust_remote_code=True, 
      torch_dtype=torch.bfloat16
  )
  
  model.eval()
  print("MODELLO PRONTO PER INFERENZA")
  
  input_text = """<|im_start|>system
  Sei un Assistente AI che ha accesso ai seguenti tools:
  
  Use the function 'stock_search' to: Get stock analisys and values
  {
    "name": "stock_search",
    "description": "Get stock values",
    "parameters": {
      "ticker": {
        "param_type": "string",
        "description": "Identifier of the ticker, es: AAPL or list for multiple tickers [‘ticker1’, ‘ticker2’]“,
        "required": true
      },
      "start": {
        "param_type": "string",
        "description": "Range start date, es: 2022-01-01",
        "required": true
      },
      "end": {
        "param_type": "string",
        "description": "Range end date, es: 2022-01-01",
        "required": true
      }
    }
  }
  Genera in formato JSON la chiamata necessaria per soddisfare la richiesta dell'utente.<|im_end|>
  <|im_start|>user
  Vorrei sapere com’è andata microsoft (MSFT) e tesla (TSLA) nel corso di luglio 2024?<|im_end|>
  <|im_start|>assistant"""
  
  
  inputs = tokenizer(input_text, return_tensors="pt").to('cuda')
  # tokenizer.eos_token = "<|im_end|>"
  # tokenizer.eos_token_id = 151645
  print(tokenizer.pad_token)
  print(tokenizer.eos_token)
  
  print("=============== inizio generazione ===============")
  with torch.no_grad():
      outputs = model.generate(
          input_ids=inputs["input_ids"],
          max_length=100,
          num_return_sequences=1,
          do_sample=False,
          temperature=0.1,
          pad_token_id=tokenizer.pad_token_id,
          eos_token_id=tokenizer.eos_token_id
      )
  
  
  generated_text = tokenizer.decode(outputs[0], skip_special_tokens=False)
  print(generated_text)
  ```
</details>


## Training

we trained this model on a single RTX A6000 48GB for about 5 hours

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)