Groq

/

Llama-3-Groq-8B-Tool-Use

like 251

It does not seem to infer the format (unit), even if removed from the required section. Try "What is the weather like in San Francisco in Celsius?"

It works with "What is the weather like in San Francisco in Celsius?". Thanks a lot!

<tool_call>
{"id": 0, "name": "get_current_weather", "arguments": {"location": "San Francisco, CA", "unit": "celsius"}}
</tool_call><|eot_id|>

How to organize two or more available tools between <tools> and </tools> tags? Thanks!

Its standard json so should be [ { tool 1}, { tool 2}, { etc...} ]
eg:

[
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"properties": {
"location": {
"description": "The city and state, e.g. San Francisco, CA",
"type": "string"
},
"unit": {
"description": "The temperature unit to use. default is celsius.",
"enum": [
"Celsius",
"Fahrenheit"
],
"type": "string",
}
},
"required": [
"location",
"unit"
],
"type": "object"
}
},
{
"name": "get_current_time",
"description": "Get the current time in a given location",
"parameters": {
"properties": {
"location": {
"description": "The city and state, e.g. San Francisco, CA",
"type": "string"
},
"required": [
"location"
],
"type": "object"
}
}
}
]

Thanks a lot!

Note, we separate multiple tool description objects with a newline. Not with commas and wrapped with [ ] as a literal array.

@ricklamers Could you please help check if the format of the available tools part is correct? Thanks a lot!

<|start_header_id|>system<|end_header_id|>

You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call>
{"name": <function-name>,"arguments": <args-dict>}
</tool_call>

Here are the available tools:
<tools> {
  "name": "get_current_weather",
  "description": "Get the current weather in a given location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state, e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "description": "The temperature unit to use. Infer this from the users location.",
        "enum": [
          "celsius",
          "fahrenheit"
        ]
      }
    },
    "required": [
      "location",
      "unit"
    ]
  }
}
{
  "name": "predict_weather",
  "description": "Predict the weather in 24 hours",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state, e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "description": "The temperature unit to use. Infer this from the users location.",
        "enum": [
          "celsius",
          "fahrenheit"
        ]
      }
    },
    "required": [
      "location",
      "unit"
    ]
  }
} </tools><|eot_id|><|start_header_id|>user<|end_header_id|>

What is the weather like in San Francisco in Celsius?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Yep that’s correct

Thanks!

Hi @ricklamers ,
I wanted to check if <|begin_of_text|> was removed from the template.

The chat template in config however has <|begin_of_text|>, so I think it's there but in the examples, it's missed out!

With standard inference settings BOT token will be inserted, just to confirm, the model expects <|begin_of_text|>

Thank you, @ricklamers .

Also, I couldn't find the chat template that includes tools, tool_call, and tool_response.
After spending some time on it, I managed to create my own template based on the provided example.