support use of tokenizer.apply_chat_template

README.md

@@ -29,7 +29,9 @@ Runpod one click template [here](https://runpod.io/gsc?template=edxvuji38p&ref=j
 ## Inference Scripts
 See below for sample prompt format.
 
-Out-of-the-box inference scripts are available for purchase [here](https://trelis.com/enterprise-server-api-and-inference-guide/).
+Complete inference scripts are available for purchase [here](https://trelis.com/enterprise-server-api-and-inference-guide/):
+- Easily format prompts using tokenizer.apply_chat_format (starting from openai formatted functions and a list of messages)
+- Automate catching, handling and chaining of function calls.
 
 ## Prompt Format
 ```
@@ -38,7 +40,96 @@ B_INST, E_INST = "[INST] ", " [/INST]" #Llama style
 prompt = f"{B_INST}{B_FUNC}{functionList.strip()}{E_FUNC}{user_prompt.strip()}{E_INST}\n\n"
 ```
 
-## Sample Prompt and Response:
+### Using tokenizer.apply_chat_template
+For an easier application of the prompt, you can set up as follows:
+
+Set up `messages`:
+```
+[
+    {
+        "role": "function_metadata",
+        "content": "FUNCTION_METADATA"
+    },
+    {
+        "role": "user",
+        "content": "What is the current weather in London?"
+    },
+    {
+        "role": "function_call",
+        "content": "{\n    \"name\": \"get_current_weather\",\n    \"arguments\": {\n        \"city\": \"London\"\n    }\n}"
+    },
+    {
+        "role": "function_response",
+        "content": "{\n    \"temperature\": \"15 C\",\n    \"condition\": \"Cloudy\"\n}"
+    },
+    {
+        "role": "assistant",
+        "content": "The current weather in London is Cloudy with a temperature of 15 Celsius"
+    }
+]
+```
+
+with `FUNCTION_METADATA` as:
+```
+[
+    {
+        "type": "function",
+        "function": {
+            "name": "get_current_weather",
+            "description": "This function gets the current weather in a given city",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "city": {
+                        "type": "string",
+                        "description": "The city, e.g., San Francisco"
+                    },
+                    "format": {
+                        "type": "string",
+                        "enum": ["celsius", "fahrenheit"],
+                        "description": "The temperature unit to use."
+                    }
+                },
+                "required": ["city"]
+            }
+        }
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "get_clothes",
+            "description": "This function provides a suggestion of clothes to wear based on the current weather",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "temperature": {
+                        "type": "string",
+                        "description": "The temperature, e.g., 15 C or 59 F"
+                    },
+                    "condition": {
+                        "type": "string",
+                        "description": "The weather condition, e.g., 'Cloudy', 'Sunny', 'Rainy'"
+                    }
+                },
+                "required": ["temperature", "condition"]
+            }
+        }
+    }    
+]
+```
+and then apply the chat template to get a formatted prompt:
+```
+tokenizer = AutoTokenizer.from_pretrained('Trelis/Llama-2-7b-chat-hf-function-calling-v3', trust_remote_code=True)
+
+prompt = tokenizer.apply_chat_template(prompt, tokenize=False)
+```
+If you are using a gated model, you need to first run:
+```
+pip install huggingface_hub
+huggingface-cli login
+```
+
+### Manual Prompt:
 ```
 [INST] You have access to the following functions. Use them if required: