bartowski

/

LongWriter-llama3.1-8b-GGUF

like 22

looks like they updated the original model card, the tokenizer_config.json doesn't define one but they gave:

<>\n{system prompt}\n<>\n\n[INST]{query1}[/INST]{response1}[INST]{query2}[/INST]{response2}

gonna update the model card to match

emmmm, yor template didnt work, heres my, seems ok

TEMPLATE """
{{- range $i, $_ := .Messages }}
{{- if eq .Role "user" }}
{{- if and $.Tools (le (len (slice $.Messages $i)) 2) }}[AVAILABLE_TOOLS] {{ $.Tools }}[/AVAILABLE_TOOLS]
{{- end }}[INST] {{ if and $.System (eq (len (slice $.Messages $i)) 1) }}{{ $.System }}

{{ end }}{{ .Content }}[/INST]
{{- else if eq .Role "assistant" }}
{{- if .Content }} {{ .Content }}{{ if not (eq (len (slice $.Messages $i)) 1) }}{{ end }}
{{- else if .ToolCalls }}[TOOL_CALLS] [
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}]
{{- end }}
{{- else if eq .Role "tool" }}[TOOL_RESULTS] {"content": {{ .Content }}} [/TOOL_RESULTS]
{{- end }}
{{- end }}
"""

PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"

oh apparently HF hid some of my tags

it's meant to be <<SYS>>\n{system prompt}\n<</SYS>>\n\n[INST]{query1}[/INST]{response1}[INST]{query2}[/INST]{response2}

whether that works or not, I couldn't say, I don't like when models don't define a template in their tokenizer_config.json

whether that works or not, I couldn't say, I don't like when models don't define a template in their tokenizer_config.json

I second to that, thx bro