rhysjones

/

phi-2-orange-v2

like 27

Hey @moc1pher , thanks for noticing this. The model is quite sensitive to the prompt format and system message, so I suspect it's related to that.

One suggestion for Text Gen:
After loading the model, if you go to the Parameters -> Instruction Template tab, and then paste in the following in the "Instruction template" text area:

{%- set ns = namespace(found=false) -%}
{%- for message in messages -%}
    {%- if message['role'] == 'system' -%}
        {%- set ns.found = true -%}
        {%- set ns.system = message['content'].rstrip() -%}
    {%- endif -%}
{%- endfor -%}
{%- for message in messages %}
    {%- if message['role'] != 'system' -%}
      {%- if ns.found and loop.last -%}
        {{- '<|im_start|>system\n' + ns.system + '<|im_end|>\n' -}}
      {%- endif -%}
        {%- if message['role'] == 'user' -%}
            {{-'<|im_start|>user\n' + message['content'].rstrip() + '<|im_end|>\n'-}}
        {%- else -%}
            {{-'<|im_start|>assistant\n' + message['content'] + '<|im_end|>\n' -}}
        {%- endif -%}
    {%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
    {{-'<|im_start|>assistant\n'-}}
{%- endif -%}

and also put a System prompt in the "Custom system message" above it, e.g. something like:

You are a helpful assistant. Reply in a short and direct manner.

You can test it's all working by clicking on the "Send to notebook" button - you should see a very basic ChatML prompt there. (If it shows Error, then re-try copying and pasting the above, or let me know). If it's all good, you can then click back to the Parameters -> Instruction Template tab and click on the Save icon, calling it e.g. phi-2-orange-2.yaml, so that you can load it back again at any time later.

Then, when you're on the top main Chat tab, selecting the "Instruct" option at the very bottom should make the model follow the above ChatML prompt format with the system prompt that you gave.