MaziyarPanahi

/

Meta-Llama-3-8B-Instruct-GGUF

like 76

after 2? I am not sure, but if you support multi-turn then it should work as long as we have context length.

So you can test the model on HuggingFace.co/chat. After a few conversations, this is how the template looks like:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHi<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHi! It's nice to meet you. Is there something I can help you with or would you like to chat?<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWhat's your name?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nI don't have a personal name, but you can call me \"Assistant\" or \"Helper\" if you like. I'm here to assist you with any questions or tasks you may have, so feel free to think of me as your go-to helper!<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nMy name is Maziyar<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nNice to meet you, Maziyar! It's great to have a name to associate with our conversation. How's your day going so far? Is there anything on your mind that you'd like to talk about or ask for help with? I'm all ears!<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWhat's 25-4*2+3?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nTo evaluate this expression, we need to follow the order of operations (PEMDAS):\n\n1. Multiply 4 and 2: 4*2 = 8\n2. Subtract 8 from 25: 25 - 8 = 17\n3. Add 3 to 17: 17 + 3 = 20\n\nSo, 25-4*2+3 equals 20!<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nthanks<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n

I have also tested in LM Studio, it can go beyond 2 conversations.

I am asking too much of these AI systems, I suppose.-)

I have noticed with several AI that it is possible to have a conversation involving three people, for example: two humans and one AI. With systems like WizardLM-2, you can quite simply adapt the prompt and the AI recognizes that there are two humans. Example:

USER1: message from USER1
ASSISTANT: response to USER1
USER2: message from USER2
ASSISTANT: response to USER2
...

I haven't yet tested a three-way conversation with WizardLM-2, but other IA using the same type of prompt can achieve it.

I was also able to do something similar with Mixtral by specifying the conversation context as follows:

<s> [INST] Context definition (multi-user conversation) [/INST]
<|USER1|> message from USER1
<|ASSISTANT|> response to USER1
<|USER2|> message from USER2
<|ASSISTANT|> response to USER2
...

However, with Llama-3's prompt, I don't really see what to do.
In my idea, it would be necessary to replace "user" with the names of users (user1, user2) in "<|start_header_id|>user<|end_header_id|>". However, this is impossible because I would need to know in advance which one, USER1 or USER2, will send the next message to change their name in "<|start_header_id|>user<|end_header_id|>".

I don't know if my explanation is clear.

@tsalvoch more than 2 people conversation! That's interesting question and I think I have seen people fine-tuning Llama-3 models for that via ChatML template. There is a place they use now, in front of the role, you can add a name. Like user Maziyar etc.

This has to be done via a better prompt template. I see if I can find that model for this purpose, I'll share it here

Thank you @MaziyarPanahi .
Knowing that this kind of feature (multi-user) is not a real need, it's just that I have implemented a Chatbot in a chat application (Nextcloud Talk) which allows multi-user. The AI that I am currently using (Mixtral 8x7B) manages to get by. And while looking to integrate Llama-3, I realized that with their prompt it would be complicated.
I am afraid that fine-tuning to revise their prompt system may result in a loss of quality in the responses. But if a model comes out, I would be happy to test it.

Thank you @MaziyarPanahi .
Knowing that this kind of feature (multi-user) is not a real need, it's just that I have implemented a Chatbot in a chat application (Nextcloud Talk) which allows multi-user. The AI that I am currently using (Mixtral 8x7B) manages to get by. And while looking to integrate Llama-3, I realized that with their prompt it would be complicated.
I am afraid that fine-tuning to revise their prompt system may result in a loss of quality in the responses. But if a model comes out, I would be happy to test it.

I have fine-tuned the 8B-Instruct with a DPO, it has ChatML template if you want to try multi-user workaround to see if it was possible? https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-DPO-v0.1
It's a very interesting use case, I would love to see which template is the best and fine-tune the Instruct enough to follow the new template