mistralai/Mixtral-8x22B-Instruct-v0.1

Apr 19

•

Hi everyone,

I tried running the examples from the readme and those did not appear to be working correctly.

Can someone please confirm whether https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1/commit/ebb919ac9e9f7f9a900644621bae7963bc593f4f is indeed correct?

I'm using the latest versions of transformers/tokenizers:
transformers-4.40.0-py3-none-any.whl
tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Thank you!

Jofthomas

Apr 20

Hey, which example are you trying to run ?

I updated the tokenizer_config and I'll update the readme

Can you try the following code ( only works with transformer>4.39.0):

from transformers import AutoTokenizer

# Initialize model ID and tokenizer
model_id = "mistralai/Mixtral-8x22B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Define a conversation example
conversation = [
    {"role": "user", "content": "What's the weather like in Paris?"},
    {"role": "assistant", "content": "The current temperature in Paris, France is 22 degrees Celsius."},
    {"role": "user", "content": "What about San Francisco?"}
]

# Generate the tool use prompt using the tokenizer
tool_use_prompt = tokenizer.apply_chat_template(
    conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Print the generated prompt
print(tool_use_prompt)

sophiamyang changed discussion status to closed Apr 24

mistralai
/

Mixtral-8x22B-Instruct-v0.1

Examples correct?