CohereForAI

/

aya-expanse-8b

like 285

Follow

Cohere For AI 630

Hey @lucasjin , can you try again ? Just do "pip install transformers". The code snippet shared in our model card to run the model should work.

Hi, I found use_fast must set to False in AutoTokenizer from.

Can u teach me why is a must while 99% other models acutually not strict constraint with this.

(This actually the root reason, not tokenizer not import)

Hi @lucasjin

I just tried with

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereForAI/aya-expanse-8b"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format the message with the chat template
messages = [{"role": "user", "content": "Anneme onu ne kadar sevdiğimi anlatan bir mektup yaz"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Anneme onu ne kadar sevdiğimi anlatan bir mektup yaz<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>

gen_tokens = model.generate(
    input_ids,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.3,
    )

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

Note that I am explicitly using the fast tokenizer in tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True), which is True by default, and it worked. For reference, I am using transformers 4.45.2.

Can you share the complete stacktrace or error message you are getting? And what version of Transformers are you using?