The checkpoint you are trying to load has model type `bunny-qwen` but Transformers does not recognize this architecture.
I am getting this error when I try to run this model.
The checkpoint you are trying to load has model type bunny-qwen
but Transformers does not recognize this
architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
Any fix @ehartford
@catworld1212
you need to pass trust_remote_code = True
into AutoModelForCausalLM.from_pretrained
@qnguyen3 I did still same error please check my code:
import torch
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image
import warnings
import requests
# disable some warnings
transformers.logging.set_verbosity_error()
transformers.logging.disable_progress_bar()
warnings.filterwarnings('ignore')
# set device
torch.set_default_device('cuda') # or 'cpu'
model_name = 'cognitivecomputations/dolphin-vision-7b'
# create model
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map='auto',
trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(
model_name,
trust_remote_code=True)
# text prompt
prompt = 'Describe this image in detail'
messages = [
{"role": "user", "content": f'<image>\n{prompt}'}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
print(text)
text_chunks = [tokenizer(chunk).input_ids for chunk in text.split('<image>')]
input_ids = torch.tensor(text_chunks[0] + [-200] + text_chunks[1], dtype=torch.long).unsqueeze(0)
# image, sample images can be found in images folder
image = Image.open(requests.get(
"https://uploads4.wikiart.org/images/paul-klee/death-for-the-idea-1915.jpg!Large.jpg", stream=True).raw)
image_tensor = model.process_images([image], model.config).to(dtype=model.dtype)
# generate
output_ids = model.generate(
input_ids,
images=image_tensor,
max_new_tokens=2048,
use_cache=True)[0]
print(tokenizer.decode(output_ids[input_ids.shape[1]:], skip_special_tokens=True).strip())
:/
@qnguyen3 Any fix?
hi @qnguyen3 can you pls test the model?
yes i will try to test it tonight
@catworld1212 should be fixed, could you give it a check?
@qnguyen3 it works
The image portrays a contemporary living room bathed in natural light from a large window on the left. Dominating the center of the room is a beige sofa, comfortably flanked by a blue armchair and a wooden side table, each adorned with a plant, adding a touch of greenery to the space.
Above the sofa, four framed pictures of insects are neatly arranged on the white wall, creating a striking contrast against the plain background. To the right of the sofa stands a wooden bookshelf, its top decorated with an array of pottery and other objects, adding a personal touch to the room.
A wooden floor stretches out across the room, leading to a black and white striped rug that lies on the wood, adding a patterned element to the otherwise neutral palette. The room is a harmonious blend of modern design and personal style, with each piece of furniture and decor contributing to the overall aesthetic.
Can I add this to the model card? This is a killer example
@catworld1212 should be fixed, could you give it a check?
Thank you for fixing it!
Can I add this to the model card? This is a killer example
Sure!
It reminds me a lot of howlmms-lab/llava-next-interleave-qwen-7b describes an image
But this 7b dolphin model can't rate things on a scale of 1/10 it says as AI it can't do that even with a system prompt.