AutoModelForCausalLM.from_pretrained("liuhaotian/llava-v1.6-vicuna-7b") fail
when i use model = AutoModelForCausalLM.from_pretrained("liuhaotian/llava-v1.6-vicuna-7b"),i got
Unrecognized configuration class <class 'transformers.models.llava.configuration_llava.LlavaConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Same Issue
This version only allows loading via the code in the github repository (https://github.com/LLaVA-VL/LLaVA-NeXT) and not via the transformers.
https://huggingface.co/llava-hf/llava-v1.6-vicuna-7b-hf Here is the hf version of the model weights, which can be loaded directly via AutoModelForCausalLM.
For another approach, refer to https://github.com/huggingface/transformers/blob/main/src/transformers/models/llava_next/convert_llava_next_weights_to_hf.py converts this version of weights to the hf version.