General discussion.
may be a noob question, but how do i load the mmproj file in Ollama? I installed the gguf via Open WebUI, but the model insists it is not able to recognize images.
I have other multimodal models that run fine in Ollama, so i'm not 100% sure why this one isn't working.
I've never used ollama, but in Koboldcpp you just load it in the mmproj field of the file selector. Though tbh, this model is old as heck and probably not worth using (I created the model.) I'm pretty sure this was just a standard Llava Mistral 1.6 adapter and will underperform at most tasks.
ahh that would explain why the llm part gave such repetitive and wonky responses.
If you don't mind me asking,
Are you aware of any good multimodal models that are not censored? LLaVA is so overly censored it's actually frustrating to use, baklava is recommended but it gives some of the wall responses that make no sense. I'm genuinely surprised that i'm having such a difficult time finding an uncensored multimodal model that works with ollama.
minstrel:nemo has been great for chat, but i'm working on setting up a new service on my network that needs vision capabilities for tagging and OCR.
my GPU has 16GB vram so i'm usually good running models around 13B or a little higher since the gpu is dedicated to running llms. My biggest hurdle is that's is AMD on linux and the only docker containers i can find with built in AMD support is Ollama and LocalAI, anything else, including KoboldCPP would have to be manually compiled.