Loading gguf model for inference

by Rasi1610 - opened Jun 11

Jun 11

I'm new to this Quantized model ,Could anyone provide me the code for how to load this .gguf model and inference.
It would be really helpful

Jul 8

use ollama

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment