brittlewis12

/

SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF

like 1

@Roronin hey there, thanks for the heads up!

I created these from b2879 in llama.cpp, but I don't immediately see any recent changes that might've caused this. Will investigate further and let you know if i find anything.

I tested the file against the same version of llama.cpp in terms of inference and it seems to work fine — can you share any more detail about that Modelfile you're using? I'm not super familiar with ollama's format or what it encodes.

@Roronin Thanks for the edit! Will further look into the IQ4_NL (& perhaps other I-quant) variant(s)!

@Roronin was able to run the IQ4_NL no problem, even using another llama.cpp client (my app, cnvrs) which is currently based on b2789 from May 4, 2024.