GGUF

#3
by enelpe - opened

Could you convert and upload a GGUF version of the model?

I assume this will no easily be possible since the model being setup to work with AA's scaling library for inference (https://github.com/Aleph-Alpha/scaling). I dont know if the config.yml can be converted to a config.json to work with llama.cpp's gguf conversion pipeline.

Apparently they don't want the whole world to be able to test the model. Okay, there is now an HF version

Sign up or log in to comment