AWQ-Variante

#2
by SebastianBodza - opened

Hallo,

wäre es möglich, dass Ihr eine AWQ Variante bereitstellt?

Danke schonmal

GGUF would also be nice :)

LAION LeoLM org

@TheBloke Can you please add this model (LeoLM/leo-hessianai-70b-chat) to your quantization queue? :)

LAION LeoLM org

For now, you guys can also check out DiscoResearch/DiscoLM-70b, which is the Leo-70b model trained on mostly English instruction data, but it still speaks German very well. There are already quantization available for this.

For now, you guys can also check out DiscoResearch/DiscoLM-70b, which is the Leo-70b model trained on mostly English instruction data, but it still speaks German very well. There are already quantization available for this.

DiscoLM exhibits very strange behavior in German. Many words are duplicated. I cannot understand why. It seems like something goes wrong with token sampling.

Sign up or log in to comment