AWQ-Variante
#2
by
SebastianBodza
- opened
Hallo,
wäre es möglich, dass Ihr eine AWQ Variante bereitstellt?
Danke schonmal
GGUF would also be nice :)
For now, you guys can also check out DiscoResearch/DiscoLM-70b, which is the Leo-70b model trained on mostly English instruction data, but it still speaks German very well. There are already quantization available for this.
For now, you guys can also check out DiscoResearch/DiscoLM-70b, which is the Leo-70b model trained on mostly English instruction data, but it still speaks German very well. There are already quantization available for this.
DiscoLM exhibits very strange behavior in German. Many words are duplicated. I cannot understand why. It seems like something goes wrong with token sampling.