i got messy code
when i say hello , i got the response:
'Hittahnaire Bobbiehl ArchitectureadenProfileottishingly improv laytex enforcement prepasatabsicile KaisTAGozhovLS dol catchphrdupaghframagliakerzd kle Cop JupLCarpbasguardaviahsisto GallStatnisse Murphybladexexecuteigaaescloudflare Controllerignon SeverRORza城abe mu Cov Routexlburyennenobaitschaval SomCODEslug hur Days macROR BartlettenbergDelegate dispar Domin Hun Хронологија Venyept atmospher synchronousnesses Ricakre tre Ford IB regressionmerceTube settledPU Bassimerudi Arab instilsquieruder Roth conscienceatorinelblattgeometryashi Eg Lub Oaklynsel interpolpec diag county Ci Kidweis Cultbuff paintings Limpossible conformityerneriels Elania communes Bah Jenkinsagan Lap Gilbert Lem Tor fratern SahGV MunimiWAYS Alice Гор pairogegebrazeugh inventory Foreign Risks представ soapSBN descruttyped Ble bod prob SchlesikelustaVB Patrick nominationardoyter rows ej ké RevoløLayout'
i know that i should down the gptq, but i had have it in my repositories
AFAIK this happens when you are trying to run an 4-bit model in 8-bit. You should be able to either set it in the model panel of web-ui when loading the model itself, or when starting up the web-ui itself, e.g.: by running
python server.py --model TheBloke_WizardLM-30B-Uncensored-GPTQ --wbits 4 --groupsize 128
If you're running on the command line it's --groupsize -1 - for no groupsize. groupsize 128 will give errors.
@Tiankong2023 what version of GPTQ-for-LLaMa do you have?
Is Wizard supported by ooba? It is not LLaMA based mode, right?
WizardLM models are Llama based, yes. And yes this model is supported in text-generation-webui - there's instructions in the README for that. And I recommend using the ooba CUDA fork of GPTQ-for-LLaMa at this time.