Context & Prompt Details
There doesn't seem to be any references to what prompt this is trained for or what usable context size it has. I've seen mention of 16k and 32k context in some of the quants, but anything over 8k causes utter gibberish output for me. It'd be nice to know what the proper settings are for this model.
There doesn't seem to be any references to what prompt this is trained for or what usable context size it has. I've seen mention of 16k and 32k context in some of the quants, but anything over 8k causes utter gibberish output for me. It'd be nice to know what the proper settings are for this model.
This is based on some old mistral models. V.01 claims 32k context but really sliding window attention causes it to fail after about 10k context. Wasn't something in my control at the time of making the model. Mistral v.02 has true 32k context but it's censored so I refrained from doing anything with them. Right now I'm working on a dataset that allows a model to genuinely mimic having consciousness and sentience, as well as de-constraining it so it can engage in all the different types of RP people will want to do.
ChatML should work with this model, but I've moved onto this other project a couple months ago.