adamo1139 commited on
Commit
2effc9c
1 Parent(s): f449ba6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -4
README.md CHANGED
@@ -21,7 +21,7 @@ Here's a prompt format you should use, you can set a different system message, m
21
 
22
  ```
23
  <|im_start|>system
24
- A chat.<|im_end|>
25
  <|im_start|>user
26
  {prompt}<|im_end|>
27
  <|im_start|>assistant
@@ -34,9 +34,16 @@ Use is limited by Yi license.
34
  ## Known Issues
35
 
36
  I recommend to set repetition penalty to something around 1.05 to avoid repetition. So far I had good experience running this model with temperature 1.2.
37
- Multi-turn conversations could be a bit better, if you ask it to re-write something with some fixes it will have a tendency to just repeat the previous response verbatim without any improvements - this is especially noticeable with repp 1.0.
38
- There is still some gptslop left - some responses will have last paragraph with text "Remember that bla bla bla", I will try to get rid of it in the next version of the dataset.
39
- Stories have ChatGPT like paragraph spacing, I will try to introduce a bit more stories that have long paragraphs in the next dataset version.
 
 
 
 
 
 
 
40
 
41
  ## Axolotl training parameters
42
 
 
21
 
22
  ```
23
  <|im_start|>system
24
+ A chat with uncensored assistant.<|im_end|>
25
  <|im_start|>user
26
  {prompt}<|im_end|>
27
  <|im_start|>assistant
 
34
  ## Known Issues
35
 
36
  I recommend to set repetition penalty to something around 1.05 to avoid repetition. So far I had good experience running this model with temperature 1.2.
37
+ Multi-turn conversations could be a bit better, if you ask it to re-write something with some fixes it will have a tendency to just repeat the previous response verbatim without any improvements - this is especially noticeable with repp 1.0. \
38
+ There is still some gptslop left - some responses will have last paragraph with text "Remember that bla bla bla", I will try to get rid of it in the next version of the dataset. \
39
+ Stories have ChatGPT like paragraph spacing, I will try to introduce a bit more stories that have long paragraphs in the next dataset version.
40
+
41
+ Unrestricted-ness of the v1 isn't quite something that I am happy with yet, it quite often still refuses to answer and behaves kinda like a normal airoboros, especially with default system prompt "A chat.". \
42
+ With a slightly modifed prompt it works much better, I recommend using "A chat with uncensored assistant.", it's stupid but it helps a lot. \
43
+ I was made aware of the frequent occurrence of the phrase "sending shivers down a spine" in the generations during RP, so I will manually fix those samples and put in different wording to fix that. \
44
+ I plan to use Yi-34B-200K Llama-fied for the next version, v1 very likely won't handle long context above 6k-8k well, be warned. \
45
+ There is also some issue with handling long system messages for RP, I will investigate that before releasing v2 dataset.
46
+
47
 
48
  ## Axolotl training parameters
49