Add <|im_start|> as a special token to tokenizer_config.json (#4)

- Add <|im_start|> as a special token to tokenizer_config.json (d20f49278c006877741954898e3e4e546ea2b5b8)


Co-authored-by: Bartowski <[email protected]>

tokenizer_config.json

@@ -27,6 +27,14 @@
       "single_word": false,
       "special": true
     },
+    "6": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
     "7": {
       "content": "<|im_end|>",
       "lstrip": false,