CatGPT-IT / added_tokens.json
baiges's picture
Upload tokenizer
7762769 verified
raw
history blame
193 Bytes
{
"<fim_end>": 32773,
"<fim_middle>": 32772,
"<fim_prefix>": 32771,
"<im_end>": 32770,
"<im_start>": 32769,
"<|endoftext|>": 32768,
"<|im_end|>": 32775,
"<|im_start|>": 32774
}