kgsf-rec-new / tokenizer_config.json
ZhankuiHe's picture
Upload tokenizer
fffc58c verified
raw
history blame contribute delete
No virus
166 Bytes
{
"tokenizers": {
"item": "EntityTokenizer",
"entity": "KGSFBaseTokenizer",
"concept": "KGSFBaseTokenizer"
},
"tokenizer_key_for_decoding": "item"
}