charlesxsh
/

custom-model

Model card Files Files and versions Community

custom-model / tokenizer_config.json

charlesxsh's picture

yo

dcf70a0 2 months ago

138 Bytes

	{
	"auto_map": {
	"AutoTokenizer": ["custom_tokenizer.CustomTokenizer", null]
	},
	"tokenizer_class": "CustomTokenizer"

	}