Spaces:

xu-song
/

tokenizer-arena

Running

update

751936e about 1 year ago

105 Bytes

	"""

	SentencePiece(unigram)

	https://huggingface.co/docs/transformers/tokenizer_summary#sentencepiece
	"""