llama3-8b-wiki2-mixed-lang-sentence / tokenizer_config.json

Commit History

Upload tokenizer
5d6e7f2
verified

yangsibo commited on