Upload tokenizer.json

#2
by vrdn23 - opened
Cisco org

New tokenizer.json that converts to lowercase and splits at a character level.

vrdn23 changed pull request status to merged

Sign up or log in to comment