Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /baichuan2

2 contributors

History: 2 commits

xu-song's picture

add more tokenizers

f4973d4 12 months ago

Baichuan2-7B-Chat
add more tokenizers 12 months ago
__init__.py

597 Bytes

add more tokenizers 12 months ago
bug.py

797 Bytes

add more tokenizers 12 months ago
encode.py

672 Bytes

add more tokenizers 12 months ago
special_token.py

486 Bytes

add more tokenizers 12 months ago