Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /glm_chinese

2 contributors

History: 2 commits

xu-song's picture

add compress rate

814ee6b 7 months ago

chinese_sentencepiece
update about 1 year ago
README.md

487 Bytes

update about 1 year ago
__init__.py

1.34 kB

add compress rate 7 months ago
convert_vocab_to_txt.py

689 Bytes

update about 1 year ago
file_utils.py

8.38 kB

update about 1 year ago
glm_chinese.vocab.txt

659 kB

update about 1 year ago
sp_tokenizer.py

4.67 kB

update about 1 year ago
test.py

115 Bytes

add compress rate 7 months ago
test_glm.py

2.5 kB

update about 1 year ago
tokenization.py

51.9 kB

update about 1 year ago
tokenization_gpt2.py

13.5 kB

update about 1 year ago
utils.py

213 Bytes

update about 1 year ago
wordpiece.py

15.5 kB

update about 1 year ago