Hugging Face's logo

merve
/
google-tokenizers

Model card Files Community
merve HF staff commited on
Commit
353c896
•
1 Parent(s): 0077a7b

Upload 5 files

Browse files
Files changed (5) hide show
  1. c4_en/sentencepiece.model +3 -0
  2. cc_all/sentencepiece.model +3 -0
  3. cc_all_extra/sentencepiece.model +3 -0
  4. mc4/sentencepiece.model +3 -0
  5. mc4_extra/sentencepiece.model +3 -0
c4_en/sentencepiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e5036bed065526c3c212dfbe288752391797c4bb1a284aa18c9a0b23fcaf8ec
3
+ size 798330
cc_all/sentencepiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
3
+ size 791656
cc_all_extra/sentencepiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:839ffa4b9afae8d77834a88b87781849aa021975d6063dec6085633fcaf7171c
3
+ size 794346
mc4/sentencepiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef78f86560d809067d12bac6c09f19a462cb3af3f54d2b8acbba26e1433125d6
3
+ size 4309802
mc4_extra/sentencepiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef78f86560d809067d12bac6c09f19a462cb3af3f54d2b8acbba26e1433125d6
3
+ size 4309802