philschmid
/

meta-llama-3-tokenizer

File size: 73 Bytes

e9c36ce

{
  "bos_token": "<|begin_of_text|>",
  "eos_token": "<|end_of_text|>"
}