Update tokenizer
#11
by
alvarobartt
HF staff
- opened
Description
Updates both tokenizer.json
and special_tokens_map.json
as in https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct/commit/e04e3022cdc89bfed0db69f5ac1d249e21ee2d30
Closes #9
alvarobartt
changed pull request title from
Update special_tokens_map.json
to Update tokenizer
alvarobartt
changed pull request status to
merged