Update special_tokens_map.json
#10
by
fxmarty
- opened
Following https://github.com/huggingface/transformers/pull/23909 it appears that this filed is relied on for special tokens. Not sure how it works internally, but I am not getting a 2 token at the beginning of a sentence with transformers>=4.34 instead of the rightful 1 before.
This update fixes the issue. Other models may be affected.
Ooh, base model must have misconfigured that. Thanks for the fix!
TheBloke
changed pull request status to
merged