FinTwitBERT / added_tokens.json
StephanAkkerman's picture
Improved tokenizer
88e3087
raw
history blame
118 Bytes
{
"@USER": 30522,
"[CLS]": 101,
"[MASK]": 103,
"[PAD]": 0,
"[SEP]": 102,
"[UNK]": 100,
"[URL]": 30523
}