# What does this PR do?
Argument name is totally debatable. Will also require a pull request in `tokenizers`. 
The goal is to be able to simply activate and de-activate the special token splitting. Feature was asked in #22490, and is required for some production type cases, where users pass inputs and we don't want them to be able to hack them
[`split_special_tokens`] Add support for `split_special_tokens` argument to encode. # What does this PR do?
Argument name is totally debatable. Will also require a pull request in `tokenizers`. 
The goal is to be able to simply activate and de-activate the special token splitting. Feature was asked in #22490, and is required for some production type cases, where users pass inputs and we don't want them to be able to hack them
xuRɎ@̙h)DVbnh&mߚfY`$פNOWŪZ"QhX	j"%Țb@(&KBMyT CB鐉(Rj,0L(LzoӊhF%Ӓe
aT-)hvΪ'5]DX#q06=/eţ|K69KĴ%XY+6^aaڹոlBMsu۽a`9
ӭg=hSt8<[fa83c[͎+[U5Y烹nGn]G7ݦ/vDbTiu1;d$݊ڶt~Uyu4Hy	Fzz\k
uَO2TP¶&NgMg<sDf?s`=\z(si믿jw)٧7f\BXiUIvX`z
y`\<:E|?!ůxuL;1ab%zQ.MVgIH*̗3
xuRɎ@̙h)DVbnh&mߚfY`$פNOWŪZ"QhX	j"%Țb@(&KBMyT CB鐉(Rj,0L(LzoӊhF%Ӓe
aT-)hvΪ'5]DX#q06=/eţ|K69KĴ%XY+6^aaڹոlBMsu۽a`9
ӭg=hSt8<[fa83c[͎+[U5Y烹nGn]G7ݦ/vDbTiu1;d$݊ڶt~Uyu4Hy	Fzz\k
uَO2TP¶&NgMg<sDf?s`=\z(si믿jw)٧7f\BXiUIvX`z
y`\<:E|?!ůxuL;1ab%zQ.MVgIH*̗3