The original BERT implementation<cite> (Devlin et al., 2019)</cite> uses a character-level BPE vocabulary of size 30K, which is learned after preprocessing the input with heuristic tokenization rules.