Subword tokenization.This class of algorithms subsumes all data-driven tokenization approaches which can decompose words into subwords/multiple tokens and currently represent the established tokenization approach upon which LLMs rely (Kudo and Richardson, 2018; Petrov et al., 2023). Because subword tokenizers decompose words into subwords, they can process out-of-vocabulary words by merging subwords from the vocabulary (Kudo and Richardson, 2018). Examples of popular subword tokenizers are WordPiece (Schuster and Nakajima, 2012), BPE (Gage, 1994; Sennrich et al., 2015), Byte-Level BPE (BBPE) (Wang et al., 2020), and Unigram (Kudo, 2018).