\(\bullet\)_Byte-Pair Encoding (BPE) tokenization._ BPE was originally proposed as a general data compression algorithm in 1994 [221], and then adapted to NLP for tokenization [222]. It starts with a set of basic symbols (_e.g.,_ the alphabets and boundary characters), and iteratively combine frequent pairs of two consecutive tokens in the corpus as new tokens (called _merge_). For each merge, the selection criterion is based on the co-occurrence frequency of two contiguous tokens: the top frequent pair would be selected. The merge process continues until it reaches the predefined size. Further, Byte-level BPE has been used to improve the tokenization quality for multilingual corpus (_e.g.,_ the text containing non-ASCII characters) by considering _bytes_ as the basic symbols for merge. Representative language models with this tokenization approach include GPT-2, BART, and LLaMA.