Byte Pair Encoding (BPE) [1]} is a simple data compression technique that iteratively replaces the most frequent pair of bytes in a sequence with a single, unused byte.
We adapt this algorithm for word segmentation.
Instead of merging frequent pairs of bytes, we merge characters or character sequences.
