Systems that are most efficient at restoring punctuations usually exploit both prosodic and lexical features with hybrid models [1]}, [2]}, [3]}.
Up until the appearance of BERT-like models, lexical features were primarily processed by recurrent neural networks [4]}, [5]}, [6]}, [7]}, [8]}, while more recent approaches use the transformer [9]} architecture [10]}, [11]}, [12]}.
The current state-of-the-art method by [13]} is a pretrained BERT, which aggregates multiple predictions for the same token, resulting in higher accuracy and significant parallelism.
