where the output size of \(FFN_{punc}\)  layer is the number of punctuation label.
The punctuation prediction layer feeds output vector \(s_i\)  into a conditional random field layer [1]}.
Negative log likelihood loss \(\mathcal {L}_{punc}\)  is used for punctuation prediction task optimization.
