private static String buildText ( String text ) { // <JAR><JAR> to PARAGRAPH mark in unicode text = doubleLineBreak . matcher ( text ) . replaceAll ( PARAGRAPH ) ; // <JAR> to " " text = lineBreak . matcher ( text ) . replaceAll ( " " ) ; return text ; }
Builds the text for Segmentation and Tokenization . This is original text namely the string used to calculate the character offsets .