private String linesToCharsMunge ( String text , List < String > lineArray , Map < String , Integer > lineHash ) { int lineStart = 0 ; int lineEnd = - 1 ; String line ; StringBuilder chars = new StringBuilder ( ) ; // Walk the text, pulling out a substring for each line. // text.split('\n') would would temporarily double our memory footprint. // Modifying text would create many large strings to garbage collect. while ( lineEnd < text . length ( ) - 1 ) { lineEnd = text . indexOf ( ' ' , lineStart ) ; if ( lineEnd == - 1 ) { lineEnd = text . length ( ) - 1 ; } line = text . substring ( lineStart , lineEnd + 1 ) ; lineStart = lineEnd + 1 ; if ( lineHash . containsKey ( line ) ) { chars . append ( String . valueOf ( ( char ) ( int ) lineHash . get ( line ) ) ) ; } else { lineArray . add ( line ) ; lineHash . put ( line , lineArray . size ( ) - 1 ) ; chars . append ( String . valueOf ( ( char ) ( lineArray . size ( ) - 1 ) ) ) ; } } return chars . toString ( ) ; }
Split a text into a list of strings . Reduce the texts to a string of hashes where each Unicode character represents one line .