<commit_before>USING: help.syntax help.markup strings byte-arrays math.order ;
IN: unicode.collation

ARTICLE: "unicode.collation" "Collation and weak comparison"
"The " { $vocab-link "unicode.collation" "unicode.collation" } " vocabulary implements the Unicode Collation Algorithm. The Unicode Collation Algorithm (UTS #10) forms a reasonable way to sort strings when accounting for all of the characters in Unicode. It is far preferred over code point order when sorting for human consumption, in user interfaces. At the moment, only the default Unicode collation element table (DUCET) is used, but a more accurate collation would take locale into account. The following words are useful for collation directly:"
{ $subsections
    sort-strings<commit_msg>unicode.collation: fix typo in docs.<commit_after>USING: help.syntax help.markup strings byte-arrays math.order ;
IN: unicode.collation

ARTICLE: "unicode.collation" "Collation and weak comparison"
"The " { $vocab-link "unicode.collation" } " vocabulary implements the Unicode Collation Algorithm. The Unicode Collation Algorithm (UTS #10) forms a reasonable way to sort strings when accounting for all of the characters in Unicode. It is far preferred over code point order when sorting for human consumption, in user interfaces. At the moment, only the default Unicode collation element table (DUCET) is used, but a more accurate collation would take locale into account. The following words are useful for collation directly:"
{ $subsections
    sort-strings
{'fraction_non_alphanumeric': 0.06188281764318631, 'fraction_numerical': 0.0026333113890717576, 'mean_word_length': 5.255144032921811, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7650366', 'n_tokens_mistral': 403, 'n_tokens_neox': 369, 'n_words': 192}