<commit_before><?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE topic PUBLIC "-//OASIS//DTD DITA Topic//EN" "topic.dtd">
<topic id="how_it_works">
  <title>How It Works</title>
  <body>
    <p>Saxon 9 supports custom collators through the use of custom collator URIs that are resolved
      through a custom collation URI resolver.</p>
    <p>Custom collation URI resolvers are configured on Saxon by transformer factories.</p>
    <p>Thus the i18n library provides a custom transformer factory that returns Saxon configured
      with the i18n library's custom collation URI resolver.</p>
    <p>When Saxon encounters a <xmlatt>collation</xmlatt> attribute on
        <xmlelement>xsl:sort</xmlelement> it calls the configured collation URI resolvers to resolve
      the URI. The i18n library's URI resolver then returns the custom Java zhCnAwareCollator, which
      implements Saxon's collation-related APIs, which are in turn used by Saxon's implementation of
      the <xmlelement>xsl:sort</xmlelement> element.</p>
    <p>The ZhCnAwareCollator integrates the ICU4J libraries for doing locale-aware collation
      generally and then provides a Simplified-Chinese-specific collator that uses the CEDICT
      dictionary along with the ICU4J zh-CN word breaker to construct pinyin sort keys from
      Simplified Chinese text.</p>
    <p>The i18n Java code also provides two Java classes that serve as Saxon 9.1 extension function
      libraries: <codeph>org.ditacommunity.i18n.textanalysis.TextAnalyzer</codeph> and
        <codeph>org.ditacommunity.i18n.textanalysis.TextMetrics</codeph>. These libraries, along
      with the ZhCnAwareCollator, back the functions in the <filepath>i18n-utils.xsl</filepath>
      module.</p>
  </body>
</topic><commit_msg>Added details about service configuration<commit_after><?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE topic PUBLIC "-//OASIS//DTD DITA Topic//EN" "topic.dtd">
<topic id="how_it_works">
  <title>How It Works</title>
  <body>
    <p>Saxon 9 supports custom collators through the use of custom collator URIs that are resolved
      through a custom collation URI resolver.</p>
    <p>Custom collation URI resolvers are configured on Saxon by transformer factories.</p>
    <p>Thus the i18n library provides a custom transformer factory that returns Saxon configured
      with the i18n library's custom collation URI resolver. </p>
    <p>The i18n jar includes a services configuration that declares that the jar includes a JAXP
      transformer factory. As long as the i18n jar comes before Saxon's jar in the JAVA class path,
      the custom transformer factory will be used and the collator will be available to Saxon.</p>
    <p>When Saxon encounters a <xmlatt>collation</xmlatt> attribute on
        <xmlelement>xsl:sort</xmlelement> it calls the configured collation URI resolvers to resolve
      the URI. The i18n library's URI resolver then returns the custom Java zhCnAwareCollator, which
      implements Saxon's collation-related APIs, which are in turn used by Saxon's implementation of
      the <xmlelement>xsl:sort</xmlelement> element.</p>
    <p>The ZhCnAwareCollator integrates the ICU4J libraries for doing locale-aware collation
      generally and then provides a Simplified-Chinese-specific collator that uses the CEDICT
      dictionary along with the ICU4J zh-CN word breaker to construct pinyin sort keys from
      Simplified Chinese text.</p>
    <p>The i18n Java code also provides two Java classes that serve as Saxon 9.1 extension function
      libraries: <codeph>org.ditacommunity.i18n.textanalysis.TextAnalyzer</codeph> and
        <codeph>org.ditacommunity.i18n.textanalysis.TextMetrics</codeph>. These libraries, along
      with the ZhCnAwareCollator, back the functions in the <filepath>i18n-utils.xsl</filepath>
      module.</p>
  </body>
</topic>
{'fraction_non_alphanumeric': 0.07660347961568424, 'fraction_numerical': 0.012464294988314724, 'mean_word_length': 4.002597402597402, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 2, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 14}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2541868', 'n_tokens_mistral': 1184, 'n_tokens_neox': 1105, 'n_words': 429}