<commit_before><sect1 id="zend.search.lucene.query-language">
    <title>Query Language</title>

    <para>
        Java Lucene and Zend_Search_Lucene provide quite powerful query languages.
    </para>

    <para>
        These languages are mostly the same with some minor differences, which are mentioned below.
    </para>

    <para>
        Full Java Lucene query language syntax documentation can be found
        <ulink url="http://lucene.apache.org/java/2_0_0/queryparsersyntax.html">here</ulink>.
    </para>

    <sect2 id="zend.search.lucene.query-language.terms">
        <title>Terms</title>

        <para>
            A query is broken up into terms and operators. There are three types of terms: Single Terms, Phrases,
            and Subqueries.
        </para>
        <para>
            A Single Term is a single word such as "test" or "hello".
        </para>
        <para>
            A Phrase is a group of words surrounded by double quotes such as "hello dolly".
        </para>
        <para>
            A Subquery is a query surrounded by parentheses such as "(hello dolly)".
        </para>
        <para>
            Multiple terms can be combined together with boolean operators to form complex queries (see below).
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.fields">
        <title>Fields</title>

        <para>
            Lucene supports fields of data. When performing a search you can either specify a field, or use
            the default field. The field names depend on indexed data and default field is defined
            by current settings.
        </para>
        <para>
            The first and most significant difference from Java Lucene is that terms are searched through
            <emphasis>all fields</emphasis> by default.
        </para>
        <para>
            There are two static methods in the Zend_Search_Lucene class which allow the developer to configure these settings:
        </para>
        <programlisting role="php"><![CDATA[<?php
$defaultSearchField = Zend_Search_Lucene::getDefaultSearchField();
...
Zend_Search_Lucene::setDefaultSearchField('contents');]]>
        </programlisting>

        <para>
            The <code>null</code> value indicated that the search is performed across all fields. It's the default setting.
        </para>
        <para>
            You can search specific fields by typing the field name followed by a colon ":" followed by the term you
            are looking for.
        </para>
        <para>
            As an example, let's assume a Lucene index contains two fields- title and text- with text as the default field.
            If you want to find the document entitled "The Right Way" which contains the text "don't go this way",
            you can enter:
        </para>
        <programlisting role="querystring"><![CDATA[title:"The Right Way" AND text:go]]>
        </programlisting>
        <para>
            or
        </para>
        <programlisting role="querystring"><![CDATA[title:"Do it right" AND go]]>
        </programlisting>
        <para>
            Because "text" is the default field, the field indicator is not required.
        </para>

        <para>
            Note: The field is only valid for the term, phrase or subquery that it directly precedes,
            so the query
            <programlisting role="querystring"><![CDATA[title:Do it right]]>
            </programlisting>
            Will only find "Do" in the title field. It will find "it" and "right" in the default field (if the default field is set)
            or in all indexed fields (if the default field is set to <code>null</code>).
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.wildcard">
        <title>Wildcards</title>
        <para>
            Lucene supports single and multiple character wildcard searches within single terms (but not within phrase queries).
        </para>

        <para>
            To perform a single character wildcard search use the "?" symbol.
        </para>

        <para>
            To perform a multiple character wildcard search use the "*" symbol.
        </para>

        <para>
            The single character wildcard search looks for string that match the term with the "?" replaced by any single character.
            For example, to search for "text" or "test" you can use the search:
            <programlisting role="querystring"><![CDATA[te?t]]>
            </programlisting>
        </para>

        <para>
            Multiple character wildcard searches look for 0 or more characters when matching strings against terms. For example, to search for test,
            tests or tester, you can use the search:
            <programlisting role="querystring"><![CDATA[test*]]>
            </programlisting>
        </para>

        <para>
            You can use "?", "*" or both at any place of the term:
            <programlisting role="querystring"><![CDATA[*wr?t*]]>
            </programlisting>
            It searches for "write", "wrote", "written", "rewrite", "rewrote" and so on.
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.modifiers">
        <title>Term Modifiers</title>

        <para>
            Lucene supports modifying query terms to provide a wide range of searching options.
        </para>
        <para>
            Zend_Search_Lucene currently supports the "~" modifier only for phrases.
            <footnote>
                <para>
                    The "~" modifier for terms is used for fuzzy search, but fuzzy search queries are not currently supported.
                </para>
            </footnote>
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.range">
        <title>Range Searches</title>

        <para>
            Range queries allow the developer or user to match documents whose field(s) values are between the lower and upper bound specified by the range query.
            Range Queries can be inclusive or exclusive of the upper and lower bounds. Sorting is performed lexicographically.
            <programlisting role="querystring"><![CDATA[mod_date:[20020101 TO 20030101]]]><commit_msg>Zend_Search_Lucene: * documentation fix (wildcard and range queries are marked as available with SVN version). Closes [ZF-2202].<commit_after><sect1 id="zend.search.lucene.query-language">
    <title>Query Language</title>

    <para>
        Java Lucene and Zend_Search_Lucene provide quite powerful query languages.
    </para>

    <para>
        These languages are mostly the same with some minor differences, which are mentioned below.
    </para>

    <para>
        Full Java Lucene query language syntax documentation can be found
        <ulink url="http://lucene.apache.org/java/2_0_0/queryparsersyntax.html">here</ulink>.
    </para>

    <sect2 id="zend.search.lucene.query-language.terms">
        <title>Terms</title>

        <para>
            A query is broken up into terms and operators. There are three types of terms: Single Terms, Phrases,
            and Subqueries.
        </para>
        <para>
            A Single Term is a single word such as "test" or "hello".
        </para>
        <para>
            A Phrase is a group of words surrounded by double quotes such as "hello dolly".
        </para>
        <para>
            A Subquery is a query surrounded by parentheses such as "(hello dolly)".
        </para>
        <para>
            Multiple terms can be combined together with boolean operators to form complex queries (see below).
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.fields">
        <title>Fields</title>

        <para>
            Lucene supports fields of data. When performing a search you can either specify a field, or use
            the default field. The field names depend on indexed data and default field is defined
            by current settings.
        </para>
        <para>
            The first and most significant difference from Java Lucene is that terms are searched through
            <emphasis>all fields</emphasis> by default.
        </para>
        <para>
            There are two static methods in the Zend_Search_Lucene class which allow the developer to configure these settings:
        </para>
        <programlisting role="php"><![CDATA[<?php
$defaultSearchField = Zend_Search_Lucene::getDefaultSearchField();
...
Zend_Search_Lucene::setDefaultSearchField('contents');]]>
        </programlisting>

        <para>
            The <code>null</code> value indicated that the search is performed across all fields. It's the default setting.
        </para>
        <para>
            You can search specific fields by typing the field name followed by a colon ":" followed by the term you
            are looking for.
        </para>
        <para>
            As an example, let's assume a Lucene index contains two fields- title and text- with text as the default field.
            If you want to find the document entitled "The Right Way" which contains the text "don't go this way",
            you can enter:
        </para>
        <programlisting role="querystring"><![CDATA[title:"The Right Way" AND text:go]]>
        </programlisting>
        <para>
            or
        </para>
        <programlisting role="querystring"><![CDATA[title:"Do it right" AND go]]>
        </programlisting>
        <para>
            Because "text" is the default field, the field indicator is not required.
        </para>

        <para>
            Note: The field is only valid for the term, phrase or subquery that it directly precedes,
            so the query
            <programlisting role="querystring"><![CDATA[title:Do it right]]>
            </programlisting>
            Will only find "Do" in the title field. It will find "it" and "right" in the default field (if the default field is set)
            or in all indexed fields (if the default field is set to <code>null</code>).
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.wildcard">
        <title>Wildcards<footnote><para>Wildcard queries  support is available with SVN and latest snapshot versions and 
               doesn't included into current release.</para>></footnote></title>
        <para>
            Lucene supports single and multiple character wildcard searches within single terms (but not within phrase queries).
        </para>

        <para>
            To perform a single character wildcard search use the "?" symbol.
        </para>

        <para>
            To perform a multiple character wildcard search use the "*" symbol.
        </para>

        <para>
            The single character wildcard search looks for string that match the term with the "?" replaced by any single character.
            For example, to search for "text" or "test" you can use the search:
            <programlisting role="querystring"><![CDATA[te?t]]>
            </programlisting>
        </para>

        <para>
            Multiple character wildcard searches look for 0 or more characters when matching strings against terms. For example, to search for test,
            tests or tester, you can use the search:
            <programlisting role="querystring"><![CDATA[test*]]>
            </programlisting>
        </para>

        <para>
            You can use "?", "*" or both at any place of the term:
            <programlisting role="querystring"><![CDATA[*wr?t*]]>
            </programlisting>
            It searches for "write", "wrote", "written", "rewrite", "rewrote" and so on.
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.modifiers">
        <title>Term Modifiers</title>

        <para>
            Lucene supports modifying query terms to provide a wide range of searching options.
        </para>
        <para>
            Zend_Search_Lucene currently supports the "~" modifier only for phrases.
            <footnote>
                <para>
                    The "~" modifier for terms is used for fuzzy search, but fuzzy search queries are not currently supported.
                </para>
            </footnote>
        </para>
    </sect2>

    <sect2 id="zend.search.lucene.query-language.range">
        <title>Range Searches<footnote><para>Range queries support is available with SVN and latest snapshot versions and 
               doesn't included into current release.</para>></footnote></title>
        <para>
            Range queries allow the developer or user to match documents whose field(s) values are between the lower and upper bound specified by the range query.
            Range Queries can be inclusive or exclusive of the upper and lower bounds. Sorting is performed lexicographically.
            <programlisting role="querystring"><![CDATA[mod_date:[20020101 TO 20030101]]]>
{'fraction_non_alphanumeric': 0.08910047995045672, 'fraction_numerical': 0.004954327295246942, 'mean_word_length': 2.2088922006954794, 'pattern_counts': {'":': 2, '<': 229, '<?xml version=': 0, '>': 229, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5327290', 'n_tokens_mistral': 3423, 'n_tokens_neox': 3192, 'n_words': 1338}