<commit_before>[[query-string-syntax]]

==== Query string syntax

The query string ``mini-language'' is used by the
<<query-dsl-query-string-query>> and by the
`q` query string parameter in the <<search-search,`search` API>>.

The query string is parsed into a series of _terms_ and _operators_. A
term can be a single word -- `quick` or `brown` -- or a phrase, surrounded by
double quotes -- `"quick brown"` -- which searches for all the words in the
phrase, in the same order.

Operators allow you to customize the search -- the available options are
explained below.

===== Field names

As mentioned in <<query-dsl-query-string-query>>, the `default_field` is searched for the
search terms, but it is possible to specify other fields in the query syntax:

* where the `status` field contains `active`

    status:active

* where the `title` field contains `quick` or `brown`.
  If you omit the OR operator the default operator will be used

    title:(quick OR brown)
    title:(quick brown)

* where the `author` field contains the exact phrase `"<NAME>"`

    author:"<NAME>"

* where any of the fields `book.title`, `book.content` or `book.date` contains
  `quick` or `brown` (note how we need to escape the `*` with a backslash):

    book.\*:(quick brown)

* where the field `title` has any non-null value:

    _exists_:title

===== Wildcards

Wildcard searches can be run on individual terms, using `?` to replace
a single character, and `*` to replace zero or more characters:

    qu?ck bro*

Be aware that wildcard queries can use an enormous amount of memory and
perform very badly -- just think how many terms need to be queried to
match the query string `"a* b* c*"`.

[WARNING]
=======
Allowing a wildcard at the beginning of a word (eg `"*ing"`) is particularly
heavy, because all terms in the index need to be examined, just in case
they match.  Leading wildcards can be disabled by setting
`allow_leading_wildcard` to `false`.
=======

Only parts of the analysis chain that operate at the character level are
applied. So for instance, if the analyzer performs both lowercasing and
stemming, only the lowercasing will be applied: it would be wrong to perform
stemming on a word that is missing some of its letters.

By setting `analyze_wildcard` to true, queries that end with a `*` will be
analyzed and a boolean query will be built out of the different tokens, by
ensuring exact matches on the first N-1 tokens, and prefix match on the last
token.
<commit_msg>Clarify pure wilcard matching with `query_string` (#26814)<commit_after>[[query-string-syntax]]

==== Query string syntax

The query string ``mini-language'' is used by the
<<query-dsl-query-string-query>> and by the
`q` query string parameter in the <<search-search,`search` API>>.

The query string is parsed into a series of _terms_ and _operators_. A
term can be a single word -- `quick` or `brown` -- or a phrase, surrounded by
double quotes -- `"quick brown"` -- which searches for all the words in the
phrase, in the same order.

Operators allow you to customize the search -- the available options are
explained below.

===== Field names

As mentioned in <<query-dsl-query-string-query>>, the `default_field` is searched for the
search terms, but it is possible to specify other fields in the query syntax:

* where the `status` field contains `active`

    status:active

* where the `title` field contains `quick` or `brown`.
  If you omit the OR operator the default operator will be used

    title:(quick OR brown)
    title:(quick brown)

* where the `author` field contains the exact phrase `"<NAME>"`

    author:"<NAME>"

* where any of the fields `book.title`, `book.content` or `book.date` contains
  `quick` or `brown` (note how we need to escape the `*` with a backslash):

    book.\*:(quick brown)

* where the field `title` has any non-null value:

    _exists_:title

===== Wildcards

Wildcard searches can be run on individual terms, using `?` to replace
a single character, and `*` to replace zero or more characters:

    qu?ck bro*

Be aware that wildcard queries can use an enormous amount of memory and
perform very badly -- just think how many terms need to be queried to
match the query string `"a* b* c*"`.

[WARNING]
=======
Pure wildcards `\*` are rewritten to <<query-dsl-exists-query,`exists`>> queries for efficiency.
As a consequence, the wildcard `"field:*"` would match documents with an emtpy value
 like the following:
```
{
  "field": ""
}
```
\... and would **not** match if the field is missing or set with an explicit null
value like the following:
```
{
  "field": null
}
```
=======

[WARNING]
=======
Allowing a wildcard at the beginning of a word (eg `"*ing"`) is particularly
heavy, because all terms in the index need to be examined, just in case
they match.  Leading wildcards can be disabled by setting
`allow_leading_wildcard` to `false`.
=======

Only parts of the analysis chain that operate at the character level are
applied. So for instance, if the analyzer performs both lowercasing and
stemming, only the lowercasing will be applied: it would be wrong to perform
stemming on a word that is missing some of its letters.

By setting `analyze_wildcard` to true, queries that end with a `*` will be
analyzed and a boolean query will be built out of the different tokens, by
ensuring exact matches on the first N-1 tokens, and prefix match on the last
token.

{'fraction_non_alphanumeric': 0.09464715688090387, 'fraction_numerical': 0.0012965363956288201, 'mean_word_length': 4.10879848628193, 'pattern_counts': {'":': 2, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2976185', 'n_tokens_mistral': 1563, 'n_tokens_neox': 1484, 'n_words': 792}