<commit_before>[[runtime]]
== Runtime fields
A _runtime field_ is a field that is evaluated at query time. Runtime fields
enable you to:

* Add fields to existing documents without reindexing your data
* Start working with your data without understanding how it’s structured
* Override the value returned from an indexed field at query time
* Define fields for a specific use without modifying the underlying schema

You access runtime fields from the search API like any other field, and {es}
sees runtime fields no differently. You can define runtime fields in the
<<runtime-mapping-fields,index mapping>> or in the
<<runtime-search-request,search request>>. Your choice, which is part of the
inherent flexibility of runtime fields.

Runtime fields are useful when working with log data
(see <<runtime-examples,examples>>), especially when you're unsure about the
data structure. Your search speed decreases, but your index size is much
smaller and you can more quickly process logs without having to index them.

[discrete]
[[runtime-benefits]]
=== Benefits
Because runtime fields aren't indexed, adding a runtime field doesn't increase
the index size. You define runtime fields directly in the index mapping, saving
storage costs and increasing ingestion speed. You can more quickly ingest
data into the Elastic Stack and access it right away. When you define a runtime
field, you can immediately use it in search requests, aggregations, filtering,
and sorting.

If you make a runtime field an indexed field, you don't need to modify any
queries that refer to the runtime field. Better yet, you can refer to some
indices where the field is a runtime field, and other indices where the field
is an indexed field. You have the flexibility to choose which fields to index
and which ones to keep as runtime fields.

At its core, the most important benefit of runtime fields is the ability to
add fields to documents after you've ingested them. This capability simplifies
mapping decisions because you don't have to decide how to parse your data up
front, and can use runtime fields to amend the mapping at any time. Using
runtime fields allows for a smaller index and faster ingest time, which
combined use less resources and reduce your operating costs.

[discrete]
[[runtime-compromises]]
=== Compromises
Runtime fields use less disk space and provide flexibility in how you access
your data, but can impact search performance based on the computation defined in
the runtime script.

To balance search performance and flexibility, index fields that you'll
commonly search for and filter on, such as a timestamp. {es} automatically uses
these indexed fields first when running a query, resulting in a fast response
time. You can then use runtime fields to limit the number of fields that {es}
needs to calculate values for. Using indexed fields in tandem with runtime
fields provides flexibility in the data that you index and how you define
queries for other fields.

Use the <<async-search,asynchronous search API>> to run searches that include
runtime fields. This method of search helps to offset the performance impacts
of computing values for runtime fields in each document containing that field.
If the query can't return the result set synchronously, you'll get results
asynchronously as they become available.

IMPORTANT: Queries against runtime fields are considered expensive. If
<<query-dsl-allow-expensive-queries,`search.allow_expensive_queries`>> is set
to `false`, expensive queries are not allowed and {es} will reject any queries
against runtime fields.

[[runtime-mapping-fields]]
=== Map a runtime field
You map runtime fields by adding a `runtime` section under the mapping
definition and defining
<<modules-scripting-using,a Painless script>>. This script has access to the
entire context of a document, including the original `_source` and any mapped
fields plus their values. At query time, the script runs and generates values
for each scripted field that is required for the query.

.Emitting runtime field values
****
When defining a Painless script to use with runtime fields, you must include
the {painless}/painless-runtime-fields-context.html[`emit` method] to emit
calculated values.
****

For example, the script in the following request calculates the day of the week
from the `@timestamp` field, which is defined as a `date` type. The script
calculates the day of the week based on the value of `timestamp`, and uses
`emit` to return the calculated value.

[source,console]
----
PUT my-index/
{
  "mappings": {
    "runtime": {
      "day_of_week": {
        "type": "keyword",
        "script": {
          "source": "emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))"
        }
      }
    },
    "properties": {
      "@timestamp": {"type": "date"}
    }
  }
}
----

The `runtime` section can be any of these data types:

// tag::runtime-data-types[]
* `boolean`
* `date`
* `double`
* `geo_point`
* `ip`
* `keyword`
* `long`
// end::runtime-data-types[]

Runtime fields with a `type` of `date` can accept the
<<mapping-date-format,`format`>> parameter exactly as the `date` field type.

If <<dynamic-field-mapping,dynamic field mapping>> is enabled where the
`dynamic` parameter is set to `runtime`, new fields are automatically added to
the index mapping as runtime fields:

[source,console]
----
PUT my-index
{
  "mappings": {
    "dynamic": "runtime",
    "properties": {
      "@timestamp": {
        "type": "date"
      }
    }
  }
}
----

[[runtime-fields-scriptless]]
==== Define runtime fields without a script
You can define a runtime field in the mapping definition without a
script. At query time, {es} looks in `_source` for a field with the same name
and returns a value if one exists. If a field with the same name doesn’t
exist, the response doesn't include any values for that runtime field.

[source,console]
----
PUT my-index/
{
  "mappings": {
    "runtime": {
      "day_of_week": {
        "type": "keyword"
      }
    }
  }
}
----

[[runtime-updating-scripts]]
==== Updating and removing runtime fields

You can update or remove runtime fields at any time. To replace an existing<commit_msg>[DOCS] Expand information on using a runtime field without a script (#73219)<commit_after>[[runtime]]
== Runtime fields
A _runtime field_ is a field that is evaluated at query time. Runtime fields
enable you to:

* Add fields to existing documents without reindexing your data
* Start working with your data without understanding how it’s structured
* Override the value returned from an indexed field at query time
* Define fields for a specific use without modifying the underlying schema

You access runtime fields from the search API like any other field, and {es}
sees runtime fields no differently. You can define runtime fields in the
<<runtime-mapping-fields,index mapping>> or in the
<<runtime-search-request,search request>>. Your choice, which is part of the
inherent flexibility of runtime fields.

Runtime fields are useful when working with log data
(see <<runtime-examples,examples>>), especially when you're unsure about the
data structure. Your search speed decreases, but your index size is much
smaller and you can more quickly process logs without having to index them.

[discrete]
[[runtime-benefits]]
=== Benefits
Because runtime fields aren't indexed, adding a runtime field doesn't increase
the index size. You define runtime fields directly in the index mapping, saving
storage costs and increasing ingestion speed. You can more quickly ingest
data into the Elastic Stack and access it right away. When you define a runtime
field, you can immediately use it in search requests, aggregations, filtering,
and sorting.

If you make a runtime field an indexed field, you don't need to modify any
queries that refer to the runtime field. Better yet, you can refer to some
indices where the field is a runtime field, and other indices where the field
is an indexed field. You have the flexibility to choose which fields to index
and which ones to keep as runtime fields.

At its core, the most important benefit of runtime fields is the ability to
add fields to documents after you've ingested them. This capability simplifies
mapping decisions because you don't have to decide how to parse your data up
front, and can use runtime fields to amend the mapping at any time. Using
runtime fields allows for a smaller index and faster ingest time, which
combined use less resources and reduce your operating costs.

[discrete]
[[runtime-compromises]]
=== Compromises
Runtime fields use less disk space and provide flexibility in how you access
your data, but can impact search performance based on the computation defined in
the runtime script.

To balance search performance and flexibility, index fields that you'll
commonly search for and filter on, such as a timestamp. {es} automatically uses
these indexed fields first when running a query, resulting in a fast response
time. You can then use runtime fields to limit the number of fields that {es}
needs to calculate values for. Using indexed fields in tandem with runtime
fields provides flexibility in the data that you index and how you define
queries for other fields.

Use the <<async-search,asynchronous search API>> to run searches that include
runtime fields. This method of search helps to offset the performance impacts
of computing values for runtime fields in each document containing that field.
If the query can't return the result set synchronously, you'll get results
asynchronously as they become available.

IMPORTANT: Queries against runtime fields are considered expensive. If
<<query-dsl-allow-expensive-queries,`search.allow_expensive_queries`>> is set
to `false`, expensive queries are not allowed and {es} will reject any queries
against runtime fields.

[[runtime-mapping-fields]]
=== Map a runtime field
You map runtime fields by adding a `runtime` section under the mapping
definition and defining
<<modules-scripting-using,a Painless script>>. This script has access to the
entire context of a document, including the original `_source` and any mapped
fields plus their values. At query time, the script runs and generates values
for each scripted field that is required for the query.

.Emitting runtime field values
****
When defining a Painless script to use with runtime fields, you must include
the {painless}/painless-runtime-fields-context.html[`emit` method] to emit
calculated values.
****

For example, the script in the following request calculates the day of the week
from the `@timestamp` field, which is defined as a `date` type. The script
calculates the day of the week based on the value of `timestamp`, and uses
`emit` to return the calculated value.

[source,console]
----
PUT my-index/
{
  "mappings": {
    "runtime": {
      "day_of_week": {
        "type": "keyword",
        "script": {
          "source": "emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))"
        }
      }
    },
    "properties": {
      "@timestamp": {"type": "date"}
    }
  }
}
----

The `runtime` section can be any of these data types:

// tag::runtime-data-types[]
* `boolean`
* `date`
* `double`
* `geo_point`
* `ip`
* `keyword`
* `long`
// end::runtime-data-types[]

Runtime fields with a `type` of `date` can accept the
<<mapping-date-format,`format`>> parameter exactly as the `date` field type.

If <<dynamic-field-mapping,dynamic field mapping>> is enabled where the
`dynamic` parameter is set to `runtime`, new fields are automatically added to
the index mapping as runtime fields:

[source,console]
----
PUT my-index
{
  "mappings": {
    "dynamic": "runtime",
    "properties": {
      "@timestamp": {
        "type": "date"
      }
    }
  }
}
----

[[runtime-fields-scriptless]]
==== Define runtime fields without a script
Runtime fields typically include a Painless script that manipulates data in some
way. However, there are instances where you might define a runtime field
_without_ a script. For example, if you want to retrieve a single field from `_source` without making changes, you don't need a script. You can just create
a runtime field without a script, such as `day_of_week`:

[source,console]
----
PUT my-index/
{
  "mappings": {
    "runtime": {
      "day_of_week": {
        "type": "keyword"
      }
    }
  }
}
----

When no script is provided, {es} implicitly looks in `_source` at query time
for a field with the same name as the runtime field, and returns a value if one
exists. If a field with the same name doesn’t exist, the response doesn't
include any values for that runtime field.

In most cases, retrieve field values through
<<doc-values,`doc_values`>> whenever possible. Accessing `doc_values` with a
runtime field is faster than retrieving values from `_source` because of how
data is loaded from Lucene.

However, there are cases where retrieving fields from `_source` is necessary.
For example, `text` fields do not have `doc_values` available by default, so you
have to retrieve values from `_source`. In other instances, you might choose to
disable `doc_values` on a specific field.

NOTE: You can alternatively prefix the field you want to retrieve values for
with `params._source` (such as `params._source.day_of_week`). For simplicity,
defining a runtime field in the mapping definition without a script is the
recommended option, whenever possible.

[[runtime-updating-scripts]]
==== Updating and removing runtime fields

You can update or remove runtime fields at any time. To replace an existing
{'fraction_non_alphanumeric': 0.07230157561478427, 'fraction_numerical': 0.0003681342953909586, 'mean_word_length': 4.299648849005072, 'pattern_counts': {'":': 36, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7472801', 'n_tokens_mistral': 3480, 'n_tokens_neox': 3393, 'n_words': 1890}