<commit_before>.. _methodbuilder:

MethodBuilder
=============

.. note:: Currently there is ``MethodBuilder`` support for HTTP and ThriftMux.
          We are waiting on user interest before expanding to more protocols.

``MethodBuilder`` is a collection of APIs for client configuration at a higher
level than the  :ref:`Finagle 6 APIs <finagle6apis>` while improving upon the deprecated
``ClientBuilder``. ``MethodBuilder`` provides:

- :ref:`Logical <mb_logical_req>` success rate metrics.
- Retries based on application-level requests and responses (e.g. an HTTP
  503 response code or a Thrift exception).
- Configuration of per-attempt and total timeouts.

All of these can be customized per method (or endpoint) while sharing a single
underlying Finagle client. Concretely, a single service might offer both
`GET statuses/show/:id` as well as `POST statuses/update`, whilst each having wildly
different characteristics. The `GET` is idempotent and has a tight latency distribution
while the `POST` is not idempotent and has a wide latency distribution. If users want
different configurations, without ``MethodBuilder`` they must create separate
Finagle clients for each grouping. While long-lived clients in Finagle are not
expensive, they are not free. They create duplicate metrics and waste heap,
file descriptors, and CPU.

.. _mb_example_http:

Turning the example into code, first in Scala:

.. code-block:: scala

  import com.twitter.conversions.time._
  import com.twitter.finagle.{http, Http, Service}
  import com.twitter.finagle.service.{ReqRep, ResponseClass}
  import com.twitter.util.Return

  // other StackClient configuration done here, e.g. `withSessionQualifier`
  val stackClient = Http.client
    .withLabel("example_client")

  val builder = stackClient.methodBuilder("inet!localhost:8080")

  // a `Service` for "GET statuses/show/:id" with relatively tight timeouts
  // and a liberal retry policy.
  val statusesShow: Service[http.Request, http.Response] = builder
    // 25 millisecond timeout per attempt. this applies to the initial
    // attempt and each retry, if any.
    .withTimeoutPerRequest(25.milliseconds)
    // 50 milliseconds timeout in total, including retries
    .withTimeoutTotal(50.milliseconds)
    // retry all HTTP 4xx and 5xx responses
    .withRetryForClassifier {
      case ReqRep(_, Return(rep)) if rep.statusCode >= 400 && rep.statusCode <= 599 =>
        ResponseClass.RetryableFailure
    }
    // build the service
    .newService(methodName = "get_statuses")

  // a `Service` for "POST statuses/update" with relatively loose timeouts
  // and a retry policy tailored to this endpoint.
  val statusesUpdate: Service[http.Request, http.Response] = builder
    // 200 millisecond timeouts per attempt. this applies to the initial
    // attempt and each retry, if any.
    .withTimeoutPerRequest(200.milliseconds)
    // 500 milliseconds timeouts in total, including retries
    .withTimeoutTotal(500.milliseconds)
    // retry only a specific HTTP response code
    .withRetryForClassifier {
      case ReqRep(_, Return(rep)) if rep.statusCode == 418 => ResponseClass.RetryableFailure
    }
    // build the service
    .newService(methodName = "update_status")

A similar example, for Java:

.. code-block:: java

  import com.twitter.finagle.Http;
  import com.twitter.finagle.Service;
  import com.twitter.finagle.http.Request;
  import com.twitter.finagle.http.Response;
  import com.twitter.util.Duration;

  Service<Request, Response> exampleService =
    Http.client().methodBuilder("localhost:8080")
      .withTimeoutTotal(Duration.fromMilliseconds(50))
      .withTimeoutPerRequest(Duration.fromMilliseconds(25))
      .newService("java_example");

Retries
-------

``MethodBuilder`` defaults to using the client's :ref:`classifier <response_classification>`
to retry failures that are marked as retryable
(``com.twitter.finagle.service.ResponseClass.RetryableFailure``).

A budget is used to prevent retries from overwhelming
the backend service. The budget is shared across clients created from
an initial ``MethodBuilder``. As such, even if the retry rules
deem the request retryable, it may not be retried if there is insufficient
budget.

Finagle automatically retries failures that are known to be safe
to retry via :src:`RequeueFilter <com/twitter/finagle/service/RequeueFilter.scala>`.
This includes ``com.twitter.finagle.WriteException WriteExceptions`` and
:ref:`retryable nacks <glossary_nack>`. As these should have already been retried,
``MethodBuilder`` will avoid retrying them again at this layer.

The :ref:`classifier <response_classification>` set by ``withRetryForClassifier`` is
used to determine which requests are successful. This is the basis for measuring
the :ref:`logical <mb_logical_req>` success metrics of the method and for logging_
unsuccessful requests.

Timeouts
--------

For per-request timeouts the defaults come from the client's configuration
for :src:`TimeoutFilter.Param <com/twitter/finagle/service/TimeoutFilter.scala>`
which is typically set on a client via ``com.twitter.finagle.$Protocol.withRequestTimeout``.

For total total timeouts, the defaults come from the client's configuration
for :src:`TimeoutFilter.TotalTimeout <com/twitter/finagle/service/TimeoutFilter.scala>`.

The total timeout is how long the :ref:`logical request <mb_logical_req>` is given to complete. This includes
the time spent on developer configured retries as well as automatic retries issued by
Finagle. Per request timeouts apply to each attempt issued, irrespective of if
it is the initial request, a Finagle requeue, or a retry based on the developer's policy.

Take a ``MethodBuilder`` configured with 100 ms per-request timeout,<commit_msg>finagle/doc: MethodBuilder docs for idempotency<commit_after>.. _methodbuilder:

MethodBuilder
=============

.. note:: Currently there is ``MethodBuilder`` support for HTTP and ThriftMux.
          We are waiting on user interest before expanding to more protocols.

``MethodBuilder`` is a collection of APIs for client configuration at a higher
level than the  :ref:`Finagle 6 APIs <finagle6apis>` while improving upon the deprecated
``ClientBuilder``. ``MethodBuilder`` provides:

- :ref:`Logical <mb_logical_req>` success rate metrics.
- Retries based on application-level requests and responses (e.g. an HTTP
  503 response code or a Thrift exception).
- Configuration of per-attempt and total timeouts.

All of these can be customized per method (or endpoint) while sharing a single
underlying Finagle client. Concretely, a single service might offer both
`GET statuses/show/:id` as well as `POST statuses/update`, whilst each having wildly
different characteristics. The `GET` is idempotent and has a tight latency distribution
while the `POST` is not idempotent and has a wide latency distribution. If users want
different configurations, without ``MethodBuilder`` they must create separate
Finagle clients for each grouping. While long-lived clients in Finagle are not
expensive, they are not free. They create duplicate metrics and waste heap,
file descriptors, and CPU.

.. _mb_example_http:

Turning the example into code, first in Scala:

.. code-block:: scala

  import com.twitter.conversions.time._
  import com.twitter.finagle.{http, Http, Service}
  import com.twitter.finagle.service.{ReqRep, ResponseClass}
  import com.twitter.util.Return

  // other StackClient configuration done here, e.g. `withSessionQualifier`
  val stackClient = Http.client
    .withLabel("example_client")

  val builder = stackClient.methodBuilder("inet!localhost:8080")

  // a `Service` for "GET statuses/show/:id" with relatively tight timeouts
  // and a liberal retry policy.
  val statusesShow: Service[http.Request, http.Response] = builder
    // 25 millisecond timeout per attempt. this applies to the initial
    // attempt and each retry, if any.
    .withTimeoutPerRequest(25.milliseconds)
    // 50 milliseconds timeout in total, including retries
    .withTimeoutTotal(50.milliseconds)
    // retry all HTTP 4xx and 5xx responses
    .withRetryForClassifier {
      case ReqRep(_, Return(rep)) if rep.statusCode >= 400 && rep.statusCode <= 599 =>
        ResponseClass.RetryableFailure
    }
    // can reduce tail latency by sending 1% extra load to the backend
    .idempotent(maxExtraLoad = 0.01)
    // build the service
    .newService(methodName = "get_statuses")

  // a `Service` for "POST statuses/update", which is non-idempotent and has
  // relatively loose timeouts
  val statusesUpdate: Service[http.Request, http.Response] = builder
    // 200 millisecond timeouts per attempt. this applies to the initial
    // attempt and each retry, if any.
    .withTimeoutPerRequest(200.milliseconds)
    // 500 milliseconds timeouts in total, including retries
    .withTimeoutTotal(500.milliseconds)
    // no retries (except on write exceptions), and no backup requests
    .nonIdempotent
    // build the service
    .newService(methodName = "update_status")

A similar example, for Java:

.. code-block:: java

  import com.twitter.finagle.Http;
  import com.twitter.finagle.Service;
  import com.twitter.finagle.http.Request;
  import com.twitter.finagle.http.Response;
  import com.twitter.util.Duration;

  Service<Request, Response> exampleService =
    Http.client().methodBuilder("localhost:8080")
      .withTimeoutTotal(Duration.fromMilliseconds(50))
      .withTimeoutPerRequest(Duration.fromMilliseconds(25))
      .idempotent(0.01)
      .newService("java_example");

Retries
-------

``MethodBuilder`` defaults to using the client's :ref:`classifier <response_classification>`
to retry failures that are marked as retryable
(``com.twitter.finagle.service.ResponseClass.RetryableFailure``).

A budget is used to prevent retries from overwhelming
the backend service. The budget is shared across clients created from
an initial ``MethodBuilder``. As such, even if the retry rules
deem the request retryable, it may not be retried if there is insufficient
budget.

Finagle automatically retries failures that are known to be safe
to retry via :src:`RequeueFilter <com/twitter/finagle/service/RequeueFilter.scala>`.
This includes ``com.twitter.finagle.WriteException WriteExceptions`` and
:ref:`retryable nacks <glossary_nack>`. As these should have already been retried,
``MethodBuilder`` will avoid retrying them again at this layer.

The :ref:`classifier <response_classification>` set by ``withRetryForClassifier`` is
used to determine which requests are successful. This is the basis for measuring
the :ref:`logical <mb_logical_req>` success metrics of the method and for logging_
unsuccessful requests.

Idempotency
-----------

``MethodBuilder`` provides ``idempotent`` and ``nonIdemptotent`` methods for a client to signal
whether it's safe to resend requests that have already been sent. 

If a client is configured with ``idempotent``, a protcol-dependent
:src:`ResponseClassifier <com/twitter/finagle/service/ResponseClassifier.scala>` is combined with
any exisiting classifier to also reissue requests on failure (Thrift exceptions for ThriftMux
clients, and 500s for HTTP clients). The parameter to ``idempotent``, ``maxExtraLoad``, is used to
configure :src:`BackupRequestFilter <com/twitter/finagle/client/BackupRequestFilter.scala>`
(a value of 0.0 disables the filter).
:src:`BackupRequestFilter <com/twitter/finagle/client/BackupRequestFilter.scala>` can reduce tail
latency by sending a second, backup request if the original request has not been satisfied within
a certain time limit (calculated dynamically based on latency). The filter will never send more than
``maxExtraLoad`` more requests (i.e., for a value of 0.01, the filter will not send more than 1%
additional requests).

If a client is configured with ``nonIdempotent``, any existing configured
:src:`ResponseClassifier <com/twitter/finagle/service/ResponseClassifier.scala>` is removed
and replaced with the default
:src:`ResponseClassifier <com/twitter/finagle/service/ResponseClassifier.scala>`, which only retries
on write exceptions (wherein the request was never sent to the server). Any configured
:src:`BackupRequestFilter <com/twitter/finagle/client/BackupRequestFilter.scala>` is also disabled,
since it's not safe to reissue requests.

Timeouts
--------

For per-request timeouts the defaults come from the client's configuration
for :src:`TimeoutFilter.Param <com/twitter/finagle/service/TimeoutFilter.scala>`
which is typically set on a client via ``com.twitter.finagle.$Protocol.withRequestTimeout``.

For total total timeouts, the defaults come from the client's configuration
for :src:`TimeoutFilter.TotalTimeout <com/twitter/finagle/service/TimeoutFilter.scala>`.

The total timeout is how long the :ref:`logical request <mb_logical_req>` is given to complete. This includes
the time spent on developer configured retries as well as automatic retries issued by
Finagle. Per request timeouts apply to each attempt issued, irrespective of if
it is the initial request, a Finagle requeue, or a retry based on the developer's policy.

Take a ``MethodBuilder`` configured with 100 ms per-request timeout,
{'fraction_non_alphanumeric': 0.0858574356638579, 'fraction_numerical': 0.008729977985272907, 'mean_word_length': 4.582203389830508, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 36, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '949815', 'n_tokens_mistral': 3646, 'n_tokens_neox': 3448, 'n_words': 1406}