<commit_before>Extending Finagle
=================

In this section we'll examine the internals of Finagle by implementing
a client and server for a simple newline-delimited, string-based
protocol. We'll build a client capable of sending simple strings to a
server, which in turn may reply with another.

While we're going to dig a bit deeper here than in the rest of the
user's guide — implementing even a simple protocol requires a deeper
appreciation of Finagle's internals — the material presented here is not
necessary to understand how to use Finagle productively. However,
expert users may find the material useful.

If you haven't yet, first read the :doc:`quickstart<Quickstart>` to
understand some of Finagle's core concepts (i.e. :doc:`Futures
<Futures>`, :doc:`Services and Filters<ServicesAndFilters>`).

The entire example is available, together with a self-contained script
to launch `sbt`, in the Finagle git repository:

::

  $ git clone https://github.com/twitter/finagle.git
  $ cd finagle/doc/src/sphinx/code/client-server-anatomy
  $ ./sbt run

Stack
-----

Finagle's clients and servers comprise many relatively simple
components, arranged together in a stack. Each component is a
:api:`ServiceFactory <com.twitter.finagle.ServiceFactory>` that
composes other service factories. We've seen this before:
:doc:`filters <ServicesAndFilters>` are a kind of component.

This allows us to create simple components that we arrange into a
sophisticated whole. For example, we define timeout behavior as a
separate module; we then arrange to insert this module into the
composite stack wherever timeout behavior is required. Some of the
components included in Finagle's clients are discussed in the
:ref:`client documentation <client_modules>`.

While we can imagine stacking such modules together in a manual
fashion—for example, by building it bottoms-up, passing each
``ServiceFactory`` into the constructor of the next—this technique quickly
becomes onerous; it become very difficult to:

- parameterize each component, for example to set the sizes of
  connection pools
- inject dependencies such as
  :util:`StatsReceiver <com.twitter.finagle.stats.StatsReceiver>` implementations
- rearrange any part of the stack, for example to remove an
  unused timeout filter
- modify the stack for a specific use, for example to replace
  the connection pooling implementation or the load balancer

Traditionally, the :api:`ClientBuilder
<com.twitter.finagle.builder.ClientBuilder>` and :api:`ServerBuilder
<com.twitter.finagle.builder.ServerBuilder>` performed this duty in
the manner just described. While it worked well for a while, it
ultimately proved inflexible. With the needs and requirements of the
various protocol implementations like :ref:`Mux's <mux>`, as well as
more sophisticated new features, the builders became unworkable
monoliths.

:api:`Stack <com.twitter.finagle.Stack>` formalizes the concept of a
*stackable* component and treats a sequence of stackable components as
a first-class (immutable) value that may be manipulated like any other
collection. For example, modules may be inserted or removed from the
stack; you can map one stack to another, for example when
parameterizing individual modules.

The stack abstraction also formalizes the concept of a *parameter*—i.e.
a map of values used to construct an object. :api:`Stack.Params <com.twitter.finagle.Stack$.Params>`
is a kind of type-safe map used to hold parameters.

``Stack`` and ``Param`` work together like this: ``Stack`` represents the stack
of modules. These modules aren't *materialized*. Rather, they represent
a constructor for a ``ServiceFactory`` that accepts a ``Params`` map. Thus,
to extract the final ``ServiceFactory`` that represents the entirety of the stack,
we simply call ``stack.make(params)``.

Finagle defines default (polymorphic) stacks for both
:api:`clients <com.twitter.finagle.client.StackClient$.newStack>` and
:api:`servers <com.twitter.finagle.server.StackServer$.newStack>`.

We'll now discuss the constituent parts of Finagle's clients and servers.

.. _transport_interface:

Transport Layer
---------------

Finagle represents the OSI transport layer as a typed stream that may
be read from and written to asynchronously. The noteworthy methods in
the interface are defined as such:

.. code-block:: scala

  trait Transport[In, Out] {
    def read(): Future[Out]
    def write(req: In): Future[Unit]
    ...
  }

Most Transports are implemented using `Netty <http://netty.io>`_
for I/O multiplexing and protocol codecs.

You have a choice of whether to use `Netty 3 or Netty 4 <http://netty.io/wiki/new-and-noteworthy-in-4.0.html>`_
for your protocol implementation. As we have migrated our existing protocols<commit_msg>finagle-doc: Fix Links for Clients and Servers on Extending Page<commit_after>Extending Finagle
=================

In this section we'll examine the internals of Finagle by implementing
a client and server for a simple newline-delimited, string-based
protocol. We'll build a client capable of sending simple strings to a
server, which in turn may reply with another.

While we're going to dig a bit deeper here than in the rest of the
user's guide — implementing even a simple protocol requires a deeper
appreciation of Finagle's internals — the material presented here is not
necessary to understand how to use Finagle productively. However,
expert users may find the material useful.

If you haven't yet, first read the :doc:`quickstart<Quickstart>` to
understand some of Finagle's core concepts (i.e. :doc:`Futures
<Futures>`, :doc:`Services and Filters<ServicesAndFilters>`).

The entire example is available, together with a self-contained script
to launch `sbt`, in the Finagle git repository:

::

  $ git clone https://github.com/twitter/finagle.git
  $ cd finagle/doc/src/sphinx/code/client-server-anatomy
  $ ./sbt run

Stack
-----

Finagle's clients and servers comprise many relatively simple
components, arranged together in a stack. Each component is a
:api:`ServiceFactory <com.twitter.finagle.ServiceFactory>` that
composes other service factories. We've seen this before:
:doc:`filters <ServicesAndFilters>` are a kind of component.

This allows us to create simple components that we arrange into a
sophisticated whole. For example, we define timeout behavior as a
separate module; we then arrange to insert this module into the
composite stack wherever timeout behavior is required. Some of the
components included in Finagle's clients are discussed in the
:ref:`client documentation <client_modules>`.

While we can imagine stacking such modules together in a manual
fashion—for example, by building it bottoms-up, passing each
``ServiceFactory`` into the constructor of the next—this technique quickly
becomes onerous; it become very difficult to:

- parameterize each component, for example to set the sizes of
  connection pools
- inject dependencies such as
  :util:`StatsReceiver <com.twitter.finagle.stats.StatsReceiver>` implementations
- rearrange any part of the stack, for example to remove an
  unused timeout filter
- modify the stack for a specific use, for example to replace
  the connection pooling implementation or the load balancer

Traditionally, the :api:`ClientBuilder
<com.twitter.finagle.builder.ClientBuilder>` and :api:`ServerBuilder
<com.twitter.finagle.builder.ServerBuilder>` performed this duty in
the manner just described. While it worked well for a while, it
ultimately proved inflexible. With the needs and requirements of the
various protocol implementations like :ref:`Mux's <mux>`, as well as
more sophisticated new features, the builders became unworkable
monoliths.

:api:`Stack <com.twitter.finagle.Stack>` formalizes the concept of a
*stackable* component and treats a sequence of stackable components as
a first-class (immutable) value that may be manipulated like any other
collection. For example, modules may be inserted or removed from the
stack; you can map one stack to another, for example when
parameterizing individual modules.

The stack abstraction also formalizes the concept of a *parameter*—i.e.
a map of values used to construct an object. :api:`Stack.Params <com.twitter.finagle.Stack$.Params>`
is a kind of type-safe map used to hold parameters.

``Stack`` and ``Param`` work together like this: ``Stack`` represents the stack
of modules. These modules aren't *materialized*. Rather, they represent
a constructor for a ``ServiceFactory`` that accepts a ``Params`` map. Thus,
to extract the final ``ServiceFactory`` that represents the entirety of the stack,
we simply call ``stack.make(params)``.

Finagle defines default (polymorphic) stacks for both
:api:`clients <com.twitter.finagle.client.StackClient$@newStack[Req,Rep]:com.twitter.finagle.Stack[com.twitter.finagle.ServiceFactory[Req,Rep]]>` and
:api:`servers <com.twitter.finagle.server.StackServer$@newStack[Req,Rep]:com.twitter.finagle.Stack[com.twitter.finagle.ServiceFactory[Req,Rep]]>`.

We'll now discuss the constituent parts of Finagle's clients and servers.

.. _transport_interface:

Transport Layer
---------------

Finagle represents the OSI transport layer as a typed stream that may
be read from and written to asynchronously. The noteworthy methods in
the interface are defined as such:

.. code-block:: scala

  trait Transport[In, Out] {
    def read(): Future[Out]
    def write(req: In): Future[Unit]
    ...
  }

Most Transports are implemented using `Netty <http://netty.io>`_
for I/O multiplexing and protocol codecs.

You have a choice of whether to use `Netty 3 or Netty 4 <http://netty.io/wiki/new-and-noteworthy-in-4.0.html>`_
for your protocol implementation. As we have migrated our existing protocols
{'fraction_non_alphanumeric': 0.08004938779709847, 'fraction_numerical': 0.0008231299516411153, 'mean_word_length': 4.930445393532642, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1852928', 'n_tokens_mistral': 2616, 'n_tokens_neox': 2499, 'n_words': 1222}