<commit_before>
.. include:: global.rst.inc

.. _Endpoints:

===========
 Endpoints
===========

*Endpoints* are objects that represent the server-side of http transactions.
They are instances of the DeproxyEndpoint_ class.
Endpoint receive HTTP requests and return HTTP responses.
The responses are generated by :ref:`handlers <Handlers>`, which can be static functions, object instance methods, or closures.
Handlers can be set when the endpoint is created, or specified on a per-client-request basis.
Endpoints are created using the addEndpoint_ method of the Deproxy_ class. (Instantiating a DeproxyEndpoint object via the construct is discouraged.)

Endpoints are very flexible. You can create intricate testing situations with them in combination with custom handlers.

How To: Single Server
---------------------

In the simplest case, a proxy sits in between the client and a server.
::

  ________                         ________                        ________
 |        |  --->  Request  --->  |        |  ---> Request  --->  |        |
 | Client |                       | Proxy  |                      | Server |
 |________|  <---  Response <---  |________|  <--- Response <---  |________|


Since we're testing the proxy, we want to be able to control the responses that the server sends in reaction to requests from the proxy.
We can simulate the server using a single DeproxyEndpoint.
By default, the endpoint will simply return 200 OK responses, unless it is given a different handler upon creation.


::

    Deproxy deproxy = new Deproxy()

    DeproxyEndpoint endpoint = deproxy.addEndpoint(9999)

    def theProxy = new TheProxy(port: 8080,
                                targetHostname: "localhost",
                                targetPort: 9999)

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1




How To: Auxiliary Service
-------------------------

A more complicated case is when the proxy has to call out to some auxiliary service for additional information.

::

  ________                         ________                        ________
 |        |  --->  Request  --->  |        |  ---> Request  --->  |        |
 | Client |                       | Proxy  |                      | Server |
 |________|  <---  Response <---  |________|  <--- Response <---  |________|

                                    |    ^
                            Request |    |  Response
                                    v    |
                                   ________
                                  |  Aux.  |
                                  |Service |
                                  |________|

An excellent example would be an authentication system.
The client sends the request to the proxy and includes credentials.
In order to determine if the credentials are valid, the proxy makes a separate HTTP request to the auth service, which then responds with yea or nay.
Depending on whether the credentials are valid or not, the proxy will either forward the request on to the server, or return an error back to the client.

In this setup, we can simulate both the server and the auxiliary service with DeproxyEndpoint objects.
The endpoint representing the auth service would have to be given a custom handler, that could interpret and respond to the authentication requests that the proxy makes according to whatever contract is necessary.

::

    Deproxy deproxy = new Deproxy()

    def endpoint = deproxy.addEndpoint(9999)

    def authResponder = new AuthResponder()
    def authService = deproxy.addEndpoint(7777, defaultHandler: authResponder.handler)

    def theProxy = new TheProxy(port: 8080,
                                targetHostname: "localhost",
                                targetPort: 9999,
                                authServiceHostname: "localhost",
                                authServicePort: 7777)

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource",
                                 headers: ['X-User': 'valid-user'])

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint
    assert mc.orphanedHandlings.size() == 1
    assert mc.orphanedHandlings[0].endpoint == authService

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource",
                                 headers: ['X-User': 'invalid-user'])

    assert mc.receivedResponse.code == "403"
    assert mc.handlings.size() == 0
    assert mc.orphanedHandlings.size() == 1
    assert mc.orphanedHandlings[0].endpoint == authService



How To: Multiple Servers
------------------------

Sometimes, a proxy might be set up in front of multiple servers.

::

                                                                   ________
                                    .------------> Request  --->  |        |
                                    |                             | Server1|
                                    |    .-------- Response <---  |________|
                                    |    |
                                    |    v
  ________                         ________                        ________
 |        |  --->  Request  --->  |        |  ---> Request  --->  |        |
 | Client |                       | Proxy  |                      | Server2|
 |________|  <---  Response <---  |________|  <--- Response <---  |________|

                                    ^    |
                                    |    |                         ________
                                    |    `-------- Request  --->  |        |
                                    |                             | Server3|
                                    `------------- Response <---  |________|

This might be the case, for example, if it is acting as a load balancer, or providing access to different versions of a ReST api based on uri.
This is simple enough to simulate by creating multiple endpoint objects, and configuring the proxy to forward client requests to them.


::

    Deproxy deproxy = new Deproxy()

    def endpoint1 = deproxy.addEndpoint(9999)
    def endpoint2 = deproxy.addEndpoint(9998)
    def endpoint3 = deproxy.addEndpoint(9997)

    def theProxy = new TheProxy(port: 8080,
                                targets: [
                                    ['hostname': "localhost", port: 9999],
                                    ['hostname': "localhost", port: 9998],
                                    ['hostname': "localhost", port: 9997]],
                                loadBalanceBehavior: Behavior.RoundRobin)

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint1

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint2

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint3


Routing
-------

Even more complex situations can be created using the Route_ built-in handler, to route requests to existing servers.
::
  ________          ________          ________        ________
 |        |  --->  |        |  --->  |  Fake  | ---> |  Real  |
 | Client |        | Proxy  |        | Server |      | Server |
 |________|  <---  |________|  <---  |________| <--- |________|

                     |    ^
                     v    |
                    ________
                   |Fake Aux|
                   |Service |
                   |________|

                     |    ^
                     v    |
                    ________
                   |Real Aux|
                   |Service |
                   |________|


This will work for requests from the proxy to an auxiliary service, or from the client/proxy to the server.
It can save us the trouble of implementing our own handlers to simulate the server or auxiliary service.
But why this round-about way of doing it? Why not just configure the proxy to send requests to those locations directly?
The real advantage to this method is that the requests go through an endpoint, so the Request_ and Response_ get captured and attached to a MessageChain_.
When makeRequest_ returns, we can make assertions against those requests and responses, which would be entirely invisible to us if the proxy had sent them directly.


Endpoint Lifecycle
------------------

1. When an endpoint is created, it opens a socket on the designated port and spawns a thread to listen for connections to that socket.
2. Whenever a new connection is made, the listener thread will spawn a new handler thread.
3. The handler thread will proceed to service HTTP request, like so:
    a. First the incoming request is read from the socket, and parsed into a Request_ object.
    b. The endpoint will examine the request headers for a ``Deproxy-Request-ID`` header, and then try to match it to an existing MessageChain_ (created before in a call to makeRequest_).
    c. The endpoint will then determine which handler to use (see :ref:`Handler Resolution Procedure <handlerResolutionProcedure>`), and pass the Request object to the handler to get a Response_ object.<commit_msg>Add code example.<commit_after>
.. include:: global.rst.inc

.. _Endpoints:

===========
 Endpoints
===========

*Endpoints* are objects that represent the server-side of http transactions.
They are instances of the DeproxyEndpoint_ class.
Endpoint receive HTTP requests and return HTTP responses.
The responses are generated by :ref:`handlers <Handlers>`, which can be static functions, object instance methods, or closures.
Handlers can be set when the endpoint is created, or specified on a per-client-request basis.
Endpoints are created using the addEndpoint_ method of the Deproxy_ class. (Instantiating a DeproxyEndpoint object via the construct is discouraged.)

Endpoints are very flexible. You can create intricate testing situations with them in combination with custom handlers.

How To: Single Server
---------------------

In the simplest case, a proxy sits in between the client and a server.
::

  ________                         ________                        ________
 |        |  --->  Request  --->  |        |  ---> Request  --->  |        |
 | Client |                       | Proxy  |                      | Server |
 |________|  <---  Response <---  |________|  <--- Response <---  |________|


Since we're testing the proxy, we want to be able to control the responses that the server sends in reaction to requests from the proxy.
We can simulate the server using a single DeproxyEndpoint.
By default, the endpoint will simply return 200 OK responses, unless it is given a different handler upon creation.


::

    Deproxy deproxy = new Deproxy()

    DeproxyEndpoint endpoint = deproxy.addEndpoint(9999)

    def theProxy = new TheProxy(port: 8080,
                                targetHostname: "localhost",
                                targetPort: 9999)

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1




How To: Auxiliary Service
-------------------------

A more complicated case is when the proxy has to call out to some auxiliary service for additional information.

::

  ________                         ________                        ________
 |        |  --->  Request  --->  |        |  ---> Request  --->  |        |
 | Client |                       | Proxy  |                      | Server |
 |________|  <---  Response <---  |________|  <--- Response <---  |________|

                                    |    ^
                            Request |    |  Response
                                    v    |
                                   ________
                                  |  Aux.  |
                                  |Service |
                                  |________|

An excellent example would be an authentication system.
The client sends the request to the proxy and includes credentials.
In order to determine if the credentials are valid, the proxy makes a separate HTTP request to the auth service, which then responds with yea or nay.
Depending on whether the credentials are valid or not, the proxy will either forward the request on to the server, or return an error back to the client.

In this setup, we can simulate both the server and the auxiliary service with DeproxyEndpoint objects.
The endpoint representing the auth service would have to be given a custom handler, that could interpret and respond to the authentication requests that the proxy makes according to whatever contract is necessary.

::

    Deproxy deproxy = new Deproxy()

    def endpoint = deproxy.addEndpoint(9999)

    def authResponder = new AuthResponder()
    def authService = deproxy.addEndpoint(7777, defaultHandler: authResponder.handler)

    def theProxy = new TheProxy(port: 8080,
                                targetHostname: "localhost",
                                targetPort: 9999,
                                authServiceHostname: "localhost",
                                authServicePort: 7777)

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource",
                                 headers: ['X-User': 'valid-user'])

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint
    assert mc.orphanedHandlings.size() == 1
    assert mc.orphanedHandlings[0].endpoint == authService

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource",
                                 headers: ['X-User': 'invalid-user'])

    assert mc.receivedResponse.code == "403"
    assert mc.handlings.size() == 0
    assert mc.orphanedHandlings.size() == 1
    assert mc.orphanedHandlings[0].endpoint == authService



How To: Multiple Servers
------------------------

Sometimes, a proxy might be set up in front of multiple servers.

::

                                                                   ________
                                    .------------> Request  --->  |        |
                                    |                             | Server1|
                                    |    .-------- Response <---  |________|
                                    |    |
                                    |    v
  ________                         ________                        ________
 |        |  --->  Request  --->  |        |  ---> Request  --->  |        |
 | Client |                       | Proxy  |                      | Server2|
 |________|  <---  Response <---  |________|  <--- Response <---  |________|

                                    ^    |
                                    |    |                         ________
                                    |    `-------- Request  --->  |        |
                                    |                             | Server3|
                                    `------------- Response <---  |________|

This might be the case, for example, if it is acting as a load balancer, or providing access to different versions of a ReST api based on uri.
This is simple enough to simulate by creating multiple endpoint objects, and configuring the proxy to forward client requests to them.


::

    Deproxy deproxy = new Deproxy()

    def endpoint1 = deproxy.addEndpoint(9999)
    def endpoint2 = deproxy.addEndpoint(9998)
    def endpoint3 = deproxy.addEndpoint(9997)

    def theProxy = new TheProxy(port: 8080,
                                targets: [
                                    ['hostname': "localhost", port: 9999],
                                    ['hostname': "localhost", port: 9998],
                                    ['hostname': "localhost", port: 9997]],
                                loadBalanceBehavior: Behavior.RoundRobin)

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint1

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint2

    def mc = deproxy.makeRequest(url: "http://localhost:8080/path/to/resource")

    assert mc.receivedResponse.code == "200"
    assert mc.handlings.size() == 1
    assert mc.handlings[0].endpoint == endpoint3


Routing
-------

Even more complex situations can be created using the Route_ built-in handler, to route requests to existing servers.
::
  ________          ________          ________        ________
 |        |  --->  |        |  --->  |  Fake  | ---> |  Real  |
 | Client |        | Proxy  |        | Server |      | Server |
 |________|  <---  |________|  <---  |________| <--- |________|

                     |    ^
                     v    |
                    ________
                   |Fake Aux|
                   |Service |
                   |________|

                     |    ^
                     v    |
                    ________
                   |Real Aux|
                   |Service |
                   |________|


This will work for requests from the proxy to an auxiliary service, or from the client/proxy to the server.
It can save us the trouble of implementing our own handlers to simulate the server or auxiliary service.
But why this round-about way of doing it? Why not just configure the proxy to send requests to those locations directly?
The real advantage to this method is that the requests go through an endpoint, so the Request_ and Response_ get captured and attached to a MessageChain_.
When makeRequest_ returns, we can make assertions against those requests and responses, which would be entirely invisible to us if the proxy had sent them directly.

::

    Deproxy deproxy = new Deproxy()

    def endpoint = deproxy.addEndpoint(9999,
            defaultHandler: Handlers.Route("real.server.example.com"))

    def authService = deproxy.addEndpoint(7777,
            defaultHandler: Handlers.Route("real.auth.service.example.com"))

    def theProxy = new TheProxy(port: 8080,
                                targetHostname: "localhost",
                                targetPort: 9999,
                                authServiceHostname: "localhost",
                                authServicePort: 7777)


Endpoint Lifecycle
------------------

1. When an endpoint is created, it opens a socket on the designated port and spawns a thread to listen for connections to that socket.
2. Whenever a new connection is made, the listener thread will spawn a new handler thread.
3. The handler thread will proceed to service HTTP request, like so:
    a. First the incoming request is read from the socket, and parsed into a Request_ object.
    b. The endpoint will examine the request headers for a ``Deproxy-Request-ID`` header, and then try to match it to an existing MessageChain_ (created before in a call to makeRequest_).
    c. The endpoint will then determine which handler to use (see :ref:`Handler Resolution Procedure <handlerResolutionProcedure>`), and pass the Request object to the handler to get a Response_ object.
{'fraction_non_alphanumeric': 0.09322935129386742, 'fraction_numerical': 0.014280650225350687, 'mean_word_length': 1.6486051502145922, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 72, 'word_list_counts': {'cursed_substrings.json': 56, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3696778', 'n_tokens_mistral': 5083, 'n_tokens_neox': 4582, 'n_words': 1734}