<commit_before># Improving the performance of Warp again

As you may remember, I improved the performance of Warp in 2012 and wrote [some blog articles](http://www.yesodweb.com/blog/2012/09/improving-warp) on this web site. Based on these articles, Michael and I wrote an article ["Warp"](http://aosabook.org/en/posa/warp.html) for POSA (Performance of Open Source Applications).

After working with Andreas last year, I came up with some new ideas to yet again improve Warp's performance, and have implemented them in Warp 2.0. In this article, I will explain these new techniques. If you have not read the POSA article, I recommend to give a look at it beforehand.

Here are the high level ideas to be covered:

- Better thread scheduling
- Buffer allocation to receive HTTP requests
- Buffer allocation to send HTTP responses
- HTTP request parser

## witty

When I was testing [multicore IO manager](http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask035-voellmy.pdf) with Mighty (on Warp), I noticed bottlenecks of network programs complied by GHC. To show such bottlenecks actually exist, I wrote a server program called [witty](https://github.com/kazu-yamamoto/witty) based on Andreas's "[SimpleServer](https://github.com/AndreasVoellmy/SimpleServer)".

"witty" provides seven command line options to switch standard methods to alternatives. My experiment with "witty" disclosed the following bottlenecks:

- Thread scheduling (the "-y" option)
- `Network.Socket.ByteString.recv` (the "-r" option)

## Better thread scheduling

GHC's I/O functions are optimistic.
For instance, `recv` tries to read incoming data immediately.
If there is data available, `recv` succeeds.
Otherwise, `EAGAIN` is returned.
In this case, `recv` asks the IO manager to notify it when data is available.

If a network server repeats receive/send actions,
`recv` just after `send` will usually fail because
there is a time lag for the next request from the client.
Thus the IO manager is often required to do notification work.
Here is a log of "strace":

    recvfrom(13, )                -- Haskell thread A
    sendto(13, )                  -- Haskell thread A
    recvfrom(13, ) = -1 EAGAIN    -- Haskell thread A
    epoll_ctl(3, )                -- Haskell thread A (a job for the IO manager)
    recvfrom(14, )                -- Haskell thread B
    sendto(14, )                  -- Haskell thread B
    recvfrom(14, ) = -1 EAGAIN    -- Haskell thread B
    epoll_ctl(3, )                -- Haskell thread B (a job for the IO manager)

To achieve better scheduling, we need to call `yield` after `send`.
`yield` pushes its Haskell thread onto the end of thread queue.
In the meanwhile, other threads are able to work.
During the work of other threads, a new request message can arrive,
and therefore the next call to `recv` will succeed.
With the `yield` hack, a log of "strace" becames as follows:

    recvfrom(13, )                -- Haskell thread A
    sendto(13, )                  -- Haskell thread A
    recvfrom(14, )                -- Haskell thread B
    sendto(14, )                  -- Haskell thread B
    recvfrom(13, )                -- Haskell thread A
    sendto(13, )                  -- Haskell thread A

In other words, `yield` makes the IO manager work less frequently.
This magically improves throughput!
This means that even multicore IO manager still has unignorable overhead.
It uses `MVar` to notify data availability to Haskell threads.
Since `MVar` is a lock, it may be slow.
Or more precisely, allocation of the `MVar` may be slow.

Unfortunately when I first added `yield` to Warp, no performance improvement were
gained. It seems to me that Monad stack (`ResourceT`) handcuffs the `yield`
hack. So, Michael removed `ResourceT` from WAI. This is why the type of
`Application` change from:

    type Application = Request -> ResourceT IO Response

to:

    type Application = Request -> IO Response<commit_msg>a tiny fix for new-warp.<commit_after># Improving the performance of Warp again

As you may remember, I improved the performance of Warp in 2012 and wrote [some blog articles](http://www.yesodweb.com/blog/2012/09/improving-warp) on this web site. Based on these articles, Michael and I wrote an article ["Warp"](http://aosabook.org/en/posa/warp.html) for POSA (Performance of Open Source Applications).

After working with Andreas last year, I came up with some new ideas to yet again improve Warp's performance, and have implemented them in Warp 2.0. In this article, I will explain these new techniques. If you have not read the POSA article, I recommend to give a look at it beforehand.

Here are the high level ideas to be covered:

- Better thread scheduling
- Buffer allocation to receive HTTP requests
- Buffer allocation to send HTTP responses
- HTTP request parser

## witty

When I was testing [multicore IO manager](http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask035-voellmy.pdf) with Mighty (on Warp), I noticed bottlenecks of network programs complied by GHC. To show such bottlenecks actually exist, I wrote a server program called [witty](https://github.com/kazu-yamamoto/witty) based on Andreas's "[SimpleServer](https://github.com/AndreasVoellmy/SimpleServer)".

"witty" provides seven command line options to switch standard methods to alternatives. My experiment with "witty" disclosed the following bottlenecks:

- Thread scheduling (the "-y" option)
- `Network.Socket.ByteString.recv` (the "-r" option)

## Better thread scheduling

GHC's I/O functions are optimistic.
For instance, `recv` tries to read incoming data immediately.
If there is data available, `recv` succeeds.
Otherwise, `EAGAIN` is returned.
In this case, `recv` asks the IO manager to notify it when data is available.

If a network server repeats receive/send actions,
`recv` just after `send` will usually fail because
there is a time lag for the next request from the client.
Thus the IO manager is often required to do notification work.
Here is a log of "strace":

    recvfrom(13, )                -- Haskell thread A
    sendto(13, )                  -- Haskell thread A
    recvfrom(13, ) = -1 EAGAIN    -- Haskell thread A
    epoll_ctl(3, )                -- Haskell thread A (a job for the IO manager)
    recvfrom(14, )                -- Haskell thread B
    sendto(14, )                  -- Haskell thread B
    recvfrom(14, ) = -1 EAGAIN    -- Haskell thread B
    epoll_ctl(3, )                -- Haskell thread B (a job for the IO manager)

To achieve better scheduling, we need to call `yield` after `send`.
`yield` pushes its Haskell thread onto the end of thread queue.
In the meanwhile, other threads are able to work.
During the work of other threads, a new request message can arrive,
and therefore the next call to `recv` will succeed.
With the `yield` hack, a log of "strace" becames as follows:

    recvfrom(13, )                -- Haskell thread A
    sendto(13, )                  -- Haskell thread A
    recvfrom(14, )                -- Haskell thread B
    sendto(14, )                  -- Haskell thread B
    recvfrom(13, )                -- Haskell thread A
    sendto(13, )                  -- Haskell thread A

In other words, `yield` makes the IO manager work less frequently.
This magically improves throughput!
This means that even multicore IO manager still has unignorable overhead.
It uses `MVar` to notify data availability to Haskell threads.
Since `MVar` is a lock, it may be slow.
Or perhaps, allocation of the `MVar` is slow.

Unfortunately when I first added `yield` to Warp, no performance improvement were
gained. It seems to me that Monad stack (`ResourceT`) handcuffs the `yield`
hack. So, Michael removed `ResourceT` from WAI. This is why the type of
`Application` change from:

    type Application = Request -> ResourceT IO Response

to:

    type Application = Request -> IO Response
{'fraction_non_alphanumeric': 0.07618080243778567, 'fraction_numerical': 0.01244286439817166, 'mean_word_length': 3.1176163094615785, 'pattern_counts': {'":': 2, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '778661', 'n_tokens_mistral': 2320, 'n_tokens_neox': 2081, 'n_words': 1043}