Q:

How to decide the sequence of actions in Netty channel pipeline

For example, if I want to build a websocket server, I wonder what should be put in the initChannel method. Then I found the websocket example in netty's sourcecode, in which I need to do the following:
public void initChannel(final SocketChannel ch) throws Exception {
                ch.pipeline().addLast(
                    new HttpRequestDecoder(),
                    new HttpObjectAggregator(65536),
                    new HttpResponseEncoder(),
                    new WebSocketServerProtocolHandler("/websocket"),
                    new CustomTextFrameHandler());
            }

But I have no idea why I need to put the objects in such an order. In the description of HttpObjectAggregator I found something like this: 
Be aware that you need to have the {@link HttpResponseEncoder} or {@link HttpRequestEncoder} before the {@link HttpObjectAggregator} in the {@link ChannelPipeline}.
But in the above code HttpObjectAggregator object is before the HttpResponseEncoder object. I am confused. How do I know I am putting those objects in a correct order?

A:

TLDR; You should put HttpServerCodec into your init method, to keep things simple. Do that before HttpObjectAggregator if you choose to use the aggregator.
I'm pretty sure the advice about putting encoders before the HttpObjectAggregator is a typo.  The encoders are outbound only handlers, while the HttpObjectAggregator is an inbound only handler, which means an event will never interact with both of them; so it makes no sense that their relative order would matter.  
The caveat here is that HttpObjectAggregator will write HttpObjects out (mainly a 100 CONTINUE) in certain cases, and for that HttpObject to be converted to a byte[] that can be sent on the wire it needs a HttpResponseEncoder before it in the pipeline.  On outgoing the pipeline is traversed in reverse, so the encoder before it will receive a message sent by the aggregator, but an encoder after it won't.  The sample code you posted has a bug in it that will only be hit if a 100 CONTINUE would need to be sent.  Looks like that bug was fixed by replacing the encoder/decoder with a HttpServerCodec before the aggregator.
A decoder, like HttpRequestDecoder or HttpResponseDecoder is an inbound only handler, and they need to be before the HttpObjectAggregator for it to function properly.  That's because those two decoders transform a byte[] into an HttpObject, while the HttpObjectAggregator is really a Message to Message decoder that transforms a HttpObject into a FullHttpMessage. 
Netty introduced the HttpServerCodec which is a combination of HttpRequestDecoder and HttpResponseEncoder in one class.  If you put that before your aggregator you'll save yourself a line of code and make sure you have the proper encoder and decoder for your server.
Good reference on understanding how a message works in the pipeline for inbound vs outbound handlers: https://netty.io/4.0/api/io/netty/channel/ChannelPipeline.html
Issue where this wording was first introduced (notice no mention of encoding, only decoding): https://github.com/netty/netty/issues/2401
Issue where this wording is pointed out as a typo/bug: https://github.com/netty/netty/issues/2471

