brief introduction

In the previous series of articles, we mentioned that the channel in netty only accepts ByteBuf objects. If it is not a ByteBuf object, it needs to be converted with a coder and decoder. Today, let's talk about the problems that need to be paid attention to in the implementation of netty's custom coder and decoder.

Implementation of custom encoder and decoder

Before introducing netty's own encoder and decoder, let's tell you how to implement a custom encoder and decoder.

All encoders and decoders in netty are derived from ChannelInboundHandlerAdapter and ChannelOutboundHandlerAdapter.

The two most important classes for the ChannelOutboundHandlerAdapter are MessageToByteEncoder and MessageToMessageEncoder.

MessageToByteEncoder encodes messages into ByteBuf. This class is also the most commonly used class for our custom coding. You can directly inherit this class and implement the encode method. Notice that this class has a generic type, which specifies the object type of the message.

For example, we want to convert Integer to ByteBuf, which can be written as follows:

       public class IntegerEncoder extends MessageToByteEncoder<Integer> {
            @Override
           public void encode(ChannelHandlerContext ctx, Integer msg, ByteBuf out)
                   throws Exception {
               out.writeInt(msg);
           }
       }

MessageToMessageEncoder is used to convert between messages. Because messages cannot be written directly to the channel, it needs to be used with MessageToByteEncoder.

The following is an example of Integer to String:

       public class IntegerToStringEncoder extends
               MessageToMessageEncoder<Integer> {
  
            @Override
           public void encode(ChannelHandlerContext ctx, Integer message, List<Object> out)
                   throws Exception {
               out.add(message.toString());
           }
       }

For ChannelInboundHandlerAdapter, the two most important classes are ByteToMessageDecoder and MessageToMessageDecoder.

ByteToMessageDecoder converts ByteBuf into the corresponding message type. We need to inherit this class and implement the decode method. The following is a method to read all readable bytes from ByteBuf and put the results into a new ByteBuf,

       public class SquareDecoder extends ByteToMessageDecoder {
            @Override
           public void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out)
                   throws Exception {
               out.add(in.readBytes(in.readableBytes()));
           }
       }
   

MessageToMessageDecoder is the conversion between messages. Similarly, you only need to implement the decode method, as shown in the following conversion from String to Integer:

       public class StringToIntegerDecoder extends
               MessageToMessageDecoder<String> {
  
            @Override
           public void decode(ChannelHandlerContext ctx, String message,
                              List<Object> out) throws Exception {
               out.add(message.length());
           }
       }

ReplayingDecoder

The above code looks very simple, but there are still some problems to pay attention to in the implementation process.

For the Decoder, we read the data from ByteBuf and then convert it. However, the data changes in ByteBuf are unknown during reading. It is possible that ByteBuf is not ready during reading, so it is necessary to judge the size of readable bytes in ByteBuf during reading.

For example, we need to parse a data structure. The first four bytes of this data structure are an int, which indicates the length of the following byte array. We need to first judge whether there are four bytes in the byte buf, then read these four bytes as the length of the byte array, and then read the byte array of this length, and finally get the result to be read. If there is a problem in one step, In other words, if the readable byte length is not enough, you need to return directly and wait for the next reading. As follows:

   public class IntegerHeaderFrameDecoder extends ByteToMessageDecoder {
  
      @Override
     protected void decode(ChannelHandlerContext ctx,
                             ByteBuf buf, List<Object> out) throws Exception {
  
       if (buf.readableBytes() < 4) {
          return;
       }
  
       buf.markReaderIndex();
       int length = buf.readInt();
  
       if (buf.readableBytes() < length) {
          buf.resetReaderIndex();
          return;
       }
  
       out.add(buf.readBytes(length));
     }
   }

This judgment is complex and may be wrong. In order to solve this problem, netty provides a ReplayingDecoder to simplify the above operations. In the ReplayingDecoder, it is assumed that all bytebufs are ready and can be read directly from the middle.

The above example is rewritten with ReplayingDecoder as follows:

   public class IntegerHeaderFrameDecoder
        extends ReplayingDecoder<Void> {
  
     protected void decode(ChannelHandlerContext ctx,
                             ByteBuf buf, List<Object> out) throws Exception {
  
       out.add(buf.readBytes(buf.readInt()));
     }
   }

Its implementation principle is to try to read the corresponding byte information. If it is not read, an exception will be thrown. After receiving the exception, the ReplayingDecoder will call the decode method again.

Although ReplayingDecoder is very simple to use, it has two problems.

The first problem is the performance problem, because the decode method will be called repeatedly. If the ByteBuf itself does not change, it will cause the same ByteBuf to be decoded repeatedly, resulting in a waste of performance. The solution to this problem is to decode in stages. For example, in the above example, we need to read the length of byte array first, and then read the real byte array. Therefore, after reading the sum of byte array lengths, you can call the checkpoint() method to make a savepoint. When the decode method is executed next time, you can skip this savepoint and continue the subsequent execution process, as shown below:

   public enum MyDecoderState {
     READ_LENGTH,
     READ_CONTENT;
   }
  
   public class IntegerHeaderFrameDecoder
        extends ReplayingDecoder<MyDecoderState> {
  
     private int length;
  
     public IntegerHeaderFrameDecoder() {
       // Set the initial state.
       super(MyDecoderState.READ_LENGTH);
     }
  
      @Override
     protected void decode(ChannelHandlerContext ctx,
                             ByteBuf buf, List<Object> out) throws Exception {
       switch (state()) {
       case READ_LENGTH:
         length = buf.readInt();
         checkpoint(MyDecoderState.READ_CONTENT);
       case READ_CONTENT:
         ByteBuf frame = buf.readBytes(length);
         checkpoint(MyDecoderState.READ_LENGTH);
         out.add(frame);
         break;
       default:
         throw new Error("Shouldn't reach here.");
       }
     }
   }

The second problem is that the decode method of the same instance may be called multiple times. If we have a private variable in ReplayingDecoder, we need to consider cleaning the private variable to avoid data pollution caused by multiple calls.

summary

By inheriting the above classes, we can implement the encoding and decoding logic ourselves. But there seems to be a problem. Is custom coding and decoder too complicated? You also need to determine the size of the byte array to be read. Is there a simpler way?

Yes, please look forward to the next article in the netty series: netty's own encoder and decoder

Examples of this article can be referred to: learn-netty4

This article has been included in http://www.flydean.com/14-netty-cust-codec/

The most popular interpretation, the most profound dry goods, the most concise tutorial, and many tips you don't know are waiting for you to find!

Welcome to my official account: "those things in procedure", understand technology, know you better!