brief introduction

In the last article, we talked about how to customize the codec and decoder in netty, but the customization implementation is still very complex. Generally, when there is no special need, we all hope that the simpler the better. The difficulty is to find the segmentation points in ByteBuf and divide ByteBuf into manageable units. Today, this article will talk about the partition processing mechanism in netty.

Frame detection

In the previous chapter, we mentioned the need for a means to distinguish different data in ByteBuf, that is, to find the segmentation points of different data in ByteBuf. If ByteBuf is first divided into independent ByteBuf, and then the independent ByteBuf is processed, it will be much easier.

netty provides encoders for four segmentation points, which we can call Frame detection. They are DelimiterBasedFrameDecoder, FixedLengthFrameDecoder, LengthFieldBasedFrameDecoder, and LineBasedFrameDecoder.

These classes are subclasses of ByteToMessageDecoder. Let's introduce them one by one.

DelimiterBasedFrameDecoder

The first is the DelimiterBasedFrameDecoder. You can see from the name that this is the decoder that divides bytebuf according to the delimiter. What is a delimiter?

netty has a Delimiters class, which specifically defines the characters to be segmented. There are mainly two Delimiters, nuldefiniter and lineDelimiter:

public static ByteBuf[] nulDelimiter() {
        return new ByteBuf[] {
                Unpooled.wrappedBuffer(new byte[] { 0 }) };
    }

    public static ByteBuf[] lineDelimiter() {
        return new ByteBuf[] {
                Unpooled.wrappedBuffer(new byte[] { '\r', '\n' }),
                Unpooled.wrappedBuffer(new byte[] { '\n' }),
        };
    }

nullDelimiter is used to process 0x00. It is mainly used to process Flash XML socket or other similar protocols.

lineDelimiter is used to handle carriage returns and line breaks, mainly in the processing of text files.

For the DelimiterBasedFrameDecoder, if there are multiple delimiters, the shortest ByteBuf will be selected. For example, if we use the DelimiterBasedFrameDecoder(Delimiters.lineDelimiter()), because there are actually two segmentation methods in the lineDelimiter, carriage return + line feed or line feed. If the following conditions are encountered:

   +--------------+
   | ABC\nDEF\r\n |
   +--------------+

The DelimiterBasedFrameDecoder will select the shortest segmentation result, that is, divide the above content into:

   +-----+-----+
   | ABC | DEF |
   +-----+-----+

instead of

   +----------+
   | ABC\nDEF |
   +----------+

FixedLengthFrameDecoder

This class will divide ByteBuf into fixed lengths. For example, the following four pieces of byte information are received:

   +---+----+------+----+
   | A | BC | DEFG | HI |
   +---+----+------+----+

If a FixedLengthFrameDecoder(3) is used, the above ByteBuf will be divided into the following parts:

   +-----+-----+-----+
   | ABC | DEF | GHI |
   +-----+-----+-----+

LengthFieldBasedFrameDecoder

This class is a little more flexible. You can take out the subsequent byte array according to the length field in the data. LengthFieldBasedFrameDecoder is very flexible. It has four properties to control them: lengthFieldOffset, lengthFieldLength, lengthAdjustment and initialBytesToStrip.

lengthFieldOffset is the starting position of the length field, lengthFieldLength is the length of the length field itself, lengthAdjustment is to adjust the length of the target data, and initialBytesToStrip is the number of bytes to be deleted during decryption. Can't you understand? It doesn't matter. Let's give a few examples.

Let's start with the simplest one:

   lengthFieldOffset   = 0
   lengthFieldLength   = 2
   lengthAdjustment    = 0
   initialBytesToStrip = 0 

   BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes)
   +--------+----------------+      +--------+----------------+
   | Length | Actual Content |----->| Length | Actual Content |
   | 0x000C | "HELLO, WORLD" |      | 0x000C | "HELLO, WORLD" |
   +--------+----------------+      +--------+----------------+

The above setting indicates that the length starts from bit 0 and the length is 2 bytes. Where Ox00C=12, which is also the length of "HELLO, WORLD".

If you do not want the length field, you can delete the length by setting initialBytesToStrip:

   lengthFieldOffset   = 0
   lengthFieldLength   = 2
   lengthAdjustment    = 0
   initialBytesToStrip = 2 (= length Length of field)
  
   BEFORE DECODE (14 bytes)         AFTER DECODE (12 bytes)
   +--------+----------------+      +----------------+
   | Length | Actual Content |----->| Actual Content |
   | 0x000C | "HELLO, WORLD" |      | "HELLO, WORLD" |
   +--------+----------------+      +----------------+

lengthAdjustment is to adjust the value of the Length field. In some cases, the Length field may contain the Length of the whole data, that is, Length + content, so it needs to be adjusted during parsing. For example, in the following example, the real Length is 0x0C, but the incoming Length is 0x0E, so the Length of the Length field needs to be subtracted by 2, That is, set lengthAdjustment to - 2.

   lengthFieldOffset   =  0
   lengthFieldLength   =  2
   lengthAdjustment    = -2 (= Length Length of field)
   initialBytesToStrip =  0

   BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes)
   +--------+----------------+      +--------+----------------+
   | Length | Actual Content |----->| Length | Actual Content |
   | 0x000E | "HELLO, WORLD" |      | 0x000E | "HELLO, WORLD" |
   +--------+----------------+      +--------+----------------+

LineBasedFrameDecoder

LineBasedFrameDecoder deals exclusively with the end of a line in a text file. That is "\ n" and "\ r\n", which are similar to the DelimiterBasedFrameDecoder, but the DelimiterBasedFrameDecoder is more general.

summary

With the above four Frame detection devices, you can first add these Frame detection in pipline, and then add a custom handler, so that the problem of reading the length of ByteBuf does not need to be considered in the custom handler.

For example, in the StringDecoder, if the LineBasedFrameDecoder has been used, in the decode method, it can be assumed that the incoming ByteBuf is a line of string, so it can be used directly as follows:

    protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
        out.add(msg.toString(charset));
    }

Isn't it simple?

This article has been included in http://www.flydean.com/15-netty-buildin-frame-detection/

The most popular interpretation, the most profound dry goods, the most concise tutorial, and many tips you don't know are waiting for you to find!

Welcome to my official account: "those things in procedure", understand technology, know you better!