Zero copy

Let's first look at its definition:

"Zero-copy" describes computer operations in which the CPU does not perform the task of copying data from one memory area to another. This is frequently used to save CPU cycles and memory bandwidth when transmitting a file over a network.

The so-called zero copy means that when operating data, there is no need to copy the data buffer from one memory area to another. There is less memory copy, which reduces the execution of cpu and saves memory bandwidth.

Operating system level zero copy

At the OS level, zero copy usually refers to avoiding copying data back and forth between user space and kernel space.

• For example, the mmap system call provided by Linux can map a section of user space memory to the kernel space. When the mapping is successful, the user's modifications to this section of memory area can be directly reflected to the kernel space;

• Kernel space changes to this area also directly reflect user space. Because of this mapping relationship, we do not need to copy data between user space and kernel space, which improves the efficiency of data transmission.

The zero copy in netty is different from the zero copy on the OS level mentioned above. Netty's zero copy is completely in the user state (Java level), and its zero copy is more inclined to the concept of optimizing data operations

Zero copy of Netty

• Netty provides the CompositeByteBuf class, which can combine multiple bytebufs into one logical ByteBuf, avoiding the copy between each ByteBuf.

• Through the wrap operation, we can wrap byte [] array, ByteBuf, ByteBuffer, etc. into a Netty ByteBuf object, thus avoiding the copy operation.

• ByteBuf supports slice operation, so ByteBuf can be decomposed into multiple bytebufs sharing the same storage area, avoiding memory copy.

• Filechannel wrapped through FileRegion Tranferto realizes file transfer and can directly send the data in the file buffer to the target Channel, avoiding the memory copy problem caused by the traditional circular write method.

Zero copy via CompositeByteBuf

Suppose we have a protocol data, which consists of a header and a message body, which are stored in two bytebufs, namely:

ByteBuf header = ...
ByteBuf body = ...

In code processing, you usually want to combine header and body into a ByteBuf for easy processing. The usual methods are as follows:

ByteBuf allBuf = Unpooled.buffer(header.readableBytes() + body.readableBytes());
allBuf.writeBytes(header);
allBuf.writeBytes(body);

As you can see, we have copied the header and body to the new allBuf, which virtually adds two additional data copying operations. So is there a more efficient and elegant way to achieve the same goal? Let's take a look at how CompositeByteBuf implements such requirements

ByteBuf header = ...
ByteBuf body = ...
CompositeByteBuf compositeByteBuf = Unpooled.compositeBuffer();
compositeByteBuf.addComponents(true, header, body);

In the above code, we define a CompositeByteBuf object and then call it.

public CompositeByteBuf addComponents(boolean increaseWriterIndex, ByteBuf... buffers) {
...
}

Method combines header and body into a logical ByteBuf, that is:

However, it should be noted that although it seems that CompositeByteBuf is composed of two bytebufs, the two bytebufs exist separately within CompositeByteBuf, and CompositeByteBuf is only a logical whole

In the above CompositeByteBuf code, it is worth noting that we call addComponents(boolean increaseWriterIndex, ByteBuf... buffers) to add two bytebufs. The first parameter is true, which means that when a new ByteBuf is added, the writeIndex of CompositeByteBuf is automatically incremented.

In addition to directly using the CompositeByteBuf class above, we can also use unpooled Wrappedbuffer method, which encapsulates the compositebytebuffer operation at the bottom, so it is more convenient to use:

ByteBuf header = ...
ByteBuf body = ...
ByteBuf allByteBuf = Unpooled.wrappedBuffer(header, body);

Zero copy through wrap operation

We have a byte array. We want to convert it into a ByteBuf object for subsequent operations. The traditional method is to copy this byte array into ByteBuf, that is:

byte[] bytes = ...
ByteBuf byteBuf = Unpooled.buffer();
byteBuf.writeBytes(bytes);

Obviously, there is also an additional copy operation in this way. We can use the relevant methods of Unpooled to wrap the byte array and generate a new ByteBuf instance without copying. The above code can be changed to:

byte[] bytes = ...
ByteBuf byteBuf = Unpooled.wrappedBuffer(bytes);

Through unpooled Wrappedbuffer method to wrap bytes into an UnpooledHeapByteBuf object, and there will be no copy operation in the packaging process That is, the generated ByteBuf object shared the same storage space as the bytes array, and the modification of bytes will also be reflected in the ByteBuf object

Zero copy through slice operation

Slice operation is the opposite of wrap operation, unpooled Wrappedbuffer can merge multiple bytebuffs into one, while slice operation can slice a bytebuff into multiple bytebuff objects sharing a storage area
ByteBuf provides two slice operation methods:

public ByteBuf slice();
public ByteBuf slice(int index, int length);

Slice method without parameters is equivalent to buf slice(buf.readerIndex(), buf. Readablebytes()), which returns the slice of the readable part in buf slice(int index, int length) method is relatively flexible. We can set different parameters to obtain slices in different regions of buf

ByteBuf byteBuf = ...
ByteBuf header = byteBuf.slice(0, 5);
ByteBuf body = byteBuf.slice(5, 10);

There is no copy operation in the process of generating header and body with slice method. The header and body objects actually share different parts of byteBuf storage space internally Namely:

Zero copy via FileRegion

Nety uses FileRegion to realize zero copy of file transfer, but the underlying FileRegion depends on Java NiO filechannel Zero copy function of transfer

First, let's start with the most basic Java IO Assuming that we want to realize the function of copying a file, using the traditional method, we have the following implementation:

public static void copyFile(String srcFile, String destFile) throws Exception {
    byte[] temp = new byte[1024];
    FileInputStream in = new FileInputStream(srcFile);
    FileOutputStream out = new FileOutputStream(destFile);
    int length;
    while ((length = in.read(temp)) != -1) {
        out.write(temp, 0, length);
    }
    in.close();
    out.close();
}

The above is a typical code implementation of reading and writing binary files Needless to say, we all know that in the above code, we constantly read fixed length data from the source file into the temp array, and then write the contents of temp into the destination file. Such copying operation does not have much impact on small files, but if we need to copy large files, frequent memory copying operations will consume a lot of system resources, Let's take a look at how FileChannel using Java NIO implements zero copy:

public static void copyFileWithFileChannel(String srcFileName, String destFileName) throws Exception {
    RandomAccessFile srcFile = new RandomAccessFile(srcFileName, "r");
    FileChannel srcFileChannel = srcFile.getChannel();
    RandomAccessFile destFile = new RandomAccessFile(destFileName, "rw");
    FileChannel destFileChannel = destFile.getChannel();
    long position = 0;
    long count = srcFileChannel.size();
    srcFileChannel.transferTo(position, count, destFileChannel);
}

It can be seen that after using FileChannel, we can directly copy (transfer to) the contents of the source file to the destination file without using an additional temporary buffer to avoid unnecessary memory operations. Let's take a look at how FileRegion is used to realize zero copy transmission of a file in Netty:

@Override
public void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
    RandomAccessFile raf = null;
    long length = -1;
    try {
        // 1. Open a file through RandomAccessFile
        raf = new RandomAccessFile(msg, "r");
        length = raf.length();
    } catch (Exception e) {
        ctx.writeAndFlush("ERR: " + e.getClass().getSimpleName() + ": " + e.getMessage() + '\n');
        return;
    } finally {
        if (length < 0 && raf != null) {
            raf.close();
        }
    }
    ctx.write("OK: " + raf.length() + '\n');
    if (ctx.pipeline().get(SslHandler.class) == null) {
        // SSL not enabled - can use zero-copy file transfer.
        // 2. Call RAF Getchannel() gets a FileChannel
        // 3. Encapsulate FileChannel into a DefaultFileRegion
        ctx.write(new DefaultFileRegion(raf.getChannel(), 0, length));
    } else {
        // SSL enabled - cannot use zero-copy file transfer.
        ctx.write(new ChunkedFile(raf));
    }
    ctx.writeAndFlush("\n");
}

You can see that the first step is to open the file through RandomAccessFile, and then Netty uses DefaultFileRegion to encapsulate a FileChannel, that is:

new DefaultFileRegion(raf.getChannel(), 0, length)

java zero copy

The "zero" of zero copy means that the number of copies of data between user state and kernel state is zero.

Traditional data copy (file to file, client to server, etc.) involves four user state kernel state switches and four copies. Among the four copies, two copies between user state and kernel state require CPU participation, and two copies between kernel state and IO device are DMA mode without CPU participation. Zero copy avoids copy between user state and kernel state and reduces two user state kernel state switches.

• Java zero copy is mostly used in network applications. Java libaries supports zero copy in linux and unix. The key api is Java nio. channel. transferTo(), transferFrom() method of filechannel.

• These two methods can be used to transfer bytes directly from the calling channel to another writable byte channel without passing data through the application, so as to improve the efficiency of data transfer.

Using zero copy technology in Web Environment

Many web applications will provide users with a large amount of static content, which means that a lot of data will be transmitted to users intact through socket s after being read out from the hard disk. This operation may not seem to consume much CPU, but it is actually inefficient.

Original copy technology

Kernal reads the data from the disk, then transfers it to the user level application, and then the application sends the same content back to the socket at kernal level. In fact, the application is only used as an inefficient intermediate medium to transfer the data of disk file to the socket.

Zero copy technology

Every time data passes through the user kernel boundary, it will be copied, which will consume CPU and RAM bandwidth. Therefore, you can use a technology called zero copy to remove these unnecessary copies.

• The application uses zero copy to request the kernel to directly transfer the disk data to the socket, rather than through the application. Zero copy improves application performance and reduces context switching between kernel and user modes.
• Using the kernel buffer as an intermediary (instead of directly transferring data to the user buffer) seems inefficient (an extra copy). However, in fact, kernel buffer is used to improve performance.

Disadvantages of zero copy

During read operations, the kernel buffer acts as a read ahead cache. When the data size of the write request is smaller than that of the kernel buffer, this can significantly improve the performance. When writing, the existence of kernel buffer can make the write request completely asynchronous.

Unfortunately, when the requested data size is much larger than the kernel buffer size, this method itself becomes a performance bottleneck. Because data needs to be copied many times between disk, kernel buffer and user buffer (the whole buffer is filled each time).

Zero copy improves performance by eliminating these redundant data copies.

Traditional methods and context switching involved

Transfer a file to another program through the network. Within the OS, the copy operation undergoes four context switches between user mode and kernel mode, and even the data is copied four times,
The specific steps are as follows:

• The read () call causes a context switch from user mode to kernel mode. Sys was called internally_ read() to read data from the file. The first copy is completed by DMA (direct memory access). The file contents are read out from the disk and stored in the buffer of the kernel.

• Then, the requested data is copied to the user buffer. At this time, read() returns successfully. The return of the call triggers the second context switch: from kernel to user. So far, the data is stored in the user's buffer.

• send() Socket call brings the third context switch, this time from user mode to kernel mode. At the same time, the third copy occurred: put the data into the kernel adress space. Of course, the kernel buffer this time is different from the buffer in the first step.

• Finally, the send() system call returned, which also caused the fourth context switch. At the same time, the fourth copy occurs, and DMA egine copies the data from the kernel buffer to the protocol engine. The fourth copy is independent and asynchronous.

zero copy mode and context conversion involved

In Linux kernel 2.4 and above (such as linux 6 or centos 6 or above), the developer modified the socket buffer descriptor to enable the network card to support gather operation and further reduce the data copy operation through the kernel. This method not only reduces the context switch, but also eliminates the data copy related to the CPU. The user level usage method has not changed, but the internal principle has changed:

The transferTo() method causes the file contents to be copied to the kernel buffer, which is completed by the DMA engine. No data is copied to the socket buffer. Instead, the socket buffer is appended with some descriptor information, including the location and length of data. Then the DMA engine transfers data directly from the kernel buffer to the protocol engine, which eliminates the only copy operation that needs to occupy CPU.

Java NIO zero copy example

FileChannel in NIO has two methods, transferTo and transferFrom, which can directly copy the data in FileChannel to another Channel or directly copy the data in another Channel to FileChannel. This interface is often used for efficient network / file data transmission and large file copy.

When supported by the operating system, this method does not need to copy the source data from the kernel state to the user state, and then from the user state to the kernel state of the target channel. At the same time, it also avoids two context switches between the user state and the kernel state, that is, it uses "zero copy", so its performance is generally higher than that of the method provided in Java IO.

Transfer a file from the client to the server through the network:

/**
 * disk-nic Zero-copy 
 */
class ZerocopyServer {
    ServerSocketChannel listener = null;
    protected void mySetup() {
        InetSocketAddress listenAddr = new InetSocketAddress(9026);
        try {
            listener = ServerSocketChannel.open();
            ServerSocket ss = listener.socket();
            ss.setReuseAddress(true);
            ss.bind(listenAddr);
            System.out.println("Listening port:" + listenAddr.toString());
        } catch (IOException e) {
            System.out.println("Port binding failed : " + listenAddr.toString() + " The port may already be in use,Error reason: " + e.getMessage());
            e.printStackTrace();
        }

    }

    public static void main(String[] args) {
        ZerocopyServer dns = new ZerocopyServer();
        dns.mySetup();
        dns.readData();
    }

    private void readData() {
        ByteBuffer dst = ByteBuffer.allocate(4096);
        try {
            while (true) {
                SocketChannel conn = listener.accept();
                System.out.println("Created connection: " + conn);
                conn.configureBlocking(true);
                int nread = 0;
                while (nread != -1) {
                    try {
                        nread = conn.read(dst);
                    } catch (IOException e) {
                        e.printStackTrace();
                        nread = -1;
                    }
                    dst.rewind();
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

class ZerocopyClient {
    public static void main(String[] args) throws IOException {
        ZerocopyClient sfc = new ZerocopyClient();
        sfc.testSendfile();
    }

    public void testSendfile() throws IOException {
        String host = "localhost";
        int port = 9026;
        SocketAddress sad = new InetSocketAddress(host, port);
        SocketChannel sc = SocketChannel.open();
        sc.connect(sad);
        sc.configureBlocking(true);
        String fname = "src/main/java/zerocopy/test.data";
        FileChannel fc = new FileInputStream(fname).getChannel();
        long start = System.nanoTime();
        long nsent = 0, curnset = 0;
        curnset = fc.transferTo(0, fc.size(), sc);
        System.out.println("Total bytes sent:" + curnset + " time consuming(ns):" + (System.nanoTime() - start));
        try {
            sc.close();
            fc.close();
        } catch (IOException e) {
            System.out.println(e);
        }
    }
}

File to file zero copy

/**
 * disk-disk Zero-copy 
 */
class ZerocopyFile {
    @SuppressWarnings("resource")
    public static void transferToDemo(String from, String to) throws IOException {
        FileChannel fromChannel = new RandomAccessFile(from, "rw").getChannel();
        FileChannel toChannel = new RandomAccessFile(to, "rw").getChannel();
        long position = 0;
        long count = fromChannel.size();
        fromChannel.transferTo(position, count, toChannel);
        fromChannel.close();
        toChannel.close();
    }
    @SuppressWarnings("resource")
    public static void transferFromDemo(String from, String to) throws IOException {
        FileChannel fromChannel = new FileInputStream(from).getChannel();
        FileChannel toChannel = new FileOutputStream(to).getChannel();
        long position = 0;
        long count = fromChannel.size();
        toChannel.transferFrom(fromChannel, position, count);
        fromChannel.close();
        toChannel.close();
    }
    public static void main(String[] args) throws IOException {
        String from = "src/main/java/zerocopy/1.data";
        String to = "src/main/java/zerocopy/2.data";
        // transferToDemo(from,to);
        transferFromDemo(from, to);
    }
}