BIO, NIO and AIO are not suitable for students with poor foundation. They need to be proficient in Java OOP programming, have certain programming thinking, be familiar with Java multithreading programming, Java IO stream programming, Java network programming, and be familiar with common Java design patterns |
---|
Most of the people in BIO have been in contact with each other. They will give a brief introduction and focus on the explanation of NIO |
---|
The ups and downs of communication architecture |
---|
- In Java software development, communication architecture is inevitable. Network communication technology, such as games, is required for data interaction between different systems or processes, or communication scenarios with high concurrency
- There are some defects in the early network communication architecture. The most annoying one is the low-performance synchronous blocking I/O communication BIO (the bottom layer is based on IO)
- With the development of the Internet, the communication performance requirements become higher. java began to support non blocking I/O communication technology NIO (bottom buffer based) in 2002
- Communication in LAN
- Underlying message passing mechanism between multiple systems
- Communication scenario of high parallel transmission and large amount of data
- Game industry, mobile game server, large online games
- Specify which channel or communication mode and architecture to use for data transmission. Java supports three network programming I/O models: BIO, NIO and AIO
- Select different I/O models under different business scenarios and performance requirements
Java bio (synchronous and blocking): traditional blocking |
---|
- The server implementation mode is one thread per connection. When the client has a connection request, the server has to start a thread processing. If the connection does nothing, it will cause unnecessary thread overhead.
- It is applicable to the architecture with small and fixed connections, has high requirements for server resources, and concurrency is limited to applications. Jdk1 The only choice before 4, the program is simple
Java NiO (synchronous non blocking) |
---|
- A thread processes multiple requests (connections). The connection requests sent by the client will be registered with the multiplexer (selector). The multiplexer polls the connection. If the currently polled connection has I/O requests, it will process them (open a thread for processing). If there are no requests, it will not wait, and BIO will be blocked all the time
- It is applicable to the architecture with a large number of connections and short connections (light operation), such as chat server, bullet screen system, inter server communication, etc. the programming is complex, jdk1 4 start support
Java AIO(NIO.2, asynchronous, asynchronous, non blocking) |
---|
- A valid request is a thread. After the I/O requests of the client are completed by the OS, the server application is notified to start the thread for processing
- It is applicable to architectures with a large number of connections and long connections (re operation), such as album server. It fully calls the OS to participate in concurrent operation. The programming is more complex, and JDK7 starts to support it
1, Java BIO
- Traditional java io programming, related classes and interfaces are in Java In io package
- BIO(blocking I/O): synchronous blocking. The server implementation mode is one connection and one thread. That is, when the client has a connection request, the server needs to start a thread for processing. If the connection does not do anything, it will cause unnecessary thread overhead. It can be improved through the thread pool mechanism (multiple clients connect to the server)
1. What is synchronous blocking
- In this case, the server will always wait for the client message. If the client does not send the message, the server will always block
- The server receives the message in any way (for example, by line), and the client must send it in the same way (also by line). Otherwise, the server may not receive the message normally or throw an exception directly
- The server can receive messages repeatedly, and the client can also send messages repeatedly
- When the accept() method of the server receives the connection, it will always establish a connection with him. It will not establish a connection with the next client until the accept() method is blocked again
- Therefore, if you want the server to process multiple client requests, you need to introduce threads. Each client request is processed by one thread
- Pseudo asynchronous IO adopts thread pool to avoid the problem of thread resource depletion caused by creating an independent thread for each request. However, the bottom layer still uses synchronous blocking BIO model, which can not fundamentally solve the problem
- If the processing of a single message is slow, or all threads in the server thread pool are blocked, subsequent socket IO messages will be queued in the queue, new socket requests will be rejected, and a large number of connection timeouts will occur at the client.
2. Port forwarding idea: implementation of group chat function
Source code: Src / main / Java / bio / install_ Under the messaging package, just look at it yourself. It's very simple. Just understand the basic ideas |
---|
Realize that a client message can be sent to all clients (group chat system): |
---|
- Client login function: to start client login, you need to enter user name and server ip address
- Real time update of online number: after the client logs in, the contact information bar of all clients is updated synchronously
- Offline number update: after detecting that the client is offline, synchronously update the contact information bar of all clients
- Group chat: push any client message to all current clients for reception
- Private chat: select a client and click private chat to send messages that can be received by the client alone
- @Message: Send a message @ a client, which can be known by others
- Message user and message time point: the server can record the user's message time point in real time for multi-channel forwarding or selection of messages
- Open the server
- idea enable client concurrent execution
- Open three clients: kk, Xiao Liu and Xiao Hong (write all ip addresses 127.0.0.1)
- Use any client to send bbbb, and everyone can see it
- Select a person and send @ message so that others can see it
- Select a person and chat privately
- The server saves the logged in socket with the container, and pushes the message to the specified socket client according to the container
2, Java NIO
New IO or non blocking IO, jdk1 4. It supports buffer oriented and channel based IO operations, and NIO can read and write files in a more efficient way |
---|
- BIO is blocked and waiting for the client message. A thread connecting with the client will always wait for it to complete. Whether it has IO operations or not, it will always occupy the thread without any operations
- NIO keeps polling and scheduling through the selector. Whoever has an IO request will find a thread to process who. If the client does not operate, it will not waste resources on it
- The IO event itself is not blocked, but the select() method to get the IO event needs to be blocked
- BIO will block IO operations, NIO will block event acquisition, and there will be no IO without events. From a high-level perspective, IO will not be blocked
- The essence of NIO is to delay IO operations until IO actually occurs, rather than BIO. As long as the IO stream is open, it will wait for IO operations
NIO has three core parts. NIO has many classes and components, but the three cores constitute the core API. Other components Pipe and FileLock are just tool classes used together with the three core components |
---|
- Channels channel
- The Stream in Channel and IO is almost the same level, but the Stream is unidirectional, such as InputStream or OutputStream.
- The Channel is bidirectional. It can perform both read and write operations
- Channels in NIO mainly include filechannel (file IO), datagram channel (UDP), SocketChannel (TCP, Client) and ServerSocketChannel (TCP, Server)
- Buffers buffer
- Key Buffer implementations include: ByteBuffer(byte), CharBuffer(char), DoubleBuffer(double), FloatBuffer(float), IntBuffer(int), LongBuffer(long) and ShortBuffer(short)
- Selectors selector
- Single thread handles multiple channels. When your application opens multiple channels, but the traffic of each connection is very low, it is very convenient to use Selector
- For example, using Selector in chat server, you have to register Channel with Selector, and then call its select() method. This method will keep blocking until a registered channel has an event ready.
- Once this method returns, the thread can handle these events
1. File channel
1. General
- Data can be read and written through the Channel. Just like water pipes, network data can be read and written through the Channel
- Channels are different from streams. Channels are bidirectional, and streams can only have one direction (InputStream or OutputStream subclass). Channels can read, write or read and write at the same time. Channels are full duplex, which better maps the underlying operating system API than streams
- Channel encapsulates the operation of the data source. The data source can be operated through channel, but the specific physical structure of the data source does not need to be concerned. This data source may be multiple (file, network socket)
- In most applications, channel corresponds to file descriptor or socket one by one. Channel is used to efficiently transfer data between a byte buffer and an entity (usually a file or socket) on the other side of the channel. In other words, it is the channel between buffer and entity, and buffer --------- channel --------- entity.
Channel interface source code analysis |
---|
- Two methods, ispen() returns whether the channel is open, true indicates open, and close() closes the channel
Characteristics of channel |
---|
- Unlike the buffer, the API of channel is mainly specified by the interface. There are fundamental differences in Channel Implementation on different operating systems, so the channel API only describes what can be done. Therefore, channel implementations often use the native code of the operating system. The channel interface allows access to the underlying I/O services in a controlled and portable manner.
- Channel is an object through which data can be read and written. Just like a stream, all data is processed through the Buffer object. Bytes will never be written directly to the channel. On the contrary, data will be written to the Buffer containing one or more bytes. When reading bytes, data is also read from the channel into the Buffer, and then bytes are obtained from the Buffer
- channel can read and write asynchronously
Four implementation classes of channel |
---|
- Filechannel (file IO): read and write data from a file
- Datagram channel (UDP): it can read and write data in the network through UDP
- Socket channel (TCP server): it can read and write data in the network through TCP
- Server socket channel (TCP client): it can listen for new TCP connections, like a Web server. A SocketChannel will be created for each new incoming connection.
2. Filechannel (file IO)
It provides common read-write operations and scatter/gather operations (set / get) for files, as well as many new methods for files |
---|
Description( π
: Not commonly used/ π: (common) | method |
---|
π Read data from Channel to ByteBuffer | int read(ByteBuffer dst) |
π "Scatter" the data in the Channel to ByteBuffer [] | long read(ByteBuffer[] dsts) |
π Write data in ByteBuffer to Channel | int write(ByteBuffer src) |
π "Aggregate" the data in ByteBuffer [] to Channel | long write(ByteBuffer[] srcs) |
π
Returns the file location for this channel | long position() |
π
Set the file location for this channel | FileChannel position(long p) |
π
Returns the current size of the file for this channel | long size() |
π
Truncate the file of this channel to the given size | FileChannel truncate(long s) |
π
Force all file updates for this channel to be written to the storage device | void force(boolean metaData) |
The buffer will be used later, but it must be used now. The usual operation is as follows |
---|
- Write data to buffer
- Call buffer Flip () reverses the read-write mode, which reads data into the buffer. After conversion, you can write directly without creating another output stream
- Read data from buffer
- Call buffer Clear() or buffer Compact() clears the contents of the buffer
1. Introduction case
Using FileChannel step 1: open FileChannel |
---|
- FileChannel cannot be opened directly. You need to obtain a FileChannel instance through an InputStream, OutputStream or RandomAccessFile
Using FileChannel step 2: read data and write the same |
---|
- Channel is only a channel. To read data, you need to use the Buffer to read into the Buffer
- FileChannel. The read () method reads data from the channel into the Buffer
- The read() method returns an int value, indicating how many bytes are read into the Buffer, and returning - 1 indicates that the read is complete
ByteBuffer buf = ByteBuffer.allocate(1024);//Byte buffer, size 1024byte
int byteCount = channel.read(buf);//Read data to buffer
Case, e: / 1 Txt file: Src / main / Java / NiO / Channel / filechanneltest java |
---|
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.StandardCharsets;
public class FileChannelTest {
public static void main(String[] args) {
FileChannelTest fileChannelTest = new FileChannelTest();
try {
fileChannelTest.test();
fileChannelTest.test2();
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* FileChannel Write the contents of the buffer to the file
*/
public void test() throws IOException{
System.out.println("===========================================Data write start================================");
//Open fileChannel
RandomAccessFile aFile = new RandomAccessFile("E:/1.txt", "rw");//Open file in read-write mode
FileChannel channel = aFile.getChannel();//Get channel
//Create Buffer
ByteBuffer buf = ByteBuffer.allocate(1024);//Use byte buffer
//What to write
String str = new String("Thief Shuang, this is NIO in FileChannel What to write 1123444 alsdkfjlaksdf");
//Empty it. Be safe. What if there's something in it
buf.clear();
//write in
buf.put(str.getBytes(StandardCharsets.UTF_8));
//Read write conversion
buf.flip();
System.out.println("buf.flip():Switch read / write mode,The above is to read the data into the buffer. After conversion here, you can write directly without creating another output stream");
//If there is content in the Buffer, it will be written all the time
while(buf.hasRemaining()){//If there is content readable
channel.write(buf);//Reading and writing is completed by the channel calling Buffer, not channel
}
buf.clear();//Clear buffer
channel.close();//Off channel
aFile.close();//Closed flow
System.out.println("===========================================Data write complete================================");
}
/**
* FileChannel Read data into buffer
*/
public void test2() throws IOException {
System.out.println("===========================================Data read start================================");
//Open FileChannel
RandomAccessFile aFile = new RandomAccessFile("E:/1.txt", "rw");//Open file in read-write mode
FileChannel channel = aFile.getChannel();//Get channel
//Create Buffer
ByteBuffer buf = ByteBuffer.allocate(1024);//Use byte buffer
//Read the data into the buffer, which is basically consistent with the basic IO
int byteCount = 0;//Read a few bytes
while((byteCount = channel.read(buf))!=-1){
System.out.print("Read content:"+new String(buf.array(),0,byteCount));
buf.flip();//Switch read / write mode -------- the above is to read the data into the buffer. After conversion here, you can write directly without creating another output stream
System.out.println("buf.flip():Switch read / write mode,The above is to read the data into the buffer. After conversion here, you can write directly without creating another output stream");
while(buf.hasRemaining()){//If there is content readable
System.out.print(buf.get());
}
buf.clear();
}
aFile.close();
System.out.println("========Read data complete=======");
}
}
2. Common methods
1. position() method: when you need to read and write data in a specific location of FileChannel, you can obtain the current location of FileChannel through position() method, or call position(long pos) method to set the current location of FileChannel |
---|
- If you set the location after the file terminator and then try to read data from the file channel, the read method returns - 1
- If you write data, the file will be expanded to the current position and written, which may lead to empty files and gaps between the data written in the physical files on the disk
long pos = channel.position();//Gets the current location of the channel
channel.position(pos+123);//Set channel position
2. size() method: returns the file size associated with the instance |
---|
long fileSize = channel.size();
3. truncate() method: intercept a file. When intercepting, the following part of the specified length in the file will be deleted |
---|
channel.truncate(1024);//Intercept the first 1024 bytes of the file
4. force() method: the data in the channel that has not been written to the disk is forced to be written to the disk |
---|
- In order to improve performance, the operating system caches the data in memory, so it is impossible to guarantee that the data written to FileChannel will be written to disk immediately. Just call the force() method
- The force() method has a boolean type parameter that indicates whether to write the file metadata (permission information, etc.) to the disk at the same time.
5. transferTo() and transferFrom() methods: if one of the two channels is FileChannel, you can directly transfer data from one channel to another |
---|
3. transferFrom() method and transferTo() method
transferFrom(): means to transfer things from... To other channels |
---|
- Transfer data from the source channel to FileChannel (explanation in JDK: transfer bytes from a given readable byte channel to the file of this channel)
transferTo(): it means to transfer to... And engage in other people's business |
---|
1. Transfer data from FileChannel to other channels
Example of FileChannel completing file replication |
---|
//Using transferFrom
public void test3()throws IOException{
//fromChannel, copied
RandomAccessFile aFile = new RandomAccessFile("E:/1.txt", "rw");//Open file in read-write mode
FileChannel fromChannel = aFile.getChannel();//Get channel
//toChannel, copy something
RandomAccessFile bFile = new RandomAccessFile("E:/2.txt", "rw");//Open file in read-write mode
FileChannel toChannel = bFile.getChannel();//Get channel
long position = 0;//Starting position
long count = fromChannel.size();//File size associated with the instance
//a.transferFrom(b,...,...); Transfer from B to a is to get things to yourself
toChannel.transferFrom(fromChannel,position,count);
aFile.close();
bFile.close();
System.out.println("E:/1.txt File content to E:/2.txt Copy complete!!!!");
}
//Using transferTo
public void test4()throws IOException{
//fromChannel, copied
RandomAccessFile aFile = new RandomAccessFile("E:/1.txt", "rw");//Open file in read-write mode
FileChannel fromChannel = aFile.getChannel();//Get channel
//toChannel, copy something
RandomAccessFile bFile = new RandomAccessFile("E:/3.txt", "rw");//Open file in read-write mode
FileChannel toChannel = bFile.getChannel();//Get channel
long position = 0;//Starting position
long count = fromChannel.size();//File size associated with the instance
//transferTo
fromChannel.transferTo(position,count,toChannel);
aFile.close();
bFile.close();
System.out.println("E:/1.txt File content to E:/3.txt Copy complete!!!!");
}
2. Socket channel
1. General
The Socket channel class is different from the Socket of traditional BIO |
---|
- It can run non blocking mode and is optional. It can activate large programs (network servers and middleware components) with great scalability and flexibility
- It is no longer necessary to use one thread for each socket connection to avoid the context exchange overhead required to manage a large number of threads.
- With the new NIO class, one or more threads can manage hundreds of active socket connections. And there is little or no performance loss.
- All socket channel classes (DatagramChannel, SocketChannel and ServerSocketChannel) inherit Java nio. channels. AbstractSelectableChannel in SPI package. That is, we can use a Selector object to perform the ready selection of socket channel
- Datagram channel and SocketChannel implement interfaces that define read and write functions, but ServerSocketChannel does not. ServerSocketChannel is responsible for listening to incoming connections and creating new SocketChannel objects. It does not transmit data itself.
Relationship between socket and socket channel |
---|
- Channel: connect the I/O service and provide the method of exchange with the service. For a socket, it will not implement the socket protocol API in the corresponding socket channel class again, but Java Net can be reused by most protocol operations
- Socket channel class, when instantiated, will create a peer socket object. It's all Java Net (datagramchannel = = > > > datagramsocket, socketchannel = = > > > socket, serversocketchannel = = = > > > ServerSocket). They have been updated to identify channels. Peer sockets can be obtained from a channel by calling the socket() method. In addition, these three Java Net class has getChannel() method.
socket in non blocking mode |
---|
- It relies on the public superclass SelectableChannel of all socket channel classes. readiness selection is a mechanism that can be used to query a channel to determine whether the channel is ready to perform a target operation, such as read or write.
- Non blocking I/O and selectivity are closely linked, which is why the API code for managing blocking mode is defined in the SelectableChannel superclass.
- Setting or resetting the blocking mode of a channel is very simple. Just call the configureBlocking() method. Pass the parameter true to set it as blocking mode, the parameter value is false, and the value is set as non blocking mode. You can call the isBlocking() method to determine which mode a socket channel is currently in.
- AbstractSelectableChannel inherits from SelectableChannel. The source code of configureBlocking() method is as follows: if you pass true, it means blocking mode. If it is the same as the current mode, return this without change, otherwise change the current mode to true (blocking)
2. ServerSocketChannel(TCPοΌServer)
ServerSocketChannel, the peer socket object is Java net. ServerSocket |
---|
- Channel based socket listener (it is a listener if it does not transmit data), and Java net. ServerSocket performs the same task, but it adds channel semantics, so it can run in non blocking mode.
- ServerSocketChannel does not have a bind() method. You must take out the peer socket and use it to bind to a port to start listening for connections. We use the API of ServerSocket to set other socket options as needed
- And Java net. Like ServerSocket, ServerSocketChannel also has an accept() method. Once a ServerSocketChannel is created and bound with a peer socket, you can call accept on one of them
- If you choose to call the accept() method on the ServerSocket, it behaves like any other ServerSocket: it always blocks and returns a Java net. Socket object
- If you choose to call the accept() method on ServerSocketChannel, you will return an object of SocketChannel type, which can run in non blocking mode.
- The accept () method of other sockets will return a Socket object. If ServerSocketChannel is called in non blocking mode, ServerSocketChannel will be blocked when there is no incoming connection waiting accept() will immediately return null. The mechanism of checking connection but not blocking fully reflects scalability, selectivity and reduces complexity
- We can also use a selector to register the ServerSocketChannel object to automatically notify when new links arrive.
Turn ServerSocketChannel on / off |
---|
//ServerSocketChannel.open() opens the channel
ServerSocketChannel ssc = ServerSocketChannel.open();
//close
ssc.close()
Switching between blocking mode and non blocking mode |
---|
//Set it to non blocking mode. Configure blocking (Boolean b) method. B is true, indicating blocking and false indicating non blocking
ssc.configureBlocking(false);
Listen for new connections |
---|
- Call the accept() method on the ServerSocketChannel to listen for new connections. When the accept() method returns, a SocketChannel containing the new connection will be returned,
- If the current mode is blocking mode, the accept() method will block until a new connection arrives. In non blocking mode, it will execute directly and automatically downward
- Usually not just listens to a connection, the general while loop calls the accept() method.
SocketChannel sc = ssc.accept();//If the accept() method is called on the ServerSocketChannel, it will return
Use case (non blocking, blocking is BIO): Src / main / Java / NiO / Channel / serversocketchanneltest java |
---|
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.nio.charset.StandardCharsets;
/**
* ServerSocketChannel test
*/
public class ServerSocketChannelTest {
public static void main(String[] args) {
ServerSocketChannelTest serverSocketChannelTest = new ServerSocketChannelTest();
try {
serverSocketChannelTest.test();
} catch (Exception exception) {
exception.printStackTrace();
}
}
/**
* Test ServerSocketChannel
*/
public void test() throws Exception {
//Define port number
int port = 8080;
//Buffer, play channel, must use buffer
ByteBuffer buffer = ByteBuffer.wrap("What to write alkdsjfl234324k123sajdf".getBytes(StandardCharsets.UTF_8));
//ServerSocketChannel.open() opens the channel
ServerSocketChannel ssc = ServerSocketChannel.open();
//Bind ip and port numbers. The channel needs to bind peer socket objects, but the channel itself has no binding method. It needs to use the binding method of peer socket objects. InetSocketAddress is a tool class that provides local ip + ports
ssc.socket().bind(new InetSocketAddress(port));
//Set it to non blocking mode. Configure blocking (Boolean b) method. B is true, indicating blocking and false indicating non blocking
ssc.configureBlocking(false);
//Start listening for connections
while(true){
System.out.println("Waiting for connections");
SocketChannel sc = ssc.accept();//If the accept() method is called on ServerSocketChannel, an object of SocketChannel type will be returned, which can run in non blocking mode
if(sc == null){
System.out.println("null,No connection passed in");
Thread.sleep(2000);
}else{//There are links
System.out.println("Incoming connection from:"+sc.socket().getRemoteSocketAddress());
buffer.rewind();//The pointer is set to 0
sc.write(buffer);//Writes the contents of the buffer through the channel
sc.close();
}
}
}
}
3. SocketChannel(TCPοΌClient)
SocketChannel, the peer socket object is Java net. Socket |
---|
- The channel connected to TCP network sockets is an optional channel for stream connection sockets sockets. SocketChannel is used to connect Socket sockets
- Channel mainly used to handle network I/O
- Transmission based on TCP connection
- Implement optional channels that can be multiplexed
- SocketChannel cannot be created for an existing socket
- The Channel created by providing the open interface has no network cascade, so it needs to use the connect interface to connect to the specified address
- NotYetConnectedException will be thrown when I/O is performed on a SocketChannel that is not connected
- SocketChannel supports two I/O modes: blocking and non blocking
- SocketChannel supports asynchronous shutdown. When a SocketChannel is read blocked on one thread and another thread calls shutdownInput on the SocketChannel, the thread reading the blocking will directly return - 1, indicating that no data has been read. If a SocketChannel is write blocked on one thread and another thread calls shutdownWrite on the SocketChannel, The write blocking thread is called to throw AsynchronousCloseException
- SocketChannel supports setting parameters
- SO_SNDBUF socket send buffer size
- SO_RCVBUF socket receive buffer size
- SO_KEEPALIVE keep alive connection
- O_REUSEADDR multiplex address
- SO_ Delay closing the Channel when linger has data transmission (available in non blocking mode)
- TCP_NODELAY disables the Nagle algorithm
There are two ways to create a SocketChannel (different scenarios). One is to establish a TCP connection directly, and the other is not to establish a TCP connection directly. Call connect when necessary |
---|
//Method 1: through socketchannel Open creates a socketchanle. InetSocketAddress is a tool class that specifies the ip and port
SocketChannel sc = SocketChannel.open(new InetSocketAddress("www.baidu.com", 80));//Direct TCP connection
//Method 2: through SocketChannel Open the SocketChannel and specify the connection through connect
SocketChannel sc2 = SocketChannel.open();//Create SocketChannel without direct TCP connection
sc2.connect(new InetSocketAddress("www.baidu.com", 80));//Make TCP connection when necessary
sc.isOpen();//Test whether the SocketChannel is open
sc.isConnected();//Test whether the SocketChannel is connected
sc.isConnectionPending();//Test whether the SocketChannel is connecting
sc.finishConnect();//Verify that the SocketChannel for which the socket connection is in progress has completed the connection
Switch blocking / non blocking mode |
---|
//3. Non blocking mode
sc.configureBlocking(false);
Setting and getting parameters |
---|
sc
.setOption(StandardSocketOptions.SO_KEEPALIVE,Boolean.TRUE)//Set keep alive connection parameters
.setOption(StandardSocketOptions.TCP_NODELAY,Boolean.TRUE);//Socket receive buffer size parameter settings
sc.getOption(StandardSocketOptions.SO_KEEPALIVE);
sc.getOption(StandardSocketOptions.SO_RCVBUF);
4. DatagramChannel(UDP)
Datagram channel. The peer socket object is datagram channel |
---|
- SocketChannel simulates connection oriented flow protocols (such as TCP/IP), while datagram channel simulates packet oriented Connectionless Protocols (such as UDP/IP)
- Connectionless, each datagram is a self-contained entity that has its own destination address and does not rely on the data load of other datagrams. Unlike stream oriented socket s, datagram channel can send separate datagrams to different destination addresses.
- Packets can also be received from any address, and each arriving datagram contains the source address (from where)
Open DatagramChannel. In the following example, open port 10086 to receive UDP packets |
---|
DatagramChannel dc = DatagramChannel.open();
dc.socket().bind(new InetSocketAddress(10086));
Receive data: receive() receives UDP packets, and SocketAddress can obtain the ip, port and other information of the contract, which can be viewed with toString (Format: / 127.0.0.1:57126) |
---|
ByteBuffer buffer = ByteBuffer.allocate(64);//buffer
buffer.clear();//empty
SocketAddress receiveAddr = dc.receive(buffer);//Receive UDP packets
System.out.println(receiveAddr.toString());//Print information
DatagramChannel dc = DatagramChannel.open();//Open channel
ByteBuffer sendBuffer = ByteBuffer.wrap("client send".getBytes(StandardCharsets.UTF_8));//buffer
dc.send(sendBuffer,new InetSocketAddress("127.0.0.1",10086));//Send UDP packets to the specified port
UDP does not have a real connection. The connection here is to receive and send packets to a specific service address with read and write |
---|
//Read() and write() can only be used after connect(), otherwise NotYetConnectionException will be thrown. When receiving with read(), if no package is received, it will be thrown
//PortUnreachableException
dc.connect(new InetSocketAddress("127.0.0.1",10086));
int readSize = dc.read(sendBuffer);
dc.write(sendBuffer);
Case, simulating server and client: Src / main / Java / NiO / Channel / datagram channeltest java |
---|
- Let the sender send
- After the receiving end is turned on, it will receive all the time
import org.junit.Test;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.DatagramChannel;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
public class DatagramChannelTest {
//Send message
@Test
public void sendDatagram() throws Exception{
//Open datagram channel
DatagramChannel sendChannel = DatagramChannel.open();
//send out
while(true){
ByteBuffer buffer = ByteBuffer.wrap("I want to send it UPD message".getBytes(StandardCharsets.UTF_8));
sendChannel.send(buffer,new InetSocketAddress("127.0.0.2",9999));
System.out.println("Send complete");
Thread.sleep(1000);
}
}
//Receive message
@Test
public void receiveDatagram() throws Exception{
//Open datagram channel
DatagramChannel receiveChannel = DatagramChannel.open();
InetSocketAddress receiveAddress = new InetSocketAddress(9999);
//Binding is different from ServerSocketChannel. This can be bound directly
receiveChannel.bind(receiveAddress);
//buffer
ByteBuffer receiveBuffer = ByteBuffer.allocate(1024);
//receive
while(true){
receiveBuffer.clear();//Clear it first, so that there is something inexplicable in the buffer
//receive() receives UDP packets, and SocketAddress can obtain the ip, port and other information of the contract, which can be viewed with toString
SocketAddress socketAddress = receiveChannel.receive(receiveBuffer);
System.out.println(socketAddress.toString());
receiveBuffer.flip();//Read write conversion
//Charset.forName("UTF-8").decode(str), encoding str in UTF-8
System.out.println(Charset.forName("UTF-8").decode(receiveBuffer));
}
}
}
3. Scatter / gather of channels
- Used to describe the operation of reading from or writing to the channel
- Scatter: reading from the channel refers to writing the read data to multiple buffers during the read operation. Therefore, the channel "scatters" the read data into multiple buffers
- Aggregate: write to channel means that data from multiple buffers is written to the same channel during write operations. Therefore, the channel "aggregates" the data from multiple buffers and sends it to the channel
- scatter/gather is often used when the transmitted data needs to be processed separately. For example, when transmitting a message composed of a message header and a message body, you may disperse the message body and message header into different buffer s, so that you can easily process the message header and message body
Scattered instances: Scattering Reads refers to reading from one channel to multiple buffer s |
---|
ByteBuffer header = ByteBuffer.allocate(128);
ByteBuffer body = ByteBuffer.allocate(1024);
ByteBuffer[] bufferArray = {header,body};
channel.read(bufferArray);
/**
* Notice the two buffers: header and body. First, they are inserted into the array bufferArray, and then the array is used as channel Input parameters for read()
* read()Method writes the data read from the channel to the buffer in the order of the buffer in the array. When one buffer is full, the channel then writes to another buffer
* Scattering Reads Before moving to the next buffer, the current buffer must be filled up, which means that it is not suitable for dynamic messages (messages with variable size)
* Of course, you can set an insurance value, such as the message header and message body. If you want to process them separately, the maximum message header is 128byte and the header buffer is exactly 128byte
* If the current request header is less than 128 bytes, fill it forcibly, so that Scattering Reads can work normally
*/
Aggregation instance: Gathering Writes refers to writing data from multiple buffer s to the same channel |
---|
ByteBuffer header = ByteBuffer.allocate(128);
ByteBuffer body = ByteBuffer.allocate(1024);
ByteBuffer[] bufferArray = {header,body};
channel.write(bufferArray);
/**
* Note the two buffers: header and body. First, they are inserted into the array bufferArray, and then the array is used as channel Input parameters for write()
* write()Method writes data to channel in the order of buffer in the array
* If a buffer has a capacity of 128 bytes but only contains 58 bytes of data, only the 58 bytes of data will be written to the channel
* In contrast to Scattering Reads, Gathering Writes can handle dynamic messages better
*/
4. Buffer
- Buffer is used to interact with NIO channels. Data is read into the buffer from the channel and written into the channel from the buffer
- A buffer is essentially a block of memory from which data can be written and then read. This memory is packaged as a NIO Buffer object, which provides a set of methods to facilitate access to memory blocks. In fact, it is a container (array)
- In NIO library, all data is processed by buffer
- When reading data, it is directly read to the buffer. When writing data, it is also written to the buffer. Whenever accessing the data in NIO, it is put into the buffer. In the Stream oriented I/O system, all data is written or read directly into the Stream object
- In NIO, all Buffer types inherit the abstract class Buffer. The commonly used one is ByteBuffer, which corresponds to byte data type. Basically, each basic data type should have a specific Buffer type corresponding to it (the following is an incomplete class diagram)
There are generally four steps for Buffer to read and write data |
---|
- Write data to buffer
- Call the flip() method to switch the read / write status
- Read data from buffer
- Call the clear() method (completely empty) or compact() method (only clear the read data, move the unread data to the beginning, and continue to insert new data) to clear the cache
1. Three attributes and types of buffer
Three attributes of Buffer |
---|
- Capacity: the fixed size value of buffer. Buffer is a memory block and can only write byte, long, char and other data into it. Once the buffer is full, you need to empty it (read data or call clear() or compact()) to continue writing data
- Positon
- When writing data to the Buffer: position indicates the current position of the data to be written. The initial value is 0. When a byte, long... And other data is written to the Buffer, the position will move to the next Buffer unit where data can be inserted. The position can be up to capacity-1 (because the initial value is 0)
- When reading data to the Buffer, position indicates the current position of the read data. For example, position = 2 indicates that three bytes have been read, or the third byte has been read. Via ByteBuffer When flip() switches to the read mode, position will be reset to 0. After the Buffer reads data from position, position will move down to the next data Buffer unit that can be read
- limit
- When writing data, limit indicates the maximum number of data that can be written to the Buffer. In write mode, limit is equal to Buffer capacity
- When reading data, limit indicates the maximum number of readable data (not null data) in the Buffer
- The meaning of position and limit depends on whether the Buffer is in read mode or write mode, but the meaning of capacity is the same whether it is in read mode or write mode
- In the above figure, the write mode, limit indicates the maximum number of writes, and position indicates the location that can be written
- After switching to read mode, the value of position in write mode is the value of limit, indicating that this place is the boundary of the read area, and position will change to 0 again, indicating the current read data position
Buffer types: these buffer types represent different data types, that is, the bytes of the buffer can be operated through char, short, int, long, float or double types |
---|
- ByteBuffer
- MappedByteBuffer: memory mapped byte cache, faster than normal
- CharBuffer
- DoubleBuffer
- FloatBuffer
- IntBuffer
- LongBuffer
- ShortBuffer
2. Buffer allocation, data reading and writing and common methods
- The acquisition of Buffer objects requires allocation first. Each Buffer class has an allocate method
- If ByteBuffer of 48 byte capacity is allocated
ByteBuffer buf = ByteBuffer.allocate(48);
- The following is to allocate a CharBuffer that can store 1024 characters
CharBuffer cuf = CharBuffer.allocate(1024);
There are two ways to write data to the Buffer |
---|
- Write from Channel to Buffer
//inChannel is a channel, and buf fer is ByteBuffer
int bytesRead = inChannel.read(buf);//Read data from CHANLE to buf
- Write to Buffer through the put() method of Buffer. The put method has many overloads, such as writing a data to a specified location or directly writing a byte array to Buffer
//Write 127 to buf buffer
buf.put(127);
Buffer switches the read / write mode and calls the flip() method |
---|
- The flip method switches the Buffer from the current mode to another mode (read - > write, write - > read)
- Calling the flip() method will perform corresponding operation transformation on position and limit. For example, after switching from write to read, position is set to 0 and limit is set to the previous position, indicating the readable boundary position
buf.flip();//Read write switching
- Read data from Buffer to Channel
int bytesWritten = inChannel.write(buf);
- Use the get() method to read data from the Buffer; Like put, the get method has many overloads. You can specify the position to read, or read data from the Buffer to the byte array
byte aByte = buf.get();
Description( π
: Not commonly used/ π: (common) | method |
---|
π Reset the read-write pointer and set position back to 0 to reread or rewrite the data in the Buffer. The limit remains unchanged and still represents the read-write limit | rewind() |
π Reset the Buffer. position is set to 0 and limit is set to capacity. On the surface, the Buffer is cleared. In fact, the data is not cleared | clear() |
π Clear the read data, copy the unread data to the beginning of the Buffer, set position to the back of the current unread data (where the data can be inserted), and set limit to capacity. It is applicable to that the data is not read, but you want to write first | compact() |
π Mark a specific position, and then you can use buffer The reset () method reverts to this position | mark() |
π Restore to a specific position marked by the mark() method | reset() |
3. Buffer operation
1. Buffer separator
In NIO, slice() method creates a sub buffer. In addition to allocating or wrapping a buffer object, it can also create a sub buffer according to the existing buffer object, that is, cut a piece out of the existing buffer as a new buffer, but the existing buffer and the created sub buffer share data at the underlying array level, that is, The sub buffer is equivalent to a view window of an existing buffer |
---|
- That is to divide a buffer into multiple parts. We can operate on each individual part
- Granulate a buffer to improve efficiency
Here is an example of a partition: Src / main / Java / NiO / buffer / bufferforexample java |
---|
import org.junit.Test;
import java.nio.ByteBuffer;
/**
* Buffer Basic usage examples
*/
public class BufferForExample {
/**
* Buffer partition
*/
@Test
public void test1() throws Exception{
//10 byte buffers of byte size
ByteBuffer buffer = ByteBuffer.allocate(10);
//Add data to this buffer
for(int i = 0;i<buffer.capacity();i++){
buffer.put((byte)i);
}
System.out.println("===========================Output entire buffer data=============================");
//Reset pointer
buffer.position(0);
buffer.limit(buffer.capacity());
while(buffer.remaining()>0){
System.out.print(buffer.get()+" ");
}
System.out.println();
//Create sub buffer
buffer.position(3);//Starting subscript 3
buffer.limit(7);//The limit is 7
ByteBuffer slice = buffer.slice();//Sub buffer 3 ~ 7 (excluding 7)
//Change the content of the sub buffer. Each data in the sub buffer is * 10
System.out.println("===========================Change sub buffer content,Every data in the sub buffer*10:(3~7((excluding 7))=============================");
for(int i = 0;i < slice.capacity();i++){
byte b = slice.get(i);
b *= 10;
slice.put(i,b);
}
//Reset pointer
buffer.position(0);
buffer.limit(buffer.capacity());
//Output the entire buffer data, and the remaining() method returns the value of limit - position, that is, the number of accessible elements remaining
System.out.println("===========================Output entire buffer data=============================");
while(buffer.remaining()>0){
System.out.print(buffer.get()+" ");
}
}
}
2. Read only buffer
Make the buffer read-only, which can only be read and cannot be written. You can convert any regular buffer into a read-only buffer by calling the buffer asReadOnlyBuffer() method. This method returns a buffer exactly the same as the original buffer and shares data with the original buffer, but it is read-only. If the contents of the original buffer are changed, the contents of the read-only buffer are also changed |
---|
For example, if you can only read this, you will not be tested. There is no relevant method of reading. Test the change of the original buffer, and the read-only buffer will change accordingly |
---|
/**
* Read only buffer
*/
@Test
public void test2() throws Exception{
//10 byte buffers of byte size
ByteBuffer buffer = ByteBuffer.allocate(10);
//Add data to this buffer
for(int i = 0;i<buffer.capacity();i++){
buffer.put((byte)i);
}
System.out.println("===========================Output entire buffer data=============================");
System.out.println(Arrays.toString(buffer.array()));
//Create read-only buffer
ByteBuffer readOnlyBuffer = buffer.asReadOnlyBuffer();
//Change original buffer
System.out.println("===========================The value of the original buffer subscript 3 is changed to 7777=============================");
buffer.put(3,(byte)7777);
System.out.println("===========================Output read-only buffer contents=============================");
//Add data to this buffer
for(int i = 0;i<buffer.capacity();i++){
System.out.print(readOnlyBuffer.get(i)+" ");
}
}
3. Direct buffer zone
It is used to speed up I/O and allocate memory in a special way. The JDK document describes that given a direct byte buffer, the Java virtual machine will try its best to directly perform local I/O operations on it. It will try to avoid copying the contents of the buffer to or from an intermediate buffer before (or after) calling the native I/O operation of the underlying operating system. To allocate a direct buffer, you need to call the allocateDirect() method instead of the allocate() method, which is no different from ordinary buffers |
---|
//Allocate direct buffer
ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
//Allocate normal buffer
ByteBuffer buffer = ByteBuffer.allocate(1024);
//Use the same method
4. Memory mapped file I/O
There are more ways to read and write file data than conventional stream based or channel based I/O blocks. Instead of reading the entire file into memory, only the actually read or written part of the file will be mapped into memory |
---|
//Memory mapped byte cache MappedByteBuffer
MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_WRITE, start, size);
/**
* Memory mapped file I/O
*/
static private final int start = 0;
static private final int size = 1024;
@Test
public void test3() throws Exception{
RandomAccessFile raf = new RandomAccessFile("E:/1.txt","rw");
FileChannel fc = raf.getChannel();
//Memory mapped byte cache MappedByteBuffer
MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_WRITE, start, size);
mbb.put(0,(byte)97);
mbb.put(1023,(byte)122);
raf.close();
}
5. Selector
Note: when used with the Selector, the Channel must be in the non blocking mode, otherwise the IllegalBlockingModeException will be thrown, and the FileChannel cannot be switched to the non blocking mode, so it cannot be used with the Selector, and the channels related to socket s can be used |
---|
- Also known as multiplexer, it is one of the core components of Java NIO. It is used to check whether the state of one or more NiO channels is readable and writable
- Realize single thread management of multiple channels, that is, single thread can manage multiple network links
- As shown in the figure above, using fewer threads to manage multiple channels avoids the overhead of thread context switching compared with using multiple threads
SelectableChannel selectable channel (an abstract class) |
---|
- Not all channels can be reused by the Selector, such as FileChannel
- Judge whether a channel can be reused by the Selector. The premise is whether the channel inherits the SelectableChannel abstract class. If so, it can be reused, otherwise it cannot be reused.
- SelectableChannel provides the public methods required to implement channel selectivity. The parent classes of all channel classes that support readiness check and all socket channels inherit the SelectableChannel class are optional. It includes channels obtained from Pipe objects, but FileChannel does not inherit SelectableChannel, so it is not optional
- A channel can be registered on multiple selectors, but each Selector can only be registered once. The relationship between channel and Selector is completed by registration. SelectableChannel can be registered on the Selector object. During registration, you need to specify which operations of the channel are of interest to the Selector (only listen to the operations of interest).
How does the Channel register with the Selector? |
---|
- Using channel Register (selector SEL, int OPS) method registers a channel with a selector. The first parameter specifies the selector registered by the channel, and the second parameter specifies the operation of interest to the selector
- Select the operation of interest
- SelectionKey.OP_READ: readable
- SelectionKey.OP_WRITE: writable
- SelectionKey.OP_CONNECT: Connect
- SelectionKey.OP_ACCEPT: receive
- If the Selector is interested in multiple operations of the channel, it can be implemented with the bit or operator, for example:
//Pay attention to reading and writing at the same time
int key = SelectionKey.OP_READ|SelectionKey.OP_WRITE;
- After the above two concerns are configured, when the channel is ready to read or write, the selector will listen and respond
- The selector queries not the operation of the channel, but some ready state of the channel (ready to read, ready to write, etc.)
- OP_ACCEPT: receive ready status
- OP_READ: read ready status
- OP_WRITE: write ready status
- A channel does not necessarily support all 4 operations. For example, ServerSocketChannel supports Accept connection operations, but socketchannel does not
- The validOps() method of the channel can obtain all supported operation sets under the channel
- After the Channel is registered, once the Channel is in a ready state, it can be queried by the Selector. It needs to be completed by using the select() method of the Selector. The function of the select method is to query the ready state of the interested Channel operation
- The Selector will poll the registered channels and monitor the interested States of each Channel. Once the interested operations are ready, they will be selected by the Selector and put into the selection key set
- A selection key first contains the type of channel operation registered in the Selector, such as selectionkey OP_ READ. It also contains the registration relationship between a specific channel and a specific Selector.
- NIO programming is to process different business logic according to the corresponding selection keys
1. Basic usage
Selector selector = Selector.open();
2. Register the Selector to the Channel |
---|
@Test
public void test()throws Exception{
//1. Get Selector selector
Selector selector = Selector.open();
//2. Get channel
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
//3. Set to non blocking
serverSocketChannel.configureBlocking(false);
//4. Binding connection
serverSocketChannel.bind(new InetSocketAddress(9999));
//5. Register the channel on the selector and specify the listening event as "receive" event
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
}
3. Polling query ready operation |
---|
- The select() method of the Selector can query the ready channel operations. These ready status sets are stored in a Set set whose element is the SelectionKey object. The select() method has multiple overloads
- select(): blocking until at least one channel is ready for the event you registered
- select(long timeout): the same as select, but the maximum blocking time is timeout milliseconds
- selectNow(): non blocking. It returns as soon as a channel is ready
- The return value type of the select method is int, which indicates how many channels are ready at present (not including those counted by the previous select method, but only from the last selection execution to the current selection execution). As long as the return value is not 0, we can iteratively select the key collection through the selectedKeys() method in the Selector and complete the corresponding operation according to the ready operation type
//6. Query the ready channel operations, and then traverse the collection to process these operations
Set<SelectionKey> selectionKeys = selector.selectedKeys();
//Traversal set
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while(iterator.hasNext()){
SelectionKey key = iterator.next();
//Determine the status of the key
if(key.isAcceptable()){//Ready new socket connection
//@TODO ready new socket connection status action code
}else if(key.isConnectable()){
//@TODO ready socket connection complete or incomplete status action code
}else if(key.isReadable()){
//@TODO ready read status operation code
}else if(key.isWritable()){
//@TODO ready write status operation code
}
iterator.remove();//Remove traversed elements
}
Stop selecting methods: when the selector performs selection, the system bottom layer will ask whether each channel is ready in turn. This process may cause the calling thread to enter the blocking state. We have the following methods to wake up the blocked thread in the select() method |
---|
- wakeup() method: call the wakeup() method of the Selector object to make the select() method in the blocked state return immediately, and force the first selection operation on the Selector that has not returned to return immediately. If there is no selection operation in progress, the next call to the select() method will return immediately
- close() method: by closing the Selector through the close() method, any thread blocked in the selection operation can be awakened (similar to wakeup()), and all channels registered with the Selector will be logged off, and all keys will be cancelled, but the Channel itself will not be closed
2. Simulate server / client instance
@Test
public void server() throws Exception{
ServerSocketChannel ssc = ServerSocketChannel.open();
ssc.socket().bind(new InetSocketAddress("127.0.0.1",8000));
ssc.configureBlocking(false);
Selector selector = Selector.open();
// Register the channel and specify that the event of interest is Accept
ssc.register(selector,SelectionKey.OP_ACCEPT);
//buffer
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
ByteBuffer writeBuffer = ByteBuffer.allocate(128);
//Read something first and then let it write
writeBuffer.put("received".getBytes());
//Change read / write mode
writeBuffer.flip();
//Start polling
while(true){
int nReady = selector.select();//Blocking polling
//When the monitoring operation is ready, obtain the operation list
Set<SelectionKey> keys = selector.selectedKeys();
//Iterator iteration
Iterator<SelectionKey> it = keys.iterator();
while (it.hasNext()){
SelectionKey key = it.next();
it.remove();
if(key.isAcceptable()){
// Create a new connection and register the connection with the selector,
// Declare that this channel is only interested in read operations.
SocketChannel socketChannel = ssc.accept();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
System.out.println("isAcceptable,Listen to connection ready operation");
}else if (key.isReadable()) {
SocketChannel socketChannel = (SocketChannel) key.channel();
readBuffer.clear();
socketChannel.read(readBuffer);
readBuffer.flip();
System.out.println("received Listen to the read ready operation. The read contents are as follows: " + new String(readBuffer.array()));
key.interestOps(SelectionKey.OP_WRITE);
}else if (key.isWritable()) {
readBuffer.rewind();
SocketChannel socketChannel = (SocketChannel) key.channel();
socketChannel.write(readBuffer);
key.interestOps(SelectionKey.OP_READ);
System.out.println("isWritable Listen to the write ready operation and write back the read content");
}
}
}
}
@Test
public void ClientDemo() throws Exception{
SocketChannel socketChannel = SocketChannel.open();
socketChannel.connect(new InetSocketAddress("127.0.0.1", 8000));
//buffer
ByteBuffer writeBuffer = ByteBuffer.allocate(32);
ByteBuffer readBuffer = ByteBuffer.allocate(32);
//Read a hello
writeBuffer.put("hello".getBytes());
//Switch read / write mode
writeBuffer.flip();
//Crazy reading and writing
while (true) {
//Reset the read-write pointer and set position back to 0 to reread or rewrite the data in the Buffer. The limit remains unchanged and still represents the read-write limit
writeBuffer.rewind();
//Write the contents of the buffer to the server, and the server will be ready for reading
socketChannel.write(writeBuffer);
//Reset the Buffer. position is set to 0 and limit is set to capacity. On the surface, the Buffer is cleared. In fact, the data is not cleared
readBuffer.clear();
//Read the channel contents to the readBuffer, and the server executes write ready
socketChannel.read(readBuffer);
readBuffer.flip();
System.out.println("Things written by the server:"+new String(readBuffer.array(),0,readBuffer.limit()));
Thread.sleep(500);
}
}
6. Pipe, sink and source channels
- NIO pipe is a one-way data connection between two threads. Pipe has a source channel and a sink channel. The data will be written to the sink channel and read from the source channel.
Creating pipes: pipe Open() opens a pipe |
---|
//Open a pipe
Pipe pipe = Pipe.open();
Write to the pipeline: sink (SinkChannel) is required and the write() method calling the sink channel is written. |
---|
//Access sink channel is the internal class of Pipe
Pipe.SinkChannel sinkChannel = pipe.sink();
//Data to write
String newData = "New String to write to file..."+System.currentTimeMillis();
//buffer
ByteBuffer buf = ByteBuffer.allocate(48);
//Empty it first to avoid inexplicable data
buf.clear();
//Read content into buffer
buf.put(newData.getBytes());
//Toggle buffer read / write state
buf.flip();
//Buffer content write channel
while(buf.hasRemaining()){
sinkChannel.write(buf);
}
sinkChannel.close();
source channel is required to read from the pipeline |
---|
//Accessing the Source channel is the internal class of Pipe
Pipe.SourceChannel sourceChannel = pipe.source();
//buffer
ByteBuffer buf = ByteBuffer.allocate(48);
//Channel content read to buffer
int bytesRead = sourceChannel.read(buf);
while(bytesRead != -1){
System.out.print(new String(buf.array(),0,bytesRead));
sourceChannel.read(buf);
}
sourceChannel.close();
7. FileLock
- It is very common in OS. Multiple programs access and modify the same file at the same time. It is easy to have problems because the file data is not synchronized. Add a lock to the file. At the same time, only one program can modify the file, or multiple programs can only read the file but cannot modify it, so as to solve the synchronization problem.
- File locks are process level, not thread level. It can solve the problem of concurrent access between multiple processes and modifying the same file, but it can not solve the problem of concurrent synchronization of multiple threads in a process.
- The file lock is held by the JVM instance of the current program. Once the file lock is obtained (locking the file), you need to call release() or close the corresponding FileChannel object, or the JVM exits to release the lock
- Once a process (such as a JVM instance) locks a file, the process can no longer lock the file before releasing the lock, that is, the locks do not overlap (the process level cannot repeatedly obtain locks on the same file)
- Exclusive lock (exclusive lock): after an exclusive lock is added to a file, the process can read and write the file. The process monopolizes the file, and other processes cannot read and write the file until the process releases the file lock.
- Shared lock: after a process applies a shared lock to a file, other processes can also access the file, but these processes (including themselves) can only read and cannot write. Threads are safe. As long as another process holds a shared lock, this file can only be read and cannot be written.
Four methods of obtaining file locks |
---|
- lock(): locks the entire file. The default is exclusive. It is blocking. If the file lock is not obtained, the current thread will be blocked until the file lock is obtained
- lock(long position,long size,boolean shared): Custom locking. The first two parameters specify the part to be locked (you can lock only part of the contents of the file), and the third parameter specifies whether to use a shared lock
- tryLock(): an upgraded version of lock(), which attempts to lock the entire file. It is an exclusive lock by default. It is non blocking. If you try to obtain a file lock, you will return the lock object if successful, and null if unsuccessful. It will not block the current thread
- tryLock(long position,long size,boolean shared): Custom locking
- When sharing a lock, other threads throw an exception if they try to write this file
The two methods of FileLock. On some OS (operating system), channel mapping cannot be used for a file after locking it |
---|
- boolean isShared(): is this file a shared lock
- boolean isValid(): is the file lock still valid
FileChannel channel = new FileOutputStream("E:/1.txt").getChannel();
FileLock lock = channel.lock();//Lock the code of the following file operation
// @TODO operations on files
lock.release();//Release lock
Case: src/main/java/nio/file_lock/FileLockTest.java |
---|
@Test
public void test1() throws Exception{
String input = "FileLockTest What to write";
System.out.println("input=====>>>>"+input);
//Through the byte array, allocate the buffer. Allocate is the specified size
ByteBuffer buffer = ByteBuffer.wrap(input.getBytes(StandardCharsets.UTF_8));
//Get a Path object
String filePath = "E:/1.txt";
Path path = Paths.get(filePath);
//Open the channel, the file path is path, write operation, and append mode
FileChannel fileChannel = FileChannel.open(path, StandardOpenOption.WRITE, StandardOpenOption.APPEND);
//Locate at the end of the channel where it can be inserted
long size = fileChannel.size();//The size of the file in the current channel
fileChannel.position(size-1);//We want to append content to the back of the channel, just after the current existing content of the channel
//Lock
FileLock lock = fileChannel.lock();
System.out.println("Shared lock:"+lock.isShared());
fileChannel.write(buffer);//Write buffer contents to
fileChannel.close();//Close channel
System.out.println("Write complete!!!");
}
8. Path
The Java Path interface is a part of the update of Java NIO. Together with Java NIO, it is included in Java 6 and Java 7 and above |
---|
- However, the Java Path interface is added to Java NIO in Java 7. The Path interface is located in Java NIO. File package
- The Java Path instance represents the path in the file system. It can point to a file or directory. The path can be either absolute or relative
- java. nio. file. The Path interface is similar to Java io. File class, but there are still differences. In many cases, you can use the Path interface instead of the file class
Create Path (absolute Path): paths Get() creates a Path instance, static factory design pattern |
---|
//Note that the reference package is Java nio. file. Lower
import java.nio.file.Path;
import java.nio.file.Paths;
//Create Path instance
Path path = Paths.get("E:/1.txt");
Create Path (relative Path): paths Get (basepath, relativepath) creates a relative Path instance |
---|
//Point to directory: E:\projects
Path projects = Paths.get("E:/", "projects");
//Point to file: e: \ projects \ 1 txt
Path path1 = Paths.get("E:/", "projects/1.txt");
Path standardization, path normalize() |
---|
- Standardization means to remove the middle of all path strings And... Code, and resolve all referenced paths of the path string
//Path.normalize() normalized path
String originalPath = "E:/projects/../aa-project";
Path path2 = Paths.get(originalPath);
System.out.println(path2);//E:\projects\..\aa-project
Path normalize = path2.normalize();
System.out.println(normalize);//E:\aa-project
9. Files
Java NIO Files class (java.nio.file.Files) |
---|
- Provides several methods of operating files in the file system, similar to Java nio. file. Work with path instances
Files.createDirectory(): creates a directory based on the Path instance |
---|
Path path = Paths.get("d:\\sgg");//Path instance, understood as path
try {
Path newDir = Files.createDirectory(path);//Create directory based on path
} catch(FileAlreadyExistsException e){//The directory has an exception that will be thrown
// directory already exists
} catch (IOException e) {//If the parent directory does not exist, an IO exception will be thrown
// Other exceptions
e.printStackTrace();
}
Path sourcePath = Paths.get("E:\\test\\01.txt");//Source file path
Path destinationPath = Paths.get("E:\\test\\002.txt");//Copy to
try {
Files.copy(sourcePath, destinationPath);//After copying the file, an exception will be thrown if the file exists
//Copy the file. If the file already exists, the existing file will be overwritten with the parameter standardcopyoption REPLACE_ Existing assignment
Files.copy(sourcePath, destinationPath, StandardCopyOption.REPLACE_EXISTING);
} catch(FileAlreadyExistsException e) {
// directory already exists
} catch (IOException e) {
// Other exceptions
e.printStackTrace();
}
Files.move(): move the file and rename it according to the destination path |
---|
Path sourcePath = Paths.get("d:\\test\\01.txt");
Path destinationPath = Paths.get("d:\\test\\001.txt");
try {
//Moving and overwriting will be based on the path D: \ \ test \ \ 001 Txt, change the name to 001 txt
Files.move(sourcePath, destinationPath,
StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
//Failed to move file
e.printStackTrace();
}
Files.delete(): deletes a file or directory |
---|
Path path = Paths.get("d:\\test\\001.txt");
try {
Files.delete(path);//Delete the specified path file. If it does not exist, an exception is reported
} catch (IOException e) {
// Failed to delete file
e.printStackTrace();
}
Files.walkFileTree(): recursively traversing the directory tree, with Path instance (directory to be traversed) and FileVisitor (called during traversal) as parameters |
---|
- FileVisitor is an interface. You must implement the FileVisitor interface yourself and pass the implemented instance to the walkFileTree() method. During directory traversal, each method implemented by your FileVisitor will be called. If you do not need to implement all of these methods, you can extend the SimpleFileVisitor class, which contains the default implementation of all methods in the FileVisitor interface.
- Each of the methods of the FileVisitor interface returns a FileVisitResult enumeration instance. The FileVisitResult enumeration contains the following four options:
- CONTINUE continue
- TERMINATE termination
- SKIP_SIBLING skip siblings
- SKIP_SUBTREE skip children
Path rootPath = Paths.get("d:\\test");//root directory
String fileToFind = File.separator + "001.txt";//File to find
try {
Files.walkFileTree(rootPath, new SimpleFileVisitor<Path>() {
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws
IOException {
String fileString = file.toAbsolutePath().toString();
//System.out.println("pathString = " + fileString);
if(fileString.endsWith(fileToFind)){
System.out.println("file found at path: " + file.toAbsolutePath());
return FileVisitResult.TERMINATE;
}
return FileVisitResult.CONTINUE;
}
});
} catch(IOException e){
e.printStackTrace();
}
3, JAVA AIO
Asynchronous I/O asynchronous IO |
---|
- In Java 7, asynchronous filechannel is added to Java NIO to write data to files asynchronously
Create asynchronous filechannel: it is also created through the static method open() |
---|
/**
* Create asynchronous filechannel
*/
@Test
public void test1() throws Exception{
//Create Path path
Path path = Paths.get("E:/i.txt");
//Get AIO channel, standardopenoption READ,StandardOpenOption. Write means to perform read and write operations on the file
AsynchronousFileChannel asynchronousFileChannel = AsynchronousFileChannel.open(path, StandardOpenOption.READ,StandardOpenOption.WRITE);
}
1. Read asynchronous filechannel data
Method 1: read the data through the read() method, return the Future object, and use the Future |
---|
/**
* Asynchronously obtain 1. 0 through AIO Txt file
* @throws Exception
*/
@Test
public void test2() throws Exception{
//Create path
Path path = Paths.get("E:/1.txt");
//AIO channel, read operation
AsynchronousFileChannel asynchronousFileChannel = AsynchronousFileChannel.open(path, StandardOpenOption.READ);
//Buffer, and location 0
ByteBuffer buffer = ByteBuffer.allocate(1024);
long position = 0;
//Call the read() method of AIO channel to get the Future
Future<Integer> operation = asynchronousFileChannel.read(buffer, position);
//Because AIO is asynchronous, I want to block it artificially. The isDone() method cannot continue running until it returns true
while(!operation.isDone());
//Switch read / write mode
buffer.flip();
//Because the AIO we created can only be read, we artificially traverse the buffer
byte[] data = new byte[buffer.limit()];//Used to save the contents of the buffer
buffer.get(data);
//output
System.out.println(new String(data));
//Empty buffer
buffer.clear();
//Close channel
asynchronousFileChannel.close();
}
Method 2: use the read() method with the CompletionHandler inner class |
---|
/**
* Asynchronously obtain 1. 0 through AIO Txt file, using CompletionHandler
*/
@Test
public void test3() throws Exception{
//route
Path path = Paths.get("E:/1.txt");
//AIO channel
AsynchronousFileChannel asynchronousFileChannel = AsynchronousFileChannel.open(path, StandardOpenOption.READ);
//Buffer and location 0
ByteBuffer buffer = ByteBuffer.allocate(1024);
long position = 0;
//Read (the buffer to which bytes are to be transferred, the file location to start the transfer (> = 0), the object to be attached to the I/O operation (can be empty), and the handler for consuming the result)
//The third parameter is the attached object, attachment, which will be used as the parameter of completed()
asynchronousFileChannel.read(buffer, position, buffer,new CompletionHandler<Integer,ByteBuffer>() {
/**
* Perform operations
* @param result How many bytes were read
* @param attachment read()The third parameter of the method is the additional I/O operation object. If we don't pass it, the source code will give us one created by themselves
*/
@Override
public void completed(Integer result, ByteBuffer attachment) {
System.out.println("result = "+result);
attachment.flip();
//Contents of output buffer
byte[] data = new byte[attachment.limit()];
attachment.get(data);
System.out.println(new String(data));
attachment.clear();
}
/**
* Call failed to execute operation
*/
@Override
public void failed(Throwable exc, ByteBuffer attachment) {
}
});
asynchronousFileChannel.close();
}
2. Write data
Method 1: write data through Future |
---|
/**
* AIO Write data to AIO channel, Future
*/
@Test
public void test4() throws Exception{
//route
Path path = Paths.get("E:/1.txt");
//AIO channel, write
AsynchronousFileChannel asynchronousFileChannel = AsynchronousFileChannel.open(path, StandardOpenOption.WRITE);
//Buffer and location 0
ByteBuffer buffer = ByteBuffer.allocate(1024);
long position = 0;
//Put content into buffer
buffer.put("What to write AIO Write data to AIO Channel, Future".getBytes());
//Read write switching
buffer.flip();
//Write and get the Future
Future<Integer> operation = asynchronousFileChannel.write(buffer, position);
buffer.clear();
//Because AIO is asynchronous, I want to block it artificially. The isDone() method cannot continue running until it returns true
while(!operation.isDone());
System.out.println("Write over");
}
Method 2: use the write() method with the CompletionHandler inner class |
---|
/**
* AIO Write data to AIO channel, CompletionHandler
*/
@Test
public void test5() throws Exception{
Path path = Paths.get("E:/1.txt");
if(!Files.exists(path)){
Files.createFile(path);
}
AsynchronousFileChannel asynchronousFileChannel = AsynchronousFileChannel.open(path, StandardOpenOption.WRITE);
ByteBuffer buffer = ByteBuffer.allocate(1024);
long position = 0;
buffer.put("AIO Passage through CompletionHandler write".getBytes());
buffer.flip();
asynchronousFileChannel.write(buffer, position, buffer, new CompletionHandler<Integer, ByteBuffer>() {
@Override
public void completed(Integer result, ByteBuffer attachment) {
System.out.println("bytes written:"+result);
}
@Override
public void failed(Throwable exc, ByteBuffer attachment) {
System.out.println("Write Failed");
exc.printStackTrace();
}
});
asynchronousFileChannel.close();
}
4, Character set Charset
Charset is used in java to represent character set encoding objects. The common static methods are as follows: |
---|
Description( π
: Ordinary/ π: (static) | method |
---|
π Get Charset object by encoding type | public static Charset forName(String charsetName) |
π Obtain all coding methods supported by the system | public static SortedMap<String,Charset> availableCharsets() |
π Get the default encoding method of the virtual machine | public static Charset defaultCharset() |
π Judge whether the code type is supported | public static boolean isSupported(String charsetName) |
π
Gets the encoding type (String) of the Charset object | public final String name() |
π
Get encoder object | public abstract CharsetEncoder newEncoder() |
π
Get decoder object | public abstract CharsetDecoder newDecoder() |
@Test
public void charSetEncoderAndDecoder() throws CharacterCodingException {
//Obtain Charset object by UTF-8 encoding type
Charset charset=Charset.forName("UTF-8");
//1. Get UTF-8 encoder
CharsetEncoder charsetEncoder=charset.newEncoder();
//2. Get UTF-8 decoder
CharsetDecoder charsetDecoder=charset.newDecoder();
//3. Obtain the data to be decoded and encoded
CharBuffer charBuffer = CharBuffer.allocate(1024);
charBuffer.put("Character set encoding and decoding");
charBuffer.flip();
//4. Coding
ByteBuffer byteBuffer = charsetEncoder.encode(charBuffer);
System.out.println("After coding:");
for (int i = 0;i < byteBuffer.limit();i++) {
System.out.println(byteBuffer.get());
}
//5. Decoding
byteBuffer.flip();
CharBuffer charBuffer1=charsetDecoder.decode(byteBuffer);
System.out.println("After decoding:");
System.out.println(charBuffer1.toString());
//Other operations
System.out.println("Specify a different format to decode:");
Charset charset1=Charset.forName("GBK");
byteBuffer.flip();
CharBuffer charBuffer2 =charset1.decode(byteBuffer);
System.out.println(charBuffer2.toString());
//6. Get the character encoding supported by Charset
Map<String ,Charset> map = Charset.availableCharsets();
Set<Map.Entry<String,Charset>> set = map.entrySet();
for (Map.Entry<String,Charset> entry: set) {
System.out.println(entry.getKey() + "="+entry.getValue().toString());
}
}
5, Comprehensive case of Java NIO chat room
Source location: src/main/java/nio_chat package |
---|
1. Server
- Manage the channel through the selector. When a client connection is ready, we process the connection and output "welcome to the chat room, please pay attention to privacy and security" to the client
- When the process is ready to read (the client sends a message), we read it out and broadcast it to other channels
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.*;
import java.nio.charset.Charset;
import java.util.Iterator;
import java.util.Set;
//Server side
public class ChatServer {
//Server side startup method
public void startServer() throws IOException {
//1 create Selector
Selector selector = Selector.open();
//2. Create ServerSocketChannel channel
ServerSocketChannel serverSocketChannel =ServerSocketChannel.open();
//3. Bind the listening port for the channel
serverSocketChannel.bind(new InetSocketAddress(8000));
//Set non blocking mode
serverSocketChannel.configureBlocking(false);
//4 register the channel with the selector
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
System.out.println("The server has started successfully");
//5 cycle, waiting for a new link
//while(true)
for(;;) {
//Get the number of channel s
int readChannels = selector.select();
if(readChannels == 0) {
continue;
}
//Get available channel s
Set<SelectionKey> selectionKeys = selector.selectedKeys();
//Traversal set
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()) {
SelectionKey selectionKey = iterator.next();
//Remove current selectionKey from set
iterator.remove();
//6. According to the ready state, call the corresponding method to realize the specific business operation
//6.1 if accept status
if(selectionKey.isAcceptable()) {
acceptOperator(serverSocketChannel,selector);
}
//6.2 if readable
if(selectionKey.isReadable()) {
readOperator(selector,selectionKey);
}
}
}
}
//Process readable state operations
private void readOperator(Selector selector, SelectionKey selectionKey)
throws IOException {
//1 get the ready channel from the SelectionKey
SocketChannel socketChannel =
(SocketChannel)selectionKey.channel();
//2. Create buffer
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
//3 cycle to read client messages
int readLength = socketChannel.read(byteBuffer);
String message = "";
if(readLength >0) {
//Switch read mode
byteBuffer.flip();
//Read content
message += Charset.forName("UTF-8").decode(byteBuffer);
}
//4 register the channel on the selector again and listen for the readable state
socketChannel.register(selector,SelectionKey.OP_READ);
//5 send messages to clients and broadcast them to other clients
if(message.length()>0) {
//Broadcast to other clients
System.out.println(message);
castOtherClient(message,selector,socketChannel);
}
}
//Broadcast to other clients
private void castOtherClient(String message, Selector selector, SocketChannel socketChannel) throws IOException {
//1 get all the channel s that have been connected
Set<SelectionKey> selectionKeySet = selector.keys();
//2. Cycle through all channel broadcast messages
for(SelectionKey selectionKey : selectionKeySet) {
//Get each channel
Channel tarChannel = selectionKey.channel();
//You don't need to send it to yourself
if(tarChannel instanceof SocketChannel && tarChannel != socketChannel) {
((SocketChannel)tarChannel).write(Charset.forName("UTF-8").encode(message));
}
}
}
//Processing access state operations
private void acceptOperator(ServerSocketChannel serverSocketChannel, Selector selector) throws IOException {
//1 access status, create socketChannel
SocketChannel socketChannel = serverSocketChannel.accept();
//2 set the socketChannel to non blocking mode
socketChannel.configureBlocking(false);
//3 register the channel on the selector and listen for the readable status
socketChannel.register(selector,SelectionKey.OP_READ);
//4 reply to the client
socketChannel.write(Charset.forName("UTF-8").encode("Welcome to the chat room. Please pay attention to privacy"));
}
//Start main method
public static void main(String[] args) {
try {
new ChatServer().startServer();
} catch (IOException e) {
e.printStackTrace();
}
}
}
2. Client
- Selector listening read ready
- When writing, just write it directly to the server. The selector of the server will handle it after listening and reading
Thread class: each client needs a thread to process. Note that the client does not run on our server, but on the user's computer. Here we use a computer to simulate |
---|
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.util.Iterator;
import java.util.Set;
public class ClientThread implements Runnable {
private Selector selector;
public ClientThread(Selector selector) {
this.selector = selector;
}
@Override
public void run() {
try {
for(;;) {
//Get the number of channel s
int readChannels = selector.select();
if(readChannels == 0) {
continue;
}
//Get available channel s
Set<SelectionKey> selectionKeys = selector.selectedKeys();
//Traversal set
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()) {
SelectionKey selectionKey = iterator.next();
//Remove current selectionKey from set
iterator.remove();
//If readable state
if(selectionKey.isReadable()) {
readOperator(selector,selectionKey);
}
}
}
}catch(Exception e) {
}
}
//Process readable state operations
private void readOperator(Selector selector, SelectionKey selectionKey) throws IOException {
//1 get the ready channel from the SelectionKey
SocketChannel socketChannel =
(SocketChannel)selectionKey.channel();
//2. Create buffer
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
//3 loop reading client messages
int readLength = socketChannel.read(byteBuffer);
String message = "";
if(readLength >0) {
//Switch read mode
byteBuffer.flip();
//Read content
message += Charset.forName("UTF-8").decode(byteBuffer);
}
//4 register the channel on the selector again and listen for the readable status
socketChannel.register(selector, SelectionKey.OP_READ);
//5 send messages to clients and broadcast them to other clients
if(message.length()>0) {
//Broadcast to other clients
System.out.println(message);
}
}
}
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.util.Scanner;
//client
public class ChatClient {
//Start client method
public void startClient(String name) throws IOException {
//Connect server
SocketChannel socketChannel = SocketChannel.open(new InetSocketAddress("127.0.0.1",8000));
//Receive server response data
Selector selector = Selector.open();
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
//Create thread
new Thread(new ClientThread(selector)).start();
//Send message to server
Scanner scanner = new Scanner(System.in);
while(scanner.hasNextLine()) {
String msg = scanner.nextLine();
if(msg.length()>0) {
socketChannel.write(Charset.forName("UTF-8").encode(name +" : " +msg));
}
}
}
public static void main(String[] args) throws IOException {
//If your user name is passed in, you need to enable idea to run concurrently
new ChatClient().startClient("patriot");
}
}
3. Summary
- Every time BIO connects, it needs to start a thread to process it in the whole process, although the client does nothing after it is connected
- But NIO only needs a selector and related channels
- When listening to the interested ready operation, it will be processed instead of a thread connecting to the client all the way