Selectect and select or multiplexing to create socket links

After learning, we understand that i\o blocks I o, non-blocking I o, I O multiplexing, asynchronous I o, an I O operat io n is actually putting data into the kernel cache, and from this copy down, the user actually only calls this interface
I/O communication is actually a user invoking a kernel interface, the kernel puts a file descriptor in the cache space, and the user gets it.
Blocking is simply understood as a function accessing the cache space, waiting without events, and suspending threads until a result is returned.
Non-blocking is a function that accesses the cache space, returns a value without events, multiplexes I/O listens for multiple interfaces at the same time, and returns when one interface returns a value.
There are three ways to do this
Select:select is currently supported on almost all platforms, and its good cross-platform support is one of its advantages, but one disadvantage of select is that there is a maximum limit to the number of file descriptors that a single process can monitor. In addition, select() maintains a data structure that stores a large number of file descriptors, along with filesAs the number of descriptors increases, the cost of replication increases linearly.At the same time, due to the delay in network response time, a large number of TCP connections are inactive, but calling select() causes a linear scan of all socket s, which wastes some overhead.
Epoll: epoll can support both horizontal and edge triggers. Theoretically, edge triggers perform better, but the code implementation is fairly complex.
Epoll also tells only those ready file descriptors, and when we call epoll_wait() to get the ready file descriptor, it returns not the actual descriptor, but a value representing the number of ready descriptors. You simply go to an array specified by epoll and get the corresponding number of file descriptors in turn.Memory mapping (mmap) technology is also used here, which eliminates the overhead of copying these file descriptors on system calls.
Another fundamental improvement is that epoll uses event-based ready notifications.In select/poll, the kernel scans all monitored file descriptors only after calling certain methods, and epoll registers a file descriptor in advance through epoll_ctl(). Once a file descriptor is ready, the kernel uses a callback-like mechanism to quickly activate the fileDescriptor, notified when a process calls epoll_wait().
Poll: There is not much difference between poll and select in nature, but poll has no limit on the maximum number of file descriptors.

The disadvantage of poll and select is that arrays containing a large number of file descriptors are copied as a whole between the user state and the address space of the kernel, and their overhead increases linearly with the number of file descriptors, regardless of whether they are ready or not.In addition, after select() and poll() tell the process the ready file descriptors, if the process does not IO them, they will be reported again the next time select() and poll() are called, so they generally do not lose the ready message, which is called Level Triggered.

Select is supported under window s, but epoll is not. There is also a selector, encapsulated on the basis of select
Look first at the server-side code

import select
import socket
import sys
import queue
 
# Create a TCP/IP socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(False)
 
# Bind the socket to the port
server_address = ('localhost', 10000)
server.bind(server_address)
# Listen for incoming connections
server.listen(5) 
# Sockets from which we expect to read
inputs = [ server ] #Listen on this port first
outputs = [ ] 
message_queues = {}
while inputs:
 
    # Wait for at least one of the sockets to be ready for processing
    print( '\nwaiting for the next event')
    readable, writable, exceptional = select.select(inputs, outputs, inputs)
    for s in readable: 
        if s is server:
            # A "readable" server socket is ready to accept a connection
            connection, client_address = s.accept()
            print('new connection from', client_address)
            connection.setblocking(False)
            inputs.append(connection) 
            # Give the connection a queue for data we want to send
            message_queues[connection] = queue.Queue()
        else:
            data = s.recv(1024)
            if data:
                # A readable client socket has data
                print('received "%s" from %s' % (data, s.getpeername()) )
                message_queues[s].put(data)
                # Add output channel for response
                if s not in outputs:
                    outputs.append(s)
            else:
                # Interpret empty result as closed connection
                print('closing', client_address, 'after reading no data')
                # Stop listening for input on the connection
                if s in outputs:
                    outputs.remove(s)  #Now that the client is disconnected, I don't have to return data to it anymore, so at this point if the client's connection object is still in the outputs list, delete it
                inputs.remove(s)    #Delete also from inputs
                s.close()           #Close this connection
 
                # Remove message queue
                del message_queues[s]
    # Handle outputs
    for s in writable:
        try:
            next_msg = message_queues[s].get_nowait()
        except queue.Empty:
            # No messages waiting so stop checking for writability.
            print('output queue for', s.getpeername(), 'is empty')
            outputs.remove(s)
        else:
            print( 'sending "%s" to %s' % (next_msg, s.getpeername()))
            s.send(next_msg)
    # Handle "exceptional conditions"
    for s in exceptional:
        print('handling exceptional condition for', s.getpeername() )
        # Stop listening for input on the connection
        inputs.remove(s)
        if s in outputs:
            outputs.remove(s)
        s.close()
 
        # Remove message queue
        del message_queues[s]

The client is still the same as a normal socket

Using the selector module:
import selectors,socket

def accept(sock,mask):
conn,addr=sock.accept()
print('received', addr)
conn.setblocking(False)
sel.register(conn,selectors.EVENT_READ,read)
def read(conn,mask):
data=conn.recv(1024)
if data:
conn.send(data)
print('Send Successfully')
else:
print('disconnected', conn)
sel.unregister(conn)
conn.close()

Sock=socket.socket()
Sock.bind(('localhost',8001))
Sock.listen(100)
Sock.setblocking(False)
sel=selectors.DefaultSelector()
#Register Events
sel.register(Sock,selectors.EVENT_READ,accept)
print('wait ')
while True:
events=sel.select()#listen for events
for key,mask in events:
callback=key.data
callback(key.fileobj,mask)

Keywords: socket network

Added by Ne.OnZ on Tue, 30 Jul 2019 04:12:24 +0300