Go back to the previous text
The first chapter only starts with data structures and some transmission examples. The second chapter will talk about how to use the network transmission protocol under the Python programming scenario.
According to the above reply, we will talk about Tcp first in order, and then get to the main topic.
Review lessons: Writing network programming in Python (1)
Deeply optimized transmission protocol Tcp
Tcp standard comes from the 1980s and has been improved and optimized for more than 30 years. However, these optimizations are not in the application layer. Avoid lengthy discussions. If you are interested, you can understand them yourself.
Some of these ideas will also be verified by Python later.
Then, in the first chapter of learning network protocol, it is the data structure used by the protocol. The next step is to learn the difference between Tcp and conventional http arrowheads. First, sort out the structure. The structures are as follows:
Transmission mode (including various settings of transmission)
Link address
Transmission form
In the application layer, Http is divided into multiple domains. The whole is more complex and more complex than TCP, but the use scenario of Http is wider than TCP, so practice makes perfect and more understanding.
The bottom layers of http arrowhead and Tcp are socket s. This http layer has made a lot of restrictions and modifications in the web layer, so there are more domains.
Http arrowhead doesn't intend to introduce it here. Let's get to the point. Tcp is also a data stream, which can obtain the length of byte array bytes.
The transport form layer is made of sockets. The Python layer makes a lot of inclusion without setting very complex socket settings.
socket native functions need to pass nodename, servname and ai_flags,ai_family,ai_socktype,ai_protocol to set
ai_family:AF_INET Ipv4. At present, most of them are Ipv4. AF_INET6 is Ipv6.
ai_socktype:SOCK_STREAM is the data stream, that is, the stream setting of TCP. SOCK_DGRAM is a data packet, which is set by UDP.
def tcp_client_options():
"""
tcp Client set IPv4 And data flow (user)
:return:
"""
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #Channel settings
return sock
def tcp_server_options():
"""
tcp Server settings (Manager)
:return:
"""
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
"""
level=socket.SOL_SOCKET Which protocol layer
socket.SO_REUSEADDR The access option name is a parameter int type
"""
# SO_ The reuseaddr port can be used again immediately after it is released. 1 means it can be used when it is turned on
sock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
print(sock.getsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR))Compared with the client, the server is the manager and manages the client link, so there will be one more setsockopt to set socket rules or configuration. The client is the user, so only set the channel (the way to access the target channel).
This pre knowledge is Tcp handshake and wave. It is also an important interview question, which is not described here.
socket.SOL_SOCKET is the default, which represents the protocol layer you choose, that is, socket.
socket.SO_REUSEADDR is a setting that can be used after TCP waves. Only 1 and 0 can be set.
sock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
It is modified to 2 or - 1, and the following print is still 1. This is also the embodiment of Python's extensive encapsulation and inclusion.
socket.SO_REUSEADDR represents the control mode of optname, that is, waving to disconnect.
Http is a single request interaction mode, that is to say, when you send a data flow request, it will be immediately delivered to the server, verified to be legal, and then answered
Tcp is not. Tcp sends a group (after multi segment data flow requests), but the server does not accept it immediately. It will jam in the network cache and send it to the server after reaching a certain number.
So this is often encountered by beginners. Assuming that the server completely receives the client information, the client sends it 10 times, with a total size of 5000 bytes. The server often receives it only 3-4 times, with a total size of 5000 bytes. You can try to write an example after reading this article.
You can set the cache size, which is related to the important concept of Tcp. The previous example plus:
def tcp_server_options():
"""
tcp server setting
:return:
"""
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
print(sock.getsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR))
#Set the sending buffer holding unit
sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 16 * 1024)
print(f'Send -> {sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF)}')
print(f'Revice -> {sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)}')socket.SO_RCVBUF is determined by the protocol layer. The constant of the protocol layer is expressed by the default size of the protocol layer (pun). 65536 of 65535 + 1. Of course, this default size can be modified. It is usually recommended to modify it to 100000, which can be adjusted according to the local conditions of the machine.
socket.SO_SNDBUF is 16 * 1024. If it is not set, how much will it be. There are many puns in the socket layer, and the default is 65536.
Communication blocking can also be set. This blocking is that the server processes the socket request of the client. The blocking will not receive the next request of the same client until it is processed. However, in high concurrency scenarios, Tcp servers with multiple work threads are generally non blocking. The settings here are set globally and will take effect on the current startup of the whole server.
sock.setblocking(False) # sets non blocking.
You can also set the timeout, but don't draw fish and braised meat. Adding the timeout will conflict with the blocking setting above, which will affect the normal setting of cache size, so it is not recommended.
This is different from the timeout in the requests used by the test tool.
The general settings have been described. Now let's talk about how the server lets the client deliver data to him.
2. Link address
Basically, there is no difference. The iP + port of the address receiver is required for network transmission from one end to the other.
The client needs to know the link address of the server to deliver the message to the correct place. The server has the receiver iP, which will assign an address, that is, the port, and then bind the port to monitor the link to this address to manage the client.
The server will also have a maximum number of links supported. The client does not have this setting. In the following example, max=5000, so the relationship between the client and the server is many to one. At most, 5000 clients can be linked to the same server.
See the above, highlight the keywords bind and listen, and give an example to distinguish the client code from the server code:
def server_listener(addr: tuple, max: int):
"""
The maximum number of 5000 links supported by the server listening
:param max:Maximum number of links supported
:param addr:tuple
:return:
"""
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# It can be used after waving
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(addr)
# max links supported
sock.listen(max)
return sock
if __name__ == '__main__':
server_listener(("0.0.0.0", 12580),5000)Because the http server is well packaged, it will not have this setting. The server is a server that monitors the client fd and polls to manage these fd.
The client and server use the same set of socket options, so that the server can receive the message linking the client, FD = socket fileno().
Access method and client link code, as shown in the following example:
def create_client():
"""
Create client
:return:
"""
# The socket of the channel set by the client and server must be the same, otherwise it cannot be linked
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sockfd = sock.fileno()
print(f"client fd={sockfd}")
sock.connect(("127.0.0.1", 12580))
return sock
if __name__ == '__main__':
client = create_client()
print(client)In the client link code, the address cannot be 0.0.0.1, and the local test can only be 127.0.0.1.
connect_ If ex () is not 0, it will return the error code, which will be described in the following chapters in series with the protocol number and the error protocol number.
The code of the server listening to the client fd is as follows. The core function is accept():
def accept_client_connect(listener_sock):
"""
The core method for the server to process and confirm the client link is accept()
:param listener_sock:The server sock object
:return:
"""
new_conn = listener_sock.accept()
if new_conn == None:
return
sock, addr = new_conn
# Server monitoring can get the fd of the sock over there
print(f"---accept_client_connection, sockfd={sock.fileno()}, addr={addr}")
return sock
if __name__ == '__main__':
sock = server_listener(("0.0.0.0", 12580), 5000)
sock_ = accept_client_connect(sock)
print(sock_)Final printing
---accept_client_connection, sockfd=512, addr=('127.0.0.1', 14024)
<socket.socket fd=512, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 12580), raddr=('127.0.0.1', 14024)>Addr port is the port temporarily allocated by the server to the client. The server is managed through dictionary, sockfd and addr[1]. The following description can be understood by other languages.
ClientInfo = {"client_fd":sockfd,"fd_port":addr[1]}
ClientGroup = []ClientInfo
{"gateway_threads":ClientGroup} The work thread has different functions. The gateway service bind clientGroup array in the server has specific examples in the following chapters.
Preview & summary
Notice the process of socket contracting
Step1: socket options (Part 2)
Step 2: disconnect socket, struct+pack (data structure) + pressure packet (data structure is in the first part, and the third part is to disconnect socket and struct)
step3: various forms of compression (Part 4)
step4: various forms of encryption (Part 5)
summary
A lot of practice and deeper understanding of socket settings are needed. In fact, there are many settings for non Python http, which can also be deeply understood.
The writing method will support the client and server examples.
No socket Shutdown and struct libraries can be previewed.
>This article was first published in the tester home community by Chen Ziang, a senior game testing and development engineer. Write three articles on network programming in Python, which will be shared with you one after another. Original link: https://testerhome.com/topics...
The above is today's sharing. Have you learned it~
Want to learn more about dry goods and cutting-edge technology?
Want to get to know the test industry and elite?
Welcome to the 2022 MTSC Conference (the 10th China Internet testing and Development Conference)
The industry's evaluation of MTSC conference: landing, pragmatic, in-depth and re sharing
Testing Summit China is a technical conference of software testing and development industry initiated by TesterHome. There is also an alias: Make Testing Super Cool!
MTSC conference, with the main purpose of software quality assurance system and testing and R & D technology exchange, began in 2015 and has been successfully held for nine sessions. A total of 1000 + enterprises and 10000 + test engineers, test managers and CTO s attended the meeting, which attracted extensive attention from the whole industry and was the top conference of China's Internet quality assurance industry.
In order to ensure the quality of topics, topics will be collected half a year in advance every year, and then after several months of review, the topics to be shared at the conference will be finally determined. The topic of MTSC 2022 is still being solicited. We sincerely invite senior test technology experts, quality management managers and test rookies to submit the topic!
Issue submission method
Direct click https://www.wjx.top/vj/wZwCju... Enter the delivery entrance, fill in and submit according to the format.
Issue deadline
In order to facilitate the review team to have more time to review and optimize communication with lecturers, and present better topics for the majority of participants, the deadline for submission of topics (without ppt) is advanced to March 30, 2022
Issue solicitation and selection process
Overall process: case submission > preliminary review and approval > ppt submission > confirmation of topics > conference speech
Share the benefits of being a lecturer
- Exercise speech ability and enhance personal brand
- Get the opportunity to communicate face-to-face with industry experts and draw on their strengths
- Enhance the company's brand and increase the attractiveness of your team
- Get free conference tickets and information:
Each lecturer will receive a free ticket to the conference, and the subsequent ppts and videos will be given to the lecturer at the first time
MTSC 2022 early bird tickets have been quietly sold. Click for details.