- Course arrangement of network programming:
day01
- Understand OSI seven layer and TCP/IP four layer network model structure
- Understand common network protocol formats
- Master the conversion between network byte order and host byte order (large end method and small end method)
- Tell the communication flow of TCP server
- Output TCP client communication flow
- Write TCP server-side code independently
- Write TCP client code independently
1.1 understand OSI seven layer and TCP/IP four layer network model structure
- Protocol: triggered from the perspective of application program, the protocol can be understood as the rules of data transmission and data interpretation (the common language used for communication between hosts).
*File transfer example - FTP
- OSI seven layer model (ideal)
Router is a bridge connecting different network segments, and the network layer summarizes ip;TCP – socke (we encode), the session layer is implemented by the kernel, and the presentation layer: encoding and decoding, such as GBK,utf-8
Development application layer protocol
[] "content data network conference table"
Physical layer data layer network layer transport layer session layer presentation layer application layer
- Model used in network process
1.2 data communication process
The data communication process is a process of packaging layers by layers by the sender and unpacking layers by layers by the receiver.
1.3 common design patterns of network applications
C/S mode: client and server
B/S mode: browser and server mode
: advantages and disadvantages:
For the C/S mode, it can ensure the performance, cache the data locally to the client, improve the data transmission efficiency, and the protocol used is relatively flexible. (traditional large network program)
Disadvantages: security, workload.
B/S: the browser does not need to be developed, with small amount of development, good portability, and is not limited by the platform.
The protocol intelligently selects the http protocol and cannot cache data, which affects the efficiency.
1.4 common network protocols
- Ethernet frame:; It is a protocol packaged in the network interface layer (data link layer: data frame format). (it is not necessary to write code, but can be intercepted by bit, str,memcpy)
Judge the data type and function. ARP: IP - > MAC address, RARP: MAC - > IP
- For example, the ARP parsing process with the frame format written as 0806:
Broadcast and send ARP response after matching. - ip protocol master how to split data
- Data frame format of TCP and UDP
TCP: (connection oriented) UDP is connectionless.
32-bit serial number to ensure multi-threaded splicing of data. 32 to confirm the serial number and ensure data transmission (Security) 16 bit checksum (the signal may be tampered after being interfered to ensure) - packet loss retransmission function and tamper proof function. ACK SYN (request identification of three handshakes), FIN (close request), 16 bit window size (flow control to prevent inconsistency between the speed of Hu search section and the sending end)
Among them: 8-bit lifetime ttl, (64) next hop: to prevent network congestion.
- UDP datagram format: (port (used to distinguish applications): 21 FTP, 22 SSH, telnet (remote login) 23) red font.
1.5 Internet Literacy
- Route: the path from the source address to the destination address
The router is located in the third layer (network layer) of the seven layer model and is used to receive network packets
(function: can route table and forward)
Difference between routing and switching:
The main difference between routing and switching is that it occurs in the second layer of OSI reference model (data link layer), while routing occurs in the third layer, namely network layer. This difference determines that routing exchange needs to use different control information in the process of summarizing mobile information.
- Routing table
It is a spreadsheet or class database stored in a router or networked computer. - Default route entry: send to the next hop address according to the interface specified in the default route entry.
- Working principle of Ethernet switch: Ethernet switch is located in the second layer (data link layer) of OSI network reference model. It is a network device that completes Ethernet data frame forwarding based on MAC address identification.
- How hub works; Hub, the difference is that the hub can provide more port services, also known as multi button repeater.
- The abbreviation of DNS server (Domain Name System) is a core service of the Internet. It can map domain names and ip addresses to each other, making it easier for people to access the Internet without remembering the ip address string. (composed of parser and domain name server)
- Local area network (LAN):
The geographical coverage is small, only within a relatively independent local range
The specially laid transmission medium is used for networking, with high data transmission efficiency (10Mb/s~10Gb/s)
Communication delay period, high reliability
LAN can support a variety of transmission media - Wide area network (WAN): the remote computer network of different local area networks, such as the Internet, is the largest wide area network.
- Port: in the logical sense, it generally refers to the port of TCP/IP protocol. The port number ranges from 0 to 65535, such as port 80 for web browsing service, port 21 for FTP service, etc. the port number less than 256 is defined as a common port.
- MTU: Communication belongs to the maximum transmission unit (MTU), which refers to the maximum packet size that a communication protocol can pass through. This parameter is usually related to the network interface.
The following are MTU s for some protocols:
FDDI protocol; 4352 bytes.
Ethernet: 1500 bytes
PPPoE(ADSL protocol): 1492 bytes;
10. 25 protocol: 576 bytes.
Point to point: 4470 bytes
1.6 important interview questions
1.6.1 how TCP establishes a connection. Additional: more detailed questions.
1.6.2 [how TCP communicates]
1.6.3 how TCP closes a connection
1.6.4 what is a sliding window
Error control and flow control.
TCP uses sliding window for transmission control. The size of sliding window means how much buffer the receiver has for receiving data. The sender can determine how many bytes of data should be sent by sliding the size of the window.
1.6.5 what is semi closed
In the fully closed state, when the server sends data and wants to close the connection, the client will receive the server's request. However, due to the fully closed state, the disconnection confirmation request sent by the client to the server cannot be returned to the server again. The semi closed state is required at this time. In order to make the data completely exchanged, and to make the data between the client and the server controllable and prevent blocking, the shutdown function should be used. After the server sends the data, an EOF should be attached to let the client know that the data on the server has been sent.
In order to ensure the complete exchange of data, enough connection time should be set aside, but how long should it be set aside?
For example, when the client connects to the server, the server transmits a file to the client, and the client sends confirmation data to the server after receiving it
At this time, the server only needs to continuously transmit file data, but the client cannot know when to receive data, and the client cannot call the input function endlessly, because this may lead to program blocking (the called function does not return)
The server should send the EOF after the data is sent, indicating the end of the file. When the client receives the EOF, it will stop receiving the data and send the confirmation data to the server. Both the close function and shutdown can send EOF data to the client, but after sending EOF with close, the data transmitted by the other party cannot be received, so shutdown is used
1.6.6 how do two hosts on LAN communicate
Communication method???.
1.6.7 how do two hosts communicate on the Internet
1.6.8 how to uniquely identify a process on the internet
1.6.9 why is TCP a reliable connection and UDP unreliable
10. Differences between routers and switches
11. Point to point, end to end
2. SOCKET programming
A well marked example of network server programming: python: https://www.cnblogs.com/python58/p/10426188.html.
To make different machines communicate, you need not only the kernel IPC, but also the sockt.
Two connections are as follows: socket pair. A file name includes two buffers. Communication is between both sides. (send to receive) can be semi closed, for example, send only but not receive.
< font color = > based on TCP/UDP protocol development, SOCKET programming, establish file descriptors and operate file descriptors.
Main API functions for socket programming:
int blind()
int listen()
int accept()
int connect() the client actively calls the server
ssize_t read()
ssize_t write()
ssize_t recv()
ssize_t send()
You can directly fill in 0 for the flag bit of recv and send functions.
Note: if the write buffer is full, write will also block. If there is no data in the read buffer during read operation, it will cause blocking. "
2.1 preliminary knowledge of socket programming
2.1 network byte order conversion
Conversion of port number:
Concept of big end and small end
Big end: also known as high-end byte order, also known as network byte order. The low-order address stores high-order data, and the high-order address stores low-order data
Small end: also known as low-end byte order, low address stores low-order data and high address stores high-order data
-
Application occasions of large end and small end???
(1) More than one byte, such as int, etc.
(2) The network transmission uses the large end method. If the machine uses the small end, it needs to convert the large end to the small end. The following four functions are used for size conversion:
The function name h represents host, n represents network, s represents short, and l represents long.
#include<arga/inet> uint32_t htonl(uint32_t hostlong); uint32_t htons(uint16_t hostshort); uint32_t ntohl(uint32_t netlong); uint16_t ntohs(uint32_t netshort);
Manually calculate conversion results:
2.2 byte order and ip address conversion function
ip address translation function:
Dot decimal ip conversion to hexadecimal byte order
int inet_pton(int af,const char *src,vodi *dst) //Convert dotted decimal IP to large end mode network IP (integer 4 bytes) af : AF_INET or AF_INET6，Incoming src Last saved to dst in for example inrt_pton(AF_INET,"127.0.0.1",&serv.sinaddr.s)
2.3 important structures commonly used in socket programming
Many network programming functions of strcut sockaddr were born earlier than IPv4 protocol. At that time, sockaddr structure was used. For forward compatibility, sockaddr now degenerates into the role of (void *), passing an address to the function. As for this function, it is sockaddr_in or sockaddr_in6, determined by the address family, and then the internal type of the function is forced to be converted to the required address type.
struct sockaddr {
sa_family_t sa_family; /* address family, AF_xxx */
char sa_data[14]; /* 14 bytes of protocol address */
};
//-------------------------------------
struct sockaddr_in {
__kernel_sa_family_t sin_family; /* Address family */ Address structure type
__be16 sin_port; /* Port number */ Port number
struct in_addr sin_addr; /* Internet address */ IP address
/* Pad to size of `struct sockaddr'. */
unsigned char __pad[__SOCK_SIZE__ - sizeof(short int) -
sizeof(unsigned short int) - sizeof(struct in_addr)];
};
struct in_addr { /* Internet address. */
__be32 s_addr;
};
//------------------------------------------------------
struct sockaddr_in6 {
unsigned short int sin6_family; /* AF_INET6 */
__be16 sin6_port; /* Transport layer port # */
__be32 sin6_flowinfo; /* IPv6 flow information */
struct in6_addr sin6_addr; /* IPv6 address */
__u32 sin6_scope_id; /* scope id (new in RFC2553) */
};
//------------------------------------------------------
struct in6_addr {
union {
__u8 u6_addr8[16];
__be16 u6_addr16[8];
__be32 u6_addr32[4];
} in6_u;
#define s6_addr in6_u.u6_addr8
#define s6_addr16 in6_u.u6_addr16
#define s6_addr32 in6_u.u6_addr32
};
#define UNIX_PATH_MAX 108
struct sockaddr_un {
__kernel_sa_family_t sun_family; /* AF_UNIX */
char sun_path[UNIX_PATH_MAX]; /* pathname */
};
The process of TCP communication coding.
2.4.1 socket function
#include <sys/types.h> /* See NOTES */
#include <sys/socket.h>
int socket(int domain, int type, int protocol);
domain:
AF_INET This is most used to generate socket Protocol, using TCP or UDP To transmit, use IPv4 Address of
AF_INET6 Similar to the above, but for use IPv6 Address of
AF_UNIX Local protocol, used in Unix and Linux On the system, it is generally used when the client and server are on the same computer
type:
SOCK_STREAM This protocol is a sequential, reliable and data complete byte stream based connection. This is one of the most used socket Type, this socket Is to use TCP For transmission.
SOCK_DGRAM This protocol is a connectionless, fixed length transport call. The protocol is unreliable to use UDP To make its connection.
SOCK_SEQPACKET The protocol is a two-wire and reliable connection, which sends fixed length packets for transmission. The package must be completely accepted before it can be read.
SOCK_RAW socket This type provides a single network access socket Type use ICMP Public agreements. ( ping,traceroute (use this protocol)
SOCK_RDM This type is rarely used and is not implemented on most operating systems. It is provided to the data link layer and does not guarantee the order of data packets
protocol:
Passing 0 means using the default protocol.
Return value:
Success: returns a pointer to the newly created socket File descriptor, failed: Return-1，set up errno
socket() opens a network communication port. If it succeeds, it returns a file descriptor like open(). The application can send and receive data on the network with read/write like reading and writing files. If the socket() call is wrong, it returns - 1. For IPv4, the domain parameter is specified as AF_INET. For TCP protocol, the type parameter is specified as SOCK_STREAM, which represents a stream oriented transport protocol. If it is a UDP protocol, the type parameter is specified as SOCK_DGRAM, which represents datagram oriented transmission protocol. The introduction of the protocol parameter is omitted. Specify 0
2.4.2 bind function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen); sockfd: socket File descriptor addr: Construct IP Address plus slogan addrlen: sizeof(addr)length Return value: 0 is returned for success and 0 is returned for failure-1, set up errno
The network address and port number monitored by the server program are usually fixed. After the client program knows the address and port number of the server program, it can initiate a connection to the server. Therefore, the server needs to call bind to bind a fixed network address and port number.
The function of bind() is to bind the parameters sockfd and addr together, so that sockfd, the file descriptor used for network communication, can listen to the address and port number described by addr. As mentioned earlier, struct sockaddr * is a general pointer type. The addr parameter can actually accept sockaddr structures of various protocols, and their lengths are different. Therefore, the third parameter addrlen is required to specify the length of the structure. For example:
The function of bind() is to bind the parameters sockfd and addr together, so that sockfd, the file descriptor used for network communication, can listen to the address and port number described by addr. As mentioned earlier, struct sockaddr * is a general pointer type. The addr parameter can actually accept sockaddr structures of various protocols, and their lengths are different. Therefore, the third parameter addrlen is required to specify the length of the structure. For example:
struct sockaddr_in servaddr; bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(6666);
First clear the entire structure, and then set the address type to AF_INET, the network address is INADDR_ANY, this macro represents any local IP address, because the server may have multiple network cards, and each network card may be bound with multiple IP addresses. In this way, you can listen on all IP addresses until you establish a connection with a client, and then determine which IP address to use. The port number is 6666.
2.4.3 listen function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int listen(int sockfd, int backlog); sockfd: socket File descriptor backlog: The number of links queued to establish 3 handshake queues and just established 3 handshake queues
A typical server program can serve multiple clients at the same time. When a client initiates a connection, the server calls accept() to return and accept the connection. If a large number of clients initiate a connection and the server has no time to process it, the client that has not accepted is in the connection waiting state, and listen() declares that sockfd is in the listening state, A maximum of backlog clients are allowed to be connected. If more connection requests are received, they will be ignored. Listen() returns 0 for success and - 1 for failure.
2.4.4 accept function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen); sockdf: socket File descriptor addr: Outgoing parameter, which returns the address information of the linked client, including IP Address and port number addrlen: Incoming and outgoing parameters (values)-Results),afferent sizeof(addr)Size. When the function returns, it returns the size of the actually received address structure Return value: Successfully returned a new socket File descriptor, used to communicate with the client. Failure is returned-1，set up errno
After the three-party handshake is completed, the server calls accept() to accept the connection. If there is no connection request from the client when the server calls accept(), it will block and wait until a client connects. Addr is an outgoing parameter. When accept() returns, the address and port number of the outgoing client. The addrlen parameter is an incoming and outgoing parameter (value result argument). The incoming parameter is the length of the buffer addr provided by the caller to avoid buffer overflow. The outgoing parameter is the actual length of the client address structure (the buffer provided by the caller may not be full). If you pass NULL to the addr parameter, it means that you don't care about the address of the client.
The program structure of our server is as follows:
while (1) {
cliaddr_len = sizeof(cliaddr);
connfd = accept(listenfd, (struct sockaddr *)&cliaddr, &cliaddr_len);
n = read(connfd, buf, MAXLINE);
......
close(connfd);
}
The whole is a while loop, processing one client connection at a time. Because cliaddr_len is an incoming and outgoing parameter. The initial value should be reset before each call to accept(). The parameter listenfd of accept() is the previous listening file descriptor, and the return value of accept() is another file descriptor connfd (Communication file descriptor). Then, it communicates with the client through this connfd. Finally, close connfd and disconnect without closing listenfd. Return to the beginning of the loop again, and listenfd is still used as the parameter of accept. Accept() returns a file descriptor successfully and - 1 in case of error.
2.4.5 connect function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen); sockdf: socket File descriptor addr: Pass in parameters to specify server-side address information, including IP Address and port number addrlen: Incoming parameters,afferent sizeof(addr)size Return value: 0 is returned for success and 0 is returned for failure-1，set up errno
The client needs to call connect() to connect to the server. The parameter forms of connect and bind are the same. The difference is that the parameter of bind is its own address, while the parameter of connect is the address of the other party. Connect() returns 0 successfully and - 1 in error.
2.5 running cpp code under linux
Compile and execute C + + programs under Linux system.
vim set instruction.
2.6 server side development process (video 14)
- serve.c
#include<stdio.h>
#include<stdlib.h>
#include<sys/types.h>
#include<string.h>
#include<unistd.h>
#include<arpa/inet.h>
#include<netinet/in.h>
#include<sys/socket.h>
#include<ctype.h>
int main()
{
//1. Create a socket and return a file descriptor LFD --- socket
int lfd = socket(AF_INET,SOCK_STREAM,0);
if(lfd<0)
{
perror("socket error");
return -1;
}
//2. Bind lfd and Ip PORT ------ bind()
//int bind(int sockfd,const struct sockaddr *addr,socklen_t addrlen);
struct sockaddr_in serv; //Easy to copy
bzero(&serv,sizeof(serv));
serv.sin_family = AF_INET;
serv.sin_port = htons(8888); //Network byte order conversion, do not use ports before 1024
serv.sin_addr.s_addr = htonl(INADDR_ANY); //Indicates the use of any locally available ip inet_pton
int ret = bind(lfd,(struct sockaddr *)&serv,sizeof(serv));
if(ret<0)
{
perror("bind error");
return -1;
}
//3. Monitoring
// int listen(int socketfd,int backlog);
listen(lfd,128);
//int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen);
//3. Do not pay attention to the ip address of the client when receiving
//int cfd = accept(lfd,NULL,NULL); // If you don't care, fill in null first
//printf("lfd==[%d],cfd==[%d]\n",lfd,cfd);
//3. Pay attention to the IP address when receiving ------------- the received IP address
struct sockaddr_in client;
socklen_t len = sizeof(client);
int cfd = accept(lfd,(struct sockaddr *)&client, &len);
//Print out the port number and ip address
char sIP[16];
memset(sIP,0x00,sizeof(sIP));
printf("client-->IP:[%s],PORT:[%d]\n",inet_ntop(AF_INET,&client.sin_addr.s_addr,sIP,sizeof(sIP)),ntohs(client.sin_port));
//-----------------------------------------------------
int n = 0;
int i = 0;
char buf[128];
while(1)
{
//4. Read data
memset(buf,0x00,sizeof(buf));
n = read(cfd,buf,sizeof(buf));
if(n<=0)
{
printf("read error or client close");
break;
}
printf("n==[%d],buf==[%s]\n",n,buf);
for(i=0; i<n; i++)
{
buf[i] = toupper(buf[i]); //Case conversion function
}
//5. Send data:
write(cfd,buf,n);
}
//6. Close the listening file descriptor and communication file descriptor
close(lfd);
close(cfd);
}
2.7 client code writing (video 15)
//View connection status: netstat -anp | grep 8888
- client.c
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<sys/types.h>
#include<unistd.h>
#include<netinet/in.h>
#include<arpa/inet.h>
int main()
{
//1. Create socket -- communication file descriptor
int cfd = socket(AF_INET,SOCK_STREAM, 0);
if(cfd<0)
{
perror("socket error");
return -1;
}
//2. Connect the network end
//int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen);
struct sockaddr_in serv;
serv.sin_family = AF_INET;
serv.sin_port = htons(8888);
inet_pton(AF_INET,"127.0.0.1",&serv.sin_addr.s_addr); //The ip address is also converted into network byte order, and the corresponding high-speed communication should be iPv4 or ipv6
printf("[%x]",serv.sin_addr.s_addr);
int ret = connect(cfd,(struct sockaddr *)&serv,sizeof(serv));//Data can be read after the connection is established
if(ret<0)
{
perror("connect error");
return -1;
}
int n = 0;
char buf[256];
while(1)
{
//3. The device that sends data must be a network card that drives the bottom layer to send information
memset(buf, 0x00, sizeof(buf));
n = read(STDIN_FILENO,buf, sizeof(buf)); //Read in information from standard input
write(cfd, buf, n); //4. Read the data from the server
memset(buf, 0x00,sizeof(buf));
n = read(cfd, buf, sizeof(buf));
if(n<=0)
{
printf("read error or server closed,n==[%d]",n);
break;
}
printf("n==[%d], buf==[%s]\n",n, buf);
}
//Close socket
close(cfd);
return 0;
}