Linux - Ubuntu web server development
Linux - Ubuntu server development
The interface between the server and the client is through Socket communication
Socket communication has three elements:
- Target address of communication (ip address of server)
- Port number used (different protocols specify different ports, for example, port 25 corresponds to stmp protocol and port 80 corresponds to http protocol)
- Transport layer protocol used( TCP，UDP , these two are written in QT, but you can refer to communication logic)
Server development
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <ctype.h>
#include <arpa/inet.h>
#define SERVER_PORT 666
int main(void)
{
// Create mailbox, AF_INET is protocol family IPV4, SOCK_STREAM is a TCP connection type
int sock; // mailbox
struct sockaddr_in server_addr; // label
sock = socket(AF_INET, SOCK_STREAM, 0);
// Clear the label and write the address and port number
bzero(&server_addr, sizeof(server_addr));
server_addr.sin_family = AF_INET;
server_addr.sin_addr.s_addr = htonl(INADDR_ANY);// Listen to all local IP addresses
server_addr.sin_port = htons(SERVER_PORT); // Binding port number
// Attach the label to the mailbox
bind(sock, (struct sockaddr *)&server_addr, sizeof(server_addr));
// Open the mailbox on the network, and 128 is the number of connections that can be accepted at the same time
listen(sock, 128);
// Waiting for letters
printf("Waiting for client connection...\n");
int done = 1;
while(done)
{
// Receive the letter and get the client information
struct sockaddr_in client; // letter
int client_sock;
char client_ip[64];
socklen_t client_addr_len;
client_addr_len = sizeof(client);
client_sock = accept(sock, (struct sockaddr *)&client, &client_addr_len); // Accept connection
// Test information, examples, read, write usage
// Print client IP address and port number
printf("client ip: %s\tport: %d\n",
inet_ntop(AF_INET, &client.sin_addr.s_addr, client_ip, sizeof(client_ip)),
ntohs(client.sin_port));
// Read data sent by client
char buf[256];
int len;
len = read(client_sock, buf, sizeof(buf)-1);
buf[len] = '\0';
printf("recive: %s\n", buf);
// Returns the data sent by the client
len = write(client_sock, buf, len);
close(client_sock);
}
return 0;
}
Q: What are htonl, htons and ntohs?
A: It is a type conversion function, mainly the conversion between host byte (h) and network byte (n) (described in detail later)
Test connection
Win + R runs cmd and uses the command of telnet [IP] [PORT] to connect to the server. The server ip address can be viewed under link/ether using ip addr
If the telnet command is invalid, start the service:
- Windos 10 (same as Win7)
window settings - > find settings "control panel" - > Programs - > enable or disable Windows functions under programs and functions - > check "Telnet client"
Example: telnet 192.168.220.128 666 the ip address is self configured, and the port is set in the server program
After connection, if * * "client ip: [IP] port: [PORT]" * * is displayed in the program, it is successful
Use the Telnet client, * * ctrl +] * * to start the command mode, and enter sen hello!, It is successful to see that the server has feedback back to the test information
Linux - Ubuntu client development
Client development
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#define SERVER_PORT 666
#define SERVER_IP "127.0.0.1"
int main(int argc, char *argv[])
{
// Get parameters (content to be sent to the server)
int sockfd; // mailbox
char *message;
struct sockaddr_in servaddr; // Label, envelope
int n;
char buf[64];
if(argc != 2) // Program parameter error
{
fputs("usage: ./echo_client message\n", stderr);
exit(1);
}
message = argv[1];
//printf("message: %s\n", message);
// New Mailbox
sockfd = socket(AF_INET, SOCK_STREAM, 0);
// Clear the label and write the address and port number
memset(&servaddr, '\0', sizeof(struct sockaddr_in));
servaddr.sin_family = AF_INET;
inet_pton(AF_INET, SERVER_IP, &servaddr.sin_addr); // The IP address to be sent is connected to the local server, so 127.0.0.1 is used
servaddr.sin_port = htons(SERVER_PORT); // Send to port number
// Connect server
connect(sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr));
// send content
write(sockfd, message, strlen(message));
// Read server content
n = read(sockfd, buf, sizeof(buf)-1);
if(n > 0)
{
buf[n] = '\0';
printf("receivs: %s\n", buf);
}
else
{
perror("error!\n");
}
close(sockfd);
return 0;
}
Test connection
Opening the program requires text with parameters, such as/ a.out "This a message text!"
Usage: start the server, wait for connection - > start the client program, with parameters
supplement
- If you report an error when writing your own code, you can use: set nu in vim command mode to display the number of text lines.
- After the server is turned on, the input and output of the client connection are normal. If the server does not respond, it may be because the server may have legacy background threads running. You can use ps -ef | grep a.out(a.out is the file name) to check whether there are legacy threads in the background, and then use kill a.out (a.out is the program name) to end these legacy processes
Socket socket
Socket means "socket" in Chinese. In Linux environment, it is used to represent the special file type of network communication between processes x. It is essentially a pseudo file formed by the kernel with the help of buffer.
Since it is a file, of course, we can use the file descriptor to refer to the socket. The purpose of encapsulating it into files in Linux system is to unify the interface and make the operation of reading and writing sockets and reading and writing files consistent. The difference is that files are mainly used for reading and writing local persistent data, while sockets are mostly used for data transmission between network processes.
In TCP/IP protocol, "IP address + TCP or UDP port number" uniquely identifies a process in network communication. "IP address + port number" corresponds to a socket. The two processes that want to establish a connection each have a socket to identify, so the socket pair composed of these two sockets uniquely identifies a connection. Therefore, socket can be used to describe the one-to-one relationship of network connection.
flow chart:
Add: the TCP mentioned earlier is similar to * * "call", which is connection oriented; UDP is similar to "writing a letter" * *.
Network byte order
In the computer world, there are two byte orders:
Big endian byte order - low address high byte, high address low byte
Small end byte order - low address low byte, high address high byte
1001 = 0x03 e9: indicates that the decimal number 1001 needs to be stored in two bytes in the computer, 03 and e9 (hexadecimal) respectively
666. And 667: Address
Multi byte data in memory can be divided into large end and small end relative to memory address, and multi byte data in disk file can also be divided into large end and small end relative to offset address in file. Network data flow can also be divided into large end and small end, so how to define the address of network data flow? The sending host usually sends the data in the sending buffer in the order of memory address from low to high, and the receiving host saves the bytes received from the network in the receiving buffer in the order of memory address from low to high. Therefore, the address of the network data stream should be specified as follows: the data sent first is the low address, and the data sent later is the high address.
TCP/IP protocol stipulates that network data flow shall adopt large end byte order, that is, low address and high byte.
For example, the port number is 1001 (0x3e9), which is saved by two bytes. If the large end byte order is adopted, the low address is 0x03 and the high address is 0xe9, that is, 0x03 is sent first and then 0xe9. These 16 bits should also be 0x03 in the low address and 0xe9 in the buffer of the sending host. However, if the sending host is in small endian byte order, these 16 bits are interpreted as 0xe903 instead of 1001. Therefore, the sending host needs to convert the byte order before filling 1001 into the sending buffer. Similarly, if the receiving host is in small end byte order, the 16 bit source port number also needs to be converted in byte order. If the host is in large byte order, no conversion is required for sending and receiving. Similarly, network byte order and host byte order should also be considered for 32-bit IP addresses.
In order to make the network program portable and make the same C code run normally after being compiled on large-end and small-end computers, the following library functions can be called to convert the network byte order and host byte order.
#include <arpa/inet.h> uint32_t htonl(uint32_t hostlong); uint16_t htons(uint16_t hostshort); uint32_t ntohl(uint32_t netlong); uint16_t ntohs(uint16_t netshort);
h represents host, n represents network, l represents 32-bit long integer, and s represents 16 bit short integer.
If the host is in small endian byte order, these functions convert the parameters to large and small endian and then return them. If the host is in large endian byte order, these functions do not convert and return the parameters intact.
Supplement: IP uses long integers and PORT uses short integers
sockaddr data structure
Many network programming functions were born earlier than IPv4 protocol. At that time, sockaddr structure was used. For forward compatibility, sockaddr now degenerates into the role of (void *), passing an address to the function. As for this function, it is sockaddr_in or other, it is determined by the address family, and then the internal type of the function is forcibly converted to the required address type.
sockaddr data structure:
struct sockaddr {
sa_family_t sa_family; /* address family, AF_xxx */
char sa_data[14]; /* 14 bytes of protocol address */
};
struct sockaddr_in {
sa_family_t sin_family; /* address family: AF_INET */
in_port_t sin_port; /* port in network byte order */
struct in_addr sin_addr; /* internet address */
};
/* Internet address. */
struct in_addr {
uint32_t s_addr; /* address in network byte order */
};
The IPv4 address format is defined in netinet / in H, the IPv4 address uses sockaddr_in structure, including 16 bit port number and 32-bit IP address, but the implementation of sock API was earlier than ANSI C standardization, and there was no void * type at that time. Therefore, these parameters such as bind and accept functions are represented by struct sockaddr * type. Before passing parameters, forced type conversion is required, for example:
struct sockaddr_in servaddr; bind(listen_fd, (struct sockaddr *)&servaddr, sizeof(servaddr)); /*initialize servaddr */
IP address translation function
#include <arpa/inet.h> int inet_pton(int af, const char *src, void *dst); const char *inet_ntop(int af, const void *src, char *dst, socklen_t size);
af value can be AF_INET and AF_INET6, which corresponds to ipv4 and ipv6
Where inet_pton and inet_ntop can not only convert IPv4 in_addr, which can also convert IPv6 in6_addr.
Therefore, the function interface is void *addrptr.
Example:
#include <stdio.h>
#include <string.h>
#include <arpa/inet.h>
int main(void)
{
char ip[]="2.3.4.5"; // IPV4
char server_ip[64];
struct sockaddr_in server_addr;
// IPV4 host byte order - > network byte order inet_pton
inet_pton(AF_INET, ip, &server_addr.sin_addr.s_addr);
printf("s_addr : %x\n", server_addr.sin_addr.s_addr);
// Network byte order - > host string IP ntohl
printf("s_addr from net to host: %x\n", ntohl(server_addr.sin_addr.s_addr));
// Network byte order - > IPv4 host byte order inet_ntop
inet_ntop(AF_INET, &server_addr.sin_addr.s_addr, server_ip, 64);
printf("server ip : %s\n", server_ip);
// Macro representing all IP addresses
printf("INADDR_ANY: %d\n", INADDR_ANY);
server_addr.sin_addr.s_addr = INADDR_ANY;
inet_ntop(AF_INET, &server_addr.sin_addr.s_addr, server_ip, 64);
printf("INADDR_ANY ip : %s\n", server_ip);
return 0;
}
Output:
s_addr : 5040302 s_addr from net to host: 2030405 server ip : 2.3.4.5 INADDR_ANY: 0 INADDR_ANY ip : 0.0.0.0
Socket programming function
socket function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int socket(int domain, int type, int protocol);
domain:
AF_INET, which is most of the protocols used to generate socket s, uses TCP or UDP for transmission and IPv4 addresses
AF_INET6 is similar to the above, but it uses IPv6 address
AF_UNIX local protocol, which is used on Unix and Linux systems, is generally used when the client and server are on the same computer
type:
SOCK_STREAM is a sequential, reliable and data complete connection based on byte stream. This is the most used socket type. This socket uses TCP for transmission.
SOCK_DGRAM is a connectionless, fixed length transport call. The protocol is unreliable and uses UDP for its connection.
SOCK_SEQPACKET this protocol is a two-wire and reliable connection, which sends fixed length data packets for transmission. The package must be completely accepted before it can be read.
SOCK_RAW socket type provides single network access. This socket type uses ICMP public protocol. (ping and traceroute use this protocol)
SOCK_RDM is rarely used and is not implemented on most operating systems. It is provided to the data link layer and does not guarantee the order of data packets
protocol:
Passing 0 means using the default protocol.
Return value:
Success: returns the file descriptor pointing to the newly created socket
Failure: Return - 1, set errno
socket() opens a network communication port. If it succeeds, it returns a file descriptor like open(). The application can send and receive data on the network with read/write like reading and writing files. If the socket() call is wrong, it returns - 1. For IPv4, the domain parameter is specified as AF_INET. For TCP protocol, the type parameter is specified as SOCK_STREAM, which represents a stream oriented transport protocol. If it is a UDP protocol, the type parameter is specified as SOCK_DGRAM, which represents datagram oriented transmission protocol. The introduction of the protocol parameter is omitted. You can specify 0.
bind function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen);
sockfd:
socket file descriptor
addr:
Construct the IP address plus the end address
addrlen:
sizeof(addr) length
Return value:
0 returned successfully
Failure returns - 1, setting errno
The network address and port number monitored by the server program are usually fixed. After the client program knows the address and port number of the server program, it can initiate a connection to the server. Therefore, the server needs to call bind to bind a fixed network address and port number.
The function of bind() is to bind the parameters sockfd and addr together, so that sockfd, the file descriptor used for network communication, can listen to the address and port number described by addr. As mentioned earlier, struct sockaddr * is a general pointer type. The addr parameter can actually accept sockaddr structures of various protocols, and their lengths are different. Therefore, the third parameter addrlen is required to specify the length of the structure. For example:
struct sockaddr_in servaddr; bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(6666); //First clear the entire structure, and then set the address type to AF_INET, the network address is INADDR_ANY， //This macro represents any local IP address, because the server may have multiple network cards, and each network card may bind multiple IP addresses, //In this way, you can listen on all IP addresses. You can't determine which IP address to use until you establish a connection with a client. The port number is 6666.
listen function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int listen(int sockfd, int backlog);
sockfd:
socket file descriptor
backlog:
In Linux system, it refers to the length of queue waiting to establish handshake for 3 times
View the system default backlo:: cat /proc/sys/net/ipv4/tcp_max_syn_backlog
Change the backlog size of the system limit: VIM / etc / sysctl conf
Finally add:
net.core.somaxconn = 1024 net.ipv4.tcp_max_syn_backlog = 1024
Save and execute: sysctl -p
A typical server program can serve multiple clients at the same time. When a client initiates a connection, the server calls accept() to return and accept the connection. If a large number of clients initiate a connection and the server has no time to process it, the client that has not accepted is in the connection waiting state, and listen() declares that sockfd is in the listening state, A maximum of backlog clients are allowed to be connected. If more connection requests are received, they will be ignored. Listen() returns 0 for success and - 1 for failure.
accept function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen);
sockdf:
socket file descriptor
addr:
Outgoing parameter, which returns the address information of the linked client, including IP address and port number
addrlen:
Pass in the outgoing parameter (value result), the size of the incoming sizeof(addr), and the size of the actual received address structure when the function returns
Return value:
A new socket file descriptor is returned successfully for communication with the client. If it fails, it returns - 1 and sets errno
Three handshake processes:
After three handshakes, the server calls accept() to accept the connection. If there is no connection request from the client when the server calls accept(), it will block and wait until a client connects. Addr is an outgoing parameter. When accept() returns, the address and port number of the outgoing client. The addrlen parameter is an incoming and outgoing parameter (value result argument). The incoming parameter is the length of the buffer addr provided by the caller to avoid buffer overflow. The outgoing parameter is the actual length of the client address structure (the buffer provided by the caller may not be full). If you pass NULL to the addr parameter, it means that you don't care about the address of the client.
Our server program structure is as follows:
while (1)
{
cliaddr_len = sizeof(cliaddr);
connfd = accept(listenfd, (struct sockaddr *)&cliaddr, &cliaddr_len);
n = read(connfd, buf, MAXLINE);
// to do...
close(connfd);
}
The whole is a while loop, which processes one client connection at a time. Because cliaddr_len is an incoming and outgoing parameter. The initial value should be reset before each call to accept(). The parameter listenfd of accept() is the previous listening file descriptor, and the return value of accept() is another file descriptor connfd. After that, it communicates with the client through this connfd. Finally, close connfd and disconnect without closing listenfd. Return to the beginning of the loop again. Listenfd is still used as the parameter of accept. Accept() returns a file descriptor successfully and - 1 in case of error.
connect function
#include <sys/types.h> /* See NOTES */ #include <sys/socket.h> int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen);
sockdf:
socket file descriptor
addr:
Pass in parameters to specify server-side address information, including IP address and port number
addrlen:
Incoming parameter, incoming sizeof(addr) size
Return value:
0 returned successfully
Failure returns - 1, setting errno
The client needs to call connect() to connect to the server. The parameter forms of connect and bind are the same. The difference is that the parameter of bind is its own address, while the parameter of connect is the address of the other party. Connect() returns 0 successfully and - 1 in error.
Error handling function
We know that the system function call cannot guarantee success every time, and error handling must be carried out. On the one hand, it can ensure the normal program logic, on the other hand, it can quickly get the fault information.
#include <errno.h> #include <string.h> char *strerror(int errnum); /* See NOTES */
errnum:
The value of the error number of the passed in parameter is generally the value of errno
Return value:
Cause of error
#include <stdio.h> #include <errno.h> void perror(const char *s); /* See NOTES */
s:
Pass in parameters, custom description
Return value:
nothing
Output the error reason to stderr
-1. Set errno
The client needs to call connect() to connect to the server. The parameter forms of connect and bind are the same. The difference is that the parameter of bind is its own address, while the parameter of connect is the address of the other party. Connect() returns 0 successfully and - 1 in error.
Error handling function
We know that the system function call cannot guarantee success every time, and error handling must be carried out. On the one hand, it can ensure the normal program logic, on the other hand, it can quickly get the fault information.
#include <errno.h> #include <string.h> char *strerror(int errnum); /* See NOTES */
errnum:
The value of the error number of the passed in parameter is generally the value of errno
Return value:
Cause of error
#include <stdio.h> #include <errno.h> void perror(const char *s); /* See NOTES */
s:
Pass in parameters, custom description
Return value:
nothing
Output the error reason to stderr