During an interview
Interviewer: you use nginx for load. Do you use single nginx? What if nginx hangs up?
Me: Yes, because the business traffic is small, nginx is unlikely to hang. Considering the stability of the service, you can use kept.
Interviewer: do you know the idea and configuration of implementation?
I:...
1, Traditional high availability
Shopping rebate www.cpa5.com cnThe idea of tomcat high availability is to add a layer of load nginx in front of the tomcat cluster, as shown in the following figure:
But once nginx fails, the whole service will be paralyzed.
2, LVS thinking to solve the problem of high availability
1. What is LVS
LVS is the abbreviation of Linux Virtual Server, which means Linux Virtual Server. It is a virtual server cluster system. This project was founded by Dr. Zhang wensong in May 1998. It is one of the earliest free software projects in China -- Excerpted from Baidu Encyclopedia
LVS's personal understanding is to use multiple physical machine clusters to create a virtual ip (Virtual Server IP, referred to as VIP). The virtual ip is not an actual physical machine, so the virtual ip will not hang. The Linux kernel of LVS has helped us to implement it. The structure is as follows:
2. nginx+keepalived
On the basis of traditional high availability, multiple server clusters are used to manage LVS with keepavlied to create a virtual ip. As long as two nginx servers are not down, the service will not be paralyzed, as shown in the following figure:
3, Environmental description
- Demo machine ip: 192.168.5.11192.168.5.12
- Linux version: CentOS Linux release 7.6.1810 (Core)
- Kept version: 1.3.4
- nginx version: 1.13.1
4, keepalived specific configuration
1. Premise
Note: this step is required for both machines
1. Modify SELinux, close SELinux, open vim /etc/sysconfig/selinux, and set SELINUX=disabled. The Linux installed in VMWare and the ECS of Tencent cloud are closed by default, as shown in the following figure:
2. Install required dependent packages.
yum -y install libnl libnl-devel libnfnetlink-devel
2. keepalived installation
Note: this step is required for both machines
1. Do not use yum to install (there are bug s). Download from the official website of keepalived, upload from keepalived, or use wget to download and decompress after downloading.
wget https://www.keepalived.org/software/keepalived-1.3.4.tar.gz tar -zxvf keepalived-1.3.4.tar.gz
2. After decompression, specify the directory.
cd keepalived-1.3.4 ./configure --prefix=/usr/local/keepalived --sysconf=/etc
Note:
- --prefix: Specifies the installation directory.
- --sysconf: keepalived configuration file directory. If other configuration directories are specified, you need to specify the configuration file to start keepalived, such as: / usr/local/keepalived/sbin/keepalived -D -f configuration file path.
3. Compile and install.
make && make install
3. keepalived configuration
This step is slightly different, with one machine as the host (192.168.5.11) and the other as the backup machine (192.168.5.12)
Host (192.168.5.11) configuration
Open keepalived.exe in the keepalived configuration file directory (my name is / etc/keepalived) Conf file, configure the following information, and delete other redundant configurations.
! Configuration File for keepalived
global_defs {
router_id 192.168.5.11 # Unique identification of keepalived
}
vrrp_instance VI_1 {
state MASTER # Initial state, MASTER and BACKUP
interface ens33 # The name of the network card interface used by the system can be viewed using ip addr
virtual_router_id 51 # Group name, the same value as the configuration of the machines participating in this virtual machine ip, that is, all machines in a cluster use the same value
priority 200 # Priority: the higher the value, the higher the priority, and the highest in the group wins
advert_int 1 # Heartbeat detection once every 1 second
authentication { # Authorization without change
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.5.10 # Virtual ip
}
}
Backup machine (192.168.5.12) configuration
Similarly, open the keepalived configuration file directory and open keepalived The conf file is configured with the following information, which is different from the host_ ID, interface and priority are configured as follows:
! Configuration File for keepalived
global_defs {
router_id 192.168.5.12 # Unique identification of keepalived
}
vrrp_instance VI_1 {
state BACKUP # Initial state, MASTER and BACKUP
interface ens33 # The name of the network card interface used by the system can be viewed using ip addr
virtual_router_id 51 # Group name, the same value as the configuration of the machines participating in this virtual machine ip, that is, all machines in a cluster use the same value
priority 100 # Priority: the higher the value, the higher the priority, and the highest in the group wins
advert_int 1 # Heartbeat detection once every 1 second
authentication { # Authorization without change
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.5.10 # Virtual ip
}
}
4. Verify LVS effect
1. Start keepalived on the two machines respectively.
/usr/local/keepalived/sbin/keepalived
2. Use ip addr to check the effect. The normal result is that the virtual ip is located on the machine 192.168.5.11 because its configuration priority is higher. But result... No accident rollover, hahaha! The problem is that two machines have double VIP s. The screenshot of rollover is as follows:
I guess it's the firewall, so a wave of Google's big law, sure enough, found the reason: the firewall intercepted the vrrp broadcast, so BACKUP couldn't receive the MASTER broadcast.
Process for finding problems:
Install the packet capture tool tcpdump on any machine in the LAN. Command: yum -y install tcpdump
Execute tcpdump -i ens33 vrrp -n to check the situation. It is found that both machines are broadcasting. Normally, BACKUP should not be broadcasting.
To verify, turn off the firewalls of both machines. The command is systemctl stop firewalld Service, I found that the situation is normal.
Check the ip addr and find that the configuration is successful.
Release vrrp broadcast without turning off firewall
Under normal circumstances, we certainly don't need a firewall to run naked, so we need to release vrrp broadcasting.
# Note the network card name in the command firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface ens33 --destination 224.0.0.18 --protocol vrrp -j ACCEPT # service iptables restart firewall-cmd --reload
3. If the configuration is normal, you need to check whether the virtual ip will automatically drift to the BACKUP machine after the MASTER host goes down. Of course, in order to simulate the scenario, I just need to kill the keepalived service.
You can see that the BACKUP backup machine has taken over.
When the MASTER host restarts, the MASTER host will grab back the control of the virtual ip.
5, nginx+keepalived
After you configure keepalived, someone will ask, it seems that it has nothing to do with nginx. It does have nothing to do with drops, but the next configuration has something to do with it.
Sometimes, the server is not down, but nginx is down, and there is no relevant configuration in keepalived. In this case, if nginx in the MASTER host is down, the virtual ip will not automatically drift to the BACKUP machine. The next step is to use keepalived to monitor nginx in combination with the configuration of nginx.
1. Edit heartbeat execution script
My execution script location is saved in / usr / local / kept / chk_ nginx_ pid. SH, as follows:
#!/bin/bash
A=`ps -C nginx --no-header |wc -l` # Count the number of nginx processes. If it is 0, it indicates that nginx is killed
if [ $A -eq 0 ];then
# Restarting nginx is not the default configuration path, so you need to specify the path, which varies from person to person
/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
# If nginx fails to restart, stop the keepalived service and transfer the VIP
if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then
# Kill it and the VIP roams to another machine
killall keepalived
fi
fi
be careful:
- Give executable permission after saving: chmod +x chk_nginx_pid.sh
- Check whether the kill command can be used. If not, install psmisc: yum -y install psmisc
2. Configure the keepalived configuration file
The added configuration is shown in the figure:
vrrp_script check_nginx {
script "/usr/local/keepalived/check_nginx.sh" # Script executed by heartbeat
interval 2 # Test every 2 seconds
weight 2 # Priority change caused by script result: 10 means priority + 10- 10 means priority - 10
}
track_script {
check_nginx # Call detection script
}
3. Is the test effective
Close the processes of keepalived and nginx and restart keepalived to see if keepalived can automatically start nginx.
killall keepalived killall nginx /usr/local/keepalived/sbin/keepalived
Similarly, the BACKUP backup machine can also add the same heartbeat detection script and configuration.
The test found that no matter how to kill nginx later, nginx will be automatically restarted by keepalived, becoming an immortal Xiaoqiang, which realizes the high availability of nginx.
That's all I've read. Let's like, comment, pay attention and collect!
Author: IT Wang Xiaoer
Starting address: https://www.itwxe.com/posts/4c06301f/
Copyright notice: the content of the article follows the signature - non-commercial use - no deduction 4.0 international license. For reprint, please give the link between the author and the original text in an obvious position on the article page.