HA - Keepalived
Principle analysis
** 💯 Introduction: * * kepplived high availability technology prevents the inevitable impact of a single point of failure of the service and enhances the high availability and security of the service. For example, the failure of middleware rabbitmq, mysql, mycat, Haproxy and other servers will affect the operation process of the whole service system. Minimize the impact of software, hardware and man-made faults on the business.
characteristic
- Automatic failover
- There are two hosts (master and slave). The two servers will jointly simulate a virtual IP address, and users can access the rear service through the virtual IP.
- The master and slave will monitor the service status of each other. When the master server has a problem, the slave server will come to take over the virtual IP of the master server and continue to provide services. The user is unaware.
- Auto detect
- A core jumper is connected between the master and slave. The software on the host performs complex monitoring and logical judgment to detect each other's operation.
problem
Brain crack: two hosts (master and slave) each have Apache and keepalive. When our master service fails, the slave server will replace the service (coup)
However, when the two servers cannot communicate, they are divided into two independent nodes. Both sides think they are connected to each other, and there will be competition (resource sharing),
It will cause the servers on both sides to fail to get up, but it will also cause reading and writing (resource sharing) and data corruption (online log of database polling).
Solution
- Add redundant core jumper
- Boot disk lock
- The party serving locks the shared disk, while the other party can only read, not write.
- The service party only enables the disk lock when it finds that all the heartbeat lines are disconnected. It is not enabled at ordinary times.
- Set up arbitration mechanism
- When there are four machines, 1 master and 3 slave = = > when the master is down, the 3 slave will vote and choose the one with the largest IP
- When both parties are equal: the arbitration mechanism acts as one vote
- Monitoring and alarm: e-mail and SMS, etc
1, Keepalived deployment
**Multicast: * * send a message to a host in the same LAN. The host group receives the message and other hosts are deemed not to have received it
Multicast refers to 224.0 0.0 address as a way of communication address.
principle
N servers with the same functions form a server group, in which there is a master and multiple backups
There is a VIP providing external services on the master (the default route of the LAN where the server is located is the VIP of the server)
The master will send multicast information. When the backup service fails to receive the message, it will be considered that the master is down
Backup selects a backup to act as the master according to the priority of VRRP.
Function: the cluster management center is a service software that ensures the high availability of services and prevents single point of failure.
1. Environmental preparation
#Deploy web services on the same machine as keepalived web1/keepalived: 192.168.178.60 web2/keepalived: 192.168.178.61 # systemctl --now disable firewalld # sed -i '/^SELINUX=/cSELINUX=disabled' /etc/selinux/config # setenforce 0
2. Install software
1,Two hosts simultaneously # yum -y install keepalived httpd 2,to Web1 Add to: # echo "web1" > /var/www/html/index.html 3,to Web2 Add to: # echo "web2" > /var/www/html/index.html 4,Two hosts start: # systemctl start httpd
3,web1
3.1. Modify configuration
# cd /etc/keepalived/
# cp keepalived.conf keepalived.conf.bak
# vim keepalived.conf
! Configuration File for keepalived
global_defs { #Global configuration
router_id web1 #The identification of the device in the group can be set differently
}
vrrp_script chk_httpd { #health examination
script "/etc/keepalived/ck_httpd.sh" #Check script
interval 2 #Check frequency Seconds (execute the script every two seconds)
weight -5 #priority minus 5
fall 3 #Fail three times (try three times and execute the script every two seconds. If the script fails, write it down. After checking three times, reduce the priority by 5)
}
vrrp_instance VI_1 { #VI_ 1 (group number). The instance name is the same for both routers.
state MASTER #Master or slave state
interface ens33 #Monitoring network card (heartbeat network card)
mcast_src_ip 192.168.178.60 #Heartbeat source IP
virtual_router_id 55 #Virtual route number (group number) shall be consistent between active and standby.
priority 100 #Priority (primary application and 1 master-slave)
advert_int 1 #Heartbeat interval S (monitoring the running status of the opposite host) / can be millisecond monitoring
authentication { #Secret key authentication (1-8 bits)
auth_type PASS #Password authentication type
auth_pass 123456 #password
}
virtual_ipaddress { #VIP
192.168.178.200/24
}
track_script { #Reference script
chk_httpd
}
}
3.2. Prepare monitoring script
1,Write monitoring script:
# vim ck_httpd.sh
#!/bin/bash
#Check whether the httpd process exists
counter=$(ps -C httpd --no-heading|wc -l)
if [ "${counter}" = "0" ]; then
service httpd start
sleep 5 #Try to start httpd once, stop for 5 seconds and test again
counter=$(ps -C httpd --no-heading|wc -l)
if [ "${counter}" = "0" ]; then
systemctl stop keepalived #If the startup fails, kill keepalive to trigger the active / standby switch
fi
fi
2,Giving executive power:
# chmod +x ck_httpd.sh
3.3 copy to Web2
# scp -r keepalived.conf 192.168.178.61:/etc/keepalived/ # scp -r ck_httpd.sh 192.168.178.61:/etc/keepalived/
4,Web2
4.1. Modify configuration
# cd /etc/keepalived/ # ls ck_httpd.sh keepalived.conf # vim keepalived.conf #Find and modify route_id web2 #Route identifier state BACKUP #Identity from mcast_src_ip 192.168.178.61 #Heartbeat source, local IP priority 99 #Priority, less than the master setting
5. All hosts start service
1,Self starting service: # systemctl --now enable keepalived 2,View virtual IP On which host: # ip a | grep ens33 3,You can also observe the log and view the virtual IP Binding: # cat /var/log/messages | grep 'Sending gratuitous ARP'
6. Access test
1,As you can see, the virtual IP stay Web1 On, visit Web1 Services: # curl 192.168.178.200 web1
7. Fault test
**Resolution: * * shut down the Web1 service, view the virtual IP transfer, and transfer it to the Web2 server to access the Web2 service
1,close Web1 Upper http service,Observe in 2 seconds httpd Status, you can find that the service is pulled up again, and the effect of the script is as follows: # systemctl stop httpd # systemctl status httpd 2,modify httpd Configuration, let httpd The service doesn't work, it doesn't work VIP Transfer of: # mv /etc/httpd/conf /opt/ # systemctl stop httpd 3,Access the service again, VIP Successful transfer, # curl 192.168.178.200 web2
2, LVS + Keepalived actual combat
**Analysis: * * increase the number of LVS, form an LVS cluster, and keep the LVS highly available. In fact, users still access through the kept virtual IP. The virtual IP corresponds to the rear Web server. The kept essence is to use the LVS-DR mode.
**Architecture: * * client > lvs1 (HA) lvs2 > Web
1. Environmental preparation
LVS1: 192.168.178.60 LVS2: 192.168.178.61 Web1: 192.168.178.7 Web2: 192.168.178.16 # systemctl --now disable firewalld # sed -i '/^SELINUX=/cSELINUX=disabled' /etc/selinux/config # setenforce 0
2,LVS
2.1. Installing software
LVS1: # yum -y install ipvsadm keepalived LVS2: # yum -y install keepalived
2.2 LVS1 configuration
a. Modify profile
[root@192 ~]# cd /etc/keepalived/
[root@192 keepalived]# cp -r keepalived.conf keepalived.conf.bak
[root@192 keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS1 #Name, custom
}
vrrp_instance VI_1 {
state MASTER #The other machine is BACKUP
interface ens33 #Heartbeat network card
virtual_router_id 51 #Virtual route number shall be consistent between active and standby
priority 150 #priority
advert_int 1 #Check interval in seconds
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.178.200/24 dev ens33 #VIP and work interface
}
}
virtual_server 192.168.178.200 80 { #LVS configuration, VIP
delay_loop 3 #Time interval of service consultation,#Check real every 3 seconds_ Server status
lb_algo rr #LVS scheduling algorithm
lb_kind DR #LVS cluster mode
protocol TCP
real_server 192.168.178.7 80 { #Web1 server
weight 1
TCP_CHECK {
connect_timeout 3 #Health check method, connection timeout (connect to the Web every three seconds. If it times out, it is considered that the Web service is down)
}
}
real_server 192.168.178.16 80 { #Web2 server
weight 1
TCP_CHECK {
connect_timeout 3
}
}
}
b. Start service, restart
# scp -r keepalived.conf 192.168.178.61:/etc/keepalived/ # systemctl --now enable keepalived # reboot
2.2 LVS2 configuration
a. Modify configuration
# vim /etc/keepalived/keepalived.conf #Find and modify router_id LVS2 state BACKUP priority 145
b. Self starting service and restart
# systemctl --now enable keepalived # reboot
3,Web
Two hosts operate at the same time
3.1 installation and testing services
1,install httpd service # yum -y install httpd 2,Write test web page # Web1: echo web1 > /var/www/html/index.html # Web2: echo web2 > /var/www/html/index.html 3,Startup and self startup httpd # systemctl --now enable httpd
3.2. Configure virtual IP - > Lo: 0
# cd /etc/sysconfig/network-scripts/ # cp -r ifcfg-lo ifcfg-lo:0 # vim ifcfg-lo:0 DEVICE=lo:0 #Interface name IPADDR=192.168.178.200 #Virtual IP NETMASK=255.255.255.255 #Address uniqueness ONBOOT=yes #Self starting #Other notes
3.3. Configure routing
# vim /etc/rc.local #add to # The boot takes effect. Ensure that if the requested target IP is $VIP, the source address of the outgoing packet is also displayed as $VIP /sbin/route add -host 192.168.178.200 dev lo:0
3.4 configure ARP
1,ignore arp request,Can reply # vim /etc/sysctl.conf net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.default.arp_ignore = 1 net.ipv4.conf.default.arp_announce = 2 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2
3.5 restart effective
# reboot
3.6 test
# curl 192.168.178.200 web2 # curl 192.168.178.200 web1
3.7 fault test
1,LVS1 close keepalived，realization VIP Transfer: # systemctl stop keepalived 2,After accessing the service again, you can find VIP Transfer to LVS2 upper # curl 192.168.178.200 web2 # curl 192.168.178.200 web1