Practical operation of keepalive in linux

1, keepalive Foundation

1.1 VRRP Technology

vrrp related terms

  • Virtual Router: virtual router

  • Virtual router ID: VRID(0-255), which uniquely identifies the virtual router

  • VIP: Virtual IP

  • VMAC: Virutal MAC (00-00-5e-00-01-VRID)

  • Physical router:

  • Master: master equipment

  • backup: standby device

  • Priority: priority

1.2 VRRP related technologies

Announcement: heartbeat, priority, etc; Periodicity

Working mode: preemptive, non preemptive, delayed preemptive,

Safety certification:

  • No certification
  • Simple character authentication: preshared key
  • MD5

Working mode:

  • Active / standby: single virtual pather
  • Active / active: active / standby (virtual router 1), standby / active (virtual router 2)
#announcement:
It's to declare your sovereignty. Don't try to seize power and keep going out

#Preemptive:
The main server goes down. It's repaired after a period of time, and then grab the sovereignty

#Non preemptive:
The main server went down and was repaired after a period of time. The original main server was used as a backup

#Is preemptive or non preemptive?

#Safety certification:
If there is no security authentication, it is not in the cluster keeplive If the server sets an ultra-high priority, it will cause accidents

#Active / active: active / standby (virtual router 1), standby / active (virtual router 2), as shown in the figure below:
Environmental Science:
There are two servers
 Virtual two virtual routers
 In the first virtual router, server 1 is the master and server 2 is the standby, so the virtual router IP1 Floating on server 1, only server 1 really works
 In the second virtual router, server 2 is the main and server 1 is the standby, so the virtual router IP2 Floating on server 2, only server 2 really works
 advantage:
#Improved resource utilization:
In this way, the primary and standby servers work at the same time and can run two projects at the same time
#It also has backup function:
If server 1 breaks down, server 2 will also have virtual machines IP1 And virtual IP2


Disadvantages:
Although it has backup redundancy function, it requires very high performance of the machine. When one of them fails, one of them used to run one task, but now all businesses are pressed on one, which is very risky

2, keepalived introduction

2.1kkepalived function

  • Address flow based on vrrp protocol
  • Generate ipvs rules for the node where the vip address is located (predefined in the configuration file)
  • Perform health status detection for each RS of ipvs cluster
  • Based on the script, the calling interface completes the functions defined in the script, and then affects the cluster transactions, so as to support nginx, haproxy and other services

2.2 keepalived architecture

Official documents:

https://keepalived.org/doc/
http://keepalived.org/documentation.html

  • User space core components:
1. vrrp stack: VIP news notifications 

2. checkers: monitor real server(In short, it is to monitor the services of the real server on the back end)

3. system call: realization vrrp Function of calling script during protocol state transition

4. SMTP: Mail component (alarm mail)

5. IPVS wrapper: generate IPVS Rules (direct generation) ipvsadm)

6. Netlink Reflector: Network interface (virtual address ip(vip)Address (floating)

WatchDog: monitor the process (whether there is a problem with the whole architecture)

- Control components: provided keepalived.conf Parser, complete Keepalived to configure
- IO Multiplexer: its own thread abstraction optimized for network purposes
- Memory management component: provides access to some general memory management functions (such as allocation, reallocation, publishing, etc.)

keeplive can cooperate with software such as ngnix and reverse proxy

2.3 keepalived related documents

  • Package name: kept
  • Main program file: / usr / SBIN / kept
  • Main configuration file: / etc / kept / kept.conf
  • Configuration file example: / usr / share / Doc / kept/
  • Unit File: /lib/systemd/system/keepalived.service
  • Environment configuration file for Unit File:
  • /etc/sysconfig/keepalived CentOS

2.4 configuration composition

/Configuration composition of / etc / kept / kept.conf

  • GLOBAL CONFIGURATION

Global definitions: define mail configuration, route_id, vrrp configuration, multicast address, etc

  • VRRP CONFIGURATION

VRRP instance(s): defines the rules and basic information of each vrrp virtual router in the vrrp protocol

  • LVS CONFIGURATION (rule setting of lvs scheduling server)

Virtual server group(s)

Virtual server(s): VS and RS of LVS Cluster

2.5 global configuration

#/etc/keepalived/keepalived.conf 
global_defs {
 notification_email {
 root@localhost
 #keepalived the target mailbox of mail sending in case of failover. Multiple mailboxes can be written separately by line
 root@localhost
  360601212@qq.com 
 }
 notification_email_from keepalived@localhost  
 #Email address
 smtp_server 127.0.0.1     
 #Mail server address
 smtp_connect_timeout 30   
 #Mail server connection timeout
 router_id R1
 #The unique ID of each keepalived host. It is recommended to use the current host name, but multiple node duplicate names will not affect
 vrrp_skip_check_adv_addr  
 #Check all notification messages and compare the consumption performance. After this configuration is enabled, if the received notification message and the previous message are the same router, the check will be skipped and the default value is all check
vrrp_strict 
#Strictly abide by VRRP protocol. After enabling this item, the following conditions will not be able to start the service: 1. There is no VIP address. 2. Unicast neighbors are configured. 3. There is IPv6 address in VRRP version 2. When this item is enabled and vrrp_iptables is not configured, iptables firewall rules will be automatically enabled. VIP cannot access by default. It is recommended not to add this configuration.
 vrrp_garp_interval 0 
 #gratuitous ARP messages message sending delay, 0 means no delay
 vrrp_gna_interval 0 
 #unsolicited NA messages message sending delay
 vrrp_mcast_group4 224.0.0.18 
 #Specified multicast IP address range: 224.0.0.0 to 239.255.255.255, default: 224.0.0.18 
 vrrp_iptables        
 #When this item and vrrp_strict are enabled at the same time, firewall rules will not be added. If vrrp_strict is not configured, it is not necessary to enable this configuration
}

Address classification

Custom multicast is generally planned and cannot be configured blindly
 The Lord and the spare shall be the same
A Class: 1-126
B   128-191
C   192-223
D   224-239
E   240-


#Modify multicast
 default keepalived Multicast is used between hosts to announce messages to each other, which will cause network congestion. Unicast can be replaced to reduce network traffic
 Note: Enable vrrp_strict Unicast cannot be enabled when
#Set the IP of the opposite host in all node vrrp_instance statement blocks. It is recommended to set it as the address dedicated to the corresponding heart jumper network instead of using the service network

2.6 configuring virtual routers

vrrp_instance <STRING> {
#< string > is the instance name of vrrp, generally the business name
 configuration parameter
 ......
 }
#Configuration parameters:
state MASTER|BACKUP
#The initial status of the current node on this virtual router. The status is MASTER or BACKUP
interface IFACE_NAME 
#It is bound to the physical interface used by the current virtual router, such as eth0,bond0,br0. It can not be on the same network card as the VIP
virtual_router_id VRID 
#Unique ID of each virtual router, range: 0-255. This value must be unique for each virtual router, otherwise the service cannot be started. Multiple keepalived nodes belonging to the same virtual router must be the same. Be sure to confirm that this value must be unique in the same network
priority 100 
#The time interval of the virtual rrp advertisement of the current physical node is 1s by default
authentication { 
#Authentication mechanism
 auth_type AH|PASS   
 #AH is IPSEC authentication (not recommended), and PASS is simple password (recommended)
 auth_pass <PASSWORD> 
 #The pre shared key is only valid for the first 8 bits. Multiple keepalived nodes of the same virtual router must be the same
}
virtual_ipaddress { 
#Virtual IP, the production environment may specify hundreds of IP addresses
 <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
 192.168.200.100 
 #Specify VIP and no network card, and the default is eth0. Note: do not specify / prefix, and the default is / 32
 192.168.200.101/24 dev eth1   
 #Specify the network card of the VIP. The post card specified in the suggestion and interface instructions is not in the same network card
 192.168.200.102/24 dev eth2 label eth2:1 
 #Specify the network card label of VIP 
}
track_interface { 
#Configure the monitoring network interface. In case of failure, it will change to FAULT state to realize address transfer
 eth0
 eth1
 ...
}

3, Actual operation

3.1 start four real machines


Turn off the firewall

systemctl stop firewalld.service 
setenforc 0

3.2 installing ipvsadm

Real machine 1

[root@localhost ~]$]yum install -y ipvsadm.x86_64 keepalived.x86_64 

Plug in loaded: fastestmirror, langpacks
base                                                | 3.6 kB     00:00
extras                                              | 2.9 kB     00:00
updates                                             | 2.9 kB     00:00
Determining fastest mirrors
 * base: mirrors.163.com
 * extras: mirrors.cn99.com
 * updates: mirrors.163.com
 Resolving dependencies
--> Checking transactions
---> software package ipvsadm.x86_64.0.1.27-8.el7 Will be installed
---> software package keepalived.x86_64.0.1.3.5-19.el7 Will be installed
--> Processing dependencies ipset-libs >= 7.1，It is a software package keepalived-1.3.5-19x86_64 need
--> Processing dependencies libnetsnmpmibs.so.31()(64bit)，It is a software package keepali.3.5-19.el7.x86_64 need
--> Processing dependencies libnetsnmpagent.so.31()(64bit)，It is a software package keepal1.3.5-19.el7.x86_64 need
--> Checking transactions
---> software package ipset-libs.x86_64.0.6.29-1.el7 Will be upgraded
--> Processing dependencies ipset-libs(x86-64) = 6.29-1.el7，It is a software package ipset-1.el7.x86_64 need
--> Processing dependencies libipset.so.3()(64bit)，It is a software package ipset-6.29-1.e6_64 need
--> Processing dependencies libipset.so.3(LIBIPSET_1.0)(64bit)，It is a software package ip.29-1.el7.x86_64 need
--> Processing dependencies libipset.so.3(LIBIPSET_2.0)(64bit)，It is a software package ip.29-1.el7.x86_64 need
--> Processing dependencies libipset.so.3(LIBIPSET_3.0)(64bit)，It is a software package ip.29-1.el7.x86_64 need
---> software package ipset-libs.x86_64.0.7.1-1.el7 Will be updated
---> software package net-snmp-agent-libs.x86_64.1.5.7.2-49.el7_9.1 Will be installed
--> Processing dependencies net-snmp-libs = 1:5.7.2-49.el7_9.1，It is supported by package 1:nmp-agent-libs-5.7.2-49.el7_9.1.x86_64 need
--> Checking transactions
---> software package ipset.x86_64.0.6.29-1.el7 Will be upgraded
---> software package ipset.x86_64.0.7.1-1.el7 Will be updated
---> software package net-snmp-libs.x86_64.1.5.7.2-28.el7 Will be upgraded
---> software package net-snmp-libs.x86_64.1.5.7.2-49.el7_9.1 Will be updated
--> Resolve dependency complete

Real machine 2

[root@localhost ~]$]yum install -y ipvsadm.x86_64 keepalived.x86_64 

Plug in loaded: fastestmirror, langpacks
base                                                | 3.6 kB     00:00
extras                                              | 2.9 kB     00:00
updates                                             | 2.9 kB     00:00
Determining fastest mirrors
 * base: mirrors.163.com
 * extras: mirrors.cn99.com
 * updates: mirrors.163.com
 Resolving dependencies
--> Checking transactions
---> software package ipvsadm.x86_64.0.1.27-8.el7 Will be installed
---> software package keepalived.x86_64.0.1.3.5-19.el7 Will be installed
--> Processing dependencies ipset-libs >= 7.1，It is a software package keepalived-1.3.5-19x86_64 need
--> Processing dependencies libnetsnmpmibs.so.31()(64bit)，It is a software package keepali.3.5-19.el7.x86_64 need
--> Processing dependencies libnetsnmpagent.so.31()(64bit)，It is a software package keepal1.3.5-19.el7.x86_64 need
--> Checking transactions
---> software package ipset-libs.x86_64.0.6.29-1.el7 Will be upgraded
--> Processing dependencies ipset-libs(x86-64) = 6.29-1.el7，It is a software package ipset-1.el7.x86_64 need
--> Processing dependencies libipset.so.3()(64bit)，It is a software package ipset-6.29-1.e6_64 need
--> Processing dependencies libipset.so.3(LIBIPSET_1.0)(64bit)，It is a software package ip.29-1.el7.x86_64 need
--> Processing dependencies libipset.so.3(LIBIPSET_2.0)(64bit)，It is a software package ip.29-1.el7.x86_64 need
--> Processing dependencies libipset.so.3(LIBIPSET_3.0)(64bit)，It is a software package ip.29-1.el7.x86_64 need
---> software package ipset-libs.x86_64.0.7.1-1.el7 Will be updated
---> software package net-snmp-agent-libs.x86_64.1.5.7.2-49.el7_9.1 Will be installed
--> Processing dependencies net-snmp-libs = 1:5.7.2-49.el7_9.1，It is supported by package 1:nmp-agent-libs-5.7.2-49.el7_9.1.x86_64 need
--> Checking transactions
---> software package ipset.x86_64.0.6.29-1.el7 Will be upgraded
---> software package ipset.x86_64.0.7.1-1.el7 Will be updated
---> software package net-snmp-libs.x86_64.1.5.7.2-28.el7 Will be upgraded
---> software package net-snmp-libs.x86_64.1.5.7.2-49.el7_9.1 Will be updated
--> Resolve dependency complete

3.3 real machine 1 and 2 load the following modules

[root@localhost ~]$]modprobe ip_vs

3.4 copying files in real machine 1

[root@localhost ~]$]cd /etc/keepalived/
[root@localhost keepalived]$]ls
keepalived.conf
[root@localhost keepalived]$]cp keepalived.conf{,.bak}
[root@localhost keepalived]$]ls
keepalived.conf  keepalived.conf.bak

3.5 real machine 1 modification of configuration file





Delete everything from here‘

Before modification


After modification

3.6 modification rules

[root@localhost keepalived]$]vim /etc/sysctl.conf

Add rule
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0

[root@localhost keepalived]$]sysctl -p
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0

3.7 execution

[root@localhost keepalived]$]systemctl start keepalived.service
[root@localhost keepalived]$]
[root@localhost keepalived]$]
[root@localhost keepalived]$]ipvsadm-save >/etc/sysconfig/ipvsadm
[root@localhost keepalived]$]
[root@localhost keepalived]$]systemctl start ipvsadm.service 
[root@localhost keepalived]$]
[root@localhost keepalived]$]ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  127.0.0.1:80 rr
[root@localhost keepalived]$]systemctl restart keepalived.service 
[root@localhost keepalived]$]ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  127.0.0.1:80 rr
TCP  192.168.91.11:80 rr
  -> 192.168.91.102:80            Route   1      0          0

4, Deploy

(1) Configuration from load scheduler

systemctl stop firewalld.service && systemctl disable firewalld.service 
setenforce 0
modprobe ip_vs
cat /proc/net/ip_vs
yum install -y ipvsadm
cd /etc/sysconfig/network-scripts/ && cp ifcfg-ens33 ifcfg-ens33:0
echo > /etc/sysconfig/network-scripts/ifcfg-ens33:0

cat > /etc/sysconfig/network-scripts/ifcfg-ens33:0 <<EOF
DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.10.25
NETMASK=255.255.255.255
EOF

ifup ens33:0	#lvs_01 there are already VIP s. If you directly use ifup, the execution will fail
systemctl restart network
ifup ens33:0	#Execute again
ip a			#Confirm whether ens33:0 is effective

cat >> /etc/sysctl.conf <<EOF
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
EOF

sysctl -p
ipvsadm-save > /etc/sysconfig/ipvsadm
systemctl start ipvsadm && systemctl enable ipvsadm
ipvsadm -C
ipvsadm -A -t 192.168.10.25:80 -s rr
ipvsadm -a -t 192.168.10.25:80 -r 192.168.10.30:80 -g
ipvsadm -a -t 192.168.10.25:80 -r 192.168.10.40:80 -g
ipvsadm
ipvsadm -ln
123456789101112131415161718192021222324252627282930313233343536

(2) Master slave load scheduler installation configuration keepalived

keepalived The configuration of is roughly the same, but the name and priority are different. Pay attention to the notes and modify them one by one
yum -y install keepalived
#Install package

cd /etc/keepalived/
cp keepalived.conf keepalived.conf.bak
#backups


vim keepalived.conf
#The configuration file only needs to be from global_ For the content started by defs, the useless configuration above is also cleared
global_defs {						#Define global parameters
    #Line 10, the mail service points to the local
	smtp_server 127.0.0.1
    #In line 12, specify the name of the server (router). The names of the primary and standby servers must be different. The primary server is LVS_01, standby LVS_02
	router_id LVS_01
}

vrrp_instance VI_1 {				#Define VRRP hot spare instance parameters
#Line 20: specify the hot standby status. The primary is MASTER and the standby is BACKUP
    state MASTER
    #Line 21, specify the physical interface that hosts the vip address
    interface ens33
    #In line 22, specify the ID number of the virtual router. Each hot spare group should be consistent	
    virtual_router_id 10
    #In line 23, specify the priority. The higher the value, the higher the priority. The primary is 100 and the standby is 99
    priority 100
    advert_int 1					#Notification interval seconds (heartbeat rate)
    authentication {				#Define the authentication information and keep it consistent for each hot standby group
		auth_type PASS				#Certification Type
        #In line 27, specify the authentication password. The primary and standby servers are consistent
        auth_pass 123123
    }
    virtual_ipaddress {				#Specify cluster vip address
        192.168.10.25
    }
}
#Line 34, specify the virtual server address (VIP) and port, and define the virtual server and Web server pool parameters
virtual_server 192.168.10.25 80 {
    delay_loop 6					#Interval between health checks (seconds)
    lb_algo rr						#Specify scheduling algorithm, polling (rr)
    #Line 37, specify cluster operating mode, direct routing (DR)
    lb_kind DR
    persistence_timeout 50			#Connection hold time (seconds)
    protocol TCP					#The application service adopts TCP protocol
    #Line 41, specify the address and port of the first Web node
    real_server 192.168.10.30 80 {
        weight 1					#Weights of nodes
        #In line 43, add the following health check method and delete the unnecessary configuration		
        TCP_CHECK {
			connect_port 80			#Add target port for check
			connect_timeout 3		#Add connection timeout (seconds)
			nb_get_retry 3			#Add retry count
			delay_before_retry 3	#Add retry interval
		}
	}

	real_server 192.168.10.40 80 {		#Add the address and port of the second Web node
        weight 1
        TCP_CHECK {
			connect_port 80
			connect_timeout 3
			nb_get_retry 3
			delay_before_retry 3
		}
	}
}
#You can delete the following redundant network segments



systemctl start keepalived && systemctl enable keepalived
#The two load schedulers enable the keepalived service

ip addr show dev ens33
#View virtual network card vip
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475

Master DR network card information

From DR network card information

We can see that the master load scheduling node has a VIP, but the slave node does not, which indicates that the VIP is currently fixed in the LVS_01.

(3) Windows 10 test access

  • In the browser of the client, the Web page content can be accessed normally through the VIP address of LVS + kept cluster
  • When either the master or slave scheduler fails, the Web site can still be accessed (you may need to refresh or reopen the browser)
  • As long as two or more real servers are available, the load balance of traffic can be realized


(4) Turn off lvs_01 keepalived service test access

On the main load scheduler

yum -y stop keepalived
1

First, VIP will drift to lvs_02


In the browser access test, the load balancing is still effective, and the highly available load balancing cluster is successfully built


Restart lvs_01 kept service, VIP drifts back to lvs_01

5. Attachment

Upper master/There are too many comments from the configuration. Here is a pure version of the configuration without comments`
`According to their own IP ,Network card name and password to be set......Make modifications

(1) lvs_01

Our experiment is followed by LVS-DR Yes, I configured it earlier lvs_01 ，Only configuration is required here keepalived
yum -y install keepalived
cd /etc/keepalived/ && cp -a keepalived.conf keepalived.conf.bak
echo > /etc/keepalived/keepalived.conf

cat > /etc/keepalived/keepalived.conf<<EOF
global_defs {
   router_id lvs_01
}

vrrp_instance vi_1 {
   state MASTER
   interface ens33
   virtual_router_id 51
   priority 110
   advert_int 1
   authentication {
      auth_type PASS
      auth_pass 6666
   }
   virtual_ipaddress {
      192.168.10.25
   }
}

virtual_server 192.168.10.25 80 {
   lb_algo rr
   lb_kind DR		
   persistence_timeout 6
   protocol TCP	
 
   real_server 192.168.10.30 80 {
      weight 1			
      TCP_CHECK {
         connect_port 80
         connect_timeout 3	
         nb_get_retry 3
         delay_before_retry 3
      }
   }
   real_server 192.168.10.40 80 {
      weight 1
      TCP_CHECK {
         connect_port 80
         connect_timeout 3
         nb_get_retry 3
         delay_before_retry 3
      }
   }
}
EOF

systemctl start keepalived && systemctl enable keepalived
ip a show dev ens33
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253

(2) lvs_02

lvs_02 Is a newly added node, lvs_01 All operations must be done once, only in the configuration keepalived There are some differences
systemctl stop firewalld.service && systemctl disable firewalld.service 
setenforce 0
modprobe ip_vs
cat /proc/net/ip_vs
yum install -y ipvsadm
cd /etc/sysconfig/network-scripts/ && cp ifcfg-ens33 ifcfg-ens33:0
echo > /etc/sysconfig/network-scripts/ifcfg-ens33:0

cat > /etc/sysconfig/network-scripts/ifcfg-ens33:0 <<EOF
DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.10.25
NETMASK=255.255.255.255
EOF

ifup ens33:0
systemctl restart network
ifup ens33:0
ip a show dev ens33

cat >> /etc/sysctl.conf <<EOF
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
EOF

sysctl -p
ipvsadm-save > /etc/sysconfig/ipvsadm
systemctl start ipvsadm && systemctl enable ipvsadm
ipvsadm -C
ipvsadm -A -t 192.168.10.25:80 -s rr
ipvsadm -a -t 192.168.10.25:80 -r 192.168.10.30:80 -g
ipvsadm -a -t 192.168.10.25:80 -r 192.168.10.40:80 -g
ipvsadm
ipvsadm -ln

yum -y install keepalived
cd /etc/keepalived/ && cp -a keepalived.conf keepalived.conf.bak
echo > /etc/keepalived/keepalived.conf

cat > /etc/keepalived/keepalived.conf<<EOF
global_defs {
   router_id lvs_02
}

vrrp_instance vi_1 {
   state BACKUP
   interface ens33
   virtual_router_id 51
   priority 100
   advert_int 1
   authentication {
      auth_type PASS
      auth_pass 6666
   }
   virtual_ipaddress {
      192.168.10.25
   }
}

virtual_server 192.168.10.25 80 {
   lb_algo rr
   lb_kind DR		
   persistence_timeout 6
   protocol TCP	
 
   real_server 192.168.10.30 80 {
      weight 1			
      TCP_CHECK {
         connect_port 80
         connect_timeout 3	
         nb_get_retry 3
         delay_before_retry 3
      }
   }
   real_server 192.168.10.40 80 {
      weight 1
      TCP_CHECK {
         connect_port 80
         connect_timeout 3
         nb_get_retry 3
         delay_before_retry 3
      }
   }
}
EOF

systemctl start keepalived && systemctl enable keepalived
ip a show dev ens33

Keywords: Linux Operation & Maintenance server

Added by xbuzzx on Tue, 07 Dec 2021 14:25:19 +0200

Popular Keywords

©2024 Programming VIP